Why we Should Worry about Response Rates in Enterprise Social Systems

Why we Should Worry about Response Rates in Enterprise Social Systems 

response-rates-cartoonThis post continues our series on key SWOOP indicators. We have %Response Rate as a key performance indicator for organisations embracing problem solving and innovation within their Enterprise Social Networking (ESN) platforms. Difficult problems require deep dialogue, discussion and debate to be effectively solved. A response to a posting is hopefully the beginning of a constructive discussion, hence an important indicator of the degree to which an organisation is predisposed to solving problems online. Our ESN benchmarking of close to 50 organisations has the average response rate at 72%, but with a large range from a low of 32% to a high of 93%. response-rate-chartresponse-rate

The Response Rate widget identifies the percentage of posts that have received a written ‘reply’ and/or a ‘like’, for the period selected. It will also identify the % posts that have received no response; a measure that community managers need to monitor closely. The timeliness of the response is also reported.  

The Response Rate widget is available at all SWOOP reporting levels, from the individual, right through to the Enterprise overall. While not all posts are framed as problems, the response rate does reflect how responsive an organisation is overall. A response is a tangible signal of value received. In the absence of specific value stories, it is the most direct measure of value being facilitated on the ESN platform.  

For the individual, a poor response rate can indicate that your postings are not framed appropriately for attracting a response. For a group, a poor response rate may indicate a lack of a critical mass of members, or inadequate community management. 

Business Imperative 

It sounds obvious, but before problems can be solved, they need to be shared. Sharing a problem can be construed as a weakness. When senior management openly share a problem, they run the risk of ‘losing face’. Isn’t solving difficult problems what they are being paid to do?  Yet it is the senior management that need to lead the way in generating a culture for collaborative problem solving. As David Thodey, the former CEO of Telstra told us,Management don’t know everything…we have been guilty of releasing poor policies that have taken us years to recover from’. Thodey used the ESN to share problems that new policies were required for, and then getting feedback before finally releasing a new policy. 

The first challenge therefore is to develop a culture which respects that sharing a problem is not a weakness but a strength of character. Think about using hash tags to monitor problems posted, and their journey to a hopeful resolution. Once problems are shared freely on the ESN, the Response Rate measure can be used to measure problems solved. Many of the online technical forms are established specifically for tracking problem resolutions. There is no reason that the ESN cannot be used in a similar way. 

 

Tyranny of the ‘Long Tail’

Longtail

The advent of Internet enabled e-commerce brought an increased focus on ‘Long Tail’ distributions . Internet organisations like Amazon are able to exploit their low marginal costs by selling low volumes to the Long Tail of buyers with unique non-mainstream needs. The Long Tail has therefore been celebrated as the new opportunity of the Internet age. Even knowledge sharing systems e.g. blogs, podcasts, video have celebrated the increased reach that the Internet facilitates. The ubiquitous 90/9/1 rule acknowledges that 90% of participants are simply consumers of content.

The Pervasive “Long Tail” Distribution

Our own work with communities and social networks identifies the Long Tail effect. Our benchmarking of ‘Key Players’ with ‘off-line’ social networks identified that the majority of those with large personal networks is confined to a selected few. Our Key Player index identifies how concentrated the core of the network is by measuring the % of participants that represent 50% of all connections. For off-line communities we found that the key player index is typically between 11% and 32%. However, when we applied this measure to online Enterprise Social Networks (ESN), this range drops to from 4% to 12%, meaning as little as 4% of the community members are responsible for 50% of all connections, accentuating how online communities amplify the Long Tail effects. To further demonstrate how pervasive this long tail distribution is, in an earlier post we showed how the social cohesion within Yammer groups in one Enterprise followed the long tail power curve distribution. In a follow up analysis we dug deeper into the group we identified as the most cohesive, to better understand what was happening inside. And what we found was another long tail distribution. Of the 243 staff who had been active in this group, over a period of 18 months since launching, 70 had only a single interaction, while 12 members (5%) were responsible for nearly 68% of all interactions. So even in what are perceived the ‘best’ community groups, most of the connecting is being done by only a selected few.

Knowledge Sharing is not Enough?

Here is the issue. Just because more people are exposed to new information and knowledge, can we assume that new enterprise value is being generated? Perhaps for those organisations that measure their success through increased readership, this is fine. But I would argue that increased readership, if it doesn’t result in increased actions, is a shallow benefit at best. We experienced the same issues with Knowledge Management (KM) in the 1990s. In those days KM solutions were largely content centric. It was common to celebrate shared content. Those of us at the centre of KM programmes of the day were however continuously challenged by our executive to demonstrate real value. I can still recall our CEO addressing the knowledge team by saying “I’m not interested in awards or newspaper articles about how great our KM programme is. What I want to see is real, on the ground, impacts”. While we could see a real change in the level of knowledge sharing that was happening, evidence of real impact was limited to selected anecdotes and one off case studies. As impressive as some of these were, they were far from representative of a sustainable enterprise wide change. Interestingly, this is where many Enterprise Social Network community managers now find themselves today.

Engaging the ESN “Long Tail”

It appears that we cannot escape the ESN “Long Tail”, so what can we do to engage them in more active collaboration? We will be addressing this more comprehensively in future posts, but its suffice to say that simply appreciating the extent of its existence and then creating some targeted interventions is a good start. Taking a leaf out of Amazon’s play book, we need to accept that the needs of the “Tail” are not the same as those at the core. Likely their needs will be more diverse and unique. It’s therefore incumbent of the community leaders to ensure that there is a sufficient richness and diversity in the conversations they seed, to attract greater participation from the ‘Tail’.

Image citation: http://www.longtail.com/about.html

Need to convince someone? Bring Data (and a good story)

Big data

As Daniel Pink suggests “to sell is human”.  Even if we do not have a formal ‘selling’ role we are always looking to ‘sell’ someone on our point of view, our recommendations, our need for their help etc.. As data analysts we live and breathe data every day, whether we are looking to develop some new insights, prove a case or simply explore possibilities. In the end we are doing it to influence someone or some group. In these days of ‘evidence based decision-making’ I am wary that one person’s ‘evidence’ is another person’s ‘garbage’. You don’t have to look much further than climate change sceptics to appreciate that. I was therefore intrigued when I came across Shawn Callahan’s recent blog post on “The role of stories in data storytelling”. Shawn talks about the use of ‘story’ before, during and after data analysis.

Before data analysis stories

Before data analysis is about understanding the dominant ‘story’ before your analysis. For us a good example of this is our recent work on comparing relationship analytics with activity analytics. The dominant storyline was (and probably still is) that social analytics used in the consumer world i.e. activity measures, are sufficient for use inside the enterprise.

During data analysis stories

The ‘during the data analysis’ story is about how stories evolve from your act of data analysis. Our story in the interactions vs activity debate was about one of our clients observing some analytics provided by Swoop and finding that the measure for social cohesion was far more reflective of their view of how different communities were collaborating and performing than the activity measures reported beside them. For us the ‘stories during data analysis’ is continual. We are always looking to find the ‘story behind the data’. And this usually comes when we can talk directly to the owners of the data, in what we call ‘sense making’ sessions. As an example, we are currently looking at adoption patterns for Yammer using some of the benchmarking data that we have collected. We have learnt from experience that collaboration happens best within ‘groups’. Our prior analysis showed that the social cohesion between groups varies a lot and follows a typical ‘power curve’ distribution when sorted from best to worst. We are now looking at how these groups evolved over time. What patterns existed for those highly cohesive groups versus those that were less cohesive? Is there a story behind these different groups? Our evolving stories are merely speculations at the moment, until we can validate them with the owners of the data.

After data analysis stories

Knowing Doing GapShawn Callahan identifies these stories as needed to bridge the gap between what the data analyst ‘knows’ and what the decision makers need to act on.  He goes on to describe types of data stories, being a chronological change, explanation or discovery stories. He recommends that if you are trying to instigate change from a dominant current story, then it has to be a better story than that one. Thankfully in our case we don’t believe there is a dominant story for the use of activity analytics with Enterprise Social Networking (ESN) implementations. Of course there are supporters, some quite passionate, but the majority point of view is that they are insufficient for the needs of the Enterprise. That said, you still need to come up with a good story. And that is still work in progress for us. We can use a discovery story to relate the trigger for the data analysis we conducted being a simple comment from a client. But our sense is that we will need even more data (evidence) couched in some powerful stories told by individuals, who have changed their interaction behaviours for the better, based on the analytics that they were provided with.

I should finish by giving Shawn’s recent book “Putting Stories to Work” a plug, since I have just completed reading it to help us develop that story. So watch this space!

Can Online Personas Improve your Collaboration Behaviour?

When we hear the term “Personas” we often associate them with profiles that marketing organisations develop to categorise the buying behaviours of consumers for targeted attention. Personas are therefore strongly linked to behaviours. In the world of Enterprise Social Networking (ESN) we are also very much interested in the collaboration behaviours that have been facilitated by the ESN.

The idea of developing a way to profile ESN behaviours comes from one of our customers, Liz Green, who is a social media strategist at Telstra, and a leading facilitator of their Yammer installation. We liked the idea and decided to design and incorporate an online persona classification based on some of our core social networking analytics, which we are sharing here.

Here is the framework of Personas that we designed:


The vertical axis partitions those that are active on the platform from those that have minimal interactions. We identify those users who have interacted on the platform less than once every 2 weeks and classify them as “Observers”. For those that have interacted more than once every 2 week, we then break them up according to our Give-Receive balance measure. The give-receive balance was inspired by Adam Grant’s In the Company of Givers and Takers and Sandy Pentland’s The Science of Building Great Teams, where they find that those organisations and individuals that balance their giving and taking/receiving are the strongest performers. Our Give-Receive measure simply balances contributions made e.g. a posts, replies, likes etc. and received e.g. replies received, likes received etc.. We classify those active participants that are able to balance their giving and receiving as “Engagers”. The Engager is our aspirational profile, in that we believe these people are the heart of the network, successfully balancing talking and listening online.

For those people who are active but lean toward the “Receiver” side, we label “Catalysts”. These are people that are able to attract significant responses (replies, likes, etc) from relatively fewer contributions. You might consider a popular blogger or tweeter as catalysts for change. It is a skill and plays an important role in energising the network and attracting new participants.

For those active participants who fall toward the “Giver” side, it infers that they make far more contributions than they gain reaction for. We further partition them into “Responders” or “Broadcasters”. We use a Posts/Reply ratio measure to partition those that mainly contribute through Replies (Responders), from those that mainly contribute through posts (Broadcasters). Responders provide value through providing visible listening to contributors. They are like the ‘care givers’ in the community. Broadcasters tend to favour posting original content over participating in conversations. While this is not necessarily a bad thing, there is likely to be other channels available for broadcasting, while the ESN should be prioritised for conversation.

Targeting

For the community manager we believe that these personas can be used to characterise the overall network at a given point of time. We would envisage targets being set for Engagers (maximise), Observers (minimize), Broadcasters (limit), Catalysts and Responders (encourage). We would also encourage individuals to look at their own persona, and adjust their online behaviour toward the role they feel they are best placed to play and contribute.

We have now tested our design on several data sets. The detailed results are beyond the scope of this post and will be reported later. It is worth noting though that the overall profile make-up will change with ESN maturity and also the time period selected for assessment. With SWOOP we have already started to enable individuals to monitor these personas in real-time, enabling individuals to make adjustments as they see fit. Likewise for the community manager, it can provide an indication of trends that could be either amplified or dampened as appropriate. For consultants or advisors that assist organisations on their efforts to improve collaboration, the personas will also be a great way to target interventions where it will provide the greatest return.

So what do you think? Would you or your organisation benefit from online Persona profiling?

Relationship Mapping and Monitoring with Yammer

Yammer&MicrosoftYammer is a leading social networking platform for use inside organisations. Its recent acquisition by Microsoft is not only good for Yammer, but for the many Microsoft Enterprise clients who have been struggling to ‘connect’ via Sharepoint. What is most exciting for us is that the combination of Microsoft’s Active Directory with Yammer’s conversational platform now provides a real opportunity to implement the ‘Real-time Social Business Dashboard‘ which will enable enterprises to move beyond their current process monitoring to see how people are really collaborating (or not) to meet organisational objectives. 

So what’s wrong with the analytics that Yammer currently provide? Well nothing really, other than the fact that its only using a proportion of the intelligence available from its tracking data. Like most of its competitors, the analytics are what we would call “ego-centric”. In other words they track the activity patterns of individuals and then aggregate the data at team, department, company level to assess the level of engagement (read usage). The knowledge management community learnt a long time ago that activity doesn’t always map to productivity. Setting performance measures against ‘documents submitted’ resulted in lots of poor quality documents being uploaded just before performance appraisal time. But ego-centric measures will still reward this. In the social media space numerous postings in forums or voluminous tweets provide little indication of effectiveness unless they provoke a response. Social network measures focus on the relationship. Relationships are jointly owned. A direct response to a post creates a relationship. A heavy interchange of messages infers a stronger relationship (not necessarily friendly, but still a more established one than  where no interchange has occurred). A ‘like’ is also a connection. Counting ‘likes’ can be good for the ego, but even better when we know who is doing the ‘liking’. 

The Social Business Dashboard has ‘relationships’ at its core. That is not to say that current ego-centric measures would not be included. For example the volume of posts is clearly a useful indicator of activity. However social business is about collaboration. ‘Connected activity’ is what we are looking for, as we know that this form of activity is what leads to high productivity. The key component for a Social Business Dashboard is the Social Network Map. The map makes visible the network connections exposed through Yammer. By analysing the map one can see the flow of knowledge and information across the organisation. Accompanying analytics can identify who your key talents are, not by their CV, but who actually seeks them out for advice. We can identify the level of reliance on key players, the level of cross department collaboration, the areas where there may be bottlenecks impacting on customer service, order to cash cycles, ideas to innovation cycles and/or prospect to client conversions.

We have written previously about how network analytics can predict higher levels of efficiency, effectiveness and innovation, how social business drives ROI and what we are calling Social Analytics 2.0. We think this new relationship between Microsoft and Yammer will pay dividends by bringing ‘Social’ into mainstream enterprises, flagging a maturity in the market that we have long waited for.

Example Yammer Interactive Social Network Map

Below we provide an example Social Network map derived from a Yammer installation. The context was an “open innovation jam” where participants were drawn from across businesses to explore new energy and sustainability ideas and opportunities. ‘Connections’ are drawn from the Yammer discussion forum data. Fictitious organisation names have been added to provide an illustration as to how Active Directory profiling information would be included in the map.

The Optimice Webmapper utility enables one to interactively explore a social network map. The ‘flyout’ menu (use the orange triangle to control) allow you to select what attributes you want to colour (Organisation or Explore-Exploit) and/or filter the nodes by. You can also choose what relationships (Relationship Strength or Time) you want to explore.

[hana-code-insert name=’Yammer’ /]

The initial scenario shows the ‘Relationship Strength‘ map. The strength is determined by the number of posts made between pairs of participants. The size of the node relates to the number of posts received; a possible indicator of influence. Move the strength slider from left to right to expose only the strongest connections. The red links identify reciprocated postings. We like to think of these as another indicator of relationship strength. You can use the explore-exploit selectors to show only the explorers, or only the exploiter organisations. Clicking on any individual node will expose the network for just that individual. Select ‘Show All‘ to restore the map. If you like you can use the flyout menu to change the filter to ‘Organisation‘. You have to press ‘Update Map‘ anytime you change something on this menu. You can now turn on and off organisations.

Now try and change the ‘Relationship Strength‘ to ‘Time‘ and the ‘Strength Type‘ to ‘Strength >‘. Press ‘Update Map‘ to expose the new map. The map shown is the final state map but move the strength slider from left to right and the map will show how the connections built over time. You can now imagine how a dashboard might show this map evolve in real-time, while allowing analysts to ‘replay the past’ to diagnose impending issues and/or opportunities.

We are currently looking for organisations that have successfully integrated Yammer into their Microsoft Enterprise environment and are interested in pursuing a ‘Social Business Dashboard’ strategy, as described above.

Contact llocklee@optimice.com.au or cai.kjaer@optimice.com.au.