Tiptoe Towards PR Meassurement

This weekend I anticipate being able to analyse all the pages of designated web sites using semantic analysis. The objective is to be able to identify values systems evident in the text in the form of semantic chunks.

Using this approach one can identify those elements of text (such as sentences) that are, by virtue of containing words (concept words) identified as having enhanced value by virtue of their strength of connectedness among the words in the corpus, of greater significance than others.

There are a number of approaches one can take. For example one can identify the strength of concept words by page or from the combined texts of all the pages.

My requirement is to be able to identify those concepts that are most connected throughout the web site to yield chunks of text ranked from most significant to least significant (and to identify the URL of the pages from which they are derived).

This is one of my approaches to help provide empirical proofs to support the Relationship Value Model .

The model posits that relationships are based on values shared between actors and semantic chunks of text have characteristics akin to values. For all intent and purposes, semantic chunks in web sites are expressions of values. They are not the complete set of because other elements such as design, site uptime, photographs, video and other images are also expressions of organisational (and personal) values.

This new development will take the hard work out of identifying the value systems inherent in a web site and is the first stepping stone towards being able to identify common values between organisations and actors.

I guess that a lot of people will be interested in the values their web site presents to the public (and those of competitors ) but this is only the first step in this journey.

Semantic concepts are valuable in other directions too.

Search engines use semantic analysis of web pages as part of their algorithms to match up search terms to web sites and an example of how this works is provided by Yahoo. Its 'Search Assist' service provides lists of semantic concept words to help people using its search engine.

Thus there is a commercial value in my research at an early stage. It can show people trying to optimise web site content how effectively their content has contributed to accessibility of their site to the public through both common values and search.

The next research aim using semantically derived values is the be able to compare the commonality of values as between different web sites. Thus one can combine the web page corpus of two sites to identify all the concepts for both and the extent to which there are common concepts, unique concepts and the relative significance (lets call it rank for the time being) of different concepts and their associated semantic chunks of text.

So far so good. But can this approach go further? Let us imagine comparing the values of an organisation as expressed through its web site (the place where more important visit most often) and the values expressed in, say, the media or blog posts or in social networks.

In theory, and we will be able to test this in a few weeks, we will be able to identify those values that these media have in common with an organisation and the values that are expressed that are unique to either the media in question or the organisation. This will offer a very powerful view of whether the message is 'getting through'.

The extent to which there is convergence and divergence is, surely, a test of how close the relationship is between the organisation and its stakeholders.

Is this a measure of the effectiveness of public relations as a whole?

It certainly has potential.

Is there a relationship between profitability and web presence

I have been looking at relationships between numbers of pages indexed by search engines each year and the financial results of companies.

There are a lots more to do before one can draw conclusions but hear is an example from one company.

The key here is to see the relationship between numbers of pages that include reference to the company and the profit indicators.

Does this mean that having a big online footprint aids profits>

The Online Asset - How Obama won

In the past, I have commented on the relationship between traditional press relations and online content.

The difference between the two is that a press release has a limited long tail value whereas an article online has a long half life and contributes to the digital footprint, Google Juice and visibility of an organisational long tail asses.

The former is best considered in accounts as part of P&L and the latter as a corporate asset on the balance sheet.

The extent of goodwill in the former is less potent than the latter.

Today, with Robin Gurney, I am building a company to help organisations identify this corporate asset in its wider sense.

The argument goes like this:

Lets suppose you are the government minister for a small island in a large archipelago in the middle of the ocean and want to increase its GDP to benefit the people of the island.

You elect to invest in building the digital footprint of the island and work at it until it has an online presence ten times bigger than all the other islands nearby.

Thus when people seek to comment about, visit or invest in the group of islands, your island always comes up first in searches, in information resources and in connectivity. The result will soon see more economic activity.

There is a direct relationship between online presence and economic activity.

In concept, it is an extension of the London School of Economics Reports which showed that a 7 per cent increase in word of mouth advocacy unlocks 1 per cent additional company growth (http://tinyurl.com/aohth).

As for an island, so for any country, company, brand, person or politician.

It is not quite a case of simple counts. There are other factors.

Lets take a very simple look at the US Democratic race and see if we can add to other commentators efforts.

The Google index for Barak Obama is, today 5,3700,000 and Hillary Clinton is 5,4300,000.

If my theory was going to be correct, Clinton should have won the race and, plainly, she did not.

But, if we look closer at the results, we see that Clinton has a couple of decades of commentary that is indexed and Obama only four years worth. He made up a lot of ground very quickly.

Now lets look at blog posts:

Obama blog posts 99,859,318
Clinton blog posts 34,127,553

This is a very different picture. Here we are seeing like for like time scales and the digital footprint shows Obama attracting three times as much comment which, one can argue, helped Obama Google Juice as well as wider visibility. What is more, the digital footprint for Obama had the benefit of a steeper long tail curve (more content was more recent).

Then, again, take photographs. Online Obama has 2,990,000 photos indexed by Google and most of them are less than six years old. On the other hand Clinton only has 2,530,000 photos and hers go back to 1969 (see picture above)!

This is why Robin and I think it is important to have robust methodologies for audition online presence. It affects political and corporate outcomes and the extent of correlation between online presence and success are far too close to be co-incidental.

There are other indicators in the use of evolving channels. Twitter is one. Clinton has 4,019 followers and Obama has 33,069 followers; Obama is following 33,960 people and Hillary is following 0.

Leavering value from online presence is not hugely dependant on other marketing activity unless that serves the online presence in a virtuous circle (as Google, Facebook, eBay and Amazon attest) and so auditing online presence is important to be able to identify the value of an organisation.

In addition, it is a significant element in valuation of organisations and for establishing thier future prospects.

Obviously, there will be much more to follow as our research reveals more information but here are some interesting findings over the last few months.

1. In the UK social media interest in organisations in general changed last autumn. The rate of increase in social media comment accelerated very significantly.

2. The rate of change of inbound links is greater for more successful companies.

3. The long tail effect does have a half life. Content effect decays over time which means that having been online for a long time is not always good in itself.

4. Active sites and active social comment raises the digital footprint faster.

Fun huh!

Agnostic evaluation of media content

Image via Wikipedia

This weekend, I have been playing with a sentiment engine and thinking about people's perspectives.

This one does not take sides. It is agnostic. It decides if a press story is positive or negative from a neutral perspective.

All you do is past text into a box and it analyses the content as positive or negative and shows the structure of sentences that leads to this conclusion.

You can try it here.

The returns it makes is an academic minefield. It challenges your thinking about truth.

It has a first cousin, that can add perspectives. For example, it can make similar decisions from the perspective of the actors (company, person brand etc) which is why it was invented but the returns with no perspective are very interesting.

Sometimes this programme gets it wrong but not often.

It is able to glean content that is negative but which contributes to the positive side of the article and you can see how it does it.

I have tried news articles, book chapters, reports and even client presentations to see where the sentiment lies.

So where is the beef?

This kind of development is useful for analysing sentiment of news articles, blogs and other content, which is its primary purpose but it also has applications in evaluating style and and bias all of which are very useful to the PR industry, regulators and watchers of political sentinemt on and off line.

Try it out and be challenged.

Measuring the value of PR - the values that make relationships

Over the next few days I will be working on development of tools that allow me to look at corporate and brand values.

You see, it’s quite hard to really see what organisational values are. There are the things organisation say and claim as values and then there is the reality.

First we need to be able to see what values are claimed by organisations.

My route is, as you might expect, mechanistic, replicable and agnostic. To achieve this computer programme will examine the public face of the organisation (government department, company brand) as is evident on their web site. Yes, it does mean opening up every page, extracting the text elements and, to identify potential value statement, process the text to identify the semantically important phrases. I have elected to choose the most significant ones and limit them to a maximum of three per web page.

This will, experience shows, provide a heap of sentences and they have to be refined. Using part of speech analysis we can identify those phrases that are adjectival.

These phrases can be considered the values of the organisation.

With this smaller group of phrases (values), we can explore those that have semantically similar content and identify generic value systems on the web site.

Hey presto, this is a way of identifying the public expression of values of the organisation.

But, as the more cynical of us might imagine, these values will be those that the organisation wants us to see (they may not be in the same order of significance that a company or government department one brand manager would choose - but we are being agnostic here).

We now need to test these values in the cauldron of public opinion.

The first cauldron is that host of people who have expressed an interest in the organisation. That is, those people who have linked to it. What are their values and which ones do they have in common.

Then there are the commentators like the press, bloggers and others who don't link in.

We can now test the expressed values of the organisation against the values of its publics.

The extent to which there is dissonance is an expression of the value of the organisation's public relations.



That's the theory.



Now to see if it works.....

The Clip Book

Working part time in an agency gives an insight into how far they have regressed.

In the 1980's and 90's I ran a PR consultancy. We had a sister evaluation company (Media Measurement) which did all out work on clip book production as well as evaluation to meet a range of client needs. In those days we used a lot of trees in PR.

The evaluation team were specialist, had all the software and equipment to deliver product on time and right first time.

Professional PR people and awesome writers were not wasted on clip counting and mounting.

Easy.

Now, a decade later I find that agencies are still using PAPER! The Clipbook carbon footprint in the PR industry is massive.

Because clients want it?

Now, I don't care about the Newspaper Licensing Agency (NLA) demand for huge amounts of money to force organisations to knock over more trees. Its time to face them down.

I am not a person who believes that you can't read off the screen. We all do it.

I don't care about size for size Advertising Equivalents spreads and all that complete AVE rubbish. That was invented to satisfy the 20th century egos of Marketing Directors. In those days they had secretaries who typed letters!

Paper guard books offer so little information compared to digital ones. Today, would professional managers accept cash flow forecasts without drill down?

But what I resent most is the complete an utter waste of intelligent people’s time and on every count. Cost, wasted skill, environment damage, encouragement of lazy, typically innumerate, PR practice are all reasons to move away from paper.

Partnering with a professional evaluation company is the right direction for agencies (and I mean partner – not supplier) and this needs to be a three way, transparent relationship with the client.

In cost saving alone, it makes sense.

Search v Value

In recent weeks I have been working for Publicasity. They are developing their online skills - and the new web site, blog Twitter etc. is due in a few weeks (of course).

But we have been working on audits for a range of clients and companies and one the striking statistics that emerge is search volume.

More than once, we have un-nerved marketing directors with a graph that they recognize as sales graphs.

These search graphs are now beginning to be important. A way of being able to identify the how well companies are doing over the counter and online in terms of sales.

It struck me that there are more applications for these kind of data.

Knowing that search volume tells us about the commercial success of an organisation means that these data could also be useful to investors and others.

This conjoin between actual activity and online activity is another way of expressing the value of online work.

Finding nice bloggers - finding nasty bloggers

In recent weeks, development in using Bayesian inference on Latent Semantic Analysis of blogs has been keeping me up at nights.

From a simple engine developed by my friend Girish, I have been thinking about how one can use this information as a predictive tool and it has some promise.

Let me go back a bit.

Here is a simple application to find out the sentiment of bloggers between two competing entities. You can try it out. Try comparing 'clinton' with 'obama' and see who is getting the most positive and most negative blog posts.

You will also see that the programme also measures the extent of objectivity in the content analysed.

Because of the way we have set this application up if the comment about the candidates is neither positive or negative it produces a nul return (because of the nature of the experiment).

This experiment has moved on quite a lot and soon I hope to have a more advanced example you can try out here.

The experiment we are working on is to use the concepts identified in the blogs to monitor what subjects are growing and which are retreating and the sentiment attached to them.

The next phase is to predict where such concepts are going within a level of confidence that is acceptable.

Now comes the theoretical leap of faith. When a new concept emerges, it will be disruptive and may well provide an insight what future news will be.

Now that really will add a new dimension to PR practice.

Lies, Damned Lies, Opinion Polls

Will Beresford, strategy director at Beyond Analysis, has some interesting comments in Channel Web this week.

His thoughts come at a time when, once again, opinion polling shows its frailty in the US primaries.

Beresford points out that:

Traditional models for businesses to research their consumers are also expected to change. Customer information will be enriched by data found on the social web to supersede traditional research tools such as questionnaires and focus groups.

Feedback and influence from social networks will ultimately become more significant factors in the purchasing decision cycle.

Of course, his view has to be predicated on the niche nature of online communities and the demands, which he outlines in the article, of social networks.

This is not easy. Many organisations are dismissive of online content because it seems to them to be banal. Often this is because they have a search engine view of content which gives one reference point in what is often a longer and more structured, often networked, conversation and at other times, to quote Professor Howard Rheingold: “When you claim to be sceptical of the authenticity of online socializing and advise enthusiasts to "get a life," who made you the authority on what a life ought to be?”

We also have to think in term of flow (Kay McMahon has an interesting paper). The theory helps explain the allure of online games and social networks. Flow is the mechanism that helps engage the user and is powered by basic instincts, something I have touched on. The drivers behind this is distilled into a short video and a lengthier post or two.

So, I agree with Beresford and invite the online community to show how to be effective in finding out more about user generated 'market' segments.

Risk Management the in thing in PR

Professor Paul Argenti at Dartmouth has written a paper for Factiva about PR evaluation.

His key findings are interesting in the light of my post on risk management and its application to PR planning and management.

His paper has a these interesting findings and uncertainty and risk management seem to be the big buzz in the C suite.

• Measuring quality – as GE does through its statistics-based Six Sigma program, which has saved the company billions of dollars;
• Mitigating risks – as investment managers do when they use statistical analysis to diversify clients’ investment portfolios
• Predicting customer behavior – as the pharmaceutical industry does when it uses statistical analysis to demonstrate the effect of
spending on direct-to-consumer advertising on sales

well.... you read it (at length) here.

Evaluating Online Public Relations

Evaluating the effects of online public relations is a hot topic. There are all manner of ideas. People suggest ways that include Return on Investment, measurement of outputs, out-takes and outcomes and much more.

None seem to have a ring of truth. Its a pretty hard thing to do when everything from a blog post to twitter tweet to a mainstream news item can result from something as simple and a wiki contribution.



I have put some thought to it in a five minute lecture.
The lecture URL is: http://www.archive.org/details/5Minute-evaluation1

Here it is again because I am trying to make streampad work (so you can download the script and embed it yourself as well).

Evaluation by another name

The discussion at E-consultancy’s March 2007 roundtable on Web Analytics offers some interesting insights.

They suggests that this form of measurement is now an industry worth some £56 million (2006).

They also offer some ideas as to how outcomes can be measured in this post.

The importance of all this is that it is now part of the mix of reporting data needed for PR in our Internet Mediated world.

Does Google drive traffic?

Last September Google was ordered by a Belgian court to remove all French and German-speaking articles from the News results and cache of Google.be. Found in violation of copyright laws, The case was brought to court by Copiepresse, a Belgian copyright firm that represents French and German news journals, including Le Soir, La Libre Belgique and La Derniere-Heure.

Andrew Larks' pick up on what is happening to the traditional print media prompted me to look at Belgium again.

I was interested in the results of the action against Google believing that it would have a deleterious effect on the pages viewed.

I was wrong if Alexa is to be believed. Reach is mostly up (in some cases a lot) and readership is down but no more that other publications in other countries. Eventually the predictions of Doc Searle and Warren Buffet have to be examined. Attrition is eating away at the traditional publishers across the world. But is denying Google access to news pages going to hold back the digital tide or not?

The results look like this:

www.lalibre.be

The number of unique pages viewed per user per day for this site up 22%. Reach is up 152%




www.dhnet.be
The number of unique pages viewed per user per day for this site down 8%. Reach up 88%.



www.lecho.be
The number of unique pages viewed per user per day for this site down 13%. Reach up 159%.



www.lesoir.be
The number of unique pages viewed per user per day for this site down 6%. Reach down 11%

The UK's Daily Telegraph with its new Internet focused investment has page views down 8%. Reach is down 6%.




Reach for Lesoir is down 11% compared to the Telegraph fall of 6% but the remainder of the Belgium papers has soured.

What seems to be happening is this:The newspapers have to gain a huge increase in reach to maintain page views in a post Google era. But they have achived it.

This is an event worth following to see if there really are alternatives to search engine promotion for traditional web sites.

Use your thumb to get a grip of web stats

“As technologies like change the Internet landscape, certain measures of engagement, such as page views, are diminishing in significance for many Web properties,” said Jack Flanagan, executive vice president of comScore Media Metrix. “The introduction of these new metrics based on ‘visits’ provides an alternative for measuring user engagement that tells us how frequently visitors are actually returning to the site to view more content.”

So say Comscore.

This is useful for those PR people who take measurement and evaluation seriously and have realised that counting on all ten fingers helps.

An aside is to note that social media continues its challenge of traditional web sites among the most popular sites.

How do you know your blog is interesting?

The Uses and gratification theory, first identified in the 1940s by Lazarsfeld and Stanton (1944), attempts to explain why mass media is used and the types of gratification that media generates.

Rubin (1986) states that there are two underlying presumptions of the uses and gratifications model. First, researchers need to understand audience needs and motives for using mass media in order to comprehend the effects of the media. Second, understanding audience consumption patterns will enhance understanding of media effects.

A social cognitive theory of Internet uses and gratifications: toward a new model of media attendance by LaRose & Eastin, (2004) is very interesting and links U&G to Internet use.

Denis McQuail (McQuail, D. (1987): Mass Communication Theory: An Introduction (2nd edn.). London: Sage) offers a schemata to help establish the quality of web sites. When reviewing a site, this is a method that may be valuable to gain insights into how people will regard and use a web site (or a blog).

Morris and Ogan (1996) poit out that U&G is a comprehensive theory and is applicable to Internet mediated communication ( see also McLeod & Becker (1981).

Return on Investment of blogs - a critical examination

It would seem churlish of me to criticise Charlene Li who was brave enough to attempt to gain some idea of Return on Investment of blogging.

In particular, it is with some sadness that I make a number of points after she kindly accepted some of my papers in her initial scoping of the research and then allowing me to see her finished article.

I admire her work and the work of Forrester.

But we do have an endemic problem in marketing and it has insidiously worked its way into the industry's folk law.

At the core is a misunderstanding in the difference between advertising and editorial. I think that Charlene has fallen for this urban marketing myth. My views on the use of advertising equivelents (AVE's) is pretty well known and is based on a lot of work by many reserachers.

People are really quite clever and discerning. They do understand the difference between the two. Research by many people and notably Guy Consterdine had demonstrated time after time that people both see and read advertising, even advertising deeply disguised as advertorial, that is advertisements disguised as editorial, differently.

We also know from his research that there are special relationships between people and the publications they read which, I think we can reasonably extend to blogs. People get different gratification from different publications and that applies to blogs too.

Indeed we know this from online responses to these two forms of communication. People seek information by searching the bloggersphere and through their social links online. People do not seek out advertisements. Click through rates show that only a minute fraction of advertisements exposed to people, however much they are presented in context, are explored by the online community.

Dwell times for advertisements are also different with editorial gaining many times longer than advertising.

This reflects the work many of have done over the years with other media such as newspapers and magazines.

People see and precess editorial and advertising differently.

Then we go to the content and context of the two forms of communication. Advertising presents an semiotic image of the organisation as the organisation would like to be seen. They are offered in context as near to the ideal as the advertiser can achieve. By contrast editorial, especially editorial mediated by the Internet, is presented in the context of the editorial vehicle. It is the context of the blog, discussion list or web site. This has an influence on how people perceive the content.

Then again, the content can be presented in passing or in great detail. It might be the content of a whole and lengthy critique or in passing or it can be a fragmentary comment. It can be offered in approving or critical tones and it can be presented that endorses or subverts a point of view or third party views or analysis.

So not only is editorial different to advertising, it comes in many contexts, in many forms and in many ways. Measuring the value of these different editorial approaches is near to impossible and can at best be an extrapolation after detailed analysis of a huge proposition of a large corpus.

My only experience of attempting this was extrapolation of over a million press articles at Media Measurement and that only gave me for a moment in time some very broad conceptual certainties that could not be measured in simple monetary form.

The problem is this, even if you have some measure of values, they are of value only from the perspective you take. For example, a high value editorial to the Chief finance officer might be a nightmare for the legal team and an inexplicable shot in the foot for the salesman.

In the analysis presented by Forester there is another misconception. This is in how to calculate value.

The research offered attempted to make an assumption that in some way, the cost of advertising reflects the value of editorial. Cost and value are two different measures and should not be assumed to be the same in any way shape or form.

To get some idea of how different forms of media analysis can be used, this paper is perhaps the starting point for a professional communicator in the 21st century.

I then turn to the issue of return on investment.

ROI, even in its simplest form has many faces. The actual investment is, as Forrester found, very difficult to guess. The actual cost and alternative investment or replacement cost can be very hard to identify. Investment in one moment is of different value to another. Online, these changed values can be very quick and so what seemed like a good investment in July can judged a disaster in December. One only has to look at the financial markets to realise this and for a judgement on ROI over a period of, say, a year the numbers seldom stack up. It is fashionable to talk about ROI in marketing circles but there are very few management processes that truly reflect cost.

When ever we seek to identify ROI we have to distinguish between raw return and discounted return. In the case of the Forrester research there does not seem to be a distinction. Before a return can be used in a business context we have to discount the value of the return by the amount of return that could have been archived by using the resource in some other way. Like investing the money in a bank or other parts of the business.

These then, are my two principle issues. First there is no such thing as advertising equivalent for editorial. It is a myth that has entered the heads of the marketing community and second there is no such thing as return on investment unless you take into account the discounted value from a given perspective.

There is a lot of research available for those who wish to follow up on this topic.

Regrettably, Charlene, we differ on this occasion.

Social Media sites are taking over

It is all very well to imagine that social media is significant.

My question is: Is Social Media attracting the interest of Internet users more than traditional serach engine and consumer web sites.

I needed empirical evidence that would prove the point.

One way of doing this is to compare like for like data of page views (see note below) between differnt types of web site.

I examined data over a three year period. In this time the number of Internet users increased and home broadband penetration increased from almost zero to nearly 60% of users. This should mean that there will be a general increase across the majority of web sites.

I took information about site traffic from Alexa and looked at page views of the top 30 Alexa ranked UK sites.

This would indicate both comparative interest in these sites as well as growth of interest among Internet users.

The results are are as follows:

Page views among the sites with top ten ranked pages:

Search engines

Retail Sites

Social Media Sites

Page views among the sites with 11-20 ranked pages:

(note: there were no search engines)

Retail Sites

Social Media Sites

Page views among the sites with 21-30 ranked pages:

(note: there were no search engines)

Retail Sites

Social Media Sites

This evidence would suggest it would be interesting to see what reach these sites enjoyed comparatively and there relative Alexa rank.

My conclusion from this is that:

Social media is among the top rank of web sites of interest to Internet Users.

Social media is making considerable inroads into the pages viewed by Internet users

Search engines are not increasing their traffic in relation to the increase of Internet users.

Retailers are, mostly loosing share of page views

Social media is enjoying an exponential increase of interest.

If someone, at this late stage is looking for a dissertation subject, this one would be stunning.

Alexa Page views measure the number of pages viewed by Alexa Toolbar users. Multiple page views of the same page made by the same user on the same day are counted only once. The page views per user numbers are the average numbers of unique pages viewed per user per day by the users visiting the site. The page view rank is a ranking of all sites based solely on the total number of page views (not page views per user). The three-month changes are determined by comparing a site's current page view numbers with those from three month ago.

Page views per million indicates what fraction of all the page views by toolbar users go to a particular site. For example, if yahoo.com has 70,000 page views per million, this means that 7% of all page views go to yahoo.com. If you summed the fractional page views over all sites, you would get 100% (this is not true of reach, since each user can of course visit more than one site).

The Power Geeks - Bloggers

Forrester Research has said that the four million European internet users who write blogs should be "got on side" by advertisers wanting to succeed in the online market.

The company's study into blogger attributes has revealed that those who write blogs spend more time online than they do watching television, and that they spend 50 per cent more time online than regular internet users.

Crucially, Forrester reveals that bloggers are more welcoming of targeted advertising than most internet users, with 41 per cent saying they don't mind such adverts compared to an average figure of 34 per cent.

Bloggers are also more willing to investigate new products, and the social aspect of the medium means that almost 25 per cent of bloggers trust other blogs, compared to just ten per cent of all users.

"Active bloggers can make or break a brand in less than a day. Firms shouldn't fake a relationship with them or they will experience a backlash. To get bloggers on their side, firms should gain bloggers' trust by establishing an honest and transparent relation," said Forrester research director Jaap Favier.

What We Talk About When We Talk About Brands

Andy Lark has an found an excellent article about how PR people can listen to conversations about brand to get a better understanding of what is being said about brands.
The original article is to be found here.

Monitoring video coverage

Eric Schonfield has noted a new PR service that allows PR people to monitor video on the web (e.g. YouTube posts). He also notes some mashup opportunities too.