Monday, July 18, 2011

PR productivity sucks

It depends how you look at the numbers 

PR Week (15 July 2011) editor Danny Rogers bangs to PR drum hard this week making journalistic claims about the PR industry. He suggests to politicians that his survey, undertaken with the PRCA is proof of the massive and ever-growing value of PR to the British economy. Well, such claims make some people in the industry wince.


He is, after all a journalist and so might be forgiven for a bit of spin but the reality is that the PR is a pretty poor performer among the top flight sectors. Output per employee is dull.

The survey estimates that there are 61,600 people employed in public relations in the UK and that the industry has a turnover of £7.5 billion. This is an estimate combined with an ambiguous statement which could be interpreted as showing turnover per employee of £121,753 per annum.

The sector, like many others, is growing too but compared to most UK sectors, the PR industry has a very small upstream economic footprint. (press clipping and rudimentary intelligence, catering services and normal office overheads). Downstream economic impact is also limited.  Press relations is an example.

Compared to the top performing sectors, PR is not a great act. Here are some examples:

The UK music industry employs more than 120,000 people and has a turnover of £3.9bn per annum showing a £355,000 annual contribution per employee.

A similar sized industry, the Space Sector is also interesting. In the 2010 a sector study showed the industry with growth rates of 10.2% in the last two years turnover of £7.5 billion. With employment of nearly 25,000, economic contribution per employee £300,000.

UK Internet economy is worth £100 billion a year and Internet companies employ an estimated 250,000 staff showing £400,000 contribution per employee.

PR, it seems has a productivity problem. It needs to up its productivity by a factor of 2.5/3 to be considered top flight.

Perhaps this is the real value of the PR Week report. It has shown that, though growing (and not as fast as most of the top performers) it is not getting the best return per employee among the better exemplars.

For most practitioner (84% of practitioners have some form of general media relations responsibility and 77% write articles and newsletters), it is worth comparing journalist and  with PR salaries. Journalists are paid:


Journalism

Salary Range

Magazine£11,170 - £34,459
Publishing£12,471 - £38,233
Newspapers£11,407 - £35,489
Internet and New Media£13,150 - £40,724

PR people are paid a comparable level of between 28,384 and £36, 500 average annual salary (up to Media Manager/Account Manager level). It would seem from these data that PR practice is comparable with the work of a journalist but PR work extends to 46.5 hours per week.

This would suggest that to compete with the high flying sectors of the economy, PR has to find an alternative to writing as its core activity.

What we now need from the research is some idea of where the high productivity gains can be achieved.

In addition we need some form of breakdown of the low value activities so that we can cut the cost of production of just simply cut out the activity.

Saturday, July 16, 2011

Press relations as a PR practice - a diversity trap for practitioners

This week-end will be the first without the News of the World at our newsagents.

It marks another occasion when the number of editorial pages that could be influenced by the media relations sector of PR shrinks.

This post explores print media as an arena for future public relations practice.

The prospects do not look good.

The reality seems to be that there is a declining appetite for the kind of media to which we are accustomed.

This has consequences for both existing practitioners and the likes of Amanda Andrews of the Telegraph Media Group (since 2008) who announced this week that she is about to jump ship to Freud Communications in the role of head of media, focusing on technology, media and telecoms clients.

There already seems to be some reluctance to recruit from the media. A number of senior ex-News of the World journalists have enquired about PR jobs, reported PR Week. But recruitment consultants have warned that the paper's 'toxic' reputation could harm their chances of making the transition.

In a word, will there be any media for these journalists to talk to?

Yesterday (14 July 2011), Fraser Nelson the editor of The Spectator (and ex-political columnist for the News of the World) wrote about how British newspapers are haemorrhaging readers and influence. He said that on Sunday "we will see just how much this process has accelerated." Graphically, he showed that average circulation of daily newspapers is now lower than the worst days of the 1940's Blitz and for Sunday titles, we have to go back to the 1930's depression to find comparably low circulation.

Notably, the trends show steady decline from the 1960's and very rapid decline over the last decade.

In the same week Mark Sweney of the Guardian reported that "...the move to close the NoW was taken on political grounds in a bid to contain the phone-hacking scandal, it is nevertheless a hammer blow – potential relaunch notwithstanding – to a sector where publishers are trying to wrestle with high costs against a backdrop of declining revenues."

He quotes Adam Smith, director at WPP's media buying network Group M: "It is like taking Channel 4 off-air on Sundays, you are suddenly taking out 20%-plus of the market, it is really substantial with no other home [for advertisers] to really go to."

Smith makes the point that: "Taking the NoW's total 7 million readership out of the equation is massive, as there really is no substitute and the market is already not in great shape generally."

Rob Lynam, head of press and media agency MEC, is reported saying: "The Sunday model is busted," adding: "The cost base on Sunday titles is significantly higher than running a daily and publishers are looking to reduce overheads. That is why News International is moving to a seven-day model, that is why Guardian Media Group made changes to the Observer and so on."

In the same article we discover that in May 2007 the total circulation of Sunday newspapers was 12.5m; by May this year it had fallen 22% to 9.7m, according to the Audit Bureau of Circulations. Advertisers have also walked away, with display advertising revenue down by more than 25% over the same period, from £406m to £303m. By comparison the daily national newspaper market has seen a slight increase over the past four years – although it is a very mixed picture with the only major winners the Sun, Metro and Daily Star.

If the model for newspapers is bad, spare a thought for the celebrity titles.

The Mail Online reports that circulation figures for Hello! and OK! make sorry reading.

During the second half of last year the average net circulation for Hello! was 527,000 per issue and that for OK! 487,000 from previous levels as high as 800,000 and 600,000 respectively.

Dominic Ponsford reporting in Press Gazette earlier this year showed that second half of 2010 UK market for print consumer magazines was still in decline overall. Overall, the UK's top 100 purchased magazine titles had an average per-issue total circulation of 28,844,482 - which was a 4.1 per cent decline year on year. A continuing trend.

Is this just an issue for consumer magazines? It seems not.

A Mashable report seems to show that the magazine format has its own problems among consumers. While digital downloads may be down, compared to the print equivalent they “roughly correlate their performance on the newsstand.” Based on the numbers, the decline may not be a lack of interest in the iPad as a digital print platform but rather a general disinterest in magazines.

In the B2B sector life is not very different.

Recently, business publisher Centaur Media announced it was merging its business publishing group – which includes The Lawyer and Money Marketing and it confirmed disposal of titles like The Recruiter, The Logistics Manager and Process Engineering. In November 2009 Haymarket announced the closure of the print edition of Media Week with the axing of 18 out of 58 editorial jobs across the media group as monthly title Revolution went quarterly and the websites Marketing Direct and Promotions and Incentives were merged into Brand Republic. It is putting its network of media titles behind online paywalls from July including Brand Republic, PR Week, Marketing, Campaign and Media Week. We also heard that United Business Media has sold its licensed trade titles to rival William Reed. In addition, Accountancy Age and Computer Weekly both abandoning the weekly news magazine format after more than 40 years and going online only.

Ben Dowell at the Guardian reports that there were 4,733 UK B2B titles being published in October 2010, compared to 5,108 five years earlier.

This decline in support of the media from marketing budgets is part of a significant trend according to PR Week. It reported the the second quarter IPA/BDO Bellwether Report showing marketing budgets were to reduce spend by 4.2% and a decline in PR budgets as well.

This is reflected in a number of other reports. Wark reports on a study suggesting: "The year has proved harder going, particularly in print, and there is a similar narrative coming out of the other big European economies."

So we see the long term decline in print and this must have an effect at some time on the media relations sector of the PR industry.

Meantime, television is doing very well and radio listeners, who were found to be (PDF)  happier than TV watchers and Internet users, reached its highest audience level ever in Q1, 2011 and the internet goes from strength to strength.

There are, no doubt, a lot of people who would argue that print media is only going through a trough and it will bounce back. However, this has the smack of whistling in the dark. The trends are long term and few in publishing seem to have any mould breaking ideas.

Of course, no one should make assumptions about the UK media without reference to Michael Bromley, Visiting Professor in Journalism at City University London, UK Media Landscape study. An enthusiast for the press he may be, but has made the point about the future of newspapers and magazines very well.

Against this backdrop one might expect the PR industry to be on its knees.

But we hear from PR Week that PR is doing well. The International Communications Consultancy Organisation (ICCO) reported that the UK PR industry sector witnessed a 13% increase in overall fee revenue, a figure similarly reflected by PR Week's Top 150 PR Consultancies 2011 report, which showed the average growth for a PR agency in 2010 was 9.24 per cent.

Perhaps we find some of the answers in the 2011 PR Census, a joint PR Week and PRCA research project, published this week.

It is a thorough study of the British PR industry and shows UK PR industry contributes £7.5bn to the economy and employs some 61,600 people.

PR could be compared to the UK space sector which is also worth around £7.5 billion to the economy. Space directly employs almost 25,000 people - so there is a considerable - a factor of 2.5 - productivity gap.

At last we begin to see what is happening to the PR industry. It is really dying.

What I read in the statistics is that there are far too many people doing media relations in the shrinking press relations sector which is depressing incomes and opportunities.

85% of practitioners include 'General Media Relations' as part or all of their work (84% are also involved in Media relations strategy planning).

Perhaps it is not surprising that with the media in such dire straights that the part of the PR sector involved in media relations (at least 77% have a direct role) has such a poor productivity performance compared to more advanced industries. Working on average 46.5 hours per week, PR people are doing everything they can to sustain a form of practice that is on the way out.

The numbers offer us an even worse view. 70% of people employed in the PR sector are aged between 25 and 44. This is in the prime of their career, the springboard for their future and it looks bleak.

Indeed the profile looks like white, middle class, middle aged women fighting for a falling slice of shrinking media pages available for editorial content and paid very little for long hours in a dying sector.

We are fooled by the growth in the industry as it bounces back from the recession in 2008 and we see higher turnover based on sweated labour offering misleading and what can only be temporary margins.

In an industry that prides itself on significance to corporate governance, it is evident that this is not a very profitable career choice. Only 30% of the people employed in PR earn £50,000 or more or the going rate for the mass of qualified Financial Accountants. For the average PR CEO/director in a salary of £83,000 is good which compares to the 40%of financial directors in the FT 250 who earn between £300,000 and £499,000 per year.

Media relations would seem to be holding the PR industry back, depressing salaries and creating a diversity trap for practitioners.


Image: DEATH A GROWTH INDUSTRY from Scrape TV

Monday, July 11, 2011

iPad activism and the investing institutions


So much of the chatter about the end of The News of the World has been about reputation.

A lot of it centres on reputation as a commodity. Building reputation is, apparently, about advertising or PR'ing or connections.

Well, in the digital age such nonsense is patently, visibly and completely exposed.

Reputation is not owned by organisations it is proffered by constituents. That complex societal group that is bounded only by its interests in the values surrounding the organisation (as distinct from Publics, Stakeholders, Market segments and followers). This is not about the values of the organisation but its interaction with values held by the constituency.

Reputation management gets on well with marketing because so many people believe reputation, trust and regard are bought with pieces of silver.

Relationship management, which is a two way street, is much harder. In this street, values have teeth and bite.

Relationships and values management are the critical elements that distinguish PR from other disciplines. It is what makes PR different from marketing, advertising, propaganda, spin and publicity.

Without PR, corporate managers like Rupert Murdoch (Newscorp), Tony Hayward (BP), Pierre Beaudoin (Bombardier), Jamie Buchan (Southern Cross) have failed. They believed that PR was about spin and press releases and more fool them and more fool their shareholders for employing such people.

Mr Murdoch needs a PR manager and not a 'reputation' manger and that will allow him to begin to build trust, reputation and regards for his empire.

To do that, he needs savvy shareholders.

Sensible shareholders are now in short supply. Sensible shareholders will by now have worked out that lack of sincerity and ethics as part of the DNA of corporate culture will mean that they will take a haircut sooner or later on the bourses of the world.

Explaining this to shareholders is the job of the institutions like CIPR, PRCA, IAB etc.

It has to be explained among the investing institutions (remembering that institutional investors are in the same mould as the failed banking sector) in simple outcomes terms.

The argument is relatively simple. If the bloggers says the directors suck, fire the chairman at the AGM. If Facebook says service sucks, fire the executive board during interims and if Twitter says it is uncomfortable dump the stock fast.

This is not an ethical brief we have to give to the financial institutions. This is war. The normal citizen can and does become an iPad activist at the drop of a hat. In the case of NotW, as Robin Grant put it "brands were being bombarded in protest – most of whom will have been unused to such a spike in negative attention. This was not just happening on Twitter, with targeted brands’ Facebook pages becoming venues for significant protest too." Unless and until the financial sector gets its act together the pension funds will suffer at the hands of social media time after time. This is why it is in the interest of the financial institutions to be assured that they have proper Public Relations managers advising the Boards they invest in. Andy Coulson, is not and never can be a public relations practitioner. He is a journalist. He does not have what it takes to be effective in modern PR. Gaming the system was fine when Coulson was the editor of the biggest circulating newspaper in the UK. Try Gaming Google+ and the online world will crush the company and shareholders with it.

Why pick on G+? Because it is of a new breed of services with in built web 3.0. The semantic web. Semantic as in searching for and exposing secrets (automatically soon enough too).

A number of authors have been making this point for years (first time I published a book about it was in 2008). There is nothing new here.

What we now need are some PR institutions that are aware of what is happening in the fields of communication and who are prepared to make the point to corporate managers and investors and in public if need be.


Cartoon by the clever Vicky Woodward

Wednesday, June 15, 2011

Return on Values

Yesterday, I began to  look at ROI in a slightly different way and Philip Sheldrake and Tim Marklein maintained the argument for staying with a financial measure of public relations based on cash investment providing an incremental cash return.

For some PR trades, this is a perfectly adequate. One press release reaches a readership of a million people who in turn repay the client the cost of the press release and then some. The effect on the relationship between the organisation and the readers as well as tertiary publics such as journalists, editors and the process of WOM is ignored. Crude, better than nothing and informing management very little about effect.

Lets see if we can improve on that.

Suppose the PR practitioner was to ask the client:

  • "Do you have values?"
  • "Does your organisation have values?"
  • "Do you invest time explaining, even re-enforcing, your values and the organisation's values to the Board?"
  • "Do you and the Board invest time explaining your values and the organisation's values to your shareholders, employees, customers, vendors and other stakeholders?"
  • "Do you use your values and the organisation's values to single out your brands among consumers/customers?"

I think that most managers would agree that their values and the values of the organisation are very significant competitive differentiators and that valued have value.

Now, lets make this harder. What if you ask the CEO:


  • "What is the Return on Investment from your values and the values of the organisation?"


Ummm......

Now, in PR, we do have the answer.

Not marketers, not accountants, not business gurus.

Although many do recognise values as important even if they are not really sure how to identify values I cite: Charles Handy, Peter Drucker and Henry Mintzberg plus L Chen - 2009; H Donker 2008; M Chong 2010; J Cambra-Fierro & Y Polo-Redondo 2008; NL Trapp 2010 etc. etc).

In PR, grounded research (much better research than brand mangers have - and explained in this post), show that, among different segments of the public, there are drivers that build relationships between them and the organisation. They take the values of the organisation and where those values coincide with personal or group values, they find an affinity with the organisation.

An organisation can be described as a nexus of values and, to extent that they chime with the values of people or groups, there is a coincidence of interest.

We also know from a range of research and academic writing that organisations need to be able to understand the affinity between consumers and brand values to be effective and successful.

I cite, for example: JN Kapferer 2008; S Boo, J Busser, 2009; KL Keller & T Apéria 2008; S Srinivasan 2009; N Mizik & R Jacobson 2008; AE Cretu 2007; J Kim & JD Morris 2008 ....

For PR, deeper and more relevant measurement is to be able to identify the Return on organisational Values.

Does the organisation understand the values of its constituents? Does the organisation have values that chime with its constituency and in explications of its values, is it creating, sustaining and enriching positive relationships.

There is significant literature which explores the concept of Return on Values and much of the literature touches on matters like ethics, trust and reputation.

I cite for example: P van Beurden 2011; KS Cameron 2006; DA Waldman & MS de Luque 2006 LL Nash 2010 etc.

With rich, sustainable and supportive relationships, organisations will prosper both in the short and long term.

The return on investment in having clear, relevant, supportive and mutually acceptable values with the organisational constituency is a great deal more than cash out and cash-plus back. Yes, there is cash-plus back today but also cash-plus back tomorrow and with wider audiences. The real ROI will be seen to deliver real shareholder value, lower cost of doing business, a stable workforce with lower recruitment cost, enhanced vendor relationships and a more supportive licence to operate (Keller, Handy, etc etc).


Return on Values seems to be a much more sensible way of measuring PR.


Image from http://thefinancialbrand.com.

Tuesday, June 14, 2011

Brave new PR

In an exercise with some students we looked at a project in which the PR person was given responsibility for a university open day with the objective of increasing the number of student registration to 110% of capacity.

The university was running at less than 95% so this was quite a challenge.

The rules were simple. No budget constraints and stick to the brief.

Of course we undertook a detailed analysis of the primary publics and their interests and drivers: candidate students, existing students,the faculty etc.

By the time we had students wearing tee shirts with QR codes printed on them, it was obvious that the internal  communication activity was going to be critical.

To get fulsome buy-in from the dominant coalition and starting with the Principles, QR codes were liberally evident outside their offices linked to web sites about achievements, interests and Facebook friends to like selected with salacious care (every lecturer has glamorous Facebook friends - it goes with the territory).

In (classroom tests) it became clear that although the staff was completely oblivious to the secrets of QR codes, the students had no such barriers. Here was a subversive form of communication that was just right for the target audience but also really useful to achieve in-house buy-in (after all when you come out of your office and a bunch of giggling students are pointing phones at the notice boards, even the crustiest academic is curious).

The ideas for the programme, now an interactive university wide exhibition, continued as exemplar courses were given the QR treatment. Film and Vision has a wonderful challenge to use their studio to create YouTube events (inspiration came from here)  for visitors providing attendees with QR codes in hard and soft copy. The stars in the show, academics, students and visitors alike were offered staring roles.

Imagine proposing to a librarian that the PR department wanted to deface books with QR codes converting them into interactive documents incorporate text pictures and video. A deeper richer experience and above all fun. Perhaps even creating Slate books.

Meantime the maths majors were (hypothetically) working on creating a virtual environment about a new form of seminar with the lecturers in class and the students in the bar (sounds credible to me).

A new way of annotation for course work was thought to be a good idea for visitors to the open day.

The whole university was offered help to work on Wikipedia entries with each paragraph having its own QR code with free help from local wikipedians.The resulting codes were photo-shopped into a walk through department exhibition using a really appealing invitation for the young men expected to take an interest

The chemists had a ball developing Avatar presented activities.

Of course all universities need to keep local communities involved and a small team worked on how they could create web presence and interaction between visitors, students and the community and local enterprises. In addition there was a case for creating new ways to stay in touch with school and university alumni.

Having scoped out what was possible, the next major consideration was reaching out to the target audience. Direct contact, involvement with schools, local press, radio and television and, of course the places where young people (and their parents) go. Every new media idea that the class had come up with had a novel, interesting and compelling application for enhancing media relations and extending interest and coverage.

Progressively, this imaginary event  became bigger developing ways that people could interact with the open day exhibitions using Augmented Reality.

The need to be able to use print, email, personal visits, social media, traditional media, shops, clubs and community institutions became significant. Detailed analysis of ways to reach the, now growing range of internal and external publics needed careful analysis. From Twitter to Linkedin, building lists and approaches needed careful planning (and careful risk analysis too). Timetabling the plan was becoming harder.

The wonderful thing about students is that they suddenly come up with a new, and compelling idea.

The idea that so many communication capabilities came from the internet of things inspired by the Corning video brought a whole new range of new communication ideas.

To be able to attract a Corning demonstration, there would be a need to make the open day bigger, requiring wider involvement of the local, and, notably, commercial community.

There are two serious drivers for commerce: incremental sales or, alternatively lower cost.

What, for example would happen if these truly exciting events could extent to the wider community? What self respecting club could resist a QR code enhancement - if only on tee-shirts and wearable transfers, badges, table mats, drinks and even the bands. What is all these electronic gizmo's were all branded to keep promoting the university?  Can the open day become an interactive community interest in shopping malls and, in doing so, increase footfall?

Could visiting prospective students apply for a course using Bump technology? Could a local manufacturer use the new knowledge that the university was acquiring in communication skills for commercial gain? Is mobile phone bump technology helpful in replacing office access passes (Bump payment is now passĂ©e) and at the same time a message board on employees phones as part of the process adding internal employee communication to access control. Communication was invading and enhancing ordinary corporate function.

As the conversation progressed, each new application, every new idea offered new and interesting tertiary opportunities to engage a huge range of media from Facebook to YouTube and every news channel going.

By now, a simple open day, a normal PR activity, was becoming an agent for organisational change.

It was creating new and additional motivations for the establishment, lecturers, students and the community.

The kind of thing they would want to do because it is fun.

What we witnessed was where advertising, sales promotion and PR blur into engagement. Relationships that needed careful and structured management to gain the most potent effect.

Congratulations to the creative students at  Escola Superior de Comunicação Social.

Exhausting isn't it?

Image from http://listverse.com/.

Can PR use ROI as a form of measurement? Its harder than you think

This week three academics have presented challenges to the PR academic community.

Professor Tom Watson at Bournemouth, Richard Bailey at Leeds Metropolitan University and my co author Philip Young at Sunderland have all made interesting contributions to PR thinking ahead of the the CIPR’s new Research and Development Unit (R&DU)  meeting next week.

Before entering into the debate on Grunigian theory presented by Richard and Philip, I wanted to respond to Tom's point.

Tom makes a point: "I still have doubts as to whether ROI, other than in a strictly financial format, can be re-purposed into a more general expression of value creation or contribution to organisational efficiency.  Business managers understand what ROI is, so why would they accept a mixed-concept PR ROI."

It is important. As AMEC boldly goes for some form of measurement of PR providing a return on investment. There seems to be a belief that ROI is a simple idea.

It would seem there is a belief that ROI is a financial measure. Of course it is not. ROI is a profoundly Public Relations measure.

Lets have a look at what ROI is. It is defined in accounting terms as:

(Gain from Investment minus Cost of Investment) divided by (Cost of Investment)

Can we pause for a moment and explore what 'Investment' means. Investment requires that an organisation has cash flow, capital reserves or some other asset that can be deployed as an investment.

Organisations comprise three principle assets: capital, proprietary process and/or service and  relationships. The acquisition of capital, and development of process or service; 'vision, mission and corporate objectives' (Kaplan 2001) are a function of relationships.

It follows that to invest in anything, an organisation needs relationships of a nature that can be invested.

So lets re-draw what ROI means:

(Gain from Relationships minus Cost of Relationships) divided by (Cost of Relationships).

ROI is profoundly about relationships. In an industry called 'Public Relations', this could be of interest. In a sector called 'marketing communications' it will be pivotal because Marcoms depends on 'public relation' to optimise relationships to create capital and cash flow to pay for this, a special area of relationship management, namely marketing. In principle the same applies to the trade of 'Corporate Affairs' and other trades associated with 'Public Relations' in practice.

Which takes us back to Richard and Philip and the Grunigian excellence model coming from systems theory. We can, if we desire stay with the systems theory view because already have a grounded reserach into the nature of relationships in the work of Bruno Amaral (2009).

This, Amaral, hypothesis is that relationships  are formed at the nexus of values and using latenet semantic analysis was able to show that where there is a nexus of semantic values there is very strong evidence that they are central to the formation of convergent relationships.This empirical research supports conclusions as to the impact of public relations as relationship management offered, by  Ledingham and Bruning (2002).

Convergent values relationships have some resonance with the Grunigian position of Publics forming round issues but in the Amaral study, it was less issues as values that were key which is a broader construct.

What we have done is to extend and develop the ideas of Grunig and Ledingham and Bruning to identify an empirically based idea of what public relations can be which accommodates both theoretical perspectives.

Can we now re-draw ROI yet again.

(Gain from Nexus of Values minus Cost of Nexus of Values) divided by (Cost of Nexus of Values).

Of course, I have only taken one view as to the nature of relationships namely the empirical research of Bruno Amaral. There will be others drawn from Psychology to the Evolutionary Sciences.

What I hope to have shown is that the theoretical concepts of Public Relations have moved on and that we can, should we wish, pursue ROI but that it will require more than an AMEC Commission to come to any meaningful conclusion unless there is a great deal more by way of, notably academic, research.

And the there is the problem of getting such ideas into the heads of the PR industry's clients. But that is another story.









RS Kaplan  Nonprofit management and Leadership, 2001 - Wiley Online Library

Excellence in Public Relations and Communication Management, 1992  IABC Research Foundation Edited by James E. Grunig

Relationship management in public relations: dimensions of an organization-public relationship (1992) John A. Ledingham and Stephen D. Bruning Public Relations Review Volume 24, Issue 1, 1998, Pages 55-65

Friday, June 03, 2011

The search for hidden meanings

Throughout written history, people have engaged in finding the hidden meaning in writing.

Fascination at the hieroglyphs on the walls ancient Egyptian temples and burial sites extends back well before  4 PM on November 26, 1922 when Howard Carter’s search for hidden meanings resulted in the discovery of the 3300 year old and untouched tomb of 19 years old king Tutankhamun .

Today, we are even more fascinated with exploring our written (and spoken) language.

And it all comes down to what is known as Part-of-speech tagging (POS tagging or POST).

Most of us have done it at school by identifying words as nouns, verbs, adjectives, adverbs, etc.

Back when the Beatles were at their peak, America and its allies were embroiled in the Vietnam war, Dr Christiaan Barnard carried out the world's first human heart transplant and The Six Day War was fought in the Middle East, NASA launched an unmanned Apollo 4 test spacecraft and Britons got their first colour television programmes . But in that same year the one development that affects more people today and will do in the future is the work of Henry Kucera and W. Nelson Francis.  They published their classic work Computational Analysis of Present-Day American English (1967), which provided basic analysis about words in texts on what is known today simply as the Brown Corpus.

Henry Kucera and W. Nelson Francis did more complicated analysis than getting computers to find nouns and verbs but the principle is the same. It is a process largely based on relationships with adjacent and related words in a phrase, sentence, or paragraph. 

Once performed by hand, POS tagging is now done in the context of the son of the Brown Corpus, computational linguistics. It uses algorithms which associate discrete terms, as well as hidden parts of speech, in accordance with a set of descriptive tags or forms of description or, more recently, that are created as they are found ‘on the fly’.

The reason that Kucera and  Francis work is so important is that we have built a whole new form of society on this idea.

Clever scientists have used this idea of extracting hidden meaning to develop a new form of internet.

One of these ideas came from three academics Scott Deerwester, Susan  Dumais, George Furnas, Thomas Landauer and Richard Harshman (1990). They outlined how to analyse relationships between a set of documents and the terms they contain by producing a set of concepts related to the documents and terms. Called Latent semantic Analysis (LSA), the idea assumes that words that are close in meaning will occur close together in text.  

This idea is used by all manner of analysis programmes and helps find those hidden meanings.

In their paper they says “...Thus while LSA’s potential knowledge is surely imperfect, we believe it can offer a close enough approximation to people’s knowledge to underwrite theories and tests of theories of cognition.”  Since 1990, academics have come a long way and accuracy is getting ever closer to social reality. 

Today, the use of semantics makes the Google and Bing web search algorithms more accurate, helps newspaper journalists find the most authoritative sources for information and informs the top companies about events and their drivers to optimise financial, marketing and communication decisions.

Remember Kristen Urbahn’s story I blogged about three weeks ago? It has lots of hidden meanings. Using Extractive.com’s special search engine Kristen can find out about the relationships between different parts of the story (using automated Part of Speech tagging).

The results show the nature of some of the significant words:

PERSON (46)
│├SCREEN ACTOR (4)
││└ Kathy Griffin (4)
  she
││  her 
│├US CABINET MEMBER (1)
││└ Donald Rumsfeld
│├US PRESIDENT (3)
││└ Obama (3)
││  Obama
││  Obama
││  his 
│├ Brian Williams
│├ Dan Pfeiffer
│├ Jill Jackson (2)
  Jill Jackson
│├ Keith (6)
  Keith Urbahn
  He
 │├ Kristen Urbahn (13)
  Kristen Urbahn
   her
  Kristen Urbahn
 
  Kristen 
│├ Maggie Fox
│├ Osama Bin Laden (6)
  Osama Bin Laden
  Bin Laden 
│├ Osama Bin Ladin (5)
  Osama Bin Ladin
  He
  Osama
  he 
│└ Sohaib Athar



LOCATION (14)
│├GPE (13)
││├COUNTRY (5)
│││├ Afghanistan
│││├ Pakistan (2)
│││  Pakistan 

│││└ US (2)
││├CITY (4)
│││├ Abbottabad
│││├ Denver
│││├ Guardian
│││└ San Francisco
││└US STATE (4)
││  Kansas
││  South Carolina
││  Washington (2)
││   Washington
││   Washington 
│└ Wiltshire
ORGANIZATION (21)
│├COMMERCIAL ORG (16)
││├MEDIA ORG (7)
│││├BROADCAST NETWORK (5)
││││└TV NETWORK (5)
││││  BBC
││││  CBS
││││  CNN (2)
││││  NBC
│││├ New York Times
│││└ Washington Times
││├ Defence
││├ Google
││├ Social Media Group

││└ Twitter (5)
││  Twitter 

│├NON GOVERNMENT ORG (2)
││├ Al Qaeda
││└ Republican Leaders Office
│└UNIVERSITY (3)
  Preston University
  University of Kentucky
  Yale
CONTACT INFO (1)
│└URL (1)
 HTTP (1)
  http://goo.gl/qHnFH

OTHER (18)
│├FACILITY (4)
││└BUILDING (4)
││  White House (4)
││   White House Communication Director
││   White House 

│├LINKED OTHER (11)
││├ Capitol Hill
││├ Christian
││├ Creative Commons
││├ Dachshunds
││├ Internet
││├ Internet
││├ Mobile
││├ POTUS
││├ President Obama
││├ Royal Wedding
││└ The New York Times
│└SOFTWARE (3)
  Facebook (3)
   Facebook 

DATE-TIME (16)
│├DATE GENERAL (8)
││├DATE (2)
│││├ Aug. 18, 2009
│││└ May 1 2011
││├DAY OF MONTH (1)
│││└ 1 May
││├MONTH NAME (1)
│││└ May
││├RELATIVE DATE (2)
│││├ months ago
│││└ the evening
││└YEAR (2)
││  2006
││  2011

│└TIME (8)
  10:30 p.m. Eastern Time
  10:40 p.m.
  10:53
  11 p.m.
  11:35
  4pm EST
  9:45 p.m.
  from 10:45 p.m.-2:20 a.m.
NUMERIC (20)
 MEASUREMENT (4)
 │└DURATION (4)
   Five years
 
  days
 
  former
 
  the hours
 NUMBER (11)
 │├ 2.0
 
│├ 3,000
 
│├ 5,000
 
│├ 7.24
 
│├ millions
 
│├ more than 185
 
│├ one
 
│├ one
 
│├ six
 
│├ three
 
│└ two
 ORDINAL (5)
   Third
  
 first
  
 first
  
 second
  
 third


Here, then, are the key elements that can be extracted from the blog post.

Two people from the 18th and 19th centuries now star in this story.

Thomas Bayes (1702–1761) was the son of London Presbyterian minister with a clever mathematical brain. He came up with what can be described as a way to look at these hidden parts of text and other content and find out the extent to which a particular inference is not true. For example Twitter is a big part of the Kristen Urbahn story but it is by no means the focus of the events in Pakistan.  It was just an (important) means by which information was shared across the globe. Thomas’ clever mathematics is the means by which it is possible for computers to make decisions about the probability that information can be relied on and, in that case, the role of Twitter in news distribution.

With enough information and generous computing power, of which modern man has plenty, Bayesian probability offers something like a partial belief, rather than a frequency. This allows the application of probability to all sorts of propositions rather than just ones that come with a known structure. "Bayesian" has been used in this sense since about 1950. Advancements in computing technology have allowed scientists from many disciplines to pair traditional Bayesian statistics with other techniques to greatly increase the use of Bayes theorem in science. Now, computers can both learn from experience and are beginning to be good at prediction.

Twitter was important for the Urbahn story and so, the software might tell us, Twitter will be significant for other stories in the future.

It is such techniques that modern managers need to hand if only to be able to discover emerging trends in communication and or news and events.

Fifty years after Thomas death, George Boole  (1815 – 1864) came into this world to give us all a great way of discovering information.  George (who was married to an equally mathematically brilliant wife Mary and who was the nice of the man who gave Mount Everest its name), gave us Boolean algebra (1854). Today most people know it because it is useful when searching for information using search engines. The Boolean operations AND, OR, and NOT help narrow down searches to get more closely to the facts we seek (Kristen AND Urbahn OR Forcht).

But, the use of AND, OR, and NOT in mathematics and computing has other applications and when combined with Bayesian probability (and other similar math) which means that computers can be used to make accurate, predictive and related inferences and learn, for themselves, from the results.

In practice, we find useful tools to give us insights into events.

For example http://twitris.knoesis.org/  (created at Kno.e.sis at the College of Engineering and Computer Science at Wright State University) provides us with an ontology, related Tweets, links to highly relevant web pages, a chart of Tweet rates and much more.


In practice, a manager can keep a close eye on mentions of a company, brand or product and the reputation drivers behind the Twitter stream.

No one is pretending that business managers need to understand all the technologies. There is a need, however, to know that using such advances is now becoming central to modern management and communication.


Bibliography
Kucera, H. and Francis, W.N. (1967) Computational Analysis of Present-day American English Journal: Neuroimage - NEUROIMAGE
Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, Richard Harshman (1990). "Indexing by Latent Semantic Analysis". Journal of the American Society for Information Science 41 (6): 391–407
 Boole, George (2003) [1854]. An Investigation of the Laws of Thought. Prometheus Books. ISBN 978-1-59102-089-9.
Gruber, Thomas R. (June 1993). "A translation approach to portable ontology Specifications". Knowledge Acquisition5 (2): 199–220.


Further Reading:
Introduction to LSA http://lsa.colorado.edu/papers/dp1.LSAintro.pdf
Semantic Inference in the Human-Machine Communication http://www.springerlink.com/content/ju71rcn9pq0wcmy3/
Continuous Semantics to Analyze Real-Time Data http://wiki.knoesis.org/index.php/Continuous_Semantics_to_Analyze_Real_Time_Data
Web semantics and ontology By Johanna Wenny Rahayu http://books.google.com/books?id=K7yFJVu8NDYC


Twitter, Facebook, and dozens more sources come through Gnip's API, normalized and enriched with metadata. http://gnip.com/