Evaluation - again

There is discussion about evaluation arising from my post about CyberAlert's recent announcement.

The discussion is relevant because it calls into question different approaches to this area of Public Relations practice.

I would recommend every practitioner read Don's post and the Annenberg study posted today.

To be able to really get to the bottom of this issue we need to take on board the opening chapters of the Watson and Noble book (Watson, T., & Noble, P. (2005). Evaluating public relations: A best practice guide to public relations planning, research and evaluation. London: Kogan Page.)

Here, they begin by asking what is public relations. It is a topic I covered recently and where I gave a definition that adapts from Edward B. Tylor's definition of culture.

Knowing what PR is, helps define Public Relations evaluation there is an equal imperative to be encompassing. I am on record saying:

“Public Relations Evaluation has application in the formative analysis for setting objectives, strategy and planning; it confirms best application of resource; it aids control of the strategic and tactical public relations programme; it is a continuous and integral part of the total PR programme to inform the practitioner as to whether PR activity is optimised for success and it has application in the final review of efficacy.”

This means that PR Evaluation has to be inclusive and how I can defend a view of the CyberAlert development.

An anonymous commenter suggested said “I don't know... Garbage In, Garbage Out. It looks a lot like KDPaine's DIY dashboard charts (maybe they're partnering on it?).

What I see is simply a charting of self-reported analysis. The only thing you get for the money are pre-formatted charts most of which won't really excite many senior business executives. The clips still need to be manually read, tagged, and sorted, and all the old crap deleted - which with a web-based clipping service is a significant number. I think our industry has a looonnggg way to go before we have something truly useful for the Boardroom.

I am sure that there is a KDPaine element and her dashboard. She comments on it on her blog.

I do not subscribe to the view that the CyberAlert service provides garbage statistics. I agree that it is possible for a dishonest PR person to rate content with tinted glasses. I do not believe that most PR people are dishonest and feel that the criticism is unfounded except for the last sentence.

We do have a long way to go.

What is significant is that there are now metrics. Few people in PR have taken on-line clips seriously and now there will be numbers that are comparable to 'bog standard' print stuff including trend data (which are the most important content). Add these data to NLP data and numbers from the likes blogpulse and the data sets are impressive for analysts.

This is the intelligence that practitioners need for evaluation. Its application in the board room is irrelevant. For the most part it has no role on the boardroom table. After all does one see bought ledger analysis there?

If we want to go further in terms of content analysis then the industry has to get used to the idea of automated inference and neural networks, LSA and fuzzy logic. It is possible and I have many such tools but there is little enthusiasm for it.

One can guess why. Most PR consultants are owned by advertising agencies. Mono-cultural deserts. The Press Agentry people (typically working in a so called 'marcoms' environment) people do not want to know what is happening. It is not their job. Being part of the advertising industry press agentry, for the most part, is 'Scream Marketing'. Why count the number of people you made deaf?

The most that this kind of activity needs is a measure called thud factor (weight of press clips).

We do find excellent research among some in-house teams and it is there where the best evaluation has been done. What we need is much more of that kind work and we need a vehicle by which best practice can be discussed among practitioners.