Newsletter February 2008

Evaluating your communication

Setting and testing benchmarks for evaluation

Choosing your communication research methods

Surf club

Quotation of the month

Site map

Contact

Home

Setting and testing benchmarks for evaluation

By Jenni Metcalfe

If you want to measure your progress over time, it’s important to set benchmarks. We evaluated the progress of the Murray-Darling Basin Commission’s communication by comparing benchmarks we had set in 1999 with where they were at seven years later. We were only able to do this by setting and following some basic processes.

Here are five tips for setting and testing benchmarks for evaluating your project, program or organisation:

Set communication objectives that are SMART (simple, measurable, achievable, realistic and time bound) e.g. ‘to raise awareness of our program by 20% in 12 months’.
Identify the performance indicators that tell you if you’ve achieved these objectives e.g. ‘level of awareness of the program’.
Find a way to measure the performance indicator e.g. ‘telephone survey 500 farmers in three catchments’.
Set your benchmark or baseline data by measuring your performance indicators e.g. ‘20% of farmers are aware of our program’.
Test progress against the benchmark at an agreed time using the same measuring tools e.g. ‘After 12 months, our telephone survey results indicate that that 30% of farmers are aware of our program; so there has been a 10% increase in awareness, but not 20%’.

Evaluating for your audience

By Melanie McKenzie

We’ve all heard of writing for your audience, but have you thought of evaluating for your audience?

As part of my PhD thesis on evaluating science communication, I’m looking at how science communicators are judging the quality of their events from their own perspectives, and not from the perspectives of the people attending their events.

Take, for example, a science magazine, where the success of the magazine depends on its ability to meet the needs of its readers (after all, if it didn’t, no one would read it). To evaluate how successful it is, we need to measure just that—how well it meets the needs of its readers.

Instead, from a science communication perspective we seem to be evaluating how much the readers learn from a particular article, how often they tell others about content from the magazine, the quality of letters to the editor, or simply the number of issues sold.

Presumably, we think that by measuring these things we are somehow measuring how well we meet the readers’ needs. The problem, more often than not, is that we haven’t bothered to find out what their needs are.

So before you go ahead and evaluate how effective your communication is, don’t just assume that you know what your audience needs—ask them!

And in planning your activities, keep in mind that the best outcomes usually happen when you consider both the things you value and the things that they value.

Choosing your communication research methods

By Michelle Riedlinger

We often do communication research for our clients as it is an essential part of planning, monitoring and evaluating your communication. But there are many ways to collect and analyse information. What methods are best?

The methods we use depend on what type of information we need, who will do the research, who will use the findings and what resources are available. We often use a mix of qualitative and quantitative methods because together they help us to understand an issue or a community in greater depth as well as breadth.

Organisations often favour quantitative analyses (such as surveys) because they want to see ‘hard’ numbers. This type of research is good for generating information that we can compare over time and across regions, whereas qualitative methods (such as interviews, focus groups and desktop reviews) can give us critical insights into a project.

At the start of a project, we may do interviews, focus groups or a desktop review to help us write a well-targeted quantitative survey. Qualitative research also helps us explain some of the more curious quantitative survey results.

With quantitative research, the more data you have the more reliable your results. It’s important to have large numbers of participants.

Qualitative research focuses on depth rather than breadth. It gives you information that is directly relevant to the community or issue. You may only need ten people. The key is to keep investigating the issues that these people raise until no new information emerges. This concept is called saturation. The major drawback of qualitative methods is that they can take as lot of time.

Surf club

Go into a draw for a case of wine or a $150 ABC Shop voucher.

All you have to do is answer 5 easy questions about our website by 7 March.

Your feedback will help us make it better. Thanks!

Quotation of the month

For a responsive creature, to exist is to evaluate.

Barbara Herrnstein Smith, in Contingencies of Value

Feel free to reproduce articles from our newsletter as long as you acknowledge Econnect Communication Pty Ltd as the source.