Door-knocking with Rajar

You know those days when the weather is so cold, your body almost shuts down and concentrates on sustaining vital functions.  It was one of those.  I was admittedly, ill-dressed for the day, albeit looking smashing in my blue Fred Perry jacket. 

Wonderful Wendy was better-prepared in her ample baggy coat.  As a cheery Rajar researcher of many years standing, it was hardly surprising WW knew what she was doing.  Here was someone who'd walked miles and miles in her life on variously bitter cold and sweltering hot days, banging on doors in the name of radio. 

WW is one of a truly dedicated team of researchers, working for the research company IPSOS-MORI, which is currently contracted to carry out the fieldwork to the specifications set by RAJAR, to produce the audience figures on which UK radio relies.

Over the years, we've all had a pop at RAJAR, given from time to time it does produce a result which seems, well, puzzling to say the least. Usually, a further audience sweep or two thankfully sees the germ flushed out the system, and smiles return.  It was when the whole markets my stations serve caught what looked to me like the Winter vomiting bug, that Ipsos afforded me the rare opportunity to accompany a researcher as they carried out their task.  If only to shut me up.

'A single job' for a single researcher is to persuade about 22 people of all shapes and sizes to fill in a RAJAR diary.  The default diary now is typically online, but people are given the chance to choose papyrus should they wish.  Either way, the research journey starts with a knock on the door.

It was Thursday, and we were seeking good folk to start their listening records the following Monday.  As we wandered down the first road selected by Ipsos, clipboards in hand, I suspected  it would be simple to find a few willing candidates.  Just grab the bloke walking down the road to Londis, or the woman pruning her privet.  Reality was less straightforward: the researchers are given a list of 'starred addresses' and are obliged to bang on these doors first.  Only if they get a refusal are they allowed to go to the neighbouring address on the list.  If the door is simply not answered, researchers are asked to return later before selecting the alternatives.  To say it takes dogged persistence would be an understatement.

My days as a spotty paper boy flooded back. Where is the doorbell? Does it work? Should we go through the porch door or peer from outside? How many times and how hard should we bang?  Wonderful Wendy was an expert in all this, and mastered too how to cope with a veritable Animal Hospital of dogs and cats.

The script at the door is standard and well-rehearsed.  Declare research company name, flash badge, smile and ask them if the puzzled person holding the door slightly ajar  would like to help  give their opinions on radio programmes.  Witness the cogs whirring in the householder's head.  Some volunteer: 'I don't listen to the radio at all', and the researcher then has to insist that their views are still valid, which possibly bemuses the respondent further.  Once inside, in some cases, there is the opportunity to 'snowball' and recruit more than one person within the household, or a friend at another starred address.

I was hoping a door might be opened to us; frankly more so I might get warm than any remaining interest in the process, my survival instincts having kicked in.  Thankfully, WW's efforts were not totally in vain; she had persuaded  a couple of folk to allow her back another time.  I gather Fridays and Saturdays tend to be more productive than Thursdays.

It was agreed, prior to my trek out, that I would be able to observe a mock interview of the full procedure.  Once through the front door of the respondent, it lasts around a quarter-hour.  To add some realism, we could not resist some role-play; and our Chief Exec, Phil Riley, ideally cast, played the part of an awkward respondent.  It took him a few handfuls of fingers to answer the question 'how many radios have you in your house'. As WW keyed in his answer, her official Rajar laptop duly bleated an 'are you sure?' verification message.  It should have known that Phil always is.  

Other questions followed, for example to establish what listening platforms he had available.  One such question features a picture and description of DAB radios, to establish whether a listener has one of these sparkling devices.  Not a perfect method to establish the answer but it's tough to suggest better; albeit a listener's listening repertoire offers a further clue.

Phil suggested he wasn't too hot on the old computer stuff, so WW whipped out a good old-fashioned colourful listening diary from her huge bag.  A pack of yellow and green cards were offered; and he was invited to select the stations which he'd been listening to in various ways, to grab both active and passive listening.  

Each card bears a station name (or two names, in the case of e.g. BBC Radio 4 and 4 Extra), alongside a simple platform description e.g 'a digital radio station'.  A first calm run through all the cards.  Then a second run through some of them, and a third run through the rest.  Like Valerie Singleton, WW then peeled off labels for each of the selected stations and stuck them at the head of the columns in the Rajar colouring book, before explaining how to draw crosses on the page joined by a line, to denote a listening occasion.  Too many stations for the twelve label placings? You are asked to  drop a few from your repertoire and note that de-minimis listening as 'any other'.  The labels are left with the respondent just in case they want to add any later; with the average number stuck in across the UK being 5.4 (6.2 in the West Mids).  The full station descriptions ('Biggest Mix of Variety & Fun with Fred Flintstone in  the morning'), over which we agonise, are contained in a huge slab of text towards the end of the diary.   

WW also took the trouble to tempt Phil into completing the rear section of the book with any comments on stations.  As she stressed its importance, I felt a tad guilty that so few of us bother reading through the comments when they are sent to us.

The multiplicity of languages spoken now in the UK is a challenge for any research, and although Rajar do have some explanatory letters in all manner of languages, it is self-evidently a challenge when researcher and respondent cannot communicate.  The richer the ethnic texture of an area, the more challenges that brings in recruiting.  Having said that, Rajar stats suggest that once a diary is successfully placed with ethic minority respondents, it's probably easier to get completion than with , for example, 15-24s as a whole.  This month will also bring an update to Rajar's population weightings as the new Census results are fed in; and one imagines some pretty serious shifts in the complexion of some markets.

Had an online diary been chosen, the respondent gets a useful little booklet asking them simply

to log on and follow the instructions on screen, accompanied by a listening jotter for notes.  If they have any questions, they're asked to 'call Alex' on a dedicated number.  All their assistants are called Alex.  I like that.  The good thing about online diary-keeping too is that the researcher knows whether or not their charges are doing their homework; and polite emails prompt along the way.

It was a fascinating experience.  Hats off again, to WW and her skilled colleagues.  If it's warm enough to forsake headwear.  I'd seen many of them at a researcher briefing a few weeks prior. They all appeared polite yet insistent, dedicated, professional; and one could see many of these indomitable characters  had achieved a lot in life, with this work being a useful way of topping up income and drawing on their considerable people skills.  They planned their excursions with military precision and pride.  Several have been working on the Rajar contract for some years; and some work for Ipsos on other research projects.

If door-to-door is the best way of conducting this major survey, and Rajar appears to think so at least for now, then I am assured that the researchers are doing it in a way of we should be proud.   And the trend to online diary-keeping shortens the briefings, which has to be a good development.  Respondents cannot yet use mobile or tablet to complete the online diary but I gather that's in the pipeline.  Overall, it's still a currency which enjoys trust and is largely consistent.

Longer term, however, is door-knocking the best methodology for the 21st Century?  Were I on the Rajar committee, I suspect I'd have heard intelligent and informed debate on just this topic.  Yes, some sort of watermarked signal triggering off a measurement device in your phone, sounds a decent solution.  I also know of a long list of quite serious limitations for that too.  But seeing the increasing numbers of 'no hawkers or circulars' signs on people's doors, ever more doorstep suspicion, and people generally leading busier lives, I am not sure just how long we can continue to rely on teams of intrepid WWs to recruit a representative band of respondents to spend quite so much time having their duties explained to them.  Would our industry ever even consider a wholly online survey journey from recruit to submission, regardless of whether recall or electronic data capture?

There is doubtless an alternate long-term solution; and we have to place trust in Rajar to identify it, in due course, and move towards it safely. We then have to accept the results may well be different from now; and not moan too much.  And, judging by the above, I'd be pretty confident in the diligence of IPSOS MORI to dispense it for us.

But, hey, what an afternoon.  When I look now at the mammoth size of the Rajar sample across the UK, still one of the most extensive research projects in the UK for any product or service, I am in awe of those who manage to persuade thousands and thousands of people to fill in the diaries. To all the WWs, thank you.  We appreciate it. Honestly.

Follow my #radiomoments daily on Twitter @davidlloydradio. 

Grab my book 'How to Make Great Radio' from Biteback.