EVENT COVERAGE

PANEL DISCUSSION ON LOCAL PEOPLE METERS
By Raul Perez, Ph.D.
President, Utilis Research & Consulting
rperez@utilis-research.com

As market research and public opinion professionals, we are interested in issues related to research’s ability to accurately reflect population estimates. Therefore, the polling and market research industry in general has been interested in the developments at Nielsen Media Research (NMR) regarding local people meter (LPM) use in select local markets. As many of our readers know, some of Nielsen’s clients have challenged the LPM’s accuracy when measuring minority populations. Since the spring of 2004, the media has widely covered the controversy on the transition to local people meter data from TV viewing diaries. People meters are devices that automatically record time and channel/station being viewed, but require panel participants (i.e., each household member) to press a pre-assigned button to identify themselves. While people meters have been used for some time to measure audiences at a national level, diary measurement has been the norm in local markets until recently. Separate local market measurement is necessary to inform media buys at a regional level.

NYAAPOR invited experts on different sides of the controversy to help illuminate the issues related to local people meter adoption. NY-AAPOR invited Paul Donato (Senior VP/Chief Executive Officer, Nielsen Media Research), David F. Poltrack (Executive VP, Research & Planning, CBS Television), George Ivie (Executive Director/CEO, Media Rating Council, Inc.), and David Marans (Senior Partner, Mind Share). The panel discussion took place in late September, and was attended by over 60 NY-AAPOR members and other professionals.

At the center of Mr. Donato’s presentation was the contention that people meters are considered a better practice globally when compared to diaries. For example, diary respondents underestimate infrequent behavior, and overestimate frequent behavior. Diary panels, therefore, do not necessarily coincide with actual behavior. Data from dairies and meters would differ mainly because different methods lead to different measures of audience.

The methodological differences between local people meters and diaries led in some cases to drops in share numbers. Those clients affected wanted to see the people meter launch postponed as these drops may have been also caused by methodological factors other than the method of measurement (meter vs. diary). As David Poltrack suggested in his presentation, faulty household language weighting may be related to these differences.

Mr. Poltrack pointed to a few problems in people meters, despite the arguments mentioned by Mr. Donato in the meter’s favor. We enumerate them below:
1) Meters exclude those in the room that are not consciously watching TV.
2) Current methodology does not measure households with DVRs.
3) Current procedures exclude homes that are considered “immeasurable” for a variety of reasons.
4) Meters do not capture out-of-home viewing behavior.
5) The methodology, as currently implemented, has “unstable” language weighting, which is based on self-report.

According to Mr. Poltrack, the instability of language weighting is due to a series of factors. Language weights results will be different depending on the survey respondent (different household members may use English differently). The research conducted to determine language use does not address normal changes over a particular time period (e.g., the arrival of a new household member with different English proficiency). Therefore, language weights tend to be outdated by the time they are implemented.

Mr. Poltrack also pointed to the Hawthorne effect –the local people meter does affect viewing behavior. Mr. Donato himself talked about “Nielsen Meter Parties” organized by a few respondents at the beginning of their participation as panel members, an issue that only affects data perceptibly in the first few weeks of data collection.

David Marans made a refreshing and provocative presentation that reminded us of the context where the research endeavor takes place. The main point of his presentation was to remind us of the seemingly unreasonable (our italics) expectations we have of respondents in opinion and market research. Respondents are constantly asked demanding tasks (e.g., hard to picture rating scales, or the requirement that they press buttons at special points in time under certain conditions with high accuracy) that would make anyone wonder if participants should be a special kind of citizen who (1) is research literate to a high degree, (2) has an unbending desire to participate for little or no compensation, (3) is trained to answer with 100% accuracy. Given these high expectations researchers may have about respondents, Mr. Marans strongly supports the use of passive methods of data collection.

Finally, Mr. Ivie made a presentation of the steps his organization (Media Rating Council, Inc.) is taking to address the issues raised by the local people controversy. While defending the use of local people meters as a better practice when compared to local diaries, Mr. Ivie did acknowledge challenges with the local people meter that were short of the stronger criticisms made by Mr. Poltrack. Among the challenges, Mr. Ivie cited technology related issues and sensitivity to respondent effects. In the role of CEO of an accreditation organization of which Nielson is not a member, Mr. Ivie could only but encourage the development of further Research & Development efforts to address the challenges he and his colleagues mentioned in the panel.

The panel participants we invited, and their comments, illustrated the interconnectedness among professionals in the research industry, its users, consultants, and other industry groups. Under most research situations, when facing conflicting views, research clients would be inclined to sever ties with suppliers, but this course of action is not an option for audience measurement clients: media buyers need an unequivocal source of information from which to make decisions, and the industry as a whole agrees with this aspect. As an organization, the NYAAPOR does not have an opinion on the controversy. However, everyone present would agree that the four panelists provided a productive model for industry debate –while disagreeing on very important aspects, they exhibited respectful, even amicable deference to each other, which we consider to be an inspiring example of civility given the level of contention generated by the local people meter launch.