 |
||||
Research Panel: "Strategic Design and Impact Evaluation " EE Works, But How?The essence of EE seeks change: change in people's behavior, change in people's attitudes, change in people's lives. In the Research Panel on Strategic Design and Impact Evaluation, chaired by Suruchi Sood (USA), EE practitioners and researchers shared from their experience how they evaluated the impact of their programs on people's lives. The CDC Sees Opportunity in American SoapsIn the first presentation, Vicky Beck (USA) focused on American soap operas and how viewers learn, and act on, health information received through that medium. Based on nearly 3,000 surveys, her team at the Centers for Disease Control and Prevention found that about 20% of Americans consider themselves regular viewers of soaps, with the highest representation among low-income, low education, female, and minority groups. Interesting responses were made to the question: "Have you, in the past year, learned anything about a disease or how to prevent it on a soap opera?" Nearly half (48%) of regular viewers said "yes." Most interesting, said Beck, was that one-third of regular viewers took an action as a result of what they saw. One-fourth mentioned the health information to someone they knew, and 7% actually made a visit to a health care practitioner. "Consistently in this group of regular viewers," said Beck, "we found higher rates for health problems, for health plan restrictions, and for low assessment of health." It's obvious that this is an audience in need of health information. But, she said, while their research found that this audience seeks out health information more, they understand it less. Our overall goal, said Beck, is to have more accurate information in the messages and to have more health story lines. She explained that the CDC does this through a variety of ways, including developing and distributing resource materials, holding network briefings, offering health experts to speak at media events, and meeting with producers and writers of the shows. Consulting is free, which is a pleasant surprise for many in the entertainment business. The CDC has also developed a soap opera award, called the Sentinel for Health Awards. The awards will be given annually to soap opera writers and producers who incorporate into their program an accurate story line on a critical health issue. The winning program at the ceremony in West Hollywood was ABC-TV's One Life to Live for a storyline on breast cancer. Other winning entries addressed adolescent HIV/ AIDS, drunk driving, and disability. As a Community Changes, So Might a Nation: Radio Soaps in IndiaIn the second presentation, Arvind Singhal (USA) spoke about community-level effects of a radio soap opera called Tinka Tinka Sukh (translated as "happiness lies in small things"). This was a radio soap opera, broadcast by All India Radio in 1996 and 1997. It ran for one year, over 27 radio stations, for 104 episodes. Researchers logged 47 categories of issues, including community pride, equal treatment of boys and girls, marital issues, empowerment of women, educating women, and many other gender themes.
Dowry was a message that had been emphasized heavily in the soap. "Now, if you're a researcher and the program producer forwards you a letter like this because you are involved in a research evaluation of the program," said Singhal, "you are intrigued! So we decided to visit this village." The village has about 6,000 people, 60 radio sets, and 5 television sets. "There are very many different methods that we used in our study of this radio soap opera," said Singhal. Methods included unstructured interviews, focus group interviews, participant observation, archival record searches, and local folk form observations. They discovered that an important "spark plug" for change in this community was the village tailor. The tailor was an avid listener of radio, and his shop was located in a high traffic area of the village, so people would naturally stop by his shop to socialize, and listen to the radio soap opera. The tailor was a driving force behind getting signatures on the community letter that eventually found its way to Singhal. Other opinion leaders in the village encouraged the creation of listening groups. Villagers could go to these "listening clubs," where villagers would gather to listen, discuss the program, then create strategies for change in their community. Singhal read a representative comment his team recorded from their study of the villagers: "We would all listen to the serial and we would hold discussions afterwards. For example, when we listened in the afternoon, in the evening when we returned home we would talk about it with others. They would also say that they would not give dowry, and tell me that I would not have to give dowry. We would advise each other how we should counsel others not to give or take dowry. Now, after listening to Tinka Tinka Sukh and after we took the vow, as expressed in that letter, that we will not give or take dowry, we have formed a group to end dowry in this village. In this way, our sisters and daughters will not suffer." Another community-based initiative they noted was how the village educated its girls. The community addressed this issue by seriously considering why certain young boys and, in particular, girls did not attend school. When they found that a primary factor was childcare (girls caring for their younger siblings), the community decided to create two new cooperative childcare centers. Before the community acted, the percentage of girls in school was 10% of the classrooms. After the program was initiated, that number jumped to 40%. Singhal stressed that the findings in this one village could not be extrapolated to all of India. "But," he noted, "these findings still provide us a unique view into the process of how EE programs can spark changes, how opinion leaders act and mobilize efforts, and how a community comes to terms with social change." The Measure of Soul City: Multimedia, Mass Communication Programming
The third presenter was Sue Goldstein (South Africa). Soul City is an extremely popular, multimedia mass communication program: from their survey of 2,000 people, they calculated 14 million television viewers, 12 million radio listeners, and 10 million people reached through booklets. Among 16-to 24-year-olds, 79% had seen Soul City in some form; among urban dwellers, 75%; and among rural dwellers, 60%. Goldstein briefed the audience on Soul City's efforts in television, radio, print media, and special outreach programs (such as with telephone hotlines for female victims of domestic violence, done in collaboration with National Network on Violence Against Women). Goldstein shared preliminary findings from their evaluations, which were aimed at investigating the program's impact on individuals, how those impacted individuals interacted with their communities, and, finally, in a societal context. She described their multi-methodology strategy, which included studies in national qualitative impact assessment, cost outcome descriptions of specific programs, monitoring and analyzing media that resulted from Soul City programming, institutional and organizational impact studies, and partnership studies, among others. Goldstein reported that, generally, increased exposure to Soul City programming positively impacted an individual's behavior change. For example, she highlighted the results from a health clinic in a rural area, which attributed their policy change from distributing condoms one day a week to distributing condoms every day of the week directly to Soul City programming. Changes do not come overnight, she cautioned. "I think the kind of pressures we're under from funders is to say, 'You did this in Soul City and the HIV rates went down. ' But the complexities of behavior change are very often not recognized." She encouraged participants to explain to decision makers that changes do not occur quickly and that, in fact, programs need to be funded consistently over many years' time to see the intended results.
|
Singhal
admitted that he stumbled upon the community-level aspect of his
research by accident. He related how that, while the soap opera
was being broadcast, All India Radio received an unusual letter,
written by an entire community. The letter, about 2 feet square,
was very colorful and signed by 184 people. "There were even thumbprints
of women who could not sign their name," he noted. Besides the
intriguing concept of the letter, its contents were unusual. The
letter noted: "We've been listening to the soap opera as a community.
We do group listening. We love this program. This program has
changed the life of our community and we as a community, based
on the messages that we have heard from the soap opera, have decided
that we will not give or take dowry."
