 |
|
|
|
|
|
Chapter 14. Performing Evaluation Research Introduction In the previous chapters you learned how to use quantitative and qualitative research techniques. This chapter shows how these research techniques can be used to evaluate or assess programs and services. To help you understand how evaluation studies are conducted, we begin this chapter with a description of an evaluation of a program carried out in Uganda (Kajimu, and others, 1998). We next describe ways evaluation research differs from other social science research. With this background, we present the steps involved in most evaluations. Even though ethical issues are discussed in Chapter 6, we return to this topic because evaluation research can raise special ethical and professional concerns. This chapter concludes with a review of problems that can limit the value of an evaluation. Illustration of evaluation research Description of the program In 1991, the Islamic Medical Association of Uganda (IMAU) conducted a survey to measure knowledge about HIV infection and perceptions of the risks associated with getting the disease (Kagimu and others, 1998). Based on the survey, the IMAU designed a two-year prevention program. The program was based on the fact that the iman (mosque leader) is the teacher of family values and behavior, including those related to risks associated with HIV. Therefore, the program focused on training imans who would then conduct household visits with families and provide information that could help prevent the spread of HIV. The objectives of the program, as operationalized in the household visits, were to: (1) increase knowledge about the transmission of HIV; and (2) to change sexual behavior in ways to prevent its spread. Twenty three (23) persons were trained to become the trainers of local teams of imans and their assistants. The training sessions included 396 imans and 792 assistants. Each team then conducted 1 or 2 visits per month with approximately 15 households each. Repeated visits allowed the imans and their assistants to introduce new topics and to revisit topics discussed previously. Over the course of the 2 years, 114,634 household visits were completed. Evaluation of the household visits At the end of the two years, a follow up survey was conducted using the same questions that were asked in the original survey. This "before" and "after" quantitative measurement was supplemented with qualitative data. Intensive interviews were conducted with 25 imans, their assistants, and community leaders. In addition, 75 persons participated in 9 focus groups. Several groups consisted only of adult males aged 25 to 40; others included only younger adult men, aged 20 to 29; and several were made up of females between the ages of 15 and 19. Separate groups were organized by age and gender to encourage fuller and more honest discussion of the risks of becoming HIV infected. After reviewing both the survey and focus group results, the research team concluded that the program had achieved its objectives. There was a substantial increase in correct knowledge about HIV transmission and in methods of preventing HIV infection. In particular, participants recognized that present practices in handling dead bodies and unsterile circumcision could cause the spread of HIV. Evidence of changes in sexual behavior was found as well. There was a significant decline in the number of sexual partners reported by respondents under 45 years of age. Condom use increased among urban males, even though some religious leaders discouraged their use. Qualitative data supported these findings and showed the importance of the repeated household visits or imparting knowledge and encouraging changes in sexual behavior. The Ugandan evaluation also illustrates the ways evaluation research differs from the usual question-answering research. |