Evaluation is an important, though frequently neglected, component of any program. Some congregations use word of mouth feedback and other leaders deliberately seek out participants to assess outcomes through conversations. Others evaluate their programs through questionnaires that ask participants to rate their experience on scales of satisfaction, effectiveness, impact, or program strengths and weaknesses. Quite a few congregations are not in the habit of evaluating their programs. The visual technique known as photo elicitation offers an additional tool for the important task of program evaluation.
When measured by number of publications, photo elicitation (PE) appears to be the most popular visual research method in the social scientific study of religion. Originally developed by anthropologist John Collier in the late 1950s, this technique uses photographs as prompts in interviews to stimulate conversations about participants’ everyday lives, memories, and experiences. Increasingly, other visual materials are being used in interviews, including drawings, artwork, and objects, leading some to prefer the term “elicitation interviewing,” even though the vast majority of studies employ photographs. PE has been applied in a number of social scientific studies of religion, most recently in Nancy Ammerman’s Sacred Stories, Spiritual Tribes (2013).
Photo elicitation shows promise as a means of program evaluation. The source of images is an initial consideration. A researcher or professional photographer may document an event or aspects of a program and employ these images in an interview; alternatively, participants could chronicle a typical day in a program or be given a list of topics related to specific program outcomes to capture visually. Photos could also be taken at the beginning, midpoint, and end of a program and used in interviews at each of those points in time to explore progress, continuity, and change. Employing the same researcher-produced photographs with all program participants enables comparisons among participants, whereas photos unique to an interviewee may reveal aspects of the program under evaluation that may not otherwise have been explored.
Whatever the source of the images, they are used in an interview in which the participant is given the photos and invited to discuss them. I find it helpful to conduct photo elicitation interviews in a location where the images may be laid out on a table or desk. And before the interview starts, I ask participants to look through the photographs and to organize them in whatever way they think will be helpful for our conversation. Sometimes people group photos thematically, by location, or chronologically. With these preparations complete, I simply ask the interviewee to choose a photo and to tell me about it. As the interviewee relates the story behind their photograph, I try to nudge the conversation in directions relevant to the aims of the research. Those familiar with standard interviewing will quickly notice an important quality of photo elicitation: while interviews are typically conducted face-to-face, in PE the researcher and participant sit shoulder-to-shoulder discussing images. As a result, the researcher becomes a learner and the interviewee a teacher of sorts. I like to think of the photographs and the conversations they spark as answers to questions I may otherwise not have known to ask.
Typically, photo elicitation interviews are audio recorded and subsequently transcribed and analyzed. Handwritten notes during the interview may also be an effective way to record the conversation. These notes should be rewritten as soon as possible after the interview to fill in any gaps or details. However one decides to record the interview, the next step is analysis. At the most basic level, analyzing photos and interview transcripts/notes is similar to the exegesis of scripture: an effort is made to understand the images and words in their proper context, common themes are identified, materials are critically engaged, and comparisons are made. A researcher could print out all their interview materials and use colored highlighters or pens to identify common themes in a fashion similar to the way some Christians highlight their bibles. The goal of the analysis should be to understand what participants reveal visually and verbally about the program under evaluation with a view toward identifying its strengths and shoring up its weaknesses. Finally, a report on the program’s effectiveness should be written using the words and photos as evidence of the program’s success and to identify areas that need to be improved. In other words, use the participants’ own words and photos to tell the story of their experience in your program. Allow those involved in the program to help you (and other leaders) see it from their perspective.
This is the second post in a series by the Congregational Studies Visual Methods Fellow, Roman R. Williams, Ph.D., Assistant Professor in the Department of Sociology and Social Work at Calvin College. He is an expert in visual research methodology, and his current research looks at ways in which local congregations can use visual research methods to better understand congregational dynamics. See his previous post on Congregational Snapshots: Understanding and Engaging Congregations through Cell Phone Photographs.
