INFORMATION COLLECTION TOOLS – ADVANTAGES AND DISADVANTAGES
Information Collection Tools Observation Advantages    Collect data where and when an event or activity is occurring Does not rely on people’s willingness to provide information Directly see what people do rather than relying on what they say they do Relatively inexpensive Good source of background information Unobtrusive Provides a “behind the scenes” look at a program that may not be directly observable May bring up issues not noted by other means Useful for gaining insight and context into a topic Allows respondents to describe what is important to them Useful for gathering quotes and stories Quick and relatively easy to set up Group dynamics can provide useful information that individual data collection does not provide Is useful in gaining insight into a topic that may be more difficult to gather information through other data collection methods Disadvantages  Susceptible to observer bias  Hawthorne effect – people usually perform better when they know they are being observed  Does not increase understanding of why people behave the way they do  Information may be inapplicable, disorganized, unavailable or out of date  Could be biased because of selective survival of information  Information may be incomplete or inaccurate  Can be time consuming to collect, review, and analyze many documents  Susceptible to interview bias  Time consuming and expensive compared to other data collection methods  May seem intrusive to the respondent  Susceptible to facilitator bias  Discussion can be dominated or sidetracked by a few individuals  Data analysis is time consuming and needs to be well planned in advance  Does not provide valid information at the individual level  The information is not representative of other groups

Document Review

   

 Interviews    Focus Groups  



1

Surveys and Questionnaires





 



Town Hall Meetings and Other Large Group Events

  

Administration is comparatively inexpensive and easy even when gathering data from large numbers of people spread over wide geographic area Reduces chance of evaluator bias because the same questions are asked of all respondents Many people are familiar with surveys Some people feel more comfortable responding to a survey than participating in an interview Tabulation of closed-ended responses is an easy and straightforward process Can gather large amount of data at one time Allows respondents to describe the issues that are important to them Provides a venue where people can build on each others’ knowledge Fully depicts people’s experience in program input, process, and results Powerful way of portraying program to outsiders Fun to do and easier to get people involved because it doesn’t seem “researchy” Especially useful as a way to get people of different cultures involved or people who are more visual than verbal Powerful way to represent data – “A picture is worth a

 Survey respondents may not complete the survey resulting in low response rates  Items may not have the same meaning to all respondents  Size and diversity of sample will be limited by people’s ability to read  Given lack of contact with respondent, never know who really completed the survey  Unable to probe for additional details  Good survey questions are hard to write and they take considerable time to develop and hone  Organizing the event takes time and resources  Definitely need to have a draw to get people to attend in the form of incentives  Need to have access to people with good facilitation skills  Need to have “ducks in a row” to ensure attendance at event  Usually quite time consuming to collect information, organize and analyze it  Represents depth of information rather than breadth  Takes some technological skill/expertise to “produce” the presentation  Need to have good facilitation skills given that these methods are group work processes  May not speak to stakeholders who prefer more quantitative approach to data collection and may not be appropriate in

Case Studies

 

Illustrated Presentations – Photo Voice, Power Voice

 



2

thousand words.” Skits, Dramatizations, Other Visual Representations  An alternative approach to linear models of evaluation that may not fit well crossculturally Useful methods when part of the evaluation purpose is to move toward some form of action Fun, innovative approach to evaluation that causes people to view program through another perspective





situations where numbers are needed  Takes some courage for people who aren’t accustomed to public display  May not speak to stakeholders who prefer more quantitative approach to data collection and may not be appropriate in situations where numbers are needed

SOURCES Finn, J., & Jacobson, M. (2008). Just Practice: A Social Justice Approach to Social Work. Peosta, IL: eddie bowers publishing. Jacobson, M., Pruitt Chapin, K., & Rugeley, C. (2009). Toward Reconstructing Poverty Knowledge: Addressing Food Insecurity through Grassroots Research Design and Implementation. Journal of Poverty, 13(1), 1-19. Russ-Eft, D., & Preskill, H. (2001). Evaluation in Organizations. New York: Basic Books.

3