Community of Evaluators-Nepal

(CoE-Nepal)

Online Repository

Collect and/or Retrieve Data





  • 3,290 total views, 81 views today

This task focuses on ways to collect and/or retrieve data about activities, results, context and other factors. It is important to consider the type of information you want to gather from your participants and the ways you will analyze that information, before you choose your option. You should also consider triangulating your options in order to ensure multiple data sources and perspectives.

Options

There are five clusters of options listed under this task:

  1. Information from individuals
  2. Information from groups
  3. Observation
  4. Physical measurements
  5. Reviewing existing records and data

Information from individuals

  • Deliberative Opinion Polls: providing information about the issue to respondents to ensure their opinions are better informed.
  • Diaries: monitoring tools for recording data over a long period of time.
  • Goal Attainment Scales: recording actual performance compared to expected performance using a 5 point scale from -2 (much less than expected) to +2 (much more than expected).
  • Hierarchical card sort:Hierarchical Card Sorting (HCS) is a participatory card sorting option designed to provide insight into how people categorise and rank different phenomena.
  • Interviews with individuals:
  • Convergent Interviews: asking probing questions to interviewees and then using reflective prompts and active listening to ensure the conversation continues.
  • In-depth Interviews:using probing and multiple interview sessions to collect detailed responses from participants beyond initial answers to questions.
  • Key Informant Interviews: interviewing people who have particularly informed perspectives.
  • Keypad technology: gauging audience response to presentations and ideas in order to gain provide valuable feedback from large group settings.
  • Mobile Data CollectionTargeted gathering of structured information using devices such as smartphones, PDAs, or tablets.
  • PhotoVoice: promoting participatory photography as an empowering option of digital storytelling for vulnerable populations.
  • Photolanguage: eliciting rich verbal data where participants choose an existing photograph as a metaphor and then discuss it.
  • Polling Booth:collect sensitive information from participants anonymously
  • Postcards: collecting information quickly in order to provide short reports on evaluation findings (or an update on progress).
  • Projective Techniques: participants selecting one or two pictures from a set and using them to illustrate their comments about something (also known as photo-elicitation).
  • Questionnaires(or Surveys)
  • Email Questionnaires:distributing questionnaires online via email.
  • Face to Face Questionnaires:administering questionnaires in real time by a researcher reading the questions.
  • Internet Questionnaires: collecting data via a form (with closed or open questions) on the web.
  • Mobile Questionnaires:using mobile phones to distribute surveys, either by linking with an adapted internet-based survey or through a specific survey app.
  • Mail questionnaires:posting hard copies to participants to be returned.
  • Telephone Questionnaires: administering questionnaires by telephone.
  • Seasonal Calendars: analysing time-related cyclical changes in data.
  • Sketch Mapping: creating visual representations (‘map’) of a geographically based or defined issue.
  • Stories (Anecdote): providing a glimpse into how people experience their lives and the impact of specific projects/programs.

Information from groups

  • After Action Review: bringing together a team to discuss a task, event, activity or project, in an open and honest fashion.
  • Brainstorming: focusing on a problem and then allowing participants to come up with as many solutions as possible.
  • Card Visualization: brainstorming in a group using individual paper cards to express participants thoughts about particular ideas or issues.
  • Concept Mapping: showing how different ideas relate to each other – sometimes this is called a mind map or a cluster map.
  • Delphi Study: soliciting opinions from groups in an iterative process of answering questions in order to gain a consensus.
  • Dotmocracy: collecting and recognizing levels of agreement on written statements among a large number of people.
  • Fishbowl Technique: managing group discussion by using a small group of participants to discuss an issue while the rest of the participants observe without interrupting.
  • Future Search Conference: identifying a shared vision of the future by conducting a conference with this as its focus.
  • Interviews with groups
  • Focus Group Discussions: discovering the issues that are of most concern for a community or group when little or no information is available.
  • Mural: collecting data from a group of people about a current situation, their experiences using a service, or their perspectives on the outcomes of a project.
  • ORID: enabling a focused conversation by allowing participants to consider all that is known (Objective) and their feelings (Reflective) before considering issues (Interpretive) and decisions (Decisional).
  • Q-methodology: investigating the different perspectives of participants on an issue by ranking and sorting a series of statements (also known as Q-sort).
  • Social mapping: Identifying households using pre-determined indicators that are based on socio-economic factors.
  • SWOT Analysis: reflecting on and assessing the Strengths, Weaknesses, Opportunities and Threats of a particular strategy.
  • World Cafe: hosting group dialogue in which the power of simple conversation is emphasised in the consideration of relevant questions and themes.
  • Writeshop:a writing workshop involving a concentrated ​process of drafting, presenting, reviewing, and revising documentations of practice.

Observation

Gathering information by observing people, places and/ or processes either directly or through still or moving images (photography or video). This cluster of options involves watching and documenting the incidence of objects and/ or the behaviour of people.

These options do not involve gathering data directly from individuals or groups, but rather about observing individuals, groups and things. Evaluators of an education project may observe the physical attributes of a school, the accessibility of the site, the availability of latrines, library, and playground. The evaluator may observe the numbers of boys and girls in a classroom, the teaching techniques used and the types of resources that children use.

  • Field Trips: organizing trips where participants visit physical sites.
  • Non-participant Observation:observing participants without actively participating.
  • Participant Observation: identifying the attitudes and operation of a community by living within its environs.
  • Photography/video: discerning changes that have taken place in the environment or activities of a community through the use of images taken over a period of time.
  • Transect: gathering spatial data on an area by observing people, surroundings and resources while walking around the area or community.

Physical measurements

Measuring physical changes based on agreed indicators and measurement procedures. Examples include birth weight, nutrition levels, rain levels, and soil fertility.

  • Biophysical: measuring physical changes over a period of time related to a specific indicator by using an accepted measurement procedure.
  • Geographical:capturing geographic information about persons or objects of interest such as the locations of high prevalence of a disease or the location of service delivery points.

Existing documents and data

Reviewing existing knowledge through project documents, information on related projects, government records and publicly available statistics.

  • Big data: Large data sets that cannot be analysed using conventional methods, often produced as byproducts of engagement, such as social media data.
  • Logs and Diaries: monitoring tools for recording data over a long period of time.
  • Official Statistics:obtaining statistics published by government agencies or other public bodies such as international organizations. These include quantitative or qualitative information on all major areas of citizens’ lives such as economic and social development, living conditions, health, education, the environment.
  • Previous Evaluations and Research:using the findings from evaluation and research studies that were previously conducted on the same or closely related areas.
  • Project Records:retrieving relevant information from a range of documents related to the management of a project such as the project description, strategic and work plans, budget and procurement documents, official correspondence, minutes of meetings, description and follow-up of project participants, progress reports.
  • Reputational Monitoring Dashboard: monitoring and quickly appraising reputational trends at a glance and from a variety of different sources.