The Basics
- Funders and donors like evaluation data
- When developing a nonprofit, program, or project, think through your feasibility study, evaluation, and assessment tools
- Evaluations should take place annually
- Assessment takes place after three or five years have passed
- You can evaluate program satisfaction, audience engagement, donor attitudes, employee satisfaction
- Do not use “Yes-No” questions
- Be careful if you use Internet-based evaluation. Some of your clientele (constituents) may not have Internet access
- List evaluation fees in your program grant and major gift requests
- To preserve anonymity, ask the bare minimum of demographic questions
A Few Words on Survey Development
- You need to use the same survey year-to-year
- Use hypotheses questions that require a 1-5 or 1-7 scale, such as
- How likely are you to use our service(s) again?
- How likely are you to recommend our service(s) to another person?
- How satisfied were you with our services?
- How likely are you to volunteer again?
- How satisfied are you with working here?
- Use rating questions, e.g. Rate the following questions in order of importance
- Use bipolar questions with a 1-10 scale, e.g., How often did you speak with a counselor (Not at all… A lot)
- Provide definitions
- Write driver questions so that you can understand why a person responded to the hypothesis question in a certain way
- Research question: How satisfied are you with working here?
- Driver question: How satisfied are you with the amount of time that your supervisor spends with you?
- Make the survey available in more than one language
- Try to have an expert validate your survey before you launch it; you cannot use a survey in an evaluation without a professional check on reliability
- If your subject responses are under 25 people, you want to study averages
- If your subject responses are at or over 25 people, you want to study correlations, t-tests, and possibly ANOVA
- If your subject responses are at or over 100 people, you want to study regressions
Program Evaluations
- Generally, your goal is to learn about satisfaction
- Think about what outcomes you want to study. See Harvard Business Review’s “It’s Not Just Semantics: Managing Outcomes Vs. Outputs”
Working with a Consultant
- If you don’t have an in-house researcher, at the very least, hire a researcher to review your survey before you launch it. Look for a flat fee for this service.
- Tell your potential consultant if you modeled your survey on a pre-existing one. A good consultant will want to see the original survey.
- Test your potential consultant’s knowledge of quantitative and qualitative analysis. Ask, “We’ve heard that we need correlation, linear regression, disaggregating data, coding, and theming. What do you think?”
- If your potential consultant can’t speak to disaggregating, coding, and theming, don’t hire that person.