WHAT'S WRONG WITH THIS GRANT?

ANSWER KEY

There are at least three things wrong with this grant excerpt. The most obvious problems:

1. Confusion between outcomes and performance indicators. An outcome is a change that is likely to take place as a result of the target population's participation in your program. The "outcomes" described in rows 1-5 are not outcomes at all, but are actually summaries of the performance indicators listed in the right hand column. This table makes no distinction between performance indicators (or performance objectives), as detailed in the first five rows, and outcomes, as detailed in rows 7-10. Not being able to distinguish an outcome from a performance indicator is lethal. Any funder that cares about this at all (and DOE is certainly one that cares) is going to assume that this agency is not capable of understanding the importance of outcomes or creating a meaningful outcome evaluation.
   
   
2. Right- and left-hand columns say the same thing. Even if the writer didn't know enough to correctly distinguish the difference between outcomes and performance indicators, you'd think s/he would have tried to provide different information in each column. In the best of all worlds, this chart would show measurable performance objectives (right hand column) which are designed to bring about specific changes in students (left hand column). The writer does start to achieve this in #8, although s/he fails to provide measurable outcomes (such as how many students will show what level of improvement) and quantifiable performance indicators (frequency, length and type of activity).
   
   
3. Combines performance indicator and outcome. The confusion continues. In #5, the writer combines both outcome and performance indicators in the right-hand column. "100% of mentors provide assistance with homework, research or other academic tasks for at least 30 minutes of each weekly meeting with their student" is a good performance indicator, while "50% of the students will show improvement in academic performance by the end of the school year (or within 6 months of being matched)," is a good outcome for that performance indicator. Trouble is, they are both included as performance indicators. In #6, the performance indicator, "Students are introduced to community resources for academic assistance," is listed as an outcome, while the outcome, "50% of students will have a library card or know how to access homework help resources within 3 months of project initiation," is listed as a performance indicator. The writer seems to have listed only long-term impacts as outcomes, while including short-term or intermediate outcomes as performance indicators. This is a mistake. Many youth-serving organizations can only measure participant progress through short-term or intermediate outcomes — and these are valid benchmarks that form the foundation for long-term success.
   
   * And for extra credit . . . I don't know about you, but when I read #9, where the outcome is described as "improved perceived connection of students to their school and community," I find myself questioning the whole thing. Is it enough to know that students "perceive" themselves to be more connected, or is it actually more important to know that they are more connected? The addition of that one word (perceived) throws up a red flag. The funder is apt to start asking questions about the validity of the whole proposal, and before you know it, the whole plan unravels like a ball of yarn in a roomful of kittens. Of course, that's not really a problem here, since it was all over after that first outcome measure.