|
|
Slide 1: Evaluation Plan Components Required of All Proposals
- The specification of an external evaluator or evaluation team along with their qualifications including:
- expertise in evaluation;
- expertise in the content areas covered by the project;
- experience working with the target populations; and
- the amount of time senior members of the team will spend on the evaluation.
- Evaluation questions and an evaluation design linked to the goals and outcomes of the project.
- Reliable procedures for collecting and analyzing data to determine:
- what works;
- for whom it works; and
- the context in which it works for different groups.1
Slide 2: Some Questions to Ask When Reviewing an Evaluation Plan
- Are the goals of the evaluation appropriate and clear?
- Will the proposed evaluation collect appropriate data to measure the degree to which the proposed project's objectives are achieved?
- Does the evaluation plan indicate by what demographic characteristics the data will be disaggregated and the findings reported? Will disability status be included?
- Do the evaluators have the skills and expertise to conduct the evaluation? Do at least some evaluation team members have:
- STEM backgrounds;
- experience doing STEM program evaluation; and
- experience working with the populations targeted in the proposal?
Tip: Have a section of the panel review summary, entitled “Evaluation Plan” where comments and questions on the overview of the evaluation plan are included.
Tip: Include at least one reviewer for each review panel with expertise in evaluation.
Rationale: Currently in the initial review process, evaluation plans get short shrift. They tend to be short (1-2 pages) and most reviewers have little expertise in evaluation. By having at least one reviewer with evaluation experience on a panel and providing reviewers with some basic questions to ask about an evaluation, more attention will be paid to the quality of the plan. Having a separate section of the panel review summary focus on evaluation will provide program officers suggestions for possible questions and/or clarifications to be asked for proposals that are competitive for funding.
While there are many possible questions to ask of an evaluation plan, it is recommended that reviewers focus on questions about the evaluation plan's potential to determine if the project's objectives are met and if the evaluation team is qualified to conduct the evaluation. Because there is so little data available on STEM workforce development and people with disabilities, it is suggested that attention be paid to this in the review of the evaluation plan.
While there are many possible questions to ask of an evaluation plan, it is recommended that reviewers focus on questions about the evaluation plan's potential to determine if the project's objectives are met and if the evaluation team is qualified to conduct the evaluation. Because there is so little data available on STEM workforce development and people with disabilities, it is suggested that attention be paid to this in the review of the evaluation plan.
1 Adapted from http://hub.mspnet.org/media/data/102312Webinar.pdf?media_000000007874.pdf