Stufflebeam 2007

Document technical information

Format pdf
Size 73,7 kB
First found мая 10, 2016

Document content analysis

Language
English
Type
not defined
Concepts
no text concepts found

Transcript

CIPP EVALUATION MODEL CHECKLIST
[Second Edition]
A tool for applying the CIPP Model to assess long-term enterprises
Intended for use by evaluators and evaluation clients/stakeholders
Daniel L. Stufflebeam
March 17, 2007
The CIPP Evaluation Model is a comprehensive framework for guiding evaluations of programs, projects, personnel, products, institutions, and
systems. This checklist, patterned after the CIPP Model, is focused on program evaluations, particularly those aimed at effecting long-term,
sustainable improvements.
The checklist especially reflects the eight-year evaluation (1994-2002), conducted by the Western Michigan University Evaluation Center, of
Consuelo Foundation’s values-based, self-help housing and community development program—named Ke Aka Ho’ona—for low income families in
Hawaii (Stufflebeam, Gullickson, & Wingate, 2002). Also, it is generally consistent with a wide range of program evaluations conducted by The
Evaluation Center in such areas as science and mathematics education, rural education, educational research and development, achievement
testing, state systems of educational accountability, school improvement, professional development schools, transition to work, training and
personnel development, welfare reform, nonprofit organization services, community development, community-based youth programs, community
foundations, personnel evaluation systems, and technology.
Corresponding to the letters in the acronym CIPP, this model’s core parts are context, input, process, and product evaluation. In general, these
four parts of an evaluation respectively ask, What needs to be done? How should it be done? Is it being done? Did it succeed?
In this checklist, the “Did it succeed?” or product evaluation part is divided into impact, effectiveness, sustainability, and transportability evaluations.
Respectively, these four product evaluation subparts ask, Were the right beneficiaries reached? Were their needs met? Were the gains for the
beneficiaries sustained? Did the processes that produced the gains prove transportable and adaptable for effective use in other settings?
This checklist is designed to help evaluators evaluate programs with relatively long-term goals. The checklist’s first main function is to help
evaluators generate timely evaluation reports that assist groups to plan, carry out, institutionalize, and/or disseminate effective services to targeted
beneficiaries. The checklist’s other main function is to help evaluators review and assess a program’s history and issue a summative evaluation
report on its merit, worth, probity, and significance, and the lessons learned.
Evaluation Checklists Project
www.wmich.edu/evalctr/checklists
This checklist has 10 components. The first—contractual agreements to guide the evaluation—is followed by the context, input, process, impact,
effectiveness, sustainability, and transportability evaluation components. The last 2 are metaevaluation and the final synthesis report. Contracting for
the evaluation is done at the evaluation’s outset, then updated as needed. The 7 CIPP components may be employed selectively and in different
sequences and often simultaneously, depending on the needs of particular evaluations. Especially, evaluators should take into account any sound
evaluation information the clients/stakeholders already have or can get from other sources. CIPP evaluations should complement rather than
supplant other defensible evaluations of an entity. Metaevaluation (evaluation of an evaluation) is to be done throughout the evaluation process;
evaluators also should encourage and cooperate with independent assessments of their work. At the end of the evaluation, evaluators are advised to
give their attestation of the extent to which applicable professional standards were met. This checklist’s final component provides concrete advice for
compiling the final summative evaluation report, especially by drawing together the formative evaluation reports that were issued throughout the
evaluation.
The concept of evaluation underlying the CIPP Model and this checklist is that evaluations should assess and report an entity’s merit (i.e., its
quality), worth (in meeting needs of targeted beneficiaries), probity (its integrity, honesty, and freedom from graft, fraud, and abuse), and
significance (its importance beyond the entity’s setting or time frame), and should also present lessons learned. Moreover, CIPP evaluations and
applications of this checklist should meet the evaluation field’s standards, including especially the Joint Committee (1994) Program Evaluation
Standards of utility, feasibility, propriety, and accuracy; the Government Accountability Office (2007) Government Auditing Standards; and the
American Evaluation Association (2004) Guiding Principles for Evaluators. The model’s main theme is that evaluation’s most important purpose is
not to prove, but to improve.
Timely communication of relevant evaluation findings to the client and right-to-know audiences is another key theme of this checklist. As needed,
findings from the different evaluation components should be drawn together and reported periodically, typically once or twice a year. The general
process, for each reporting occasion, calls for draft reports to be sent to intended primary users about 10 days prior to a feedback workshop.1 At the
workshop the evaluators should use visual aids, e.g., a PowerPoint presentation, to brief the client, staff, and other members of the audience. (It is
often functional to provide the clients with a copy of the visual aids, so subsequently they can brief members of their boards or other stakeholder
groups on the most recent evaluation findings.) Those present at the feedback workshop should be invited to raise questions, discuss the findings,
and apply them as they choose. At the workshop’s end, the evaluators should summarize the evaluation’s planned next steps and future reports;
arrange for needed assistance from the client group, especially in data collection; and ask whether any changes in the data collection and reporting
plans and schedule would make future evaluation services more credible and useful. Following the feedback workshop, the evaluators should
finalize the evaluation reports, revise the evaluation plan and schedule as appropriate, and transmit to the client and other designated recipients the
finalized reports and any revised evaluation plans and schedule.
Beyond guiding the evaluator’s work, the checklist gives advice for evaluation clients. For each of the 10 evaluation components, the checklist
provides checkpoints on the left for evaluators and checkpoints on the right for evaluation clients.
The CIPP Model’s background is summarized in the appendix. For more information about the CIPP Model, please consult the references and
related checklists listed at the end of this checklist.
CIPP Evaluation Model Checklist
2
1. CONTRACTUAL AGREEMENTS
CIPP evaluations should be grounded in explicit advance agreements with the client, and these should be updated as needed throughout the
evaluation. (See Daniel Stufflebeam’s Evaluation Contracts Checklist at www.wmich.edu/evalctr/checklists)
Evaluator Activities
Client/Stakeholder Activities—Contracting
Develop a clear understanding of the evaluation job to be done.
Clarify with the evaluator what is to be evaluated, for what purpose,
according to what criteria, and for what audiences.
Secure agreements needed to assure that the right information can
Clarify with the evaluator what information is essential to the
be obtained.
evaluation and how the client group will facilitate its collection.
Clarify for the client, in general, what quantitative and qualitative
Reach agreements with the evaluator on what analyses will be most
analyses will be needed to make a full assessment of the program.
important in addressing the client group’s questions.
Clarify the nature, general contents, and approximate required timing
Assure that the planned final report will meet the needs of the
of the final summative evaluation report.
evaluation’s different audiences.
Clarify the nature, general contents, and timing of interim, formative
Assure that the evaluation’s reporting plan and schedule are
evaluation reports and reporting sessions.
functionally responsive to the needs of the program.
Reach agreements to protect the integrity of the reporting process.
Assure that the reporting process will be legally, politically, and
ethically viable.
Clarify the needed channels for communication and assistance from
Assure that the evaluation plan is consistent with the organization’s
the client and other stakeholders.
protocol.
Secure agreements on the evaluation’s time line and who will carry
Clarify for all concerned parties the evaluation roles and
out the evaluation responsibilities.
responsibilities of the client group.
Secure agreements on the evaluation budget and payment amounts
Assure that budgetary agreements are clear and functionally
and dates.
appropriate for the evaluation’s success.
Clearly define provisions for reviewing, controlling, amending, and/or
Assure that the evaluation will be reviewed periodically and, as
canceling the evaluation.
needed and appropriate, subject to modification and termination.
CIPP Evaluation Model Checklist
3
2. CONTEXT EVALUATION
Context evaluation assesses needs, assets, and problems within a defined environment.
Evaluator Activities
Client/Stakeholder Activities—Program Aims
Compile and assess background information on the intended
Use the context evaluation findings in selecting and/or clarifying the
beneficiaries’ needs and assets from such sources as health
intended beneficiaries.
records, school grades and test scores, funding proposals, and
newspaper archives.
Interview program leaders to review and discuss their perspectives
Use the context evaluation findings in reviewing and revising, as
on beneficiaries’ needs and to identify any problems (political or
appropriate, the program’s goals to assure they properly target
otherwise) the program will need to solve.
assessed needs.
Interview other stakeholders to gain further insight into the needs
Use the context evaluation findings in assuring that the program is
and assets of intended beneficiaries and potential problems for the
taking advantage of pertinent community and other assets.
program.
Assess program goals in light of beneficiaries’ assessed needs and
Use the context evaluation findings—throughout and at the
potentially useful assets.
program’s end—to help assess the program’s effectiveness and
significance in meeting beneficiaries’ assessed needs.
Engage a data collection specialist2 to monitor and record data on
the program’s environment, including related programs, area
resources, area needs and problems, and political dynamics.
Request that program staff regularly make available to the evaluation
team information they collect on the program’s beneficiaries and
environment.
Annually, or as appropriate, prepare and deliver to the client and
agreed-upon stakeholders a draft context evaluation report providing
an update on program-related needs, assets, and problems, along
with an assessment of the program’s goals and priorities.
Periodically, as appropriate, discuss context evaluation findings in
feedback sessions presented to the client and designated
audiences.
Finalize context evaluation reports and associated visual aids and
provide them to the client and agreed-upon stakeholders.3
CIPP Evaluation Model Checklist
4
3. INPUT EVALUATION
Input evaluation assesses competing strategies and the work plans and budgets of the selected approach.
Evaluator Activities
Client/Stakeholder Activities—Program Planning
Identify and investigate existing programs that could serve as a
Use the input evaluation findings to devise a program strategy that is
model for the contemplated program.
scientifically, economically, socially, politically, and technologically
defensible.
Assess the program’s proposed strategy for responsiveness to
Use the input evaluation findings to assure that the program’s
assessed needs and feasibility.
strategy is feasible for meeting the assessed needs of the targeted
beneficiaries.
Assess the program’s budget for its sufficiency to fund the needed
Use the input evaluation findings to support funding requests for the
work.
planned enterprise.
Assess the program’s strategy against pertinent research and
Use the input evaluation findings to acquaint staff with issues
development literature.
pertaining to the successful implementation of the program.
Assess the merit of the program’s strategy compared with alternative
Use the input evaluation findings for accountability purposes in
strategies found in similar programs.
reporting the rationale for the selected program strategy and the
defensibility of the operational plan.
Assess the program’s work plan and schedule for sufficiency,
feasibility, and political viability.
Compile a draft input evaluation report and send it to the client and
agreed-upon stakeholders.
Discuss input evaluation findings in a feedback workshop.
Finalize the input evaluation report and associated visual aids and
provide them to the client and agreed-upon stakeholders.
CIPP Evaluation Model Checklist
5
4. PROCESS EVALUATION
Process evaluations monitor, document, and assess program activities.
Evaluator Activities
Engage an evaluation team member to monitor, observe, maintain a
photographic record of, and provide periodic progress reports on
program implementation.
In collaboration with the program’s staff, maintain a record of
program events, problems, costs, and allocations.
Client/Stakeholder Activities—Managing and Documenting
Use the process evaluation findings to coordinate and strengthen
staff activities.
Use the process evaluation findings to strengthen the program
design.
Use the process evaluation findings to maintain a record of the
program’s progress.
Periodically interview beneficiaries, program leaders, and staff to
obtain their assessments of the program’s progress.
Use the process evaluation findings to help maintain a record of the
program’s costs.
Maintain an up-to-date profile of the program.
Use the process evaluation findings to report on the program’s
progress to the program’s financial sponsor, policy board,
community members, other developers, etc.
Periodically draft written reports on process evaluation findings and
provide the draft reports to the client and agreed-upon stakeholders.
Present and discuss process evaluation findings in feedback
workshops.
Finalize each process evaluation report (possibly incorporated into a
larger report) and associated visual aids and provide them to the
client and agreed-upon stakeholders.
CIPP Evaluation Model Checklist
6
5. IMPACT EVALUATION
Impact evaluation assesses a program’s reach to the target audience.
Evaluator Activities
Engage the program’s staff and consultants and/or an evaluation
team member to maintain a directory of persons and groups served;
make notations on their needs and record program services they
received.
Assess and make a judgment of the extent to which the served
individuals and groups are consistent with the program’s intended
beneficiaries.
Periodically interview area stakeholders, such as community leaders,
employers, school and social programs personnel, clergy, police,
judges, and homeowners, to learn their perspectives on how the
program is influencing the community.
Client/Stakeholder Activities—Controlling Who Gets Served
Use the impact evaluation findings to assure that the program is
reaching intended beneficiaries.
Use the impact evaluation findings to assess whether the program is
reaching or did reach inappropriate beneficiaries.
Use the impact evaluation findings to judge the extent to which the
program is serving or did serve the right beneficiaries.
Use the impact evaluation findings to judge the extent to which the
program addressed or is addressing important community needs.
Use the impact evaluation findings for accountability purposes
regarding the program’s success in reaching the intended
beneficiaries.
Include the obtained information and the evaluator’s judgments in a
periodically updated program profile.
Determine the extent to which the program reached an appropriate
group of beneficiaries.
Assess the extent to which the program inappropriately provided
services to a nontargeted group.
Draft an impact evaluation report (possibly incorporated into a larger
report) and provide it to the client and agreed-upon stakeholders.
As appropriate, discuss impact evaluation findings in feedback
sessions.
Report the impact evaluation findings to the client and agreed-upon
stakeholders.
CIPP Evaluation Model Checklist
7
6. EFFECTIVENESS EVALUATION
Effectiveness evaluation documents and assesses the quality and significance of outcomes.
Evaluator Activities
Client/Stakeholder Activities—Assessing/Reporting Outcomes
Interview key stakeholders, such as community leaders,
Use effectiveness evaluation findings to gauge the program’s
beneficiaries, program leaders and staff, and other interested
positive and negative effects on beneficiaries.
parties, to determine their assessments of the program’s positive
As relevant, use the effectiveness evaluation findings to gauge the
and negative outcomes.
program’s positive and negative effects on the
community/pertinent environment.
As feasible and appropriate, conduct in-depth case studies of
Use the effectiveness evaluation findings to sort out and judge
selected beneficiaries.
important side effects.
Engage an evaluation team member and program staff to supply
Use the effectiveness evaluation findings to examine whether
documentation needed to identify and confirm the range, depth,
program plans and activities need to be changed.
quality, and significance of the program’s effects on beneficiaries.
As appropriate, engage an evaluation team member to compile
Use the effectiveness evaluation findings to prepare and issue
and assess information on the program’s effects on the
program accountability reports.
community.
Engage a goal-free evaluator4 to ascertain what the program
Use the effectiveness evaluation findings to make a bottom-line
actually did and to identify its full range of effects—positive and
assessment of the program’s success.
negative, intended and unintended.
Obtain information on the nature, cost, and success of similar
Use needs assessment data (from the context evaluation findings),
programs conducted elsewhere and judge the subject program’s
effectiveness evaluation findings, and contrasts with similar
effectiveness in contrast to the identified “critical competitors.”
programs elsewhere to make a bottom-line assessment of the
program’s significance.
Compile effectiveness evaluation findings in a draft report (that
may be incorporated in a larger report) and present it to the client
and agreed-upon stakeholders.
Discuss effectiveness evaluation findings in a feedback session.
Finalize the effectiveness evaluation report and present it to the
client and agreed-upon stakeholders.
Incorporate the effectiveness evaluation findings in an updated
program profile and ultimately in the final evaluation report.
CIPP Evaluation Model Checklist
8
7. SUSTAINABILITY EVALUATION
Sustainability evaluation assesses the extent to which a program’s contributions are institutionalized successfully and continued over time.
Evaluator Activities
Interview program leaders and staff to identify their judgments
about what program successes should be sustained.
Interview program beneficiaries to identify their judgments
about what program successes should and could be sustained.
Review the evaluation’s data on program effectiveness, program
costs, and beneficiary needs to judge what program activities
should and can be sustained.
Interview beneficiaries to identify their understanding and
assessment of the program’s provisions for continuation.
Client/Stakeholder Activities: Continuing Successful Practices
Use the sustainability evaluation findings to determine whether
staff and beneficiaries favor program continuation.
Use the sustainability findings to assess whether there is a
continuing need/demand and compelling case for sustaining the
program’s services.
Use the sustainability findings as warranted to set goals and plan
for continuation activities.
Use the sustainability findings as warranted to help determine how
best to assign authority and responsibility for program
continuation.
As appropriate, use the sustainability findings (along with other
relevant information on the program) to help plan and budget
continuation activities.
Obtain and examine plans, budgets, staff assignments, and other
relevant information to gauge the likelihood that the program will be
sustained.
Periodically revisit the program to assess the extent to which its
successes are being sustained.
Compile and report sustainability findings in the evaluation’s
progress and final reports.
In a feedback session, discuss sustainability findings plus the
possible need for a follow-up study to assess long-term
implementation and results.
Finalize the sustainability evaluation report and present it to the
client and agreed-upon stakeholders.
CIPP Evaluation Model Checklist
9
8. TRANSPORTABILITY EVALUATION
Transportability evaluation assesses the extent to which a program has (or could be) successfully adapted and applied elsewhere. (This is an
optional component of a CIPP evaluation. It should be applied when the client or some other authorized party desires and arranges for such a
study. Sometimes such a transportability evaluation is an apt subject for a doctoral dissertation.)
Evaluator Activities
Client/Stakeholder Activities—Dissemination
Engage the program staff in identifying actual or potential adopters
Use the transportability evaluation findings to assess the need for
of the program by keeping a log of inquiries, visitors, and
disseminating information on the program.
adaptations of the program.
If relevant, survey a representative sample of potential adopters.
Use the transportability evaluation findings to help determine
Ask them to (1) review a description of the program and a
audiences for information on the program.
summary of evaluation findings; (2) judge the program’s relevance
Use the transportability evaluation findings to help determine what
to their situation; (3) judge the program’s quality, significance, and
information about the program should be disseminated.
replicability; and (4) report whether they are using or plan to adopt
Use the transportability evaluation findings to gauge how well the
all or parts of the program.
program worked elsewhere.
Visit and assess adaptations of the program.
Compile and report transportability evaluation findings in draft
reports.
Discuss transportability evaluation findings in a feedback session.
Finalize the transportability evaluation report and associated visual
aids and present them to the client and agreed-upon stakeholders.
CIPP Evaluation Model Checklist
10
9. METAEVALUATION5
Metaevaluation is an assessment of an evaluation, especially its adherence to pertinent standards of sound evaluation (See Stufflebeam,
Daniel. Program Evaluations Metaevaluation Checklist. www.wmich.edu/evalctr/checklists)
Evaluator Activities
Client/Stakeholder Activities–Judgment of the Evaluation
Reach agreement with the client that the evaluation will be guided
Review the Joint Committee Program Evaluation Standards and
and assessed against the Joint Committee Program Evaluation
reach an agreement with the evaluators that these standards
Standards of utility, feasibility, propriety, and accuracy and/or
and/or other standards and/or guiding principles will be used to
some other mutually agreeable set of evaluation standards or
guide and judge the evaluation work.
guiding principles.
Consider contracting for an independent assessment of the
evaluation.
Encourage and support the client to obtain an independent
assessment of the evaluation plan, process, and/or reports.
Document the evaluation process and findings, so that the
evaluation can be rigorously studied and evaluated.
Steadfastly apply the Joint Committee Standards and/or other set
of agreed-upon standards or guiding principles to help assure that
the evaluation will be sound and fully accountable.
Periodically use the metaevaluation findings to strengthen the
evaluation as appropriate.
Assess and provide written commentary on the extent to which the
evaluation ultimately met each agreed-upon standard and/or
guiding principle, and include the results in the final evaluation
report’s technical appendix.
Keep a file of information pertinent to judging the evaluation
against the agreed-upon evaluation standards and/or guiding
principles.
Supply information and otherwise assist all legitimate efforts to
evaluate the evaluation as appropriate.
Raise questions about and take appropriate steps to assure that
the evaluation adheres to the agreed-upon standards and/or other
standards/guiding principles.
Take into account metaevaluation results in deciding how best to
apply the evaluation findings.
Consider appending a statement to the final evaluation report
reacting to the evaluation, to the evaluators’ attestation of the
extent to which standards and/or guiding principles were met, to
the results of any independent metaevaluation, and also
documenting significant uses of the evaluation findings.
CIPP Evaluation Model Checklist
11
10.
THE FINAL SYNTHESIS REPORT
Final synthesis reports pull together evaluation findings to inform the full range of audiences about what was attempted, done, and
accomplished; what lessons were learned; and the bottom-line assessment of the program.
Evaluator Activities
Client/Stakeholder Activities: Summing Up
Organize the report to meet the differential needs of different audiences, e.g.,
Help assure that the planned report contents will
provide three reports in one, including program antecedents, program
appeal to and be usable by the full range of audiences.
implementation, and program results.6
Continuing the example, in the program antecedents report include discrete
Help assure that the historical account presented in the
sections on the organization that sponsored the program, the origin of the
program antecedents report is accurate, sufficiently
program being evaluated, and the program’s environment.
brief, and of interest and use to at least some of the
audiences for the overall report.
In the program implementation report include sections that give detailed,
Help assure that the account of program
factual accounts of how the main program components were planned, funded,
implementation is accurate and sufficiently detailed to
staffed, and carried out such that groups interested in replicating the program
help others understand and possibly apply the
could see how they might conduct the various program activities. These
program’s procedures (taking into account pertinent
sections should be mainly descriptive and evaluative only to the extent of
cautions).
presenting pertinent cautions.
In the program results report include sections on the evaluation design, the
Use the program results report to take stock of what
evaluation findings (divided into context, input, process, impact, effectiveness,
was accomplished; what failures and shortfalls
sustainability, and transportability), and the evaluation conclusions (divided
occurred; the extent to which the program was fully
into strengths, weaknesses, lessons learned, and bottom-line assessment of
ethical; how the effort compares with similar programs
the program’s merit, worth, probity, and significance). Contrast the program’s
elsewhere; and what lessons should be heeded in
contributions with what was intended, what the beneficiaries needed, what the
future programs.
program cost, and how it compares with similar programs elsewhere.
Use the full report as a means of preserving
institutional memory of the program and informing
At the end of each of the three reports, consider including photographs and
interested parties about the enterprise.
graphic representations that help retell the report’s particular accounts.
Supplement the main report contents, throughout, with pertinent quotations; a
prologue recounting how the evaluation was initiated; an epilogue identifying
needed further program and evaluation efforts; an executive summary;
acknowledgements; information about the evaluators; and technical
appendices containing such items as interview protocols, questionnaires,
feedback workshop agendas, data tables, and on-site evaluator’s handbook
of procedures.
CIPP Evaluation Model Checklist
12
BIBLIOGRAPHY
American Evaluation Association 2003 Ethics Committee. (2004). Guiding principles for evaluators. Available from
http://www.eval.org/Publications/GuidingPrinciples.asp
H
H
Controller General of the United States. (2002, January). Government auditing standards (2007). Washington, DC: U.S. General Accounting
Office: Author. Available from http://gao.gov/govaud/ybk01.htm
Guba, E. G., & Stufflebeam, D. L. (1968). Evaluation: The process of stimulating, aiding, and abetting insightful action. In R. Ingle & W. Gephart
(Eds.), Problems in the training of educational researchers. Bloomington, IN: Phi Delta Kappa.
Joint Committee on Standards for Educational Evaluation. (1988). The personnel evaluation standards. Newbury Park, CA: Sage.
Joint Committee on Standards for Educational Evaluation. (1994). The program evaluation standards. Thousand Oaks, CA: Sage.
Shadish, W. R., Newman, D. L., Scheirer, M. A., & Wye, C. (1995). Guiding principles for evaluators. New Directions for Program Evaluation, 66.
Stufflebeam, D. L. (1966). A depth study of the evaluation requirement. Theory Into Practice, 5(3), 121-133.
Stufflebeam, D. L. (1967, June). The use and abuse of evaluation in Title III. Theory Into Practice 6, 126-133.
Stufflebeam, D. L. (1969). Evaluation as enlightenment for decision-making. In H. B. Walcott (Ed.), Improving educational assessment and an
inventory of measures of affective behavior (pp. 41-73). Washington, DC: Association for Supervision and Curriculum Development and
National Education Association.
Stufflebeam, D. L. (1972). The relevance of the CIPP evaluation model for educational accountability. SRIS Quarterly, 5(1).
Stufflebeam, D. L. (1973). Evaluation as enlightenment for decision-making. In B. R. Worthen & J. R. Sanders (Eds.), Educational evaluation:
Theory and practice. Worthington, OH: Charles A. Jones Publishing Company.
Stufflebeam, D. L. (1983). The CIPP model for program evaluation. In G. F. Madaus, M. Scriven, & D. L. Stufflebeam (Eds.), Evaluation models
(Chapter 7, pp. 117-141). Boston: Kluwer-Nijhoff.
Stufflebeam, D. L. (1985). Stufflebeam's improvement-oriented evaluation. In D. L. Stufflebeam & A. J. Shinkfield (Eds.), Systematic evaluation
(Chapter 6, pp. 151-207). Boston: Kluwer-Nijhoff.
Stufflebeam, D. L. (1997). Strategies for institutionalizing evaluation: revisited. Occasional Paper Series #18. Kalamazoo: Western Michigan
University Evaluation Center. Available from http://www.wmich.edu/evalctr/pubs/ops/
H
H
nd
Stufflebeam, D.L. (2000). The CIPP model for evaluation. In D.L. Stufflebeam, G. F. Madaus, & T. Kellaghan, (Eds.), Evaluation models (2 ed.).
(Chapter 16). Boston: Kluwer Academic Publishers.
Stufflebeam, D. L. (2001). The metaevaluation imperative. American Journal of Evaluation, 22(2), 183-209.
Stufflebeam, D. L. (2003). The CIPP model for evaluation. In D. L. Stufflebeam, & T. Kellaghan, (Eds.), The international handbook of
educational evaluation (Chapter 2). Boston: Kluwer Academic Publishers.
CIPP Evaluation Model Checklist
13
Stufflebeam, D. L. (2003). Institutionalizing evaluation in schools. In D. L. Stufflebeam, & T. Kellaghan, (Eds.), The international handbook of
educational evaluation (Chapter 34). Boston: Kluwer Academic Publishers.
Stufflebeam, D. L., Foley, W. J., Gephart, W. J., Guba, E. G., Hammond, R. L., Merriman, H. O., & Provus, M. (1971). Educational evaluation and
decision making (Chapters 3, 7, & 8). Itasca, IL: F. E. Peacock.
Stufflebeam, D. L., Gullickson, A., & Wingate, L. (2002). The spirit of Consuelo: An evaluation of Ke Aka Ho’ona. Kalamazoo: Western Michigan
University Evaluation Center. Available from http://www.wmich.edu/evalctr/pubs/ecpub.htm#Evaluation%20Reports
H
H
Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation theory, models, and applications. (Chapter 19). San Francisco: Jossey-Bass.
Stufflebeam, D. L., & Webster, W. J. (1988). Evaluation as an administrative function. In N. Boyan (Ed.), Handbook of research on educational
administration (pp. 569-601). White Plains, NY: Longman.
U.S. Government Accountability Office (GAO). (2007). Government auditing standards. Washington, DC: Author. Available from
http://gao.gov/govaud/ybk01.htm
H
H
RELATED CHECKLISTS
(available at www.wmich.edu/evalctr/checklists)
Checklist for Negotiating an Agreement to Evaluate an Educational Program by Robert Stake
Checklist for Developing and Evaluating Evaluation Budgets by Jerry Horn
Evaluation Contracts Checklist by Daniel Stufflebeam
Evaluation Plans and Operations Checklist by Daniel Stufflebeam
Evaluation Values and Criteria Checklist by Daniel Stufflebeam
Feedback Workshop Checklist by Arlen Gullickson & Daniel Stufflebeam
Program Evaluations Metaevaluation Checklist (Based on The Program Evaluation Standards) by Daniel Stufflebeam
CIPP Evaluation Model Checklist
14
APPENDIX: BACKGROUND OF THE CIPP MODEL
This checklist represents a fifth installment of the CIPP Model. The model’s first installment—actually before all 4 CIPP parts were introduced—was
published more than 35 years ago (Stufflebeam, 1966) and stressed the need for process as well as product evaluations. The second installment—
published a year later (Stufflebeam, 1967)—included context, input, process, and product evaluations and emphasized that goal-setting should be
guided by context evaluation, including a needs assessment, and that program planning should be guided by input evaluation, including assessments
of alternative program strategies. The third installment (Stufflebeam et al., 1971) set the 4 types of evaluation within a systems/improvement-oriented
framework. The model’s fourth installment (Stufflebeam, 1972) showed how the model could and should be used for summative as well as formative
evaluation. The model’s fifth installment—illustrated by this checklist—breaks out product evaluation into the above-noted four subparts in order to
help assure and assess a program’s long-term viability. See Stufflebeam (2003-a, -b); Stufflebeam, Gullickson, and Wingate (2002); and Stufflebeam
and Shinkfield (2007).
Notes
1
The feedback workshops referenced throughout the checklist are a systematic approach by which evaluators present, discuss, and examine
findings with client groups. A checklist for planning feedback workshops can be found at www.wmich.edu/evalctr/checklists/.
2
Applications of the CIPP Model typically have included relatively low-cost evaluation team members who spend much time at the program site
systematically observing and recording pertinent information. (Their costs are relatively low because they reside in the program’s geographic area
and/or are relatively junior members of the evaluation field, such as graduate research assistants.) Called Traveling Observers when program sites
are dispersed or Resident Observers when program activities are all at one location, these evaluators help design and subsequently work from a
specially constructed Traveling Observer’s Handbook containing prescribed evaluation questions, procedures, forms, and reporting formats. Such
handbooks are tailored to the needs of the particular evaluation. While the observers focus heavily on context and process evaluations, they may
also collect and report information on program plans, costs, impacts, effectiveness, sustainability, and transportability. The use of such specialists
enhances the feasibility of regularly and closely studying a program when it would be too costly for the lead evaluators or high-cost experts to be on
site for extensive periods of time.
3
Whereas each of the seven evaluation components includes a reporting function, findings from the different components are not necessarily
presented in separate reports. Depending on the circumstances of a particular reporting occasion, availability of information from different evaluation
components, and the needs and preferences of the audience, information across evaluation components may be combined in one or more composite
reports. Especially, process, impact, and effectiveness information are often combined in a single report. The main point is to design and deliver
evaluation findings so that the audience’s needs are served effectively and efficiently.
CIPP Evaluation Model Checklist
15
4
A goal-free evaluator is a contracted evaluator who, by agreement, is prevented from learning a program’s goals and is charged to assess what the
program is actually doing and achieving, irrespective of its aims. This technique is powerful for identifying side effects or unintended outcomes both
positive and negative, also for describing what the program is actually doing, irrespective of its stated procedures.
5
See the RELATED CHECKLISTS section on page 14 to identify a number of checklists designed to guide metaevaluations.
6
Clearly, different audiences have different needs and interests regarding the range of information available from an evaluation. A report on a
program’s background, organizational setting, and geographic environment would be of considerable interest to an audience that had no previous
contact with the program; this same information would be of much less interest to an audience that possesses detailed familiarity with such matters.
Potential adopters of a program often would want detailed documentation on how the program was organized, designed, staffed, funded, and
operated, but many other persons, with interest in the program, would not require such detailed information. Likely, all audiences for the program
evaluation would want information on its outcomes and judgments of its value. In general, evaluators are advised to identify the different audiences
for an evaluation; analyze their differential needs; and—accordingly—design, prepare, and deliver modularized reports. When presented with such a
modularized report, the different audiences can turn directly to the module(s) that most interests them. Providing such easy access to the desired
information is in the interest of increasing the evaluation’s impacts.
This checklist is being provided as a free service to the user. The provider of the checklist has not modified or adapted the checklist to fit the specific
needs of the user, and the user is executing his or her own discretion and judgment in using the checklist. The provider of the checklist makes no
representations or warranties that this checklist is fit for the particular purpose contemplated by users and specifically disclaims any such warranties
or representations.
CIPP Evaluation Model Checklist
16

Similar documents

×

Report this document