HSS 3101 Lecture Notes - Lecture 12: Needs Assessment, Indirect Costs, Logic Model

49 views6 pages
Class 12: Evaluation Research and Mixed Methods
What is evaluation research (aka program evaluation)?
- Research undertaken to see whether a program or activity is meeting or has met objectives set for
it
- Purpose depends on the objectives of the program (Process = how questions)
- Evaluates the impact of social interventions: new teaching methods, innovations in parole etc.
- Evaluation research is a form of applied research—intended to have real-world e&ect.
- Emphasis on the process rather than the product
- Re'ects social scientists’ increasing desire to make a di&erence in the world
oHowever: in'uence of governmental requirements that program evaluations must
accompany the implementation of new programs
oThe availability of research funds to ful+ll those requirements
Purpose of evaluation research:
Can be looked at from di&erent vantage points:
1. The program evaluator
2. The administrator
3. The consumer or public
4. The organization
Whatever the purpose, those involved in the process must be sure to accomplish the task by using
appropriate methodologies, processes, and analyses.
The research project must be able to be replicated by other evaluation groups. (With
same/similar goals)
-Logic model: so that other programs can copy your model
- Participants interested in outcomes, organizers interested in cost bene+ts
Reasons for Evaluation Research
- Management and administrative reasons
- Assessment of the appropriateness of program changes
- Identi+cation of ways to improve the delivery of interventions
- Accountability to funding agencies
- To ful+ll planning and policy purposes
- To test innovative ideas
- To decide if programs should be curtailed or expanded, or to support one program in lieu of another
What can be evaluated?
Programs
Drugs
Health
Databases
Lifestyles (single,
married, etc.)
Software programs
Teaching styles
Workshops
Sta& and personnel
Management systems
Organizations
Needs-assessment
strategies
Evaluation models and
schemes
Topics appropriate to evaluation research
- Evaluation research is appropriate whenever some social intervention occurs or is planned
oSocial intervention: action taken in social context to produce some intended result
- Process of determining whether a social intervention has produced the intended result
- The topics appropriate for evaluation research are limitless.
- The questions appropriate for evaluation research are of great practical signi+cance: jobs,
programs, and investments as well as values and beliefs.
Steps in conducting evaluation
- They’re interactive, can work at the same time:
oWho is the client?
oWhat are the purposes of the
evaluation?
oWhat is the methodology to be
utilized?
oWhat is included in the contract?
oHow is the evaluation to be
conducted?
oWho will use the results of the
evaluation?
Unlock document

This preview shows pages 1-2 of the document.
Unlock all 6 pages and 3 million more documents.

Already have an account? Log in
oMethodological approaches in evaluation research (types of research design)
- Experimental and Quasi-Experimental Designs
- Correlation
- Surveys
- Personnel Assessment
- Expert Judgment
- Case Study
oSeveral methods may be used in any one evaluation project, as they each contribute to the total
process
oHow to choose the method
- When determining the methodology to be used, the evaluator must be familiar
with the purposes of the evaluation.
- Characteristics that must be considered when determining the feasibility of the
method:
oAvailable funds
oTime schedule
oAvailability of data
oThe legal, political, ethical, and administrative constraints when conducting an
evaluation.
oFormulating the problem
- Problem: What is the purpose of the intervention to be
evaluated?
- This question often produces vague results.
- A common problem is measuring the “unmeasurable.”
- Evaluation research is a matter of +nding out whether something
is there or not there, whether something happened or did not happen.
- To conduct evaluation research, we must be able to operationalize, observe, and measure.
oOutcome or response variable
oIf a social program is intended to accomplish something, we must be able to measure that
something
- Achieve agreements on de+nitions in advance
- Achieve agreements on how the measurements will be made
- De+nitions of a problem and a suBcient solution may be de&ned by law or by agency
regulations;
oMust be aware of such speci+cations and accommodate them
- There may be several outcome measures, for instance surveys of attitudes and behaviors, existing
statistics, use of other resources.
oNew vs Existing measures
- If the study addresses something that’s never been measured before, the choice is easy—new
measures.
- If the study addresses something that others have tried to measure, the researcher will need to
evaluate the relative worth of various existing measurement devices in terms of her or his speci+c
research situations and purpose.
oAdvantage of
creating
measure
oAdvantages of using existing measures
- Greater relevance and
validity than using existing
measures.
o
- Less time and energy
- Greater scienti+c signi+cance, possible comparisons to
previous work
- Known degrees of validity and reliability
- New measures require pre-testing
oSuccess or Failure?
- Potentially one of the most taxing aspects of evaluation research is determining whether the
program under review succeeded or failed. De+nitions of “success” and “failure” can be rather
diBcult
oTypes of Evaluation Research
- Needs assessment
- Formative evaluation
Open ended boxes = people
don’t really write things, maybe
just write more speci+c questions
Money
Time
Passion
Can’t measure everything,
just have to zone in on
objectives
Secondary data analysis
-Review existing data
-Maximizing time on
site
Unlock document

This preview shows pages 1-2 of the document.
Unlock all 6 pages and 3 million more documents.

Already have an account? Log in

Get OneClass Notes+

Unlimited access to class notes and textbook notes.

YearlyBest Value
75% OFF
$8 USD/m
Monthly
$30 USD/m
You will be charged $96 USD upfront and auto renewed at the end of each cycle. You may cancel anytime under Payment Settings. For more information, see our Terms and Privacy.
Payments are encrypted using 256-bit SSL. Powered by Stripe.