A Guide to Implementing Evidence-Based Programs and Policies

Why Does This Matter?

Policymakers and funders are increasingly asking for research evidence about programs and services aimed at children and families. While it is important to ask about effectiveness, it is equally important to understand the limits of research evidence and the complexities of implementing programs in communities.

Policy Implications

Policymakers and practitioners should understand these requirements for replicating programs that research has demonstrated to be effective:

• Programs must exist, must have been evaluated, and must have been found to be effective. Many human service programs have yet to be demonstrated to be effective.
• Programs must be described in ways that allow close replication and many are not.
• Local service providers must be willing and able to closely replicate documented programs. However, many programs need to be tailored to the needs and circumstances of the participants and environment, so actual programs often differ substantially from community to community.
• Even when programs are carefully described and replicated, there’s no guarantee they will be as effective or produce benefits comparable to the original project. Therefore, the implementation and outcome need to be studied and evaluated.

Policymakers need to know how much and what kind of evidence is needed to declare a program worthy of replication. Elements to keep in mind when considering a body of research include:

• Is the research convincing on whether the program can produce the intended benefits?
• Is the program beneficial when implemented in the field under conditions similar to those when the program was studied?
• Does the program provide sufficient benefits to enough participants to justify its cost?
• What critical elements of the program are necessary for it to be effective, and what parts can be tailored to local circumstances?
• What services and procedures, personnel, and budgets are needed to implement the program?

What the Research Says

It’s not enough to have a well-documented model service program with demonstrated effectiveness. Also needed: a process for getting communities to want the program, opportunities for modifying the program to fit local circumstances, and a plan for implementing the program with reasonable enthusiasm and fidelity. Research has identified four broad factors that contribute to the gap between science and practice:

• Implementing high-quality service programs is complex, requiring much knowledge and many skills.
• Individuals must coordinate among different agencies, and communities must be ready to adopt and maintain new strategies.
• It’s often the case that financial, technical, and personnel resources are insufficient.
• Local clienteles and circumstances may pose unusual difficulties, and there may be little guidance from research.

Facts at a Glance

• Consensus groups gather academics to address issues, review research, and describe the best consensus. The National Institutes of Health and the National Academy of Sciences regularly convene such groups.
• Groups that include practitioners, policymakers, and consumers—in addition to researchers—help fill knowledge gaps and blend research with best practices and policy realities.
• Projects that build community capacity in a collaborative planning process include the Strengths, Weakness, Opportunities, Threats analysis, which provides a structure for strategic planning, and the Getting to Outcomes process, which guides organizations and communities. The Aspen Institute’s Roundtable on Community Change is developing an infrastructure for scholarly and community change. The Pathways Mapping Initiative of the Project on Effective Interventions at Harvard University integrates traditional research evaluations of service programs with professional experience and values.