2016 Federal Index
Leadership
Did the agency have a senior staff member(s) with the authority, staff, and budget to evaluate its major programs and inform policy decisions affecting them in FY16?
Score
8
8
Administration for Children and Families (HHS)
- In late FY15, ACF established the new career position of Deputy Assistant Secretary for Planning, Research, and Evaluation to oversee its Office of Planning, Research, and Evaluation (OPRE) and support evaluation and other learning activities across the agency. ACF’s budget for research and evaluation in FY16 is approximately $142 million. ACF’s evaluation policy gives the OPRE Director (now Deputy Assistant Secretary) “authority to approve the design of evaluation projects and analysis plans; and…authority to approve, release and disseminate evaluation reports.” OPRE’s staff of 42 includes experts in research and evaluation methods as well as ACF programs and policies and the populations they serve. OPRE engages in on-going collaboration with program office staff and leadership to interpret research and evaluation findings and to identify their implications for programmatic and policy decisions. OPRE also provides written summaries of emerging findings and holds monthly meetings with agency and program leadership to discuss their implications.
- While OPRE oversees most of ACF’s evaluation activity and provides overall coordination, some ACF program offices also sponsor evaluations. ACF’s evaluation policy states, “In order to promote quality, coordination and usefulness in ACF’s evaluation activities, ACF program offices will consult with OPRE in developing evaluation activities. Program offices will discuss evaluation projects with OPRE in early stages to clarify evaluation questions and methodological options for addressing them, and as activities progress OPRE will review designs, plans, and reports. Program offices may also ask OPRE to design and oversee evaluation projects on their behalf or in collaboration with program office staff.”
Score
8
8
Corporation for National and Community Service
- CNCS’s Office of Research and Evaluation Director (R&E) oversees the development of social science research designed to measure the impact of CNCS programs and shape policy decisions; encourage a culture of performance and accountability in national and community service programs; provide information on volunteering, civic engagement, and volunteer management in nonprofit organizations; and assist in the development and assessment of new initiatives and demonstration projects. The R&E Director, who is overseeing R&E’s $4 million budget and a staff of 9 in FY16, is a member of CNCS’s Leadership Team and Policy Council. The R&E Director also meets regularly with CNCS Program Directors to identify areas where evidence can be generated and used for various decisions.
- The R&E Director meets annually with all CNCS program offices to identify priorities and negotiate which pools of funds are need to support the year’s priorities. The FY16 plan was developed through a series of formal and informal conversations.
Score
8
8
Millennium Challenge Corporation
- MCC’s Monitoring and Evaluation (M&E) Division, which falls within the Department of Policy and Evaluation (DPE), has a staff of 23 and an FY16 budget of $20.6 million in due diligence (DD) funds to be used directly for measuring high-level outcomes and impacts in order to assess the effects of its programs and activities. Departments throughout the agency have a total of $75 million in DD funds in FY16. The M&E Managing Director as well as the Departmental Vice President have the authority to execute M&E’s budget and inform policy decisions affecting independent evaluations. The M&E Managing Director participates in technical reviews of proposed investments as well as in regular monitoring meetings in order to inform policy and investment decisions. The Vice President sits on the Agency’s Investment Management Committee which examines the evidence base for each investment before it is approved by the MCC Board and conducts regular oversight over the compact (i.e., grant program) development process.
Score
8
8
U.S. Agency for International Development
- USAID has an agency-wide Evaluation Policy. The agency just released a report to mark the five-year anniversary of the policy.
- USAID field missions are required to have an evaluation plan, and all USAID missions and offices provide an internal report on an annual basis on completed, ongoing and planned evaluations, including evaluations planned to start anytime in the next three fiscal USAID provides a Performance Management Plan (PMP) Toolkit to assist missions worldwide.
- Given USAID’s decentralized structure, individual programs, offices, bureaus and missions may develop learning agendas, which several have done, including the USAID’s Bureau for Food Security for the US government’s Feed the Future initiative and USAID’s Democracy, Human Rights, and Governance (DRG) Center. All Washington Bureaus have annual evaluation action plans that look at quality and use and identify challenges and the priorities for the year ahead.
- All final USAID evaluation reports are available on the Development Experience Clearinghouse except for approximately five percent of evaluations completed each year that are not public due to principled exceptions to the presumption in favor of openness guided by OMB Bulletin 12-01 Guidance on Collection of U.S. Foreign Assistance Data.
- USAID is currently updating its operational policy for planning and implementing country programs. A key change in the policy is that missions will include a learning plan as part of their five-year strategic plan, also known as the CDCS. The plan will outline how missions will incorporate learning into their programming, including activities like regular portfolio reviews, evaluation tracking and dissemination plans, and other analytic processes to better understand the dynamics of their programs and their country contexts.
Score
8
8
U.S. Department of Education
- ED’s Institute of Education Sciences (IES), with a budget of $618 million in FY16, supports research and conducts evaluations of ED’s major programs, including impact The Director of IES and the Commissioner of the National Center for Education Evaluation and Regional Assistance (NCEE) are supported by 10 staff who oversee these evaluations. The Office of Planning, Evaluation, and Policy Development’s (OPEPD) Program and Policy Studies Services (PPSS) has a staff of 20 and serves as the Department’s internal analytics office. PPSS conducts short-term evaluations to support continuous improvement of program implementation and works closely with program offices and senior leadership to inform policy decisions with evidence. While some evaluation funding – such as that for Special Education Studies and Evaluations – is appropriated to IES ($10.8 million in FY16), most evaluations are supported by funds appropriated to ED programs. NCEE and PPSS staff work closely with program offices to design program evaluations that reflect program priorities and questions. Both IES and PPSS provide regular briefings on results to help ensure information can be used by program offices for program improvement.
- Both IES and PPSS sit on ED’s Evidence Planning Group (EPG) with other senior staff from the ED’s Office of Planning, Evaluation, and Policy Development (OPEPD) and the Office of Innovation and Improvement (OII). EPG reviews and advises programs and Department leadership on how evidence can be used to improve Department Senior officials from IES, OII, and PPSS are part of ED’s leadership structure and weigh in on major policy decisions. They play leading roles in the formation of the Department’s annual budget requests, recommendations around grant competition priorities, including evidence, and providing technical assistance to Congress to ensure that evidence informs policy design.
Score
8
8
U.S. Dept. of Housing & Urban Development
- HUD’s Office of Policy Development & Research (PD&R) informs HUD’s policy development and implementation by conducting, supporting, and sharing research, surveys, demonstrations, program evaluations, and best practices. PD&R achieves this mission through three interrelated core functions: (1) collecting and analyzing national housing market data (including with the Census Bureau); (2) conducting research, program evaluations, and demonstrations; and (3) providing policy advice and support to the HUD Secretary and program offices. PD&R is led by an Assistant Secretary who oversees six offices, about 149 staff including a team of field economists that work in HUD’s 10 regional offices across the country, and a budget of $108.1 million in FY16. The Assistant Secretary ensures that evidence informs policy development through frequent personal engagement with other principal staff, the Secretary, and external policy officials; HUDstat performance review meetings (see Question #4 below for a description); speeches to policy audiences, sponsorship of public research briefings, and policy implications memoranda. The Assistant Secretary also regularly engages with each HUD program office to ensure that metrics, evaluations, and evidence inform program design, budgeting, and implementation.
- Periodic PD&R meetings with program offices enable knowledge-sharing about evaluation progress and emerging needs for research, evaluation, and demonstrations. In recent years, Congress has authorized support for evaluations from program resources through set-asides, transfer authority, and supplemental appropriations to implement demonstrations.
Score
9
9
U.S. Department of Labor
- DOL’s Chief Evaluation Officer is a senior official with responsibility for all activities of the Chief Evaluation Office (CEO), and coordination of evaluations Department-wide. CEO includes 15 full-time staff and contractors plus 1-2 detailees at any given The CEO is responsible for the appropriated budget for the Departmental Program Evaluation ($10 million in FY16) and the Department’s evaluation set-aside funds ($30 million in FY16). In FY16, the CEO will directly oversee an estimated $40 million in evaluation funding and collaborate with DOL agencies on additional evaluations being carried out through an additional $15 million to evaluate Employment and Training Administration (ETA) pilots, demonstrations and research and evaluations of large grant programs including the Performance Partnership Pilots (P3), American Apprenticeship Initiative (AIA), the Trade Adjustment Assistance Community College and Career Training (TAACCCT) Grant Program, and Reentry Programs for Ex-Offenders. The CEO also participates actively in the performance review process during which each operating agency meets with the Deputy Secretary to review progress on performance goals established for the year required under Government Performance and Results Act (GPRA). The CEO’s role is to incorporate evidence and evaluation findings as appropriate and to identify knowledge gaps that might be filled by evaluations or convey evidence that can inform policy and program decisions or performance.