Did the agency have a senior staff member(s) with the authority, staff, and budget to evaluate its major programs and inform policy decisions affecting them in FY16?

• HUD’s Office of Policy Development & Research (PD&R) informs HUD’s policy development and implementation by conducting, supporting, and sharing research, surveys, demonstrations, program evaluations, and best practices. PD&R achieves this mission through three interrelated core functions: (1) collecting and analyzing national housing market data (including with the Census Bureau); (2) conducting research, program evaluations, and demonstrations; and (3) providing policy advice and support to the HUD Secretary and program offices. PD&R is led by an Assistant Secretary who oversees six offices, about 149 staff including a team of field economists that work in HUD’s 10 regional offices across the country, and a budget of $108.1 million in FY16. The Assistant Secretary ensures that evidence informs policy development through frequent personal engagement with other principal staff, the Secretary, and external policy officials; HUDstat performance review meetings (see Question #4 below for a description); speeches to policy audiences, sponsorship of public research briefings, and policy implications memoranda. The Assistant Secretary also regularly engages with each HUD program office to ensure that metrics, evaluations, and evidence inform program design, budgeting, and implementation.
• Periodic PD&R meetings with program offices enable knowledge-sharing about evaluation progress and emerging needs for research, evaluation, and demonstrations. In recent years, Congress has authorized support for evaluations from program resources through set-asides, transfer authority, and supplemental appropriations to implement demonstrations.

Did the agency have an evaluation policy, evaluation plan, and research/ learning agenda(s) and did it publicly release the findings of all completed evaluations in FY16?

• HUD’s evaluation policy (see 1–6, 21, 23), which guides HUD’s Research Roadmap described below, includes reaching out to internal and external stakeholders through a participatory approach; making research planning systematic, iterative, and transparent; focusing on research questions that are timely, forward-looking, policy-relevant, and leverage HUD’s comparative advantages and partnership opportunities; aligning research with HUD’s strategic goals; and using rigorous research methods including program demonstrations with randomized controlled trials as appropriate.
• HUD’s Office of Policy Development and Research (PD&R) has developed the Research Roadmap FY14-FY18, (see pp. 6-20) a strategic, five-year plan for priority program evaluations and research to be pursued given a sufficiently robust level of PD&R also integrated its evaluation plan into HUD’s FY14-FY18 Strategic Plan (see pp. 57-63) to strengthen the alignment between evaluation and performance management. During FY16, PD&R is using similar principles and methods to refresh the Roadmap to address emerging research topics.
• HUD also employs its role as convener to help establish frameworks for evidence, metrics, and future research.
• According to the Research Roadmap FY14-FY18, (see 28), as part of HUD’s annual performance report required by GPRA, “agencies should describe findings from agency-funded evaluations or other research completed during the prior fiscal year.” Further, “Agencies are expected to have a web page on the agency’s evaluations or links to other evaluations relevant to the agency’s work with summaries of the findings and specific citations.” PD&R publishes and disseminates evaluations in a timely fashion through these and other means, and also follows a policy of including language in research and evaluation contracts that allows researchers to independently publish results, even without HUD approval, after not more than 6 months.

Did the agency invest at least 1% of program funds in evaluations in FY16? (Note: Meeting this criteria requires both Agency and Congressional action.)

• For FY16, Congress appropriated $50 million for core research activities; $10 million for research, evaluations, and demonstrations; and $25 million for technical assistance in the Research & Technology This $85 million total, half of the requested amount, equals 0.19 percent of HUD’s $45.5 billion of FY16 program budget authority, net of Salaries and Expenses. The $10 million devoted to research, evaluations, and demonstrations is about 12 percent of the $85 million total. Additionally, much of the $50 million is used for surveys (especially for the American Housing Survey) and other data acquisition that indirectly support evaluation of HUD’s mission activities in domains such as affordable housing and housing finance.
• In FY10, Congress authorized the transfer of up to 1% of funds from individual HUD program funds to the Transformation Initiative (TI) Fund for: (1) research, evaluation, and program metrics; (2) program demonstrations; (3) technical assistance; and (4) information technology. After FY11, HUD no longer sought to fund information technology with the TI Fund, and Congress has not provided requested levels of evaluation funding or, since FY14, supported transfers to TI.

Did the agency implement a performance management system with clear and prioritized outcome-focused goals and aligned program objectives and measures, and did it frequently collect, analyze, and use data and evidence to improve outcomes, return on investment, and other dimensions of performance in FY16?

• HUD conducts regular data-driven performance reviews—“HUDStat” meetings—that focus on quarterly progress toward achieving each of HUD’s priority goals. The HUD Secretary and senior leadership from throughout the agency, and sometimes from partner agencies, attend these meetings to address challenges, review metrics, improve internal and external collaboration, and increase performance. Strategic goals and two-year priority goals are publicly posted. HUD documents alignment between strategic goals and supporting objectives and metrics in the consolidated Annual Performance Plan-Annual Performance Report, and identifies the staff assigned lead responsibility for each objective.

Did the agency collect, analyze, share, and use high-quality administrative and survey data – consistent with strong privacy protections – to improve (or help other entities improve) federal, state, and local programs in FY16?

• The HUD USER web portal continues to provide researchers, practitioners, and the public with PD&R datasets including the American Housing Survey, HUD median family income limits, and Picture of Subsidized Households tabulations at multiple geographic levels, as well as microdata from research initiatives on topics such as housing discrimination, the HUD-insured multifamily housing stock, and the public housing population. To help users identify which data are useful to them, reference guides identify datasets and characterize their relevance and usefulness for research in designated categories.
• HUD’s Office of Policy Development and Research (PD&R) has authority to enter into cooperative agreements with research organizations, including both funded Research Partnerships and unfunded Data License Agreements, to support innovative research projects that leverage HUD’s data assets and inform HUD’s policies and programs. A dedicated subject-matter expert is available to answer questions for those seeking a data license.
• HUD’s PD&R and the National Center for Health Statistics at the Centers for Disease Control have successfully linked HUD administrative data for assisted renters with respondents to two national health surveys and made the linked data available to researchers to begin building a picture of tenant health issues.
• HUD is involved in a wide array of data-sharing agreements, including geocoding services provided by HUD’s Geocoding Service Center; a recent agreement with the U.S. Department of Health and Human Services to pilot an administrative data linkage with Medicare and Medicaid utilization records; national compilation of local point-in-time counts of homeless individuals and administrative data from homeless service providers using Homeless Management Information Systems; collection of tenant data for Low-Income Housing Tax Credit properties from state housing finance agencies; an ongoing agreement with Actionable Intelligence for Social Policy to develop integrated data systems for policy analysis and program reform, including local education data; and a multiagency federal agreement under development about protocols for information security in data- sharing.

Did the agency use a common evidence framework, guidelines, or standards to inform its research and funding decisions and did it disseminate and promote the use of evidence-based interventions through a user-friendly tool in FY16?

• HUD’s Policy Development and Research (PD&R) office provides evidence of “what works” primarily through HUD USER, a portal and web store for program evaluations, case studies, and policy analysis and research; the Regulatory Barriers Clearinghouse; and through initiatives such as Innovation of the Day, Sustainable Construction Methods in Indian Country, and the Consumer’s Guide to Energy-Efficient and Healthy Homes. This content is designed to provide current policy information, elevate effective practices, and synthesize data and other evidence in accessible formats. Through these resources, researchers and practitioners can see the full breadth of work on a given topic (e.g., rigorous established evidence, case studies of what’s worked in the field, and new innovations currently being explored) to inform their work.

Did the agency have staff, policies, and processes in place that encouraged innovation to improve the impact of its programs in FY16?

• HUD’s Policy Development and Research (PD&R) office is conducting a number of evaluated, random-assignment program demonstrations to test new program models, which can be found in PD&R’s biennial report: the Family Options study of homelessness interventions, Family Self-Sufficiency Demonstration, Pre-Purchase Homeownership Counseling Demonstration, Support and Services at Home (SASH) Demonstration for elderly households, Supportive Services Demonstration for health services in elderly housing, Rent Reform Demonstration, and the Small Area Fair Market Rent Demonstration. The latter demonstrations are in early or middle stages; interim results and long-term follow-up results generally will be reported through HUD USER during the next 2-4 years.
• PD&R also is collaborating with the White House Social and Behavioral Sciences Team and S. Department of Education to link tenant data with records of students and individuals submitting Free Applications for Federal Student Aid, helping increase access of HUD tenants to higher education through low-cost, behaviorally informed experiments about effective outreach methods. While detailed information about these experiments is not available at present, some can be found in HUD’s 2015 Annual Report (see p. 62) and will be included in SBST’s annual report in July 2016.
• PD&R houses the Office of International and Philanthropic Innovation, and administers five types of Secretary’s Awards to encourage excellence: Public-Philanthropic Partnerships, Opportunity and Empowerment, Healthy Homes, Historic Preservation, and Housing and Community Design. The competitions are judged by juries of professionals, and bring visibility to the nation’s most compelling solutions for addressing housing and community development challenges.
• PD&R sponsors an Innovation in Affordable Housing Competition to engage multidisciplinary teams of graduate students in addressing a specific housing problem developed by an actual public housing agency. The competition increases the nation’s future human capacity to address the affordable housing crisis by exposing future designers, administrators, and policymakers to real-world challenges of a specific legal and community context, with their proposals to be evaluated by an expert jury.
• In FY16, HUD’s National Disaster Resilience Competition is providing funding for resilient housing and infrastructure projects to states and communities that suffered major natural disasters. Collaborative teams were assisted in extensively researching and developing their proposals by nine Resilience Academies developed by the Rockefeller Foundation in partnership with HUD. The in-depth, juried process is ensuring that the $1 billion of resources available for these communities in FY16 will result in more resilient housing and infrastructure and bridge the gap between social and physical vulnerabilities.

Did the agency use evidence of effectiveness when allocating funds from its 5 largest competitive grant programs in FY16?

• In FY16 HUD’s major competitive grant programs are: 1) Homeless Assistance ($1.9 billion); 2) Disaster Assistance/National Disaster Resilience Competition ($300 million); 3) Choice Neighborhoods Grants program ($125 million); 4) Service Coordinators program ($77 million); and 5) Family  Self-Sufficiency Program Coordinators ($73 million).
• The National Disaster Resilience Competition used evidence about disaster resilience, including benefit/cost analysis, to ensure that disaster funding improves communities’ ability to withstand and recover more quickly from future disasters, hazards, and shocks rather than simply recreating the same vulnerabilities.
• Decisions regarding the design, funding, and implementation of all HUD competitive grant programs are evidence-based, as specified in funding criteria in HUD’s FY16 Notice of Funding Availability (NOFA). The “Achieving Results and Program Evaluation” factor (see 13), provides funding priority for applicants that demonstrate effective use of evidence in identifying or selecting the proposed practices, strategies, or programs proposed in the application, and requires all grantees to cooperate in HUD-funded research and evaluation studies (see p. 14). Another factor, “Past Performance,” provides: “In evaluating applications for funding HUD will take into account an applicant’s past performance in managing funds, including, but not limited to…. meeting performance targets as established in Logic Models or other performance evaluation tools approved as part of the grant agreement….” (see p. 14). The “Achieving Results and Program Evaluation” factor and “Past Performance” factor are two of five factors considered that total 100 points. The maximum achievable score, with priority points and bonus points, is 106.

Did the agency use evidence of effectiveness when allocating funds from its 5 largest non-competitive grant programs in FY16? (Note: Meeting this criteria requires both Agency and Congressional action.)

• HUD’s budget contains 3 large formula grant programs for public housing authorities (PHAs): 1) the Public Housing Operating Fund ($4.5 billion in FY16), 2) the Public Housing Capital Grants ($1.8 billion in FY16), and 3) Housing Choice Voucher (HCV) Administrative Fees ($1.7 billion in FY16). Another 3 formula grant programs serve cities or tribes: 1) Community Development Block Grant Entitlement/Non-Entitlement ($3.0 billion in FY16), 2) HOME Investment Partnerships ($0.9 billion in FY16), and 3) Native American Housing Block Grants ($0.6 billion in FY16).
• Although the funding formulas are prescribed in statute, evaluation-based evidence is central to each HUD is using evidence from a 2015 Administrative Fee study of the costs that high-performing PHAs incur in administering a HCV program to propose a new FY17 approach for funding Administrative Fees while strengthening PHA incentives to improve HCV outcomes by providing tenant mobility counseling.
• HUD is also conducting a Rent Reform demonstration and a Moving To Work (MTW) demonstration to test efficiencies of changing rent rules.
• HUD also is conducting an extensive assessment of Native American, Alaska Native, and Native Hawaiian housing needs to strengthen the evidence base for the formula programs.

In FY16, did the agency shift funds away from any practice, policy, or program which consistently failed to achieve desired outcomes? (Note: Meeting this criteria requires both Agency and Congressional action.)

• HUD’s FY17 budget request includes a new formula for funding Housing Choice Voucher Administrative Fees that shifts funding away from inappropriately compensated public housing agencies and increases overall funding according to evidence about actual costs of maintaining a high- performing voucher program. (See here for more)
• The Administration’s FY17 request recommends shifting support from homeless interventions shown to have limited effectiveness toward housing vouchers that were proven effective in the Family Options study.