Randomized Controlled Trials of Multi-sectoral Programs: Lessons from Development Research

"[W]ell-designed pragmatic RCTs, informed by theory, process evaluations, qualitative research, and collaborative engagement between researchers and implementers from the design phase through the interpretation and utilization of results can generate evidence that helps partners design and implement successful multi-sectoral development programs at scale."

Over a period of two decades, the International Food Policy Research Institute (IFPRI) has designed and implemented randomised controlled trials (RCTs) to assess the effectiveness of multi-sectoral programmes in improving nutrition, food security, and other measures of well-being. This article outlines IFPRI's evaluation approach and explains how it addresses three perceived pitfalls of RCTs in order to generate evidence that helps partners design and implement multi-sectoral development programmes at scale.

Three criticisms of RCTs, and IFPRI strategies:

• The "black box" nature of the relationship between interventions and their impacts - RCTs have been criticised failing to explain how an impact was achieved. IFPRI addresses this issue by developing a conceptual framework that maps hypothesised pathways of impact. Shaped by a theory of change, this process is informed by evidence-based findings gleaned through quantitative and qualitative methods. For example, to understand whether - and how - nutrition- and gender-sensitive homestead food production programmes with behaviour change communication (BCC) improved child nutrition, IFPRI carried out a cluster-RCT to test the impacts of the programme and used mixed methods, including interviews with implementers and participants to understand barriers and facilitators, to assess impact pathways. The confirmed pathways of impact included improvements in women's agricultural production, empowerment, health and nutrition knowledge, and infant and young child feeding and care practices. These pathways supported the plausibility of findings that the programme reduced child anaemia and wasting.
• Limited external validity of findings - It has been argued that RCTs may be unable to clarify the extent to which an intervention is expected to have similar effects in a different setting. IFPRI's impact evaluations seek to explore what conditions allowed the underlying mechanisms to play out; "the conceptual framework itself is likely to have wider application and policy relevance across many settings". IFPRI also generates information on external validity by conducting RCTs of similar interventions in different contexts. For example, IFPRI used RCTs to evaluate a maternal and child health and nutrition programme in Guatemala and Burundi featuring food rations, health services strengthening, and BCC. While both settings were characterised by high levels of stunting, food insecurity was a severe problem only in Burundi. The programme reduced the prevalence of stunting in both settings, but also contributed to postpartum weight retention in the already overweight study population in Guatemala. "This finding underscores the importance of carefully analyzing context when designing programs to prevent unintended negative consequences on targeted populations."
• Challenges in translating results to impact at scale - IFPRI addresses this by working closely with implementing partners in the design, research, and dissemination processes to inform adaptation and scale-up of programmes and policies. The fact that IFPRI has research teams in Sub-Saharan Africa and South Asia facilitates engagement throughout the research process and often supports policy adoption and scale-up. For example, IFPRI has a long-term presence in Bangladesh, which underpinned a collaborative RCT carried out with the Bangladesh Ministry of Agriculture (BMoA) on the Agriculture, Nutrition, and Gender Linkages (ANGeL) Project. The evaluation showed the effectiveness of combining agricultural production, nutrition knowledge, and gender sensitisation trainings to improve production and consumption diversity, farmers' income, and women's empowerment. Dissemination and continued engagement around these results motivated the BMoA to scale up the programme nationally, which is being rolled out as a large-scale RCT designed in collaboration with IFPRI.

In conclusion, IFPRI's multi-sectoral, collaborative approach to RCTs "helps program implementers learn whether their good intentions have the envisioned benefits, while bringing together the tools of research with the voices of those whose lives are affected to advance our collective understanding of what works" to improve people's health, nutrition, and well-being.