Insights from implemation and evaluation of progams focusing on young people

In 2020, UNFPA launched the International Technical/Programmatic Guidance on Out-of School Comprehensive Sexuality Education, which expressed the need to generate more evidence on certain aspects such as the recruitment, training, and ongoing support for facilitators. In order to generate evidence on this, a multi-country study was designed in collaboration UNFPA/WHO/national partners: Implementation research to assess the feasibility, acceptability, and effectiveness of packages of actions tailored to different contexts to prepare and support facilitators to deliver CSE effectively in out-of-school settings.
The results of the research in Colombia cover two phases (2022/2025), evaluating CSE programs from UNFPA global framework program: "Reaching those most left behind through CSE for out-of-school young people". Phase I evaluated an CSE program implemented in the Department of Atlántico with the participation of young Venezuelan migrants and host populations, and Phase II evaluated a program implemented in Solano, Caquetá, with rural youth affected by the armed conflict.
The evaluation used a quasi-experimental mixed-methods design, with nine tools applied to program managers, facilitators, and young people (14 to 17), including document review, interviews, focus groups, observation lists, and pre- and post-tests.
The findings show that the selection of facilitators is fundamental to the quality of the program; the training and ongoing support of facilitators must be comprehensive, continuous, and in accordance with a rigorous plan; and it is essential to strike a balance between fidelity of implementation, which guarantees the rigor of the CSE program, and adaptations that allow for cultural contextualization. 

With the continuing expansion of internet access around the globe, social impact campaigns delivered through social media are expanding. However, most existing research evaluating their impact is limited to measuring reach or short-term effects. This presentation will discuss findings from two research projects offering differing methodologies to evaluate the impact of online media interventions on global health outcomes. While the first evaluation studies impacts of a real-world social media campaign "in the wild", the second study offers a way to isolate impacts at the level of individuals using a virtual, lab-like environment on social media to test media interventions. 


The first project, an evaluation of a multi-state ad campaign to promote malaria prevention in India (in partnership with Malaria No More), will focus on results produced from an assessment of the campaign's effectiveness at reducing real-world malaria incidence. The second, an evaluation of a multimedia effort focused on improving sexual and reproductive health behaviors, attitudes, and perceived norms in Kenya (in partnership with Girl Effect), will present findings from an upcoming randomized control trial evaluating the individual and combined effects of exposure to a character-driven video drama and a smart chatbot focused on private provision of information on sexual health-related topics. These diverse approaches offer vital lessons for rigorously measuring the impact of digital communication on health outcomes worldwide.

Pakistan's health system is constrained by structural barriers that limit Direct-to-User (D2U) communication, particularly with women and adolescents, through hierarchical gatekeeping and restrictive gender norms. Against this systemic challenge, this project deployed a Generative AI (GenAI) voice helpline for Human Papillomavirus (HPV) vaccination. Despite these barriers, provisional pre-use survey data from caregivers revealed striking demand: 76.6% were open to vaccination. Furthermore, among those expressing a helpline preference, a potential GenAI line (3321) was significantly preferred over the established human service (1166) (54.7% vs. 45.3%).


A two-stage adaptive design first ensured functional readiness. Stage one – the Optimisation Phase – achieved rapid internal improvement within eight weeks. This phase included pulse-checks, a three-arm multi-armed bandit (MAB) experiment (n=1,933 completions), and structured audits of 398 medically relevant call completions. The MAB revealed the need for a hybrid, context-adaptive greeting policy (professional voices leading to intent, elder-sister to trust). Medical accuracy reached 86.03% (v1.0), which identified systematic errors in benefits messaging and eligibility for targeted remediation.


Stage two is currently implementing the Comparative Evaluation using a quasi-experimental comparison with Inverse-Probability-of-Treatment Weighting (IPTW) (with final causal estimates available March 2026). Critically, 53% of mothers required permission to use a phone for health information, strongly indicating that an anonymous, 24/7 D2U GenAI service may help mitigate such access barriers. This demonstrates how governed AI can create alternative health communication pathways.

This study redefines vaccine communication by centering community knowledge in behavioral design. UNICEF and Viamo engaged over 18,000 parents across 14 districts in Pakistan through mobile interactive voice response (IVR) surveys to test how messenger identity and message framing influence HPV vaccine uptake. Two randomized controlled trials compared messages voiced by a mother versus a doctor and examined three framing variants: HPV prevention, cervical cancer prevention, and a blended approach.
Results showed consistently high acceptance (80–83%) but revealed important behavioral nuances. A mother's voice accelerated intent to vaccinate, while a doctor's voice reinforced confidence. Blended messages combining HPV and cancer framing generated the strongest urgency to act. These findings challenged assumptions of cultural resistance, showing that empathy and identification drive engagement even in conservative contexts.
Two innovations: the visibility gradient, guiding stigma-sensitive language across public and private channels, and the Number Needed to Accelerate (NNA), linking message design to uptake speed, translated behavioral insights into operational tools. Together, they informed UNICEF's HPV communication strategy and offered a scalable model for inclusive, evidence-based social and behavior change communication in low-literacy or high-stigma settings.


Keywords: HPV vaccine, behavioral science, visibility gradient, Number Needed to Accelerate, A/B testing, IVR (interactive voice response), low-literacy audiences, gender dynamics, stigma-sensitive messaging, timing of uptake.