When a capacity-building programme ends, the first question that is usually asked is, “Was the programme successful?” In other words, “Did the programme have an impact?”
On the surface, the answer appears straightforward. We can report how many people enrolled, completed the programme, attended live or virtual sessions and passed assessments. These indicators matter, but they only tell part of the story. A participant can achieve an excellent pass mark and finish the programme with a certificate. Yet if nothing changes in the way they work, make decisions or influence the institution around them, can we really call the programme a success? Digital Frontiers aims to design and deliver learning experiences that lead to systemic change and tangible impact.
How We Approach Impact in Programme Delivery
This brings us to a question that everyone in the capacity-building space must answer: ‘How do you measure impact?’
In truth, this is one of the hardest questions to answer. Measuring impact is challenging. It needs to be considered across four levels:
- The individual level
- The organisational level
- The sector/industry level
- The country level
At an individual level, we can track quantitative metrics from the training programme, such as attendance, completion rates and assessment scores. These are easy to measure and add useful surface insights. However, determining whether learning has resulted in better decision-making and attitude change is more difficult to infer from this data. We can supplement with qualitative data from focus groups and interviews, but it still only limits us to personal reflection and subjective experience.
Impact at organisational and industry levels is much more difficult and complex. Simple course metrics provide no insight as to whether the programme resulted in stronger institutions, more inclusive systems, or industry shifts. This requires larger reviews, surveys, and interviews with industry experts, policymakers, and regulators. Then, considering the conditions at the country level, stakeholder relationships and implementation pathways determine whether learning can translate into wider systems change.
Figure 1: Our Capacity Building Approach (Source: Digital Learning Solutions Concept Note, Digital Frontiers, 2026)
Our approach is grounded in a simple belief: learning is not the outcome; it is the mechanism that enables change. To ensure our programmes have impact, we include insight and input from all four levels in our programme design.
Every programme begins with needs analysis and consultation with experts and stakeholders to determine what is required to bring about the desired impact across all four levels. These are transformed into learning outcomes that are the foundation for the programme design.
Focusing on desired outcomes ensures that our curriculum design is intentional and that learning content, facilitation, peer engagement and implementation will deliver the real-world change we hope to see.
This approach focuses the entire programme design around ensuring that we have included mechanisms that will prompt the application of information into real-world scenarios. This has the potential to result in systemic change and actual impact, as opposed to simply transferring knowledge and hoping it is used.
At Digital Frontiers, driving impact is particularly important. We know that many of our students have gone on to become policymakers and regulators who design and implement institutional practices, policy, or oversight that ultimately affects thousands or even millions of people.
And as part of our DNA, Digital Frontiers exists to help advance a thriving Global South by empowering local changemakers and institutions to co-create inclusive digital systems that unlock opportunity, equity and lasting impact. Consequently, we are interested not only in what participants know at the end of a course, but in what they go on to do with that knowledge.
From Learning to Systems Change
Our programme design is grounded in adult learning theory and andragogical principles; we apply instructional design best practices and active learning strategies to create engaging, skills-based learning experiences that align with client needs and support long-term knowledge retention.
The outcomes identified from the needs analysis and expert and stakeholder engagement inform our programme and curriculum design. These outcomes are differentiated between knowledge and skills and the behavioural and attitude changes we hope participants attain by the end of the learning process.
Best-practice evaluation literature supports this progression. Donald and James Kirkpatrick’s Four Levels of Evaluation argues that effective programmes should ultimately be assessed by changes in behaviour and organisational results, rather than participant satisfaction alone. Likewise, Etienne Wenger’s theory of Communities of Practice recognises that meaningful learning continues through ongoing peer engagement, reflection and collaborative problem-solving.
- Knowledge and Skills Outcomes
These outcomes inform what content is included in the training course itself, conveying what the individual participant will need to know and be able to do once they have completed the programme. - Behaviour and Attitude Changes
These outcomes also influence learning content, such as including case studies, interviews and opportunities for reflection; however, they are more heavily shaped by the broader programme design. Participants need opportunities to discuss with peers and engage with technical experts, both of which are supported by including virtual class discussions and establishing ongoing communities of practice.
Putting Theory into Practice
These principles and approaches were instrumental in the design of our Risk-Based Supervision for Inclusive Digital Financial Services Programme. When developing the Monitoring, Evaluation and Learning (MEL) framework for the programme, we considered it as an extension of the learning journey to track the actual impact of the programme. This foundational design will eventually provide us the best chance to answer the question, “Was the programme successful?”
A crucial aspect of this design, which enables us to measure impact and extend the programme beyond knowledge transfer, is the inclusion of Communities of Practice (CoP) into the overall programme framework. Within each focus country where we deliver the programme, we have established a CoP. These CoP’s create spaces where participants can meet in person and continue discussing implementation challenges, share practical experiences and support one another long after formal learning has ended. These conversations help translate knowledge into action while also providing valuable evidence of emerging change. The in-country champions who lead these CoP’s also have the opportunity to monitor discussions and identify areas where training from the programme has resulted in systemic change well after the programme has concluded.
This country-level lens is especially important for programmes that aim to influence regulatory practice, policy implementation or market development. In these cases, evidence of impact may include changes to supervisory frameworks, new or revised institutional procedures, stronger collaboration between regulators and market actors, improved alignment across ecosystem stakeholders, or the adoption of approaches that affect service delivery and consumer outcomes at country scale.
Figure 2: Our Communities of Practice Model (Source: Risk-based Supervision for Inclusive Digital Financial Services Programme Brochure, Digital Frontiers, 2026)
In this instance, the framework follows participants from baseline surveys, through engagement and assessment, into structured follow-ups at 30, 90 and 180 days, combining quantitative and qualitative evidence. It examines learning gains, behaviour change, institutional strengthening and impact contribution through surveys, supervisor validation, learning logs and case studies that stem from engagements through the Communities of Practice.
Why This Matters
We believe that monitoring and evaluation should be more than measuring delivery. It should help identify where learning has become implementation, where implementation has become institutional practice and where those changes contribute to broader systems transformation. Essentially, it helps us determine impact and success across the four levels: individual, organisational, industry, and country.
We do not see programme delivery as simply delivering training. We see it as designing collaborative learning journeys that build lasting capacity, strengthen institutions and create measurable development impact. Programme success is therefore about far more than a pass mark—it is about the lasting change that follows.
Learn more about Programme Delivery at Digital Frontiers.
By Andile Bidi (Senior Programme Coordinator), Megan Carver (Learning and Development Manager), and Blaze Haselau (Senior Manager of Digital Learning Solutions)
(Article also shared on Digital Frontiers Institute on 18 September 2026)