Peace and security are two halves of the same coin. Achieving both in any meaningful way is a long-term, multi-generational effort that is never a clean, linear process. It depends heavily on a magical combination of political circumstances, committed leadership at multiple levels, investment in the sectors that most impact peace and security, and sustained momentum by donors and partner institutions. Given such intricacies, it’s important to consider complexity-aware monitoring (CAM) approaches that can help uncover the most essential information.
Traditional monitoring and evaluation (M&E) approaches alone don’t truly advance understanding of how programs can work (or not work) in dynamic environments and specifically how governance-focused initiatives can really affect change over time. To be intentional in crafting strategies and objectives—particularly given shifting national priorities and constrained budgets—it’s paramount to consider other ways of looking at this problem.
CAM is a framework that offers a broad approach to identifying how to conduct M&E in complex environments as well as for complicated programs in general. CAM is relevant when one or more of the following conditions hold: cause and effect are uncertain; a diversity of perspectives exists; contextual factors are likely to have an outsized influence on programming; new program opportunities or needs arise; or the pace of change is unpredictable.
CAM tools run the gamut according to each project’s need, what aspect of the problem the project team wants to unpack, or what type of question it wants to ask. Some commonly used approaches include:
- Sentinel Indicators: Proxy for complex processes that are difficult to study in their entirety. These are easily communicated and can signal the need for further analysis and investigation;
- Stakeholder Feedback: A family of approaches, privileging perspectives of partners, beneficiaries, or non-participants. It seeks diversity rather than consensus (complexity is diverse, knowledge of system is partial, and predictability is low.);
- Most Significant Change (MSC): Collection and analysis of stories describing the most important project outcomes. Perspectives of different stakeholder groups are represented in the criteria for determining a significant change. It captures a broad range of results, intended and unintended, positive and negative. Diverse perspectives on the results are explicit.;
- Outcome Harvesting (OH): Discovers results without reference to predetermined objectives. It works backwards to determine the contribution with an emphasis on verification and describing contribution. And it captures a broad range of outcomes. (See Dexis’ blog on OH here.)
- Process Monitoring of Impacts: Examines how a result at one level is used to achieve results at the next level using predicted and emergent processes. A log frame serves as a hypothesis, not a blueprint. An example follows.
So, how does CAM look, smell, and feel in practice and on the ground? Consider the following case study on the Security Governance Initiative (SGI). Launched in 2014, SGI was designed to enable six partner countries (Ghana, Kenya, Mali, Niger, Nigeria, and Tunisia) to provide more efficient and transparent management and oversight of security and justice sector institutions.
The SGI approach is not a traditional train-and-equip program, but instead works with partner countries to develop policies and strategies and to implement programs that address their institutional and capacity-building needs within the security and justice sectors.
Dexis selected CAM for SGI to understand what factors and key milestones within process and policy development demonstrated change and to understand impacts on overall reform. To do this, Dexis conducted in-person interviews and focus groups; performed process mapping with implementers; and triangulated with other donors and adjacent programs.
By monitoring the development of processes over time, CAM helped to identify and broaden understanding of key milestones and factors that demonstrate how process and policy development leads to reform and change. But there are limitations and considerations. For example, access to partner organizations, individuals, and leadership can be challenging with this approach, as is investigating long-term, generational change. And allocating the necessary time and labor can impact the budget.
CAM does not solve the persistent challenges with M&E for complex programs and environments, but it can deepen and augment existing M&E and implementation efforts. CAM approaches can provide a deep understanding of stakeholders and their values, a broad picture of results and changes in the environment and provide insights for policy and program stakeholders.
When utilizing CAM, some things for practitioners to note:
- CAM is not a silver bullet. Practitioners should be intentional and establish a criterion for selecting CAM as a methodology and communicate advantages and limitations to relevant stakeholders, early and often.
- CAM is an approach and way of thinking. Practitioners should consider CAM early in the planning process as it can have time, budget, and labor implications for M&E. Utilize it as a useful approach when developing evaluation questions and understanding important contextual nuances. Remember to communicate to relevant stakeholders.
- CAM is a specialist within a team. Practitioners should review all the tools and approaches within CAM to complement the existing M&E system. They can be layered and sequenced throughout planning and implementation to help inform those processes.
Sarah Kim is Senior Technical Advisor in the Monitoring, Evaluation, and Learning Division at Dexis where she provides business development and technical support for State and USAID programs focused on security assistance, governance, political transition, conflict/post-conflict, and stabilization sectors.
Photo by WOLFGANG KUMM/DPA/dpa Picture-Alliance
