COLOMBIA
Finding New Opportunities for Leadership on M&E
When Gustavo Petro’s administration took office in Colombia in August 2022, an important report awaited his policymakers – a summary of all the public policy evaluations conducted by the National Department of Planning (DNP) over the last four years, with clearly worded explanations of findings and recommendations on next steps. This report was the first of its kind in Colombia’s lauded history of efforts around national monitoring and evaluation (M&E) and was an important step in Colombia’s latest effort to increase the use of evaluation findings by the public sector and the general public. The report is also setting the stage for the country to show regional and global leadership on how to better communicate evaluation findings to policymakers.
A team from the Global Evaluation Initiative (GEI), worked closely with the DNP, Colombia’s agency tasked with national M&E efforts, on this report, producing a document quite different than past reports. What set this report apart from previous efforts was the focus on translating results from technically rigorous evaluations into straightforward, accessible language. Making sure that evaluation results are understood by stakeholders and decision-makers is an important way to ensure that successful programs and policies receive the positive attention they deserve, while those that are struggling are improved. Communicating evaluation results clearly also helps policymakers continue to view evaluations as valuable tools. This summary report of evaluation findings was part of GEI’s ongoing support to DNP, as it undertakes the latest effort in strengthening the National Management and Results Evaluation System (Sinergia).
“The Colombian evaluation system continues to have an important presence in government decisions and is regarded as a best practice example. However, there is a willingness to improve with the GEI support. We hope to count on GEI’s support to continue improving our evaluation process and to have more and better evidence for decision making.”
A History of Progress
Colombia is no stranger to being regarded as a leader in Latin America – and globally - for their national M&E system. For the last three decades, Colombia has focused on institutionalizing the role of M&E in government processes and decisions. It has long been seen as “one of the strongest M&E systems in Latin America, in terms of the types of M&E it undertakes, its overall credibility, and its utilization,” according to the World Bank.1
In 1994, DNP was given legal responsibility for reporting evaluation results annually to the National Council for Economic and Social Policy (CONPES) headed by Colombia’s President. Sinergia was created the same year and served to “operationalize the constitutional and legal mandates for evaluation by assigning responsibility for self-evaluation to all agencies in the executive branch of government”.2 In mid-2002, the government “introduced management by results, transparency, and accountability as key principles for public sector reform through the Public Administration Renewal Program. SINERGIA received new importance and became the focal point of the results-based management reform.”
Part of what makes Sinergia and DNP’s efforts around M&E a best practice example is the willingness and commitment to continue improving. Consequently, in 2020, DNP decided to focus on improving the use of evaluation findings by decision-makers and policymakers, approaching GEI to support this goal.
1 “A Diagnosis of Colombia’s National M&E System, SINERGIA.” World Bank Group. Accessed April 9, 2023. ieg.worldbankgroup.org/sites/default/files/Data/reports/sinergia_0.pdf.
2 “Case-Study #4: National Evaluation Policy In Colombia.” Parliamentarians Forum for Development Evaluation. Accessed April 9, 2023. betterevaluation.org/sites/default/files/cscolombia.pdfdefault/files/cscolombia.pdf
A Holistic Approach to Meet Objectives
For the last two years, a GEI team has been providing technical and advisory services to DNP, focusing on several intersecting factors that are influencing the uptake of evaluation findings in Colombia.
CONPES Document. Early on, Colombia decided that using the power of CONPES was the best approach to ensure cross-government buy-in. The CONPES is the “highest national planning authority and is the government’s advisory body in all aspects related to the country’s economic and social development. To achieve this, it coordinates and orients the organizations in charge of economic and social management by studying and approving documents on the development of general policies presented in the session.”4
The GEI team provided feedback to DNP on the CONPES document 40835, which is the official commitment of the Colombian government to improve its national M&E system in the period of 2022 - 2026. GEI’s recommendations included the following:
- The evaluation cycle should not only include current evaluations of policies or programs. It should also include evidence from previously conducted evaluations in Colombia - or even evaluation findings from similar programs from other countries - for the design of new programs. For this reason, DNP plans to soon open a new Analysis Unit.
- The national M&E system should include the Ministry of Finance. Ministries of Finance are key players in national M&E systems, especially if evaluation results are meant to be linked to the budgetary process. DNP included this element in the CONPES document, and it is now actively trying to be included in the budget process. This task has not been easy, but the effort is beginning to show results.
- The national M&E system should start including rapid evaluations, making the system more flexible so that it can respond to specific demands from programs and sectors in a shorter turn-around time than traditional evaluations. DNP is actively seeking to generate this type of evaluation and thus expand its offer of evaluation tools.
- The national M&E system should include concrete evaluation guidelines. Even though Colombia has sound prescriptive elements for its system, concrete guidelines for evaluations are not in place. (GEI will be supporting DNP with these guidelines.)
- To support the CONPES document, DNP, GEI, together with their partners DEval, the World Food Program and FOCELAC, suggested that Colombia set a baseline using the National Evaluation Capacities Index (INCE). INCE is a collaborative diagnostic tool that measures evaluation capacities and practices at the national level. GEI will complement the INCE with a diagnostic on the use of evaluation findings during 2023. Both results will be part of the CONPES document process.
Impact Evaluation Workshops. With the support of GEI, DNP staff and invited academics from Colombia attended an online workshop on impact evaluation, organized by GEI Associate Partner, 3ie. Workshop topics included the latest impact evaluation techniques, including big data, new double-difference evaluation methods, and methods using artificial intelligence. The DNP staff are now more prepared not only to engage in new impact evaluations but also to supervise and coordinate those evaluations. The DNP team is now working on impact evaluations for some of the new administration’s programs.
Evaluation Summary Document. The GEI team supported the development of a document that summarized the main results and recommendations of policy evaluations carried out in 2018-2022. This was a key input for the process of connecting with the incoming administration. For the first time in the history of Colombia, the evaluation system generated a non-technical document to summarize the most important findings of the evaluations for communication purposes to the general public, but especially for the new government. The DNP now has the clarity that they need to produce this type of document to increase the use of the evaluation findings. This is important not only when a new administration takes over, but also when DNP is trying to communicate the importance of the evaluation process.
Knowledge Sharing. GEI supported DNP in the organization of various events for gLOCAL Evaluation Week 2022 to share recent experiences around their goal of increasing the use of evaluations in the Colombian national M&E system.
4 “National Public Investment System of Colombia.” Regional Observatory on Planning for Developmentin Latin America and the Caribbean. Accessed April 9, 2023. https://observatorioplanificacion.cepal.org/en/planning-systems/national-public-investment-system-colombia
5 “Strengthening the Use and Institutionality of Assessments for Decision-Making in Colombia.” Document 4083 of the National Council of Economic and Social Policy Republic Of Colombia, National Planning Department. Accessed April 10, 2023. https://colaboracion.dnp.gov.co/CDT/Conpes/Econ%C3%B3micos/4083.pdf
Moving Forward Together
Looking forward, GEI’s team in Colombia has a lot planned with its DNP colleagues. To complement the findings of the INCE diagnostic conducted in 2022, GEI will be designing and implementing a custom diagnostic to measure the use and uptake of evaluation findings across the Colombian public sector. Based on the findings of this diagnostic, GEI will develop guidance for the use of evaluations and will advise DNP on how to develop an indicator for the use of evaluations.
In addition, based on the results of the diagnostic on the use of evaluation findings, the GEI will produce, together with DNP, a set of recommendations to improve the public policy decision-making process using evaluation findings. These recommendations will include ideas about adjusting the evaluation cycle during the year to match it with the budgetary process, a work that has already started at the DNP.
Lastly, GEI will be helping Colombia share its experiences through three sessions at gLOCAL Evaluation Week 2023. The three sessions will look at the evaluation of graduate education in Colombia; how to better communicate technical evaluation findings to the general public and policy makers; and, the best way to measure the use of evaluation findings.
Colombia’s leadership on national evaluation system implementation is continuing with a new – and important – phase of improvement, again setting a standard on using evidence to improve the lives of their citizens.
Story by Maria Fyodorova
Built with Shorthand