What we do
Theme 5: Monitoring, Evaluation, Learning & Sharing
Justine Wangila (Kenya), Thematic Director
See Staff section for more information.
Theme 5 objectives embody the SAA commitment to becoming an evidence-based organization that better understands and documents the impacts of its investments.
Establish a relevant, efficient and effective monitoring, evaluation, learning and sharing system to drive SAA’s evidence-based programs.
- To promote and institutionalize monitoring, evaluation, learning and sharing, involving partners for evidence-based reporting and impact assessment at SAA.
- To assess and identify farmers', other target beneficiaries' and partners' needs to prioritize SAA interventions.
- To collect effectively and efficiently and use baseline data and information on SAA interventions.
- To collect, analyze, use and report in a continuous and systematic manner, monitoring data and information from selected SAA intervention areas.
- To develop and implement appropriate strategies for periodic internally and externally commissioned evaluations to assess performance – relevance, efficiency, effectiveness, impact and sustainability – and guide decision making.
- To develop and implement appropriate methodologies and tools to measure and assess SAA program(s) and project(s)' impacts on smallholder farmers, partners and agricultural development in the four focus countries.
- To identify, capture, document and share good practices and lessons on SAA interventions.
Over much of its 25-year history, SAA did not undertake formal monitoring and evaluation of its program activities. No baseline data was collected when program activities began in each of the 15 program countries. No professional adoption studies were conducted on adoption rates and patterns from the technologies demonstrated in more than 3 million plots. There have been four external program reviews, but these reviews were largely qualitative, since an organized body of data about program activities and impacts was unavailable.
This is not to say that the SG 2000 programs had no impact on smallholder crop productivity or total production. They did. Moreover, SG 2000 staff members were legendary for the time they spent in the field with farmers, reviewing outcomes and assessing the agronomic efficacy of recommended technologies. But, the lack of an organized monitoring, evaluation, learning and sharing system was an institutional weakness, one that lessened the overall impact of SAA investments and reduced the influence the program had in international development circles.
SAA/CIMMYT Impact Assessment Project
SAA's first step towards establishing a fully-fledged MELS Program began in 2006 when The Nippon Foundation funded the International Maize and Wheat Improvement Center (CIMMYT) to undertake an independent project, ‘Knowledge System to Monitor and Assess Impacts of SAA and Partners' Activities' in two SAA focus countries – Uganda and Ethiopia.
The CIMMYT project employed a team of Social Scientists, Economists and Geographic Information Systems (GIS) Specialists to assess SAA SG 2000 interventions' impacts on the livelihoods of smallholder farmers. It covered both the direct and indirect, positive and negative, and intended and unintended impacts. Significant spillovers were also assessed, including on local non-participants, NGOs, the private sector, and on local development efforts and policies.
The project's findings and their apparent policy implications were communicated through workshops and publications, as well as through a project website (http://sg2000ia.cimmyt.org). More than 20 technical economic reports, including published drafts of international peer reviewed journal papers, can be accessed there. This project came to an end in 2010.
Beyond the Impact Assessment Project and into the Future
SAA established Theme 5 to build a relevant, efficient and effective monitoring, evaluation, learning and sharing system to document impacts and the effectiveness of all investments—both core and extra-core donor funded. To achieve these, Theme 5 works with and through the other SAA themes and the SG2000 country programs to implement the SAA MELS system(s). An interesting challenge for the MELS Theme is to become fully understood and accepted by all SAA staff, as an integral service to enable other Themes to achieve their own objectives. The danger is that MELS becomes seen as the SAA policeman.
For example, the MELS system aims to enable rapid appraisal of proposed technologies and activities; the management of key information; inform timely modification of SAA investments; and over time, through collected data and information track and document performance of interventions and partners.
Learning and Sharing
As SAA implements SG2000 interventions and activities in the field, there are lessons that are learned on challenges, problems, failures, successes, and best practices that need to be shared. Defining best practices is a challenge, since who determines or defines a good or best practice? We intend to take on, this challenge through institutionalizing a learning and sharing framework that shows how lessons and practices should be collected, verified, stored and shared; i.e., that facilitates identification of lessons and best practices and promotes adoption of the best practices from mainly the M&E outputs. Similarly, sharing results and lessons as quickly as possible is important to efficient program management and to achieving impact.
To learn and share, SAA will use various approaches, including annual reports, concepts and procedures workshops, training materials, newsletters, blogs, documentary videos, radio programs, working papers, journal articles, books, policy advice/briefs, presentations at conferences and meetings, and the biennial Borlaug Symposium ICT will drive learning and sharing processes at SAA and the SAA website will be a critical information clearing house in our strategy.