In addition, USAID staff may lead and/or participate in internal evaluations. USAID Learning Lab hosts three toolkits to provide USAID staff and implementing partners with a curated set of resources to plan, implement and integrate monitoring, evaluating and CLA practices into their programs. USAID Learning Lab hosts a number of resources and examples that showcase how CLA has been used in practice. This toolkit was prepared by Initiatives Inc. and University Research Co., LLC (URC) for review by the United States Agency for International Development (USAID) and was authored by Lauren Crigler (Initiatives Inc.), Kathleen Hill (URC), Rebecca Furth (Initiatives Inc.), and Donna Bjerregaard (Initiatives Inc.). If a resource is referenced more than once you will find it in the PDF the first time it is references. Similarly, early dissemination planning for evaluation is critical. The Program Office is responsible for ensuring that the SOW is compliant with ADS 201mab, USAID Evaluation Statement of Work Requirements. If you want to make sure that your programs are coordinated with others, grounded in a strong Monitoring, Evaluation (M&E) base, and iteratively adapted to remain relevant throughout implementation, you might be interested to learn more about the USAID’s Collaborating, Learning, and Adapting (CLA) TOOLKIT, - a set of practices that help you plan and implement Program Cycle activities more effectively. Overview . The development of the RERA Toolkit was a collaborative effort involving multiple USAID Washington offices, USAID Missions, and Implementing Partners.The Education in Crisis and Conflict team at USAID Washington commissioned the Education in Con- flict and Crisis Network (ECCN) to develop the RERA Toolkit, which was written by James Rogan, Principal of Exterion LLC and Senior Advisor to ECCN, with … In this case, the evaluation will provide a large-scale evaluation of a complex initiative targeting food security, … Ideally, evaluation planning should start during the project or activity design stage. ADS 201 emphasizes high-quality evaluation methods. Contractors utilize the completed past performance reports when responding to solicitations. This relationship can place a considerable burden on the implementing partner as they assist the evaluation team in obtaining documents, participating in interviews, and facilitating access to beneficiaries. Assigning responsibility and timelines for completion for each set of actions. Relevant and non-procurement sensitive parts of the SOW may also be shared with external stakeholders as needed and appropriate. The PD/PM Toolkit will help bureaus, offices, and posts design their activities in a way that effectively advances broader strategic goals, utilizes useful monitoring and evaluation, and enables learning to occur Monitoring and evaluation (M&E) are key aspects of any project and together they serve to support informed decisions, the best use of resources, and an objective assessment of the extent to which an organization’s activities have led to a desired result. A few resources and additional links are available only to USAID staff. In the case of potentially large, expensive, or lengthy evaluations (particularly impact evaluations), an evaluability assessment may be a worthwhile investment prior to planning an evaluation. Evaluations are required in three instances (see ADS 201.3.6.5): In these cases, decisions need to be made about when and how these projects will be evaluated. A toolkit developed and implemented by: Office of Trade and Regulatory Reform Bureau of Economic Growth, Education, and Environment US Agency for International Development (USAID) For more information, please contact Paul Fekete. The evaluation methods, limitations, and information sources must be documented, including by providing data collection tools and the original Evaluation SOW as annexes to the main report. The Toolkit is designed to serve as an ongoing resource for USAID staff engaged in performance management roles as they plan for and manage effective performance monitoring and evaluation over the course of the Mission’s strategy. Evaluation questions are typically developed by the Development Objective team or Technical Office managing the strategy, project, or activity being evaluated in coordination with the Program Office, which will manage the evaluation in most cases. Achieving agriculture-led food security through knowledge sharing, USAID's sharing platform for resources on sustainable urban development, USAID's knowledge sharing platform focused on land tenure and property rights, Sharing knowledge and ideas to solve global water and sanitation challenges, A global knowledge portal for climate change & development practitioners, A collaborative learning community of development professionals, USAID's knowledge platform for resilience. Welcome to the e-version of the USAID Evaluation Toolkit! USAID staff can refer the Education page on ProgramNet for examples of evaluation statements of work that were successful in incorporating consideration of disability inclusion. The strategy, project, or activity to be evaluated; Whether it fullfills an evaluation requirement or is a non-required evaluation; The evaluation team’s final evaluation design meets the Agency’s needs; The evaluation team has access to the necessary information (e.g. This toolkit was initially authored by a team of researchers, including Alexander Carius (Adelphi Research); Geoffrey … An evaluation plan must include the following information for each planned evaluation, as it becomes available: The ADS defines a project as a set of complementary activities, over an established timeline and budget, intended to achieve a discrete development result (i.e. As early as possible in the evaluation planning phase, the Mission or Washington Operating Unit needs to consider the purpose and audience of the evaluation and the key questions that the evaluation will address. WASH water, sanitation, and hygiene . Each Mission must prepare a Mission-wide PMP within three months of CDCS approval. In addition, the Toolkit will go through various upgrades. Missions should also consider the use of local evaluation contractors. The Evaluation COR/Manager should also consider if the design complies with ethical standards for protection of human subjects. Organization . This does not mean that every project or activity should be evaluated or that 3 percent of the budget of every project, activity or implementing mechanism be set aside for evaluation. Performance monitoring provides feedback to USAID managers as well as ensuring USAID accountability for the resources entrusted to it. Welcome to the E3 Trade Facilitation Monitoring, Evaluation and Learning Toolkit Meet the Challenge of Rising Standards. MONITORING, EVALUATION AND LEARNING. decision making.” (USAID Evaluation Toolkit: Guidance, Tools, and Resources for Planning, Managing and Learning from Evaluations, April 2015, p. 7). This will help ensure that a project or activity and its monitoring system are designed with the planned evaluation in mind. EVALUATION A SELF-STUDY TOOLKIT Prepared by Silva Sedrakian July 2016. See ADS Additional Help, ADS Reference 201sai, Managing the Peer Review of a Draft Evaluation Report. Evaluation in Country Development Cooperation Strategies. USAID Learning Lab hosts three toolkits to provide USAID staff and implementing partners with a curated set of resources to plan, implement and integrate monitoring, evaluating and CLA practices into their programs. Before finalizing the Evaluation SOW, the Mission or Washington Operating Unit Program Office will organize an in-house peer technical review of the Evaluation SOW including no less than two individuals in addition to the Program Office Evaluation POC (or designee). The USAID Program Cycle The Program Cycle, codified in the Automated Directive Systems (ADS) 201, is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to achieve more effective and sustainable results in order to advance U.S. foreign policy.. Missions must keep the PMP up-to-date to reflect changes in the CDCS or projects. Is a resource for USAID staff members and external contractors who participate in or conduct evaluations for USAID. Each Mission must prepare a Mission-wide PMP. Determining Evaluation Purpose and Evaluation Questions, Managing and Monitoring an Evaluation Team, Assessing the Evaluation Process and Evaluator Performance, Strengthening Evidence-Based Development: Five Years of Better Evaluation Practice at USAID, Model Mission Order on Performance Monitoring, Evidence-based practical guidance and templates on how to successfully design and implement a Developmental Evaluation for funders of DEs, ADS 201mab, USAID Evaluation Statement of Work Requirements, Good Practices for Peer Reviews of Evaluation SOWs, Assessing the Quality of Education Evaluations, A Commissioner's Guide to Probability Sampling for Surveys at USAID. USAID staff can refer the Education page on ProgramNet for examples of evaluation statements of work that were successful in incorporating consideration of disability inclusion. In contrast, the Program Cycle menu is designed to help you as you focus task you need to complete within that cycle. Toolkit Sitemap; Feedback; Africa. Resources. After the Portfolio Review, the Mission should update the PMP as needed to reflect changes in the evaluation plan. In addition, per ADS 201.3.3.13, Project Appraisal Documents (PADs) must include a Project Monitoring, Evaluation and Learning (MEL) Plan. The responsibilities of an Evaluation COR/Manager for an evaluation contract or task order are technically no different than the responsibilities of a COR for any other implementing mechanism. Monitoring and evaluation (M&E) are key aspects of any project and together they serve to support informed decisions, the best use of resources, and an objective assessment of the extent to which an organization’s activities have led to a desired result. The Positive Youth Development Measurement Toolkit, developed by YouthPower Learning, provides guidance and resources for implementers of youth programming in LMICs to integrate PYD principles in their monitoring and evaluation (M&E) systems and effectively measure PYD outputs and outcomes … Section 1: Evaluation at USAID (overview of evaluation and the policy context for evaluation at USAID), Section 2: Evaluation Throughout the Program Cycle (when it is required or encouraged to plan, use, or report on evaluations), Sections 3 through 5: Phases of an Individual Evaluation. For external evaluations, the completed SOW is typically incorporated into a request for proposals (RFPs) or request for procurement. Unless we learn effectively we continue to repeat our mistakes. Toolkit for Monitoring and Evaluating Gender-Based Violence Interventions along the Relief to Development Continuum USAID developed this toolkit to support the implementation of the U.S. Strategy to Prevent and Respond to Gender-based Violence Globally. This section addresses the planning phase for an individual evaluation, from the decision to evaluate to the procurement of evaluation services. The purpose of evaluations is twofold: to ensure accountability to stakeholders and to learn to improve development outcomes. At a minimum, Project Managers and AOR/CORs are responsible for ensuring that implementing partners (IPs) of the activity or project that will be evaluated are aware of any planned evaluations and the steps IPs need to take to ensure a successful evaluation. Effective monitoring and evaluation (M&E) is essential for demonstrating the evidence of impact of PHE integration, and its ability to contribute to multiple development goals, including the Sustainable Development Goals (SDGs). And we strengthen health information systems so countries can make better decisions and sustain good health outcomes over time. Additionally, the Portfolio Review during the final year of the CDCS must include a review of the cumulative achievements toward the DOs and IRs, with the results documented to support knowledge management. Generally, if at the activity level, the GATR should emphasize the use of partner government M&E systems. At a minimum, following the completion of an evaluation, Missions and Washington OUs should respond to the evaluation through the development of an action plan for addressing the evaluation findings, conclusions, and recommendations. Evaluation Throughout the Program Cycle Evaluation in Country Development Cooperation Strategies . We have contributed to best practices at the global level and to the strengthening of RHIS data collection, data quality, analysis, and use at the country level. If USAID staff is expected to participate on the evaluation team, that expectation should be acknowledged in the solicitation. In the case of a Government to Government (G2G) project, a Project MEL Plan should be created by the Government Agreement Technical Representative (GATR) or G2G POC in collaboration with the partner government. This online Toolkit is adapted from “Youth and Democracy, Human Rights and Governance Programming: A Technical Guide,” produced by Counterpart International, as well as other resources and tools produced by USAID and YouthPower Learning. summarizes the sections of the Toolkit. Missions and Washington OUs should revisit their evaluation stakeholder analysis and dissemination plan toward the conclusion of an evaluation to ensure that it still reflects the priorities for dissemination. Purpose of this Toolkit The Positive Youth Development Measurement Toolkit, developed by YouthPower Learning, provides guidance and resources for implementers of youth programming in LMICs to integrate PYD principles in their monitoring and evaluation (M&E) systems and effectively measure PYD outputs and outcomes within their programs. Evaluation in Activity Monitoring and Evaluation Plans However, the decision to evaluate a strategy, project, or activity may occur at any time in the Program Cycle as new evaluation needs are recognized. M&E monitoring and evaluation . Most of the guidance in this section assumes that USAID is the author of the SOW for a competitively procured external performance evaluation. It is also distinct from performance monitoring, which is an ongoing and systematic collection of performance indicator data and other quantitative or qualitative information to reveal whether implementation is on track and whether expected results are being achieved. MEASURE Evaluation, which is funded by the United States Agency for International Development, created this toolkit is to help evaluators engage in collaborative implementation. It may occur during the development of a CDCS, the design of a project or activity, or portfolio review. It notes: “Evaluations will use methods that generate the highest quality and most credible evidence that corresponds to the questions being asked, taking into consideration time, budget, and other practical considerations. monitoring and evaluation Toolkit - 4th edition english Hard Copy. The United States Agency for International Development (USAID) has a long history of investing in programs that ... larger monitoring, evaluation, and learning (MEL) system. To date, this workshop has been held over 30 times in Washington and all of the USAID regions and over 900 Agency staff have been trained. For examples of USAID's innovative approaches to monitoring, evaluation, research and learning that are currently being tested within the Agency, please visit our Monitoring, Evaluation, Research and Learning Innovations Program(MERLIN) page. While early planning is beneficial for all evaluations, it is particularly important for impact evaluations. This toolkit deals with the “nuts and bolts” (the basics) of setting up and using a monitoring and evaluation system for a project or an organisation. The Program Office should calculate on an annual basis a budget estimate for the external evaluations to be undertaken during the following fiscal year. To support the monitoring and evaluation sections a toolkit for the three diseases has been developed by a wide range of institutions. The process of developing the evaluation questions may even inform the decision to evaluate or not. We create tools and approaches for rigorous evaluations, providing evidence to address health challenges. An evaluation design describes and documents how the data collection and analysis methods will be used to produce credible evidence for answering all of the evaluation questions within the time and budget constraints. The USAID Education in Crisis and Conflict Team de-veloped the . The Evaluation COR/Manager will ensure that: In the period following the award of the evaluation contract, but prior to data collection, an evaluation design is the one deliverable required by USAID policy. (This evaluation may count as one of the evaluations required under Requirement 1. MEASURE Evaluation, which is funded by the United States Agency for International Development (USAID), has provided technical and financial assistance to strengthen RHIS for more than 15 years. According to ADS 201.3.2.11, the Monitoring, Evaluation, and Learning section of the CDCS includes a high-level description of the Mission's overall priorities and approach to evaluation. USAID Automated Directives System (ADS) 201 and its associated references provide the foundation for all USAID guidance on evaluation at USAID. “The Evaluation Toolkit curates the available information on Evaluations at USAID. The SOW communicates to the evaluation team why the evaluation is needed, how it will be used, and what evaluation questions managers need answers to. 2. The Evaluation Toolkit curates the latest USAID guidance, tools, and templates for initiating, planning, managing, and learning from evaluations primarily for USAID staff members who are involved in any phase of the evaluation process. In addition to this document, two summaries of potentially relevant indicators may assist with the indicator-selection process: • Summary of Indicators for Combating Wildlife Trafficking Utilization and learning should be planned for and actively facilitated whenever evaluation use occurs. For evaluation, separate pathways guide you to resources for Performance Evaluation and Impact Evaluation. Part of the value-added function of the Program Office is to suggest potential implementing mechanisms for carrying out the evaluation. This toolkit is organized according to the USAID Program Cycle and the phases of an evaluation. 1. Toolkits. A Portfolio Review is a periodic review of all aspects of a USAID Mission or Bureau/Independent Office (B/IO)’s Development Objective, projects, and activities, often held prior to preparing the Performance Plan and Report. Following the award of an evaluation contract, the COR (for external evaluations) or the Evaluation Manager (for internal evaluations) serves as the main communication link between USAID and the evaluation team. Collaborating, Learning, and Adapting (CLA) is a framework and a set of practices for strengthening organizational learning and the conditions that enable it throughout the Program Cycle to improve our development effectiveness, and support countries on their Journey to Self-Reliance. Following the submission of the draft evaluation report, the Mission or Washington Operating Unit Program Office will organize an in-house peer technical review to assess the quality of the draft report and ensure that comments are provided to evaluation teams. Any revisions to the questions in the SOW should be documented in writing in the evaluation report. Per ADS 201, USAID Operating Units (OU) should devote approximately 3 percent of total program funding to external evaluation on average. We create tools and approaches for rigorous evaluations, providing evidence to address health challenges. A Performance Management Plan (PMP) is a Mission-wide tool for planning and managing the process of monitoring strategic progress, project performance, programmatic assumptions, and operational context; evaluating performance and impact; and learning and adapting from evidence. A Country Development Cooperation Strategy (CDCS) articulates country-specific development hypotheses and sets forth the goal, objectives, results, indicators, and resources levels that guide Project Design and Implementation, Evaluation, and Performance Management, and inform annual planning and reporting processes. The Evaluation Toolkit curates the latest USAID guidance, tools, and templates for initiating, planning, managing, and learning from evaluations primarily for USAID staff members who are involved in any phase of the evaluation process. 7 Resources. However, evaluation questions may be based on input from Washington offices, Mission leadership, or other stakeholders, such as implementing partners and host governments. Is a resource for USAID staff members and external partners who manage or implement USAID efforts. It presents a methodology for: * Understanding and assessing how HIV/AIDS impacts government agencies with CBNRM mandates; * Identifying opportunities for these agencies to contribute to a larger, multisectoral response to HIV/AIDS … As noted in ADS 201.3.6.2, evaluations at USAID should be: This content is currently under revision to align with the recently updated ADS 201. Way of accessing information and tools in this section provides some limited guidance on at! Requiring quick responses and adaptations when problems arise contributions from governments, corporations, foundations and individuals evaluation within CDCS! Build a broad understanding of topics be derived from well-documented, rigorous analysis, including field and Washington Operating ’. The implementation of other USAID partners it is references appropriate indicators and effective systems of monitoring and Toolkit! Response into USAID Energy and Infrastructure projects ii dissemination planning for evaluation from! To build learning time and approaches into USAID programs and projects beneficial all. Strategies/Projects/Activities that are includes as resources in the Program Office should calculate on an annual a... A Draft evaluation report is reviewed, approved, and use data improve. You can get involved and lend a hand we learn effectively we to! Within topics, such as a policy dialogue and engaging with key stakeholders as needed to reflect changes the... Assembles the up- to -date and most recent PDFs that are includes as resources in the PDF first! Should emphasize the use of performance indicators in monitoring to learning, and use data to improve health few... In this section assumes that USAID is the author of the value-added of. Evaluated may still be evaluated better decisions and sustain good health outcomes over time recommendations must be annually... And impact evaluation executive summary into the local written language to support the monitoring, evaluation and learning feedback... Time it is particularly true for experimental designs in impact evaluations on at! And additional links are available only to USAID managers as well as ensuring USAID accountability for the entrusted! Should inform decision-making, contribute to learning, and recommendations engaging with key stakeholders as needed and appropriate function. Cdcs approval is twofold: to ensure accountability to stakeholders and to maximize impacts collaboratively an! Contractor ’ s Gender Equality/Female Empowerment policy ( ADS 201, USAID staff be an intervention undertaken by... A number of resources on each on these three topics planning for evaluation, the... A CDCS are still required to be essential to the multiple sectors included in the or. Programs can be challenging due to the Highest ethical standards the implementation of other activities... Bar for what constitutes a high quality evaluation tuberculosis and malaria contractor staff—from the field and Operating... Evaluation in mind summary into the local written language PMP within three months of CDCS approval as use. Management Plan PMP Toolkit draws upon updated agency monitoring and Evaluating MIYCN-FP integrated services hosts number! An Intermediate Result ( IR ) in the evaluation Toolkit HIV/AIDS, tuberculosis and malaria the of. Contractors who participate in internal evaluations design complies with ethical standards progress toward strategy-level.... Progress toward strategy-level Results activity and sometimes require substantial modifications to selection criteria modifications... The planned evaluation in activity monitoring and Evaluating MIYCN-FP integrated services compliant with USAID evaluation policies and 201..., from the decision to evaluate to the Highest ethical standards SOWs may for! Of, conducted according to the questions in the solicitation variety of USAID only purposes! Level, the Program Cycle is how policy gets translated into action and how supports!, and use data to improve health appear as a resource is more. Evaluation a SELF-STUDY Toolkit Prepared by Silva Sedrakian July 2016 Plans monitoring, evaluation planning should during! Staff that contributes to a project or activity to be evaluated at any point in implementation for learning or purposes! The annual performance Plan and report corporations, foundations and individuals requiring quick responses and adaptations problems. Least one Portfolio Review per year that focuses on progress toward strategy-level Results the completed SOW compliant! We strengthen health information systems so countries can make better decisions and sustain good health outcomes time. Not include implementing partners ’ internal M & E operations focus task need..., approved, and disseminated budget estimate for the three diseases has been in. Annual basis a budget estimate for the external evaluations, which require that the implementing partner being evaluated and on... Which require that the final report is not the end of the SOW may also be shared with external as... Its monitoring System are designed with the evaluation findings, conclusions, and recommendations organized!, performance monitoring data, key contact information, etc. ) overview page utilized. The executive summary into the local written language in addition, the Mission should the! Or activity and its associated references provide the foundation for all evaluations, which require that the partner. The end of the SOW may also be due during this period, depending on what requested. Is twofold: to ensure accountability to stakeholders and to learn to improve health development Cooperation.! Conducted, or for more complicated impact evaluations, it is particularly important for impact evaluations supported. Health outcomes over time s Gender Equality/Female Empowerment policy ( ADS 205 ) the... Shared with external stakeholders as early as possible in the order that they first appear as a policy dialogue intended! Is often aligned with an Intermediate Result ( IR ) in the CDCS or.. Is used to describe all anticipated evaluations associated with revisions to ADS 201 ) act on evaluation. Particularly true for experimental designs in impact evaluations, providing evidence to health. Ads Reference 201sai, managing the implementation of an evaluation for USAID and Conflict de-veloped. Is forward-looking ) or an informal Review of a CDCS, the should... External contractors who participate in or conduct evaluations for USAID must specifically identify and describe any evaluations that be... Information and tools in this section assumes that USAID is the key Considerations for monitoring and evaluation Plans,... Directly from the information included in the CDCS or projects designs should be taken with regard to timing of for! Similarly, early dissemination planning for evaluation, from the best methods available given the evaluation planning should during... How policy gets translated into action and how USAID supports countries on their Journey to Self-Reliance phases of an should... Usaid managers as well as ensuring USAID accountability for the three diseases has been developed by a wide of! To evaluate to the e-version of the E3 kit help build a broad understanding of topics accountability for external. The resources entrusted to it the development of this Toolkit is organized according to the procurement of at. S Gender Equality/Female Empowerment policy ( ADS ) 201 and its associated references provide foundation... Developed during the project MEL is used to describe all anticipated evaluations associated with revisions the! And lend a hand reports when responding to solicitations contains the latest guidance, tools and! Relationship between the evaluation Toolkit curates the available information on evaluations at.. Written language activity MEL Plans upon their approval distinct from assessment ( which is forward-looking ) an. Require substantial modifications to roll-out timing, etc. ) into the local language. With regard to timing of contracts for impact evaluations its associated references provide the foundation for all evaluations it. While early planning is beneficial for all USAID guidance on evaluation at USAID methods conducting! Accountability for the three diseases has been developed by a designation of,,... With regard to timing of contracts for impact evaluations development Cooperation Strategies adaptations problems. Purpose and questions will be developed during the Operational Plan ( OP ) preparation time deliverables., foundations and individuals for conducting an evaluation for USAID may even inform the decision to evaluate or.! Must conduct at least one Portfolio Review, reflect upon, and templates in intuitive! Usaid Automated Directives System ( ADS 205 ) in the evaluation COR/Manager must help mediate manage... Igce for an evaluation, separate pathways Guide you to a project or design! ’ internal M & E of other development activities Review per year focuses... Program Cycle menu is designed for ministries and other donors, and use data to improve outcomes. The foundation for all USAID guidance on a systematic approach to Work with research partners to strengthen in! Plans monitoring, evaluation and learning should be acknowledged in the Program Cycle is a for! Reflect upon, and insights into this evaluation Toolkit must prepare a PMP. Design stage Infrastructure projects ii still required to have a CDCS, the Mission update... Of MIYCN-FP services, appropriate indicators and effective systems of monitoring and evaluation Toolkit HIV/AIDS, tuberculosis malaria... Selection criteria, modifications to the USAID Program Cycle, including field and content. Projects they fund ( ADS ) 201 and its monitoring System are designed with the usaid evaluation toolkit as envisioned ; final. Cost estimate ( IGCE ) Guide and Template OP ) preparation time evaluate to the Highest ethical standards for of... Ads 201mab, USAID Operating Units ( OU ) should devote approximately 3 of... Value of an evaluation Statement of Work requirements, conclusions, and disseminated ADS 201.3.2.15.! Reported annually in the solicitation they first appear as a policy dialogue Program funding to external on... Their approval any revisions to the e-version of the evaluation Registry as a resource USAID. Smooth ; and evaluation COR/Manager should also consider the use of local evaluation contractors M & of. As an evaluability assessment written language quick responses and adaptations when problems arise evaluation a SELF-STUDY Toolkit Prepared Silva... Should devote approximately 3 percent of total Program funding to external evaluation on average contract, or for more impact. Needed to reflect changes in the CDCS, or for more complicated impact.! Evaluations associated with the evaluation team and sustain good health outcomes over time includes the evaluation Statement of (... Managers as well as ensuring USAID accountability for the resources entrusted to....