Reproductive Health for Refugees Consortium

Monitoring and Evaluation Tool Kit

D R A F T for FIELD TESTING

January 2003

Rounded Rectangle: CONTENTSRounded Rectangle: INTRODUCTIONRounded Rectangle: FAQsRounded Rectangle: PATHWAYRounded Rectangle: QUALITATIVERounded Rectangle: QUANTITATIVE

 

Method

·      What is the purpose of this M&E Tool Kit?

·      How is the M&E Tool Kit organized?

·      How can you use this M&E Tool Kit?

·      What data collection methods are included in this Tool Kit?

·      Why is sharing information part of the M&E process?

 

Frequently Asked Questions

·      What do you want to do?

 

Causal Pathway Framework

·      Program Planning: What is it and why is it important?

·      How do we plan effective projects?

·      Using the Causal Pathway to design projects

·      Using the Causal Pathway to monitor and evaluate projects

·      Setting project objectives

 

Addressing the needs of young people

·      Adolescent programming [under construction] (check www.rhrc.org for updates)

 

Method Protocols

Qualitative

·      Observation of staff performance

·      Focus group discussion

·      In-depth interview of key informants

·      (pdf file) Participatory learning and action

Quantitative

·      Training assessment

·      Routine service statistics

·      Client record review

·      Client exit interview

·      Community-based survey

·      Biomarker survey

·      Disease surveillance

Combination methods

·      Facility assessment

·      Situation analysis

·      Quality of care assessment

·      Rapid assessment

 

Sharing Information

·      Overview of sharing information

·      Outline of a journal article

·      Template for a slide presentation (PowerPoint file)

·      Tips for a slide presentation (PowerPoint file)

·      Guidelines for a poster presentation

·      Outline of a report

 

Glossary

 

 

 

What is the purpose of this M&E Tool Kit?

The RHRC Monitoring and Evaluation Tool Kit (M&E Tool Kit) presents a decision-oriented model for program monitoring and evaluation.  With tools from the M&E Tool Kit, managers will find it easier to learn how their program is progressing, and what to do to make it work better.  Use of the M&E Tool Kit will help managers integrate useful evaluation activity into the everyday management of their programs to improve the quality of reproductive health services in refugee situations.

The M&E Tool Kit is tailored specifically to the information and decision-making needs of managers of reproductive health programs serving refugees and other war-affected persons.  Materials in the M&E Tool Kit might also be used by field staff in other sectors, country directors or headquarters personnel – anyone who uses data to make decisions for improving programs.

How is the tool kit organized?

The M&E Tool Kit is a practical guide to monitoring and evaluation of reproductive health programs.  Protocols for how to use a variety of methods for collecting, analyzing and sharing information are presented as with straightforward explanations and examples.  The brief text is accompanied by a CD-ROM that contains examples of model forms.

It is important to plan for monitoring and evaluation at the time the program is being designed.  The M&E Tool Kit includes a description of the Causal Pathway Framework, a systematic guide to program planning.  Once a program design is outlined, the manager will develop a monitoring and evaluation plan.  The remaining materials in the M&E Tool Kit help managers choose the ways of gathering information that make the most sense in their settings.

The methods that can be used for collecting and analyzing information are grouped into three categories: “Qualitative”, “Quantitative”, and “Combination approaches.” 

The methods that can be used for collecting and analyzing information are grouped into three categories:

·         Qualitative methods are used to gain depth of understanding, to learn why people think or behave as they do.

·         Quantitative methods are used to measure how many (or what percent) of people think or behave a certain way.

·         Combination approaches use a mix of qualitative and quantitative methods.

Your information needs, as well as other factors like cost, time and personnel, will determine which methods best suit your project.

How can you use this Tool Kit?

For the manager who is ready to design and implement a monitoring and evaluation plan for her program, the materials in this M&E Tool Kit give step by step guidance on how to identify the information you need and then gather the information so that it is reliable.  This guide recommends that you:

·         Design the program using the Causal Pathway Framework (or, if the project is already going on, map the causal pathway of the project).

·         Identify the information you need to help you learn if the project is progressing as you expect; this is the monitoring and evaluation plan.

·         Follow the steps in this guide to select data collection methods that fit your information needs, and follow the suggestions for gathering and analyzing the information.

·         Make changes to improve your program based on the findings.

·         Communicate useful information from your project to other managers so that they may design and implement good reproductive health projects.

Why is sharing information part of the M&E process?

Sharing information is an integral and important part of project monitoring and evaluation.  Project results should be shared with community members, with colleagues from other organizations working in the site and with a wider professional audience through presenting papers at professional conferences and publishing articles in professional journals.  Sharing information in these ways can help your project and other organizations and is also good for your own professional development. 

Sharing information is a way to let stakeholders know the good work (and mistakes) you are making with your project, and contributes to your organization's credibility and status.  When you make conference presentations or publish articles in professional journals, your project results become part of the permanent record in the humanitarian and development fields from which others can learn.  Sharing information – with any and all interested parties – is an achievement that you can be very proud of.

 

Answers to this set of Frequently Asked Questions provide guidance on what methods may be appropriate to use when gathering information for various purposes.

How do I do an RH Needs and Resources Assessment?

˜ Rapid Assessment Protocol

˜ CDC/CERTI RAP (.pdf files)   Part 1   Part 2   Part 3   Annexes

˜ RHRC Needs Assessment Field Tools

˜ CARE PLA Manual “Embracing Participation” (.pdf files)   Part 1   Part 2   Part 3

How do I design a good RH Project?

˜ Causal Pathway Framework

˜ Program Plan Outline

How do I set objectives?

˜ Causal Pathway Framework

How do I do supervision in an RH Project?

˜ Quality of Care Assessment

˜ Routine Service Statistics

How do I measure achievement of objectives in an RH Project?

˜ Causal Pathway Framework

˜ Program Plan Outline

How do I do an evaluation of an RH Project?

˜ Causal Pathway Framework

¦ Visit the ALNAP website http://www.alnap.org/index.html.  Look for the ALNAP Quality Proforma in HUMANITARIAN ACTION: Improving performance through improved learning, ALNAP Annual Review 2002.

How do an RH program for youth?

˜ Adolescent Programming Protocol [under construction] (check www.rhrc.org for updates)

How do I do medical research on RH? (link to internet resource)

¦ Visit the CDC website http://www.cdc.gov/nccdphp/drh/sata_wshop.htm

How do I do behavioral research on RH? (link to internet resource)

˜ (.pdf file) FHI Guidelines For Repeated Behavioral Surveys In Populations At Risk Of HIV

How do I collect information on:

·   why community members act as they do or believe as they do concerning reproductive health?

Focus group discussion

Individual In-depth interview

(.pdf file) Participatory learning and action

 

·   how many or what percent of community members know, believe or act as they do concerning reproductive health?

Community-based survey

 

·   clients’ in-depth opinions concerning RH services?

                 Focus group discussion
                 Individual in-depth interview

 

·   how many or what percent of clients continue services?

                 Client record review

 

·   the characteristics of the users of RH services?

                 Client record review

                 Client exit interview

 

·   how many or what percent of clients are satisfied with the RH services?

                 Client exit interview

 

·   what RH services are available?

                 Facility assessment

                 Situation analysis

 

·   the quality of the RH services?

                 Quality of care assessment

 

·   how many or what percent of the community use RH services?

                 Community-based survey

 

·   the RH service providers’ knowledge, attitudes, skills and practices?

                 Training assessment

                 Observation

 

 

 

Contributors

The RHR Consortium M&E Program Tool Kit was written by Therese McGinn and Susan Purdin of the Heilbrunn Department of Population and Family Health, Mailman School of Public Health, Columbia University, using materials developed for use with our RHR Consortium colleagues at headquarters and in the 10 field projects associated with the M&E Program.  These projects are:

 

  • American Refugee Committee

o        Strengthening Family Planning in Montserrado, Sinoe and Grand Gedeh Counties, Liberia

o        Strengthening AIDS Prevention in Port Loko, Sierra Leone

  • International Rescue Committee
    • Community Outreach Program for Reproductive Health Promotion and Disease Prevention, Kenema District, Sierra Leone
    • Improving Reproductive Health in Achol-Pii Refugee Camp, Pader District, Northern Uganda
  • JSI Research and Training Institute
    • Improved Family Planning Services – Luena and Huambo, Angola, with International Medical Corps
    • Monitoring and Evaluation of Family Planning and Youth/Adolescent Reproductive Health Projects in Ngara District and Kigoma Region, Tanzania, with UMATI
  • Marie Stopes International
    • Sexual & Reproductive Health Care and Information Activities for the Prevention of Pregnancies among Displaced Adolescents and Host Communities in Cartagena and Barranquilla, Colombia with Profamilia
    • Reducing Maternal Mortality among Repatriated Populations along the Guatemala-Mexico Border, with Marie Stopes Mexico
  • Women’s Commission for Refugee Women and Children
    • Domestic Violence Program for Palestinian Refugees in Lebanon, with Association Najdeh
    • Reproductive Health Services Quality Improvement Project in Mae Tao Clinic, Thailand

 

Additional contributors to the M&E Tool Kit are Sara Casey and Mona Selim during their tenure as Graduate Research Assistants to the M&E Program.

 

Susan Purdin developed the CD-Rom version of this DRAFT Field Test version of the M&E Tool Kit.  January 2003.

 

The M&E Program was supported by a generous grant from the David and Lucile Packard Foundation, and additional support from the Bill and Melinda Gates Foundation.

 

 

Adobe Acrobat Reader 5

Several of the documents included in this M&E Tool Kit are in Adobe Acrobat format (.pdf).  To be able to read these files, you must have Adobe Acrobat Reader software installed on your computer.  An installation file for Adobe Acrobat Reader 5 is included on this M&E Tool Kit CD.  If you do not already have Adobe Acrobat Reader click here to install it on your computer.  NOTE: When asked if you want to run the program from its current location or save it to disk, you must save it to your computer.  I cannot run from its current location.

 

FEEDBACK FORM
DRAFT for Field Testing – Monitoring and Evaluation Tool Kit

We want to hear from you.  Please let us know what you think of this M&E Tool Kit.

Send e-mail to: info@rhrc.org

 

1. Did you encounter any computer or software related problems when trying to use the CD rom?  If so, please explain the problem, and if you were able to solve it, tell us how you did so.

 

 

2. Please tell us what tools you tried. 

What did you use them for? 

How well did they work for you? 

What problems did you have with them?

 

 

3. How can we make this tool kit more useful for you?