Required citation:

FAO. 2018. FAO Technical Guide 3 – Integrating gender into implementation and monitoring and evaluation of cash transfer and public works programmes. Rome. 48 pp.
Licence: CC BY-NC-SA 3.0 IGO.

The designations employed and the presentation of material in this information product do not imply the expression of any opinion whatsoever on the part of the Food and Agriculture Organization of the United Nations (FAO) concerning the legal or development status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The mention of specific companies or products of manufacturers, whether or not these have been patented, does not imply that these have been endorsed or recommended by FAO in preference to others of a similar nature that are not mentioned.

The views expressed in this information product are those of the author(s) and do not necessarily reflect the views or policies of FAO.

ISBN 978-92-5-131038-0
E-ISBN 978-92-5-133243-6 (MOBI)

© FAO, 2018

Some rights reserved. This work is made available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo).

Under the terms of this licence, this work may be copied, redistributed and adapted for noncommercial purposes, provided that the work is appropriately cited. In any use of this work, there should be no suggestion that FAO endorses any specific organization, products or services. The use of the FAO logo is not permitted. If the work is adapted, then it must be licensed under the same or equivalent Creative Commons license. If a translation of this work is created, it must include the following disclaimer along with the required citation: “This translation was not created by the Food and Agriculture Organization of the United Nations (FAO). FAO is not responsible for the content or accuracy of this translation. The original English edition shall be the authoritative edition.

Any mediation relating to disputes arising under the licence shall be conducted in accordance with the Arbitration Rules of the United Nations

Commission on International Trade Law (UNCITRAL) as at present in force.

Third-party materials. Users wishing to reuse material from this work that is attributed to a third party, such as tables, figures or images, are responsible for determining whether permission is needed for that reuse and for obtaining permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user.

Sales, rights and licensing. FAO information products are available on the FAO website (www.fao.org/publications) and can be purchased through publications-sales@fao.org. Requests for commercial use should be submitted via: www.fao.org/contact-us/licence-request. Queries regarding rights and licensing should be submitted to: copyright@fao.org.

Contents

Acknowledgments

Abbreviations and acronyms

Introduction

PART 1
A guide to gender-sensitive implementation of SP programmes

1.1 Why is gender-sensitive implementation important?

1.2 Key dimensions for mainstreaming gender into programme implementation

PART 2
How to plan a gender-sensitive M&E framework and impact evaluation

2.1 Why is gender-sensitive M&E important?

2.2 Definition of key concepts

2.3 How to develop a gender-sensitive M&E framework

2.4. The role of impact evaluations

ANNEX 1
Key resources

ANNEX 2
List of key informants

ANNEX 3
Learning tools

References

Acknowledgments

This document is the result of a joint effort of the Gender and Social Protection teams of the Social Policies and Rural Institutions Division (ESP) of the Food and Agriculture Organization of the United Nations (FAO) prepared by Maja Gavrilovic, Juan Gonzalo Jaramillo Mejia, Susan Kaaria and Natalia Winder-Rossi, with technical guidance from Ana Paula de la O Campos and Pamela Pozarny.

The authors are grateful to Liz Koechlein of FAO, Rebecca Holmes of the Overseas Development Institute (ODI), Professor Maxine Molyneux of the UCL Institute of the Americas, Amber Peterman of the United Nations Children’s Fund (UNICEF), and Raquel Tebaldi of the International Policy Centre for Inclusive Growth (IPCIG) for having provided very useful feedback on draft versions of this document.

Finally, the authors would like to thank Andrea Wöhr who developed the graphic design, Christine Legault and Chiara Villani of FAO who managed the publishing process and Gordon Ramsey for copyediting the document.

Abbreviations and acronyms

BISP Benazir Income Support Programme (Pakistan)

CTP Cash transfer programme

GSPVA Gender-sensitive poverty and vulnerability analysis

EPWP Expanded Public Works Programme (Rwanda)

M&E Monitoring and evaluation

MGNREGS Mahatma Gandhi National Rural Employment Guarantee Scheme (India)

NGO Non-governmental organization

PAT Progress assessment table

PSNP Productive Safety Net Programme (Ethiopia)

PWP Public works programme

SP Social protection

VUP Vision 2020 Umurenge Programme (Rwanda)

WEAI Women’s Empowerment in Agriculture Index

Introduction

Welcome to the FAO Technical Guide 3 – Integrating gender into implementation and monitoring and evaluation of cash transfer and public works programmes. This is the third of three technical guides in the Toolkit on gender-sensitive social protection programmes to combat rural poverty and hunger (see Figure 1).

Purpose of the technical guides

This Toolkit on gender-sensitive social protection programmes to combat rural poverty and hunger is designed to support social protection (SP) practitioners in their efforts to systematically apply a gender lens to SP programmes in ways that are in line with FAO commitments[1] to expand inclusive SP systems for rural populations. The Toolkit is intended to deepen the knowledge and technical skills that practitioners require to integrate gender issues effectively into the design, delivery, and monitoring and evaluation (M&E) of cash transfers and public works programmes (PWPs). The Toolkit has a specific focus on SP’s role in reducing gendered social inequalities and rural poverty and hunger. For a detailed discussion of the rationale and scope of the Toolkit, see Technical Guide 1. The Toolkit is designed for government staff involved in SP programme development and implementation. It may also be of use to FAO gender and social protection focal points in regional and country offices, development partners and SP practitioners in general.

Overview of Technical Guide 3

Technical Guide 3, which provides practical guidance on how to ensure gender is considered in the implementation and M&E frameworks of cash transfers and public works programmes (PWPs), has two parts:

Part 1: A guide to gender-sensitive implementation of
SP programmes.

Part 2: How to plan a gender-sensitive M&E framework.

Technical Guide 3 combines conceptual insights, illustrative examples and practical tools, such as checklists and exercises. It is intended for beginners and women and men with an intermediate level of experience in working in SP programme development and implementation.

How should the technical guides be used?

The technical guides in the Toolkit follow a logical programming sequence: from planning and design to implementation and M&E. Each technical guide can be used as a stand-alone resource or in combination with the other technical guides in the Toolkit. Figure 1 indicates the key issues and topics covered in the technical guides. Users can work at their own pace and according to their personal interests and learning objectives, without direct guidance or facilitation.

The technical guides can be adapted for, and used in face-to-face training workshops with a dedicated facilitator to respond to specific country demands and contexts. The combination of techniques used throughout the technical guides is designed to build on the participants’ existing knowledge and experience.

How were the technical guides prepared?

The technical guides are based on:

a comprehensive review of literature on gender, rural poverty and vulnerability, SP, and gender-sensitive SP programming. The review combined theoretical and conceptual readings with empirical and practitioner-based literature, including impact evaluations and case studies;

expert consultations with key partners within and outside FAO[2];

policy and operational insights into gender-sensitive SP, gathered during a series of webinar events on gender and SP organized by FAO and the IPC-IG;[3] and

a peer review by external experts, academics and senior FAO staff.

1.1 Why is gender-sensitive implementation important?

Technical Guide 2 discusses how gender issues can be integrated into the design of cash transfers and PWPs. To deliver on the gender commitments laid out in the design of the SP programme, it is necessary that programme staff have adequate capacities, and an efficient system is in place for programme implementation.

Evidence suggests that failure to deliver on gender equality goals typically occurs during programme implementation (Holmes and Jones, 2010; Tebaldi, 2016). This failure is partly due to limited gender awareness and insufficient skills among staff to address gender issues, inadequate funding and a lack of clear guidelines on how to implement gender-related design provisions. When SP staff are not effectively sensitized about the importance of gender, or do not have enough resources on hand, addressing gender equality can be perceived as an ‘add-on’ to the primary objectives, and consequently not be given the priority it warrants.

Poor programme delivery can undermine gender equality goals and lead to unintended gender effects.

Other factors that can hamper the programme’s delivery of gender-related outcomes include a lack of political commitment to gender equality, and cultural resistance to embrace and promote gender equality and women’s empowerment at the field level (Holmes and Jones, 2010). Limited participation by rural women and men in programme delivery and governance can also contribute to suboptimal results.

Poor programme delivery can undermine gender equality goals and lead to unintended gender effects (see Technical Guide 1, Part 6). SP programmes should strive to create an institutional and operational environment that enables staff to address gender issues effectively during programme implementation. Effective implementation is typically context- and programme-specific, but in general it includes the following core dimensions:[4]

transparent procedures and adequate resources to deliver regular and timely benefits;

gender-sensitive payment arrangements;

gender awareness and skills to address gender issues among staff and implementation partners;

inclusive programme governance and grievance mechanisms; and

political commitments and support from key stakeholders for gender-sensitive programming.

1.2 Key dimensions for mainstreaming gender into programme implementation

i. Predictability and regularity of
transfers

KEY POINTS

Regular and predictable transfers are key prerequisites for SP programmes to achieve their poverty reduction and gender equality objectives. Unreliable payment of benefits affects poor rural women disproportionately, as women tend to rely more heavily on transfers to manage basic consumption and make productive investments (Holmes and Jones, 2010; FAO expert consultations, 2016). FAO research in Rwanda on the public works component of the Vision 2020 Umurenge Programme’s (VUP) highlighted how recurrent delays in wage payments had a greater adverse impact on food security for households headed by women than for households headed by men. In some cases, women were forced to drop out of the programme and seek employment elsewhere (Pavanello et al., 2016). To address this situation, the Government of Rwanda introduced the Expanded Public Works Programme (EPWP), which has ensured that everyone is offered a guaranteed number of workdays per month and regular payment of wages. This has reduced barriers to programme participation for women who are the head of their household.

Regular and predictable transfers are key prerequisites for SP programmes to achieve their poverty reduction and gender equality objectives.

KEY ACTIONS TO STRENGTHEN THE GENDER FOCUS

Several actions can be undertaken to ensure that programme managers and staff have the capacity to deliver transfers on time, including:

the provision of clear guidance for staff regarding payment schedules;

an easy-to-use system for the accurate compilation of payroll lists;

the dissemination of adequate information to programme beneficiaries about when and how to collect payments and how much they are entitled to. Sensitization and information campaigns need to be tailored to the literacy levels of rural women, and delivered through local information channels that are accessible to women (e.g. radio programmes, village savings and loans groups, and community meetings among others); and

the establishment of a participatory monitoring system that allows programme staff and beneficiaries to report delays in payment to programme officials.

ii. Gender-sensitive payment arrangements

Social transfers can be delivered through two main payment arrangements: (i) the physical distribution of cash, and (ii) electronic payments (also known as e-payments or digital payments). Both approaches have advantages and disadvantages for gender equality and social inclusion. In this section, we discuss their implications and propose actions to make different payment arrangements more gender-sensitive.

KEY POINTS

Physical distribution of cash: In a majority of SP programmes, cash is physically distributed to beneficiaries at a fixed time at certain pay points (e.g. government offices, village committees and post offices) (Cirillo and Tebaldi, 2016). This method may promote the social inclusion of rural women in that it can build links between women and officials at the disbursement points and increase women’s visibility in the community. However, having to travel long distances and queue for long hours to collect the cash can be a substantial impediment for rural women, especially elderly women, or women who are pregnant or nursing. Mobility constraints may also prevent rural women from reaching the disbursement points, particularly if they live in very remote and/or conflict- and hazard-affected areas.

E-payments have several advantages for rural women.

Electronic payments: In a system using e-payments, cash is transferred electronically to a bank, which uses a network of different pay points to disburse cash to recipients on request. These pay points can include bank branches, automated teller machines (ATMs) or local ‘agents’. Programme recipients can obtain the money using smartcards, a personal identification number (PIN) and/or biometric identification, or through mobile phone-operated accounts (Cirillo and Tebaldi, 2016).

E-payments have several advantages for rural women. They have the potential to reduce the times and cost involved in travelling to pay points and waiting times. Recipients can choose when to collect their cash payment, which improves security and gives them more discretion over how the money is used. This is particularly important for rural women as they often lack access to and control over household incomes.

E-payments can also be used strategically to promote financial inclusion and economic advancement of rural women. For example, some programmes, such as the VUP in Rwanda and Ain El-Sira in Egypt, deliver payments through local banks and support women in opening their own bank accounts so that it is easier for them to gain direct access to the transfers. The Familias en Acción Programme in Colombia and the Child Care Grant in South Africa offer additional financial services, such as micro-insurance and microcredit to recipients through savings accounts (Holloway, Rouse and Niazi, 2017). Box 1 considers the positive gender outcomes of using e-payments.

However, low levels of education and financial illiteracy among rural women can limit the uptake of e-payments. FAO research into e-payments in Malawi’s Social Transfer Programme found that older women often faced considerable difficulties in using electronic banking to collect money. These women were unfamiliar with digital technology and mobile services, and depended on younger family members to obtain their cash and tell them when it had become available (FAO, 2015).

KEY ACTIONS TO STRENGTHEN THE GENDER FOCUS

The advantages and disadvantages of different payment arrangements must be assessed to ensure both women and men have efficient and cost-effective access to transfer payments. This can be done during the gender-sensitive poverty and vulnerability analysis (GSPVA) (see Technical Guide 2, Part 1).

Systems for the physical distribution of cash can be modified to become more gender-sensitive by:

providing collection services close to women’s homes to reduce opportunity costs and travel times; and

allowing flexibility in payment intervals and locations to accommodate rural women’s productive work and care responsibilities. This is particularly important for women in pastoral communities with nomadic lifestyles.

E-payments can be made more gender-sensitive by:

offering simplified, low-cost accounts to reduce women’s barriers to accessing payments and increase bank account ownership among women. For example, in the Mahatma Gandhi National Rural Employment Guarantee Scheme (MGNREGS) in India, wages are transferred to basic or ‘no-frills’ bank accounts in return for a fixed fee of 2 percent of the amount paid.

allowing more flexibility in the requirements for official documents (e.g. birth and marriage certificates) required to open bank accounts. SP schemes can link beneficiaries to complementary registration programmes and/or subsidize the costs of obtaining documents. For example, the Juntos Programme in Peru encourages and supports beneficiaries in obtaining identification cards as part of a ‘citizenship and rights’ drive.

adapting e-payment administrative procedures to the financial and technical literacy levels of rural women, and providing women with training support to ensure their effective use of new technologies (see Box 2). In some cases, programmes can provide women with free mobile phones to reduce technological and cost-related constraints.

ensuring that new female customers are treated fairly by banking institutions and have sufficient financial skills to be able understand and trust digital financial services enough to adopt them.

iii. Staff capacity to deliver on gender provisions

KEY POINTS

It is important that all SP programme staff, including programme managers, programme designers, and the frontline staff implementing the programme (e.g. programme promoters, desk officers, caseworkers, and M&E officers) have technical skills and knowledge about gender mainstreaming. Financial resources also have to be set aside to ensure programme commitments related to gender are met. Gender courses for SP staff are often quite limited. For example, Oxfam staff in Sri Lanka’s emergency CTP reported that they had not received any training on gender awareness due to time constraints and the heavy work demands made on them during programme implementation. As a result, staff lacked a common understanding of how to address gender issues in SP programming (Wallace and Chapman, 2011).

Financial resources also have to be set aside to ensure programme commitments related to gender are met.

KEY ACTIONS TO STRENGTHEN THE GENDER FOCUS

To deliver gender-sensitive SP programmes, it is important to make investments in capacity building at all government levels for gender mainstreaming. Several actions can be taken to achieve this.

Carry out a capacity needs assessment to identify technical, functional and financial capacity gaps within the organizations implementing the SP programme. This assessment can be conducted as part of the GSPVA (see Technical Guide 2).

Organize awareness-raising events for all managers, field staff, local community leaders and other relevant stakeholders on gender issues. Training workshops, mentoring and/or e-learning approaches (e.g. digital courses, webinars) can be used to update the skills and knowledge of core programme staff on new gender mainstreaming strategies.

Provide clear, practical guidelines and tools to field staff for integrating gender into SP programmes, and monitoring and evaluating the results.

Establish incentive systems to motivate staff to address gender in their work and offer recognition to staff whose work demonstrates heightened gender awareness.

Hire gender specialists to provide technical support and mentoring to programme staff.

iv. Gender-sensitive institutional structures and governance arrangements

KEY POINTS

Promoting the active participation of rural women and men in programme management and governance structures is another important strategy to ensure the effective implementation of gender-sensitive provisions in the SP programme.

Women should be systematically represented within all the institutional bodies in the programme. A commitment to gender balance within the programme staffing, from steering committees to frontline staff, is a key measure for increasing women’s participation in SP programmes. For example, Mexico’s Prospera Programme elected local promotoras (women community promoters) to serve as a liaison between beneficiaries and programme officials. The promotora’s role included informing participants of their responsibilities and rights, and facilitating interactions with programme officials. The promotoras significantly improved women’s participation (Molyneux and Thompson, 2012).

Ensuring the active participation of rural women and men in the programme governance mechanism (e.g. grievance systems and gender audits) ensures greater accountability in actions designed to achieve the expressed SP and gender goals.

Ensuring the active participation of rural women and men in the programme governance mechanism (e.g. grievance systems and gender audits) ensures greater accountability in actions designed to achieve the expressed SP and gender goals. For example, safe and transparent access to appeals processes can enable women and men to state their concerns and the challenges they face in accessing and benefiting from the programme. Research into SP among indigenous women in Mexico and the Andean region found that beneficiaries initially experienced poor treatment (e.g. closed clinics, long queues and racism). The situation improved, however, after they were made aware of their rights and demanded respectful treatment (Molyneux, 2017). Engaging rural women in social accountability mechanisms can also provide direct opportunities to build their leadership capacities and increase their social and political inclusion.

KEY ACTIONS TO STRENGTHEN THE GENDER FOCUS

Programme staff can undertake the following activities to strengthen women’s participation in programme operations and oversight.

Develop gender-balanced institutional structures. This could include establishing a quota for women’s representation in the governing and operational structures of the programme, such as:

– steering committees,

– the programme coordinating management
unit,

– programme design staff,

– implementing frontline staff, and

– local programme implementing committees.

Include rural women’s representatives in social and gender audits, and promote the establishment of women’s committees at different administrative levels.

Design grievance mechanisms to empower marginalized and excluded women who would ordinarily face systematic obstacles to having their voices heard.

Organize regular awareness-raising events with rural women to inform them of their rights and get them involved in governance and monitoring activities, and social and gender audits.

Use participatory methodologies, including citizen report cards and other feedback mechanisms (e.g. specially designed community meetings, go-to committees and suggestion boxes) (Holmes and Jones, 2010).

Establish open and easily accessible, women-friendly communication and information channels to promote and reinforce activities that foster accountability.

v. Addressing discriminatory socio-cultural norms and promoting progressive change

KEY POINTS

SP schemes are not implemented in a policy vacuum. The socio-cultural context in which programmes operate, and the political economy issues involved, can strongly affect the programme’s implementation and ultimately its success (Holmes and Jones, 2010). The traditional norms, beliefs and attitudes related to gender equality may affect the degree of commitment and the behaviour of the people responsible for designing and implementing SP activities. For example, even when interventions adopt a progressive gender design, the design measures might not be implemented effectively by field staff in areas where customary patriarchal practices prevail, and where objections and opposition to gender equality are strong.

SP schemes are not implemented in a policy vacuum.

In India’s MGNREGS programme, for example, single women have been turned away from labour opportunities because they ‘look too weak’ or lack a male partner to carry out the required tasks. Some local officials have also advised men not to open bank accounts in their wives’ names even though this practice is encouraged in the MGNREGS programme design (Chopra, Kelbert and Iyer, 2013; Holmes and Jones, 2010). For further discussion on ‘gendered political economy’ issues, refer to the ODI Toolkit, How to design and implement gender- sensitive social protection programmes.[5]

Key actions for strengthening gender focus

Socio-cultural attitudes and institutional motivations are never easy to address. Nonetheless, the following actions can contribute to establishing a positive environment for gender-sensitive programme implementation.

Conduct a stakeholder and institutional assessment to gauge the feasibility of, and potential constraints, to promoting gender issues on the ground. See Part 1 of Technical Guide 2 for guidance on how to conduct this analysis.

Develop an advocacy and stakeholder engagement plan for the promotion of gender-sensitive SP. Forging strong alliances between implementing agencies, gender focal points in key ministries and civil society organizations (e.g. producer organizations, cooperatives and women’s groups) is a vital means of ensuring political and community buy-in for the specific programme and for activities to reduce gender equality and empower women in general.

Organize a mobilization and awareness-raising campaign targeted at community members to increase their understanding of the programme’s gender dimensions, its objectives, rationale, evidence and expectations. Use this campaign to foster support and cooperation from the community during implementation (see Box 4).

Engage men and boys in sensitization and awareness-raising events to ensure their buy-in and commitment for the promotion of gender equality. These activities also provide the opportunity to gauge men’s views on how gender affects their experiences of poverty and vulnerability, their access to SP programmes and ultimately their well-being. This approach has been adopted within the Juntos Programme in Peru and the Bolsa Familia and Promundo Programme in Brazil.

Include gender-sensitive indicators to measure progress over time, from tracking and monitoring of budget allocations through to the implementation of gender-related programme provisions (see Part 2 on M&E).

Summary questions

What type of challenges may hinder the successful implementation of gender-sensitive programme features?

How can these implementation challenges be overcome?

Exercise 1: Assessing the opportunities and challenges for adopting gender-sensitive programme implementation (SEE ANNEX 3)

2.1 Why is gender-sensitive M&E important?

a. Rationale for gender-sensitive M&E

Many SP programmes aim to mainstream gender into their design, but few track gender-related outcomes through a systematic M&E process. Except for tracking the number of women beneficiaries, the collection of basic sex-disaggregated data is generally weak in M&E of SP interventions (IEG, 2014). However, SP programmes are likely to affect individual men and women members of the household differently, and influence existing gender dynamics for better or worse (see Technical Guide 1). Effective integration of gender into existing M&E frameworks is critical for assessing the differential impacts of programmes on rural women and men, and making adjustments to the programmes so that they reach them equitably.

b. The purpose of gender-sensitive M&E

Gender-sensitive M&E allows programme implementers to assess how, and to what extent, SP interventions affect gender equality, and achieve positive results for both women and men. More specifically, M&E helps to:

assess gender-related changes in status, roles and capacities of women and men affected by the programme over time;

analyse the participation of women and men in the programmes, and their access to and control over benefits;

measure the economic and social impacts, both positive and negative, of the programmes on rural women and men; and

assess how specific programme design and implementation processes promote (or impede) gender equality and women’s empowerment, and identify good practices that drive positive results.

There are three main motives for undertaking gender-sensitive M&E.

i. To learn from and modify existing programmes

Programmes can assess the extent to which they are meeting their gender-equality objectives, and identify any necessary adjustments in programme activities to improve performance and outcomes.

For example, the monitoring of Tajikistan’s Second Public Employment for Sustainable Agriculture and Water Management Project’s implementation phase showed there was very low participation by women. Constraints to women’s participation included the physical demands of the work; the reluctance among Tajik men in some communities to allow women to engage in the type of work offered; the women’s time poverty due to their household responsibilities; and the lack of valid identity cards among women. Because of the findings obtained through the monitoring process, measures were implemented to address these constraints and increase female employment (IEG, 2014).

ii. To inform the design of new programmes

Findings from evaluations can be used to inform the initial programme design. Pilot projects can be established explicitly to test an intervention’s design options before the final design is determined or the project is scaled up. In Burkina Faso, for instance, the Nahouri Cash Transfer Pilot Project used a randomized experimental evaluation to determine how the gender of the recipient of the benefit affected poverty outcomes. The findings were then used to inform the targeting criteria of the newly expanded programme.

iii. To inform policy dialogue on gender mainstreaming

Gender-sensitive indicators and data are important policy tools that can be used to advocate for gender equality and gender-sensitive SP programming.

Over the years, organizations, such as UNICEF, FAO and ODI, have developed important methodologies and generated information on the gender-related impacts of SP programmes. This research has been instrumental in influencing policy and programming debates around the role of cash transfers and public works in empowering rural women and men in a number of countries, including Ethiopia, Rwanda and Zambia. For example, the findings of FAO research in Rwanda into the unintended effects of the VUP’s public work component on rural women’s time use and workloads resulted in a government commitment to redesign the original VUP so that it more effectively addressed the needs of labour-constrained households (Pavanello et al., 2016).

2.2 definition of key concepts

a. What is M&E?[6]

Monitoring is the regular collection and analysis of data, usually collected by programme staff, to track how the programme is proceeding and determine whether it is on target or not (Perrin, 2012).

evaluation is the systematic and periodic assessment of a programme to establish whether the goals and objectives planned by the intervention have been achieved (Perrin, 2012). Evaluation can provide more comprehensive information than monitoring about what is taking place, why, and whether it is appropriate or not, and can then provide guidance for future programme direction. Evaluations typically use data collected through monitoring as a starting point.

Evaluation is the systematic and periodic assessment of a programme to establish whether the goals and objectives planned by the intervention have been achieved.

The most frequently used types of evaluations in SP interventions are targeting accuracy assessments, process evaluations, outcome (also known as effectiveness) evaluations and impact evaluations (Perrin, 2012)[7]. In this module, we refer mainly to impact evaluations and outcome evaluations.

b. The role of the participatory M&E

There are many different approaches to monitor and evaluate programmes. Participatory M&E, which is particularly important for gender equality and women’s empowerment, is a process through which various programme stakeholders engage in monitoring or evaluation, share control over the content and results of the M&E activities, and engage in identifying or taking corrective action in response to the findings (Ayers et al., 2012). Participatory M&E allows the process to be ‘owned’ by the group, rather than being imposed upon them. It can be designed explicitly as a strategy to empower the poorest and most marginalized women and men to take part in the programme assessment and voice their perspectives.

Participatory M&E can be designed as a strategy to empower the poorest and most marginalized women and men to take part in the programme assessment and voice their perspectives.

The M&E staff should adapt participatory methods that are based on the programme’s objectives and goals, the socio-cultural context, the resources they have and the individual needs of stakeholders. It is vital to ensure that stakeholders participating in M&E can do so in an institutional setting in which they feel comfortable providing their inputs into discussion and can trust that their feedback will be properly acknowledged and acted upon. See the in-depth case study in Rwanda in section 2.4 for discussion on how the participatory research tools were applied to solicit beneficiaries’ perspectives in the impact evaluation of the VUP programme in Rwanda.

2.3 How to develop a gender-sensitive M&E framework

This section explains how to prepare a gender-sensitive M&E framework.

i. What is an M&E framework?

An M&E framework is a table that describes what changes need to be measured, how and by whom, in order to determine if the programme is on track and being successful. It is used as a tool by the implementing organization to keep track of progress being made to reach programme targets. It should be distinguished from impact evaluations, which are usually done by independent experts and involve more complex evaluation activities to identify the causal relationships of the programme (see section 2.4). The M&E framework defines: (i) what changes to measure, (ii) the specific indicators to track the change (iii) the source of data, frequency and responsibility for data collection, and (iv) the reporting of data.

A gender-sensitive M&E framework should embed gender into all of these elements, rather than treat gender as a separate component. Sex-disaggregated data needs to be collected and reported for all expected programme results. This is crucial even for SP interventions that do not explicitly strive to have an impact on gender equality and women’s empowerment.

ii. The importance of having a good logical framework

A robust logical framework is the basis for developing an M&E framework (McCord, Holmes and Harman, 2017). Expected impacts, outcomes, outputs and activities should be expressed explicitly, and specific indicators for each result should be identified to track and assess progress. Gender needs to be considered at each stage of the results chain. If it is not, sex-disaggregated results will not be systematically measured. Table 1 provides a simplified example of a logical framework for a PWP and CTP.[8]

iii. Key steps in planning an M&E Framework

There are four key steps to follow:

Each step is illustrated in the following section using the example of a PWP. The same basic principles also apply to CTPs.

STEP 1. DECIDING WHAT TO MEASURE

The first step in designing an M&E framework is deciding what gender-related changes to monitor and evaluate. As discussed above, the selection of key issues needs to be aligned with the programme’s objectives.

The information needs of key stakeholders, including beneficiaries, programme staff and policy-makers can also inform decisions about what should be measured. A simple stakeholder analysis can be undertaken to map their specific information requirements (see Box 5).

As a general rule, it is important to prioritize information requirements and focus on the most important information in line with staff capacity and budgetary resources. Introducing gender-relevant data can be a great burden to an already overloaded administrative system. It can overwhelm the staff who are tasked with collecting the data and lead to suboptimal results.

Broadly speaking, the M&E framework can focus on two areas of analysis:

a. process-related monitoring
and

b. an assessment of programme results.

a. Process-related monitoring (activities and outputs)

At a very basic level, staff should track whether programme activities, including gender-specific provisions articulated in the programme design, have been implemented as intended, and if these activities are leading to expected outputs. Information can be collected about what is being done in the programme and how; the quality of benefits and services provided; and whether the benefits reach the poor female and male beneficiaries as intended. Box 6 suggests some questions that can assist in the planning process.

A simple tool, the progress assessment table (PAT) can be used to guide the systematic collection of data for process monitoring. When filling out the PAT, each programme-related activity (including gender-sensitive provisions) must be mapped out and entered into the table (see Table 2). The PAT records who should participate and benefit from the programme activities as opposed to who is actually reached and benefits. This information must be disaggregated by sex and other relevant socio-economic variables. Analysis of PAT data may involve regular staff discussions that take into consideration what is working and what is not, and any unanticipated shortfalls or opportunities that can be addressed by revising the original programme plans.

A simple tool, the progress assessment table (PAT) can be used to guide the systematic collection of data for process monitoring.

Data from the PAT can also be used to inform programme evaluation. With the PAT, staff can collect basic data on how the operational features and programme implementation are promoting (or impeding) gender equality and women’s empowerment. This data can later help programme evaluators to identify specific factors and mechanisms that can contribute to positive gender outcomes and promote equal access to benefits for rural women and men.

b. An assessment of programme results (outcomes and impacts)

Once the programme activities and outputs have been defined, the next step is to select the specific results – the outcomes and impacts – to measure. Outcomes refer to specific changes in the attitudes, behaviors, and/or status of programme beneficiaries that are expected to occur as a result of the programme activities and outputs. Impacts describe the overarching effects that the intervention will have at some point further into the future.

The M&E should assess and report on gender-related outcomes and impacts. It needs to capture how men and women are positively or negatively, affected by the project.

The M&E should assess and report on gender-related outcomes and impacts.

The M&E framework typically collects data on the ‘intermediate’ outcomes that the programme is trying to achieve in relation to the target population. Some impacts may be captured through routine M&E data collection. More complex impact evaluation techniques will be required to infer the programme’s impacts on beneficiaries and non-beneficiaries. For example, changes in time use, women and men’s workloads, and access to cash savings can be detected during programme implementation. Other type of impacts (e.g. women’s empowerment, or reduction in gender gaps in employment) may take a longer time to materialize, and can only be determined after the programme has ended.

A participatory process with stakeholders can be used to choose and define the specific results to be measured. A first step, involves identifying simple, positive outcome statements that express the desired results from the SP programme. They describe what a ‘successful’ programme looks like for different types of stakeholders in ways that matter to them. They can also be used in consultations with beneficiaries to ensure their input is taken into account in the formulation of key outcomes. At least one outcome statement needs to be defined for each programme output (see Table 3). Once the outcome statements are determined, change is measured through the use of indicators (see next section).

STEP 2: SELECTING INDICATORS

Once the outcome statements are defined, the next step is to choose and identify gender-sensitive indicators to measure the success of the programme.

a. Formulating indicators

Indicators are pieces of information that help programme staff to measure changes that have occurred over time. Indicators provide a way to measure achievement and reflect on the changes that may occur as a result of an intervention. Their usefulness lies in their ability to point to changes in the changes in gender roles and welfare status of women and men over time, and measure whether gender equality is being achieved (FAO, 2015). Indicators should be developed for all levels of result to monitor progress with respect to inputs, activities, outputs, outcomes and goals.

There are a number of things to keep in mind when developing indicators.

There should be at least one indicator for each activity, output, outcome and goal in your logical framework (see Table 4).

As much as possible, indicators should be disaggregated by sex, and other variables that are relevant to the programme (e.g. age, ethnicity, marital status).

Phrase indicators in such a way that results for women and men can be compared. Without comparable data for women and men, it is not possible to assess whether an initiative has been effective at targeting and benefiting one group as opposed to another, or none.

Choose a small number of indicators to make data collection more feasible.

Ensure data is regularly analysed to see what progress is being made.

Use a combination of qualitative and quantitative measures to get a full picture of the changes that are occurring.

b. The importance of a baseline data and targets

A starting point or a baseline is needed to successfully use indicators to measure change. A baseline is a measure of the situation before the programme starts. This information can be collected through qualitative research and/or quantitative survey. Gender-sensitive baseline data can be collected through a GSPVA (see Technical Guide 2). It is difficult to measure the progress being made as a result of the programme if a baseline has not been established prior to the programme’s implementation.

For each indicator, a realistic target can be defined, against which progress can be measured. A target refers to what you hope to achieve by the time the programme ends. Baseline data and targets should match the activity, outcome, output or goal indicators (see Table 4).

STEP 3: SPECIFYING THE DATA SOURCE, THE FREQUENCY OF DATA COLLECTION AND THE PARTIES RESPONSIBLE FOR COLLECTING DATA

Once the indicators have been selected and formulated, the next step involves identifying the source of data that will be used to measure progress. The M&E framework also needs to specify the frequency with which of the data should be collected and who will have responsibility for collecting the data. Box 8 presents key questions that can be considered during this stage of the planning process.

a. Data source: using the existing or new data

The first task in this step is determining where to find the data for the sexdisaggregated indicators. Data can be collected through an existing source, or a new one. There are numerous sources of secondary data, including the programme itself, government bodies and academia. These sources can provide a range of documentation (e.g. official statistics, national account data, national household surveys, and programme management and appraisal documents).

b. Data collection methods to generate new data

For some indicators, new data must be collected through primary research. There is a wide range of quantitative and qualitative methods that can be employed to generate new data. Each method has specific role to play in understanding the progress being made in addressing gender-related issues.

Quantitative methods, such as surveys are important to help discern general patterns about the populations of interest. Household surveys are typically used in SP evaluations to collect household and individual-level data. Individual-level data allows for comparison between men and women, and can help capture gender differences in beneficiaries’ perspectives of the programme in terms of service quality and constraints in accessing services. This information can also capture trends in men’s and women’s intrahousehold decision-making, asset ownership, agricultural production, income and other topics.

Quantitative methods, such as surveys are important to help discern general patterns about the populations of interest.

Surveys to collect individual-level data can be designed as stand-alone questionnaires, or existing survey modules can be modified to integrate gender-related questions.

Qualitative methods can shed light on gender issues that are difficult to measure through statistics or standard survey questionnaires. For example, qualitative methods may enable M&E staff to better understand different perceptions held by women and men in the household with regard to ‘intangible’ aspects of well-being, such as status, self-esteem, experience of vulnerability and risk, or what it means to be ‘empowered’ or ‘disempowered’ in a given society. Qualitative methods may be needed to gain a better understanding of why and how certain changes related to intrahousehold dynamics happened during the programme.

Qualitative methods can shed light on gender issues that are difficult to measure through statistics or standard survey questionnaires.

There are a variety of different qualitative methodologies that can be used to gather information (e.g. interviews, participatory rural appraisal tools and ethnographic methods). Box 9 highlights some commonly used methods for collecting qualitative data. For a discussion of participatory tools, see the in-depth case study from Rwanda at the end of this section.

Qualitative and quantitative methods have both advantages and limitations (see Table 5). Given the broad range of issues that the M&E framework needs to capture, a mixed-set of methodologies may be required to collect multidimensional data (see Box 12 for an example).

The data collection process itself must be gender-sensitive. For example, M&E officers can consider the following actions.

Identify who should be interviewed: Data will need to be collected at the individual level (e.g. interviews with both men and women in the household) in order to make gender comparisons, and capture the voices and perceptions of both women and men. For some aspects, it is also useful to collect data at the household level and include female- and male-headed households as the unit of analysis. This can enable an assessment of gender gaps, and allow for comparisons to be made in household aggregate changes.

Train staff in gender-sensitive research methodologies: The collection and analysis of data can be influenced by gender biases that arise from a lack of gender awareness, or the cultural attitudes and social practices of the staff involved in the process. M&E officers and enumerators must be trained on basic gender concepts and gender-sensitive data collection methods and analysis. This would include providing them with guidance on ethical research protocols and measures for protecting the identity and privacy of respondents.

Invest time and resources to conduct proper gender analysis: The collection of sex-disaggregated data can be done relatively easily with forward planning, and does not necessarily require gender specialists. However, the collected data should be properly analysed from a gender perspective. This requires deeper thinking and the formulation of appropriate questions to reach a more profound level of analysis. This may require that gender specialists become involved in the process.

Address the barriers that poor rural women face in participating in the research: This can be done by using men and women researchers; finding activity locations where women feel comfortable; determining the time of day most suitable for women; and using sex-segregated focus group discussions.

c. Frequency and responsibility: when and how often should data be collected and by whom?

Programme officers should determine how often the indicators will be measured. Some indicators may require regular data collection. For example, data on the implementation of job quotas can be gathered after each targeting cycle. However, data on individual employment outcomes and income may need to be collected yearly, at the end of the farming season. Activity and output indicators are usually easier to measure with greater frequency than outcome and impact indicators.

Activity and output indicators are usually easier to measure with greater frequency than outcome and impact indicators.

Finally, the M&E framework needs to specify the person or entity responsible for the collection of data for each result and indicator. As a general rule, the programme staff who are engaged in routine monitoring typically use activity and output indicators to measure progress. Independent evaluation experts focus on assessments that use outcome and impact indicators.

STEP 4: REFLECTING ON THE RESULTS AND REPORTING THE DATA

The final step in the M&E process is specifying how the data will be reported and to whom. The results of M&E activities need to be consistently reported in the project/ programme completion documents or in supervision reports, and shared with key stakeholders. This is needed to enable a regular discussion on the appropriate actions to be taken in light of the findings. A continuous process of learning and reflection on the results should be encouraged among various stakeholders. For example, quarterly reflection meetings can be organized among staff and key decision-makers to discuss the findings collected during M&E, and adjust the programme documents if necessary. These meetings need to be properly planned and budgeted for in M&E work plans.

Feedback loops and discussion forums can also be established as a means of sharing findings with a broader audience, including the general public, to promote a wider commitment to gender equality. Evidence-based advocacy campaigns can contribute to building links among different institutions; create opportunities for learning lessons between government departments, NGOs, donors and international agencies; and promote gender-sensitive SP programming. Table 6 presents a consolidated M&E framework.

2.4. The role of impact evaluations

This final section briefly explores the importance of impact evaluations, and highlights a few issues to keep in mind when designing a gender-sensitive impact evaluation.

IMPORTANT NOTE: In this section we do not propose to provide a comprehensive guide on how to design and undertake impact evaluations. There are many excellent resources that can be consulted for this purpose (see Annex 1). We simply note here several issues that evaluators need to consider when designing gender-sensitive impact evaluations of a SP programme.

i. What is an impact evaluation?

As mentioned already, M&E data is typically collected only on the beneficiaries of the programme, and not on non-beneficiaries. Consequently, M&E data cannot be used to infer programme impacts. An impact evaluation is required to do that. An impact evaluation is a systematic study that uses econometric and qualitative research techniques to infer the causal impact of a programme on its beneficiaries. Impacts are determined by comparing the outcomes for programme participants with the outcomes of non-beneficiaries (Rogers, 2014). Impact evaluations can attribute the changes in participants’ well-being to specific aspects of a SP programme. Impact evaluations can be used to generate robust evidence of how cash transfers and PWPs empower women and men, and what design components are needed to achieve gender equality outcomes.

An impact evaluation is a systematic study that uses econometric techniques to infer the causal impact of a programme on its beneficiaries.

ii. Considerations to keep in mind when planning an impact evaluation

There are four main steps that should be considered when planning a gender-sensitive impact evaluation.

The first step is to formulate the overarching gender-sensitive evaluation questions to guide the assessment. For examples of these types of questions from a SP programme in Ghana, see Box 10.

The second step is to articulate a theory of change. A well-developed theory of change is an essential tool for guiding the evaluation process and the collection of data. A theory of change explains how programme activities are expected to produce a series of immediate results (outputs) that contribute to achieving intermediate outcomes and ultimately the intended impacts. See the in-depth case study from Rwanda at the end of this section.

The third step is to decide on specific design techniques to assess impacts. Impact evaluations can employ randomized or experimental, quasi-experimental, or non-experimental designs. Experimental designs construct a treatment and control group through random assignment. Quasi-experimental designs construct a comparison group through matching, regression discontinuity, propensity scores or another means. Non-experimental designs look systematically at whether the evidence is consistent with what would be expected if the intervention was producing the impacts, and also whether other factors could provide an alternative explanation (see Rogers, 2014).

Finally, for impact evaluations, as with standard M&E activities, researchers need to identify data collection methods to build credible evidence for the impacts. Impact evaluations increasingly use mixed-method data collection approaches to enable a more complete understanding of programme results and how they were achieved. A mixed-method evaluation refers to the systematic integration of quantitative and qualitative methodologies and methods at some or all stages of an evaluation (Perrin, 2012). A mixed-method approach helps to overcome the weaknesses inherent in each approach when used in isolation. It also increases the credibility of the evaluation’s findings because the information is ‘triangulated’ from different data sources.

Qualitative and quantitative methods can be used in a mutually reinforcing way. For example, qualitative research can be conducted first to generate preliminary data that can help frame the questions for more structured, larger-scale surveys. Qualitative research can also follow the quantitative assessment to better delineate the mechanisms that drive changes in a community and explain the quantitative impacts of an intervention (see Box 11).

iii. Leveraging evidence to influence policy and programming

Impact evaluations of conditional cash transfers have tended to focus only on a few gender outcomes, such as education, nutrition and health. A more recent wave of studies has examined a broader range of outcomes, related to women’s employment and income generation, access to resources, and gender-based violence (IEG, 2014). Using the Women’s Empowerment in Agriculture Index (WEAI) methodology,[9] FAO (2015) has examined the impacts of SP interventions on complex and elusive topics, such as women’s empowerment, their economic advancement and bargaining power and agency.

A greater understanding of gender dynamics in SP programmes will enhance the ability of these programmes to be inclusive and deliver benefits to all individuals, especially the most vulnerable.

Increasing investments in the future to develop meaningful approaches to measure outcomes and changes in women’s empowerment will be critical for making SP more effective in reducing poverty and fostering gender equality. However, simply collecting data through M&E and impact evaluations is not enough to increase commitments to undertake more gender-sensitive programming. Evidence also needs to be reported and communicated more systematically to stimulate learning and discussion among policymakers and other relevant actors working in SP and gender. A greater understanding of gender dynamics in SP programmes will enhance the ability of these programmes to be inclusive and deliver benefits to all individuals, especially the most vulnerable.

Summary questions

What is M&E?

Why is participatory and gender-sensitive M&E important?

What are the key steps in undertaking gender-sensitive M&E?

Exercise 2: Developing a plan to conduct gender-sensitive M&E (SEE ANNEX 3: LEARNING TOOLS)

In-depth case study: FAO mixed-methods impact evaluation research in Rwanda

FAO conducted a case study on the public works component of the VUP in Rwanda to gain better understanding of how SP programmes can promote rural women’s empowerment as a part of a strategy for rural poverty reduction. A set of three hypotheses were developed and tested to understand the impact of public works on women’s economic advancement, power and agency:

The economic advancement hypothesis: SP programmes (in this case, VUP public works) will promote the economic advancement of women by increasing their productive resources (e.g. incomes and access to credit and savings) and improving skills and employment opportunities.

The power and agency hypothesis: SP (VUP public works) will strengthen women’s power and agency by increasing their bargaining power within the household and wider community. This will increase women’s self-confidence, their ability to engage in social networks, and their participation in decision-making in the public arena.

The operations hypothesis: Operational and design features of SP programmes (VUP public works) will ensure women’s equal access to benefits and build linkages with community-based services and livelihood interventions that will promote gender equality and women’s economic empowerment.

To test these hypotheses, the study employed a mixed-methods approach for collecting data.

The quantitative component included a household survey and individual surveys of participant households from the VUP locations, and households from a comparable community. The individual questionnaire was informed by the WEAI methodology, which measures empowerment, through aspects such as decision-making power and control over assets related to agriculture. The WEAI individual-level module covered information on the individual’s contribution to household decision-making around production and income generation; access to and control over productive assets and credit; access to agricultural extension or other training; individual leadership and influence in the community; and time use. The household survey also contained a module specific to aspects of the VUP programme design.

The design of the qualitative study was based on three methods: focus group discussions, semi-structured key informant interviews, and in-depth household case studies. In each research location, focus group discussions were conducted with the women and men who were VUP public works beneficiaries; women and men non-beneficiaries (including those eligible for the PWP); opinion leaders; and community members who contributed to VUP implementation. The key informant interviews included interviews with administrative officers; members of the administrative structures that contribute to the VUP programme (e.g. the local administrative development agency at the national level and VUP managers at the sector level); social sector professionals, including teachers, social welfare workers, and health and agricultural extension workers; and representatives of the National Council of Women, village savings and loans associations and other groups.

A number of participatory tools were also applied during the focus group discussions to collect information.

A community well-being analysis was used with opinion leaders to: (i) understand the socio-economic status of the community and the perceived gender differences between wealth groups; and (ii) understand the perceptions of the effectiveness of the VUP targeting.

A mapping exercise was undertaken to understand the differences between men and women’s access to and control over household resources, including productive assets, natural resources and family labour.

A seasonal calendar, combined with a description of the gendered division of labour, was used to explore how seasonal variations affect key agricultural and non-agricultural activities, and the gendered division of tasks among family members.

Organizational and group profile mapping (Venn diagrams) were used to understand the characteristics of the institutions, organizations and groups active in the community and their linkages; and their importance and value for men and women in the community.

Overall, the study found that the VUP only marginally promoted the economic advancement and bargaining power and decision-making of women beneficiaries. Operational issues and implementation challenges were found to pose significant obstacles to women’s empowerment. The study proposed several recommendations to improve the design and delivery mechanisms of the VUP, including the targeting of beneficiaries; sensitizing the activities to intrahousehold gender dynamics; and incorporating basic gender indicators in the management information system to measure the progress in women and men’s economic empowerment and changes in their productive livelihoods. The findings of the assessment have subsequently informed a redesign of the VUP programme to enhance its gender-sensitive design provisions and improve poverty outcomes.

Source: Pavanello et al., 2016; Warring and de la O Campos, 2016.

ANNEX 1
Key resources

FAO resources on social protection and gender

From Protection to Production (PtoP) project research publications on social protection: www.fao.org/economic/ptop/publications/reports/en/

FAO Gender publications

www.fao.org/gender/resources/publications/en/

See in particular:

de la O Campos, A.P. 2015. Empowering rural women through social protection. Rural transformations. Technical Papers Series No. 2. Rome, FAO. www.fao.org/3/a-i4696e.pdf)

FAO. 2013. Gender Focal Points: Introductory Training Module, Rome.

FAO. 2017. Social Protection Framework: Promoting Rural Development for All. Rome.

www.fao.org/publications/card/en/c/87c92abf-466d-4b01-b8ec-57ca7317ed57/)

The Innocenti and Transfer Project publications on cash transfer impact evaluations

https://transfer.cpc.unc.edu/?page_id=310

See in particular:

Davis, B., Handa, S., Hypher, N., Winder-Rossi, N., Winters, P. & Yablonski, J., eds. 2016. From Evidence to Action: The Story of Cash Transfers and Impact Evaluation in sub-Saharan Africa. Oxford, UK, FAO, UNICEF and Oxford University Press. www.fao.org/publications/card/en/c/0b24b92e-5254-4d06-a317-84ccd6babe23/).

Rogers,P., (2014), Overview of Impact Evaluation, UNICEF. http://devinfolive.info/impact_evaluation/img/downloads/Overview_ENG.pdf

International Labour Organization (ILO) resources on social protection

www.ilo.org/global/topics/social-security/lang--en/index.htm

See in particular:

ILO. 2015. World Social Protection Report 2014/15. Building economic recovery, inclusive development and social justice. Geneva, Switzerland. www.ilo.org/global/research/global-reports/world-social-security-report/2014/lang--en/index.htm)

Tanzarn, N. & Gutierrez, M.T. 2015. Illustrated Guidelines for Gender- responsive Employment Intensive Investment Programmes. Geneva, Switzerland, ILO.

www.ilo.org/global/topics/employment-intensive-investment/publications/WCMS_459976/lang--en/index.htm

World Bank resources on social protection and gender

https://openknowledge.worldbank.org/discover?query=social%20protection

See in particular:

IEG (Independent Evaluation Group). 2014. Social safety nets and gender. Learning from impact evaluations and World Bank projects. Washington, DC, World Bank. https://ieg.worldbankgroup.org/evaluations/social-safety-nets-and-gender-learning-impact-evaluations-and-world-bank-projects)

World Bank. 2012. World Development Report 2012: Gender Equality and Development. Washington, DC. https://openknowledge.worldbank.org/handle/10986/4391)

World Bank. 2015. The state of social safety nets 2015. Washington, DC. https://openknowledge.worldbank.org/handle/10986/22101)

Institute of development Studies (IdS) resources on social protection and gender

www.ids.ac.uk/idsresearch/centre-for-social-protection

ODI resources on social protection and gender

https://www.odi.org/search/site?f[0]=im_field_programme%3A38&f[1]=bundle%3Aresource&solrsort=ds_sort_date%20desc

Social Protection and Human rights platform

http://socialprotection-humanrights.org

ANNEX 2
List of key informants

External experts

Deepta Chopra

Research Fellow, IDS

Valeria Esquivel

Senior Research Fellow, UNRISD (at time of interview, Valeria was based at UNRISD)

Rebecca Holmes

Research Fellow, ODI

Nicola Jones

Research Fellow, ODI

Anna McCord

Independent Consultant

Amber Peterman

Impact Evaluation Researcher, UNICEF Office of Research – Innocenti

Pamela Pozarny

Senior Research Fellow, Governance and Social Development Resource Centre (GSDRC)/FAO (at time of interview, Pamela was based at the GSDRC)

Rachel Sabates-Wheeler Research Fellow, IDS

FAO Focal Points

Mohamed Ag Bendech

SP Focal Point, FAO Regional Office for Africa

Claudia Brito

Gender Focal Point, FAO Regional Office for Latin America and the Caribbean

AlHassan Cisse

SP Focal Point, FAO Regional Office for Africa

Abdurazakova Dono

SP Focal Point, FAO Regional Office for Europe and Central Asia

Pablo Faret

SP Focal Point, FAO Regional Office for Latin America and the Caribbean

Alfredo Impiglia

FAO Regional Office for the Near East

Flavia Lorenzon

SP Focal Point, FAO Regional Office for the Near East

Clara Park

Gender Focal Point, FAO Regional Office for Asia and the Pacific

Ishida Tomomi

SP Focal Point, FAO Regional Office for Asia and the Pacific

ANNEX 3
Learning tools

Exercise 1:
Assessing the opportunities and challenges for adopting gender-sensitive programme implementation

LEARNING OBJECTIVE: Assess the scope for, and challenges to, implementing gender-sensitive design commitments in daily work.

Timing indication

30 minutes (group breakout discussions)

20 minutes (presentation of findings)

20 minutes (plenary discussions)

Notes to the facilitator

Using a checklist of key strategies for mainstreaming gender into SP implementation systems (see discussion in Part 1) encourage participants to reflect on the scope for, and challenges to, addressing gender issues more meaningfully in their daily work. If they are already working on gender-sensitive programming, participants can discuss how gender is integrated (or not) in various stages of the programme implementation cycle; what the strengths and shortcomings associated with this approach are; and how these challenges can be overcome.

Exercise 2: developing a plan to conduct the gender-sensitive M&E (breakout groups and plenary)

LEARNING OBJECTIVE: To provide an opportunity to use the knowledge and guidance to develop an M&E plan.

Timing indication

60 minutes (group breakout discussions and development of posters)

20 minutes (presentation of findings)

20 minutes (plenary discussions)

Notes to the facilitator

Divide trainees into small teams to work on the following tasks and develop posters to present the main findings of consultations:

– Task 1: Discuss the main objectives and purpose for undertaking gender-sensitive M&E in the existing SP programme.

– Task 2: Drawing on the broad thematic areas of analysis presented in section 2.3, develop a list of relevant issues and topics to monitor and evaluate. Participants should refer to Step 1 as a guide for their discussion.

– Task 3: Decide on ‘domains of change’ and formulate ‘outcome statements’. For each ‘outcome of change’ identify a sex-disaggregated indicator. Use Tables 3 and 4 respectively to guide discussion.

– Task 4: Depending on the type of indicators identified in step 3, select the appropriate data collection methods and tools to assess progress.

– Task 5: What research dissemination approaches and strategies would be suitable to communicate the M&E findings effectively? Discuss who are the most relevant actors to lead/engage in the research communication process.

References

Ayers, J., Anderson, S., Pradhan, S. & Rossing, T. 2012. Participatory Monitoring, Evaluation, Reflection and Learning for Community-based Adaptation: PMERL Manual. A Manual for Local Practitioners. Geneva, Switzerland, CARE International. (also available at https://www.care.org/sites/default/files/documents/CC-2012-CARE_PMERL_Manual_2012.pdf)

Bonilla, J., Castro Zarzur, R., Handa, S., Howlin, C., Peterman, A., Ring, H. & Seidenfeld, D. 2017. Cash for women’s empowerment? A mixed-methods evaluation of the Government of Zambia’s Child Grant Program. World Development, 95: 55-72. (also available at www.sciencedirect.com/science/article/pii/S0305750X15311682)

Cirillo, C. & Tebaldi, R. 2016. Social Protection in Africa: Inventory of Non- Contributory Programmes. Brasilia, IPC-IG & UNICEF. (available at http://www.ipc-undp.org/joint-publications-004-social-protection-africa-inventory-non-contributory-programmes)

Chopra, D., Kelbert, A.W. & Iyer, P. 2013. A Feminist Political Economy Analysis of Public Policies Related to Care: A Thematic Review. Evidence Report No. 9. Brighton, UK, IDS. (also available at https://www.ids.ac.uk/publication/a-feminist-political-economy-analysis-of-public-policies-related-to-care-a-thematic-review)

de Groot, R. 2016. Ghana LEAP 1000 Impact Evaluation: Overview of Study Design. Innocenti Research Brief. Florence, Italy, UNICEF Office of Research. (also available at www.unicef-irc.org/publications/851-ghana-leap-1000-impact-evaluation-overview-of-study-design.html)

Emmett, B. 2012. Electronic payment for cash transfer programmes: Cutting costs and corruption or an idea ahead of its time? Pension Watch Briefing No. 8. London, HelpAge International. (also available at http://www.cashlearning.org/downloads/Pension-watch-briefing-no-8-FINAL.pdf)

FAO. 2015. The State of Food and Agriculture 2015. Social protection and agriculture: breaking the cycle of rural poverty. Rome. (also available at www.fao.org/publications/sofa/2015/en/)

FAO. 2017. Social Protection Framework: Promoting Rural Development for All. Rome. (also available at www.fao.org/publications/card/en/c/87c92abf-466d-4b01-b8ec-57ca7317ed57/)

Hall, S. 2015. Gender Assessment of Access to Social Safety Nets in Afghanistan. Final Report. Washington, DC, World Bank.

Holloway, K., Rouse, R. & Niazi, Z. 2017. Women’s Economic. Empowerment Through. Financial Inclusion. A Review of Existing Evidence and Remaining Knowledge Gaps. Innovations for Poverty Action. (also available at https://www.poverty-action.org/sites/default/files/publications/Womens-Economic-Empowerment-Through-Financial-Inclusion.pdf)

Holmes, R. & Jones, N. 2010. How to design and implement gender-sensitive social protection programmes. ODI Toolkit. London, ODI. (also available at https://www.odi.org/publications/5093-design-implement-gender-sensitive-social-protection-programmes)

IEG (Independent Evaluation Group). 2014. Social safety nets and gender. Learning from impact evaluations and World Bank projects. Washington, DC, World Bank. (also available at https://ieg.worldbankgroup.org/evaluations/social-safety-nets-and-gender-learning-impact-evaluations-and-world-bank-projects)

IFPRI (International Food Policy Research Institute). 2018. Gender, Agriculture, and Assets Project, Phase 2 (GAAP2). [online]. [Cited 28 July 2018] http://gaap.ifpri.info

McCord, A., Holmes, R. & Harman, L. 2017. Indicators to measure social protection performance: Implications for European Commission programming. Tools and Methods Series, Concept Paper No. 5. Brussels, European Commission. (also available at https://europa.eu/capacity4dev/tand-m-series/document/indicators-measure-social-protection-performance)

Molyneux, M. 2017. Social protection and gender: Where are we now and what next? Presentation prepared for FAO and UNDP-ICG webinar series on gender and social protection.

Molyneux, M. & Thompson, M. 2012. CCT programmes and women’s empowerment in Peru, Bolivia and Ecuador. CARE. (also available at https://insights.careinternational.org.uk/publications/cct-programmes-and-womensempowerment-in-peru-bolivia-and-ecuador)

Pavanello, S., Pozarny, P., de la O Campos, A.P. & Warring, N. 2016. Qualitative research on rural women’s economic empowerment and social protection. The Impacts of Rwanda Vision 2020 Umurenge Programme (VUP). Qualitative report. Rome, FAO. (also available at www.fao.org/family-farming/detail/en/c/445026/)

Peersman, G. 2014. Overview: Data Collection and Analysis Methods in Impact Evaluation. Methodological Briefs: Impact Evaluation 10. Florence, Italy, UNICEF Office of Research. (also available at https://www.unicef-irc.org/publications/755-overview-data-collection-and-analysis-methods-in-impact-evaluation-methodological.html)

Perrin, B. 2012. Linking Monitoring and Evaluation to Impact Evaluations. Impact Evaluation Notes No. 2. Inter Action. (also available at https://www.interaction.org/sites/default/files/Linking%20Monitoring%20and%20Evaluation%20to%20Impact%20Evaluation.pdf)

Tebaldi, R. 2016. Gender and social protection in sub-Saharan Africa: a general assessment of programme design. Research Brief No. 58. Brazilia, Brazil, IPC-IG. (also available at www.ipc-undp.org/pub/eng/PRB58_Gender_and_social_protection_in_sub_Saharan_Africa.pdf)

Wallace, T. & Chapman, J. 2011. Walking the Talk: Cash Transfers and Gender Dynamics. A report by Concern Worldwide and Oxfam GB. London, Oxfam GB. (also available at https://policy-practice.oxfam.org.uk/publications/walking-the-talk-cash-transfers-and-gender-dynamics-131869)

Warring, N. & de la O Campos, A.P. 2016. Research on rural women’s economic empowerment and social protection. The Impacts of Rwanda Vision 2020 Umurenge Programme (VUP). Quantitative report. Rome, FAO. (also available at www.fao.org/publications/card/en/c/71f973f4-7c34-4ab7-bb91-e2042f54c023/)

1 See the FAO Social Protection Framework (FAO, 2017) for its corporate approach and programmatic work in SP.

2 The list of experts interviewed can be found in Annex 2.

3 For further information on the webinar series see http://socialprotection.org/connect/communities/gender-sensitive-social-protection.

4 Some of these components, such as payment arrangements, can also be considered programme design issues. However, they are discussed here as part of the delivery system as they are also important during the implementation phase.

5 The ODI Toolkit is available at www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/6262.pdf

6 M&E can be used to monitor policies, programmes and projects at the field and institutional level. This section explores the implications of integrating gender into M&E mechanisms for SP programmes.

7 Targeting accuracy assessments help reveal whether programme benefits reach the poor. Process evaluations assess and explain the operations of a particular programme operations and how programme implementation has contributed to measurable impacts. Outcome/effectiveness evaluations measure programme effects in the target population by assessing progress in terms of the outcome or outcome objectives that the programme seeks to achieve (Perrin, 2012). Impact evaluations involve more complex evaluation activities to identify causal impacts of a programme and are usually done by independent experts. For more information on impact evaluations, see section 2.4.

8 To create a logical framework, download the template and example from the tools4dev web site at: www.tools4dev.org/resources/online-course-how-to-write-a-monitoring-evaluation-framework-step-by-step-lessons/.

9 Developed by Feed the Future (led by USAID), the International Food Policy Research Institute (IFPRI) and the Oxford Poverty and Human Development Initiative.