End-line Evaluation Terms of Reference SOM Family Strengthening through Cash Transfer and Child Protection in Baidoa, Somalia
End-line Evaluation Terms of Reference
SOM Family Strengthening through Cash Transfer and Child Protection in Baidoa, Somalia, 1.4.-31.12.2018
Save the Children International Somalia/Somaliland (SCI) is implementing a project called “Family Strengthening through Cash Transfer and Child Protection in Baidoa”funded by the Ministry for Foreign Affairs of Finland (MFA) to support the most vulnerable families in 14 drought and conflict affected internally displaced people’s (IDP) camps and host communities in Baidoa Somalia by addressing their urgent basic needs to prevent further food insecurity, malnutrition, poor health and gender-based violence (GBV). A monthly cash transfer via mobile phone (USD 64 per month) have been provided for 4 months to 1,405 households (HH). 347 of these targeted HHs (20 %) are from host communities from poor and vulnerable HHs with a child or an adult with a disability. An integration between food security and livelihoods (FSL) and child protection (CP) has been emphasized by providing cash transfers to additional 330 HH with child protection risks in 5 out of targeted 14 IDP camps where the project has also supported 5 child-friendly spaces (CFS). Cash transfer beneficiaries have been selected through working closely with community relief committees (CRC), village relief committees (VRC) and child welfare committees (CWC).
This project has combined cash transfers with complementary interventions (cash plus) to ensure that cash will have a maximum impact on children. The cash transfer has been linked to awareness raising on child feeding, appropriate care practices and prevention and response to violence and abuse. Children’s well-being has been promoted by improving the care of children by the households and the community. Caregivers, community groups, community leaders and service providers have been trained to prevent and respond to CP issues. Children have received support to build their resilience and to better address risks and stress they are facing by strengthening the capability of children themselves through child resilience activities and structured play and recreational sessions. Provision of psychological first aid (PFA) and psychosocial support to distress parents, caregivers and children has also be part of cash plus support. In addition, vulnerable girls and boys with child protection concerns (e.g. family separation) have been directly supported through child protection case management and referrals.
The project implementation is from 1 April to 31 December 2019 with total funding of EUR 905946
- Purpose and objectives of the evaluation
The main purpose of this End-line final project evaluation is to facilitate a process which will document project outputs and impact and to measure progress achieved, status of all indicators compared toproject baseline figures.
The end-line evaluation will also explore the impact of cash plus intervention on the lives of the targeted children. In addition to this, the end line project evaluation will document the gaps and lessons learnt of the project for improved programming and inform next phase project design.
- % of target 500 boys and girls who can identify where to go to if they have protection concerns (Target = 70%)
- % of target 330 households implementing better care practices (Target = 70%)
- % of target 1,735 households using cash transfers to support children’s well-being (Target = 90%)
To achieve this objective, the project has two results described below. These are the indicators that will be compared between the baseline survey figures and end line survey figures.
Improved access to income through provision of monthly unconditional cash transfers (UCT)
· % of targeted households who have sufficient access to food during the cash transfers period (Target = 90%)
· % reduction in coping strategy index (CSI) among targeted vulnerable households (Target = 30%)
· % of boys and girls in targeted households who maintain at least 2 meals per day during cash transfer period (Target = 90%)
Girls and boys affected by drought and conflict have better protection from violence and recruitment to armed groups through prevention and response services.
· # of suspected cases of separation, violence, abuse, exploitation or neglect identified through CWCs, community education committees (CECs), and IDP committees, and supported through case management (target = 330)
· % trained staff from other sectors who reported and referred cases of CP violations (target = 70%)
· # of boys and girls provided with direct support (legal, medical, counselling and reintegration) (target = 550)
· % of children and caregivers targeted through child resilience program report improved psychological wellbeing (target = 65%)
· % of surveyed community based structures which have better knowledge to prevent and respond to CP concerns (Target = 80%)
- To assess the project’s performance and achievements vis-à-vis the project’s overall objectives, the project indicators in the logical framework and the baseline data for the indicators gathered at the start of the project.
- To assess the relevance, effectiveness, efficiency, impact, sustainability, coordination, scalability/replicability and gender effects of the project.
- Collect qualitative data on the impact of the program including benefits of combining the cash support with cash plus activities and integrating FSL support with child protection interventions (case management, support to child welfare committees and child-friendly spaces); collectcase studies that demonstrate change in the lives of children.
- To document unintended impact of the project (negative and positive impacts).
2.2. KEY QUESTIONS
Following evaluation questions should be at least included in the evaluation criteria. The evaluation willbe based on OECD DAC criterion.
The extent to which the objectives of the project interventions are consistent with the needs and interest of the beneficiaries.
- a) To what extent were the project’s activities adapted to meet the needs of particularly vulnerable children in the target project area?
- How relevant were the project objectives and outcomes in benefiting vulnerable target children in Baidoa?
- To what extent weretheproject’s deliverables aligned with Local administration and South West Regional Government Ministry of Gender and Family Affairs’ priorities?
- b) Effectiveness
Extent to which the objectives of project interventionhave been achieved.
- a) To what extent did the project attain its objectives and results described in the project proposal?
- b) What good practices, success stories, lessons learnt and replicable experiences have been identified in the project implementation?
- c) What was the contribution of theproject’s stakeholders including the South West Regional Government Ministry of Gender and Family Affairs and project committees to the achievement of the projectobjectives and results?
- c) Efficiency
- a) Did the project use the financial and human resources as per the project document to deliver the intended results?
- b) Did the project offer better value for money considering its impact on the lives of the vulnerable children?
- d) Impact
- Positive and negative effects of the intervention on the target vulnerable children.
- a) To what extent did the project have an impact on the targeted beneficiaries?
b)What are the key short term and long term changes produced bythe project, positive or negative and what are the key factors behind these changes?
- e) Sustainability
Probability of the benefits of the intervention continuing in the long term.
- a) Which mechanisms already existed and which have been put in place by the project to ensure results and impact, policy coordination mechanisms, partnerships, networks?
- b) To what extent has the capacity of beneficiaries (institutional and/or individual) been strengthened such that they are resilient to external shocks and/or do not need support in the long term?
- c) To what extent will the projectbe replicated or scaled up at local or national levels?
- g) Coordination
- a) How have the project activities been coordinated within different stakeholders including the Government line Ministries, project committees and other organizations (LNGOS, INGOS, UN) in the target area to achieve overall objective?
- h) Scalability/Replicability
- a) Is there any likely ability of the project or its components to be scaled or replicated in other programme areas?
b)Who are the main actors in the scale-up/replication and how has the project engaged with them to date?
- Gender equality
- a) To what extent has the project been gender sensitive or transformative in design and implementation as per SCI Gender Equality Guidance?
- b) What concrete measures were taken in the project to increase gender equality and reduce inequalities?
SCI Somalia/Somaliland will select a local research consultant/institution to conduct the End-line evaluation. The selected research consultant will be required to prepare detailed research methodology in partnership with SCI Somalia/Somaliland.
End-line survey will be proposed to apply both quantitative and qualitative data collection methods. A total of 14 IDPs and host communities in Baidoa district will be target areas for this End-line evaluation.
Based on the project log frame (objective, results), project team is intended to have baseline data as secondary data source to compare to end line key findings. The detailed data source and methods, and full questionnaires are required from consultant team. HH survey questionnaires are based on indicators of objective and outcomes which are shown in above table. Consultant team should submit the draft questionnaires when they provide the technical proposal to SCI for tender selection process.
The End-lineevaluation team will use all project documents; secondary data sources will be used for further data analysis.
PrimarydataanddataanalysisprocesswillbedisaggregatedbythesexandtargetCWC, VRCs and CRC.Draftfindingswillbepresentedwithprojectteamandkeypartnerstovalidate.
As a minimum, the evaluation process will include the following key steps:
- Review of relevant literature related to project(list of reference materials provided below) and drafting an inception report before the evaluation exercise at the field.
- Application of appropriate data collection tools (e.g. questionnaire, checklist etc.) for interviews and focus group discussion;
- Data analysis and Evaluation Report writing; and
- Presentation of key evaluation findings;
4.2. REFERENCE MATERIALS
- Baidoa MFA child protection project narrative proposal
- Baidoa MFA child protection Baseline Report
- Baidoa MFA child protection Quarterly and Final Reports
- Project monitoring and evaluation plan
- Monthly Reports
- Project MEAL Plan
4.3 EVALUATION TEAM
- Experience in social protection programing specially cash program and child protection, DRR and related fields. Experience in Somalia context is desirable.
- Prior experience in conducting cash based programing, child protection assessments. Having conducted similar assignment in Somalia will be an advantage.
- Solid analytical, result based planning, report writing, communication and diplomacy skills with ability to correlate different data sets to actionable conclusions.
- English language skills (report should be written in English). Ability to make clear presentations and disseminate findings to both technical and non-technical audience
The consultant is expected to submit the following documents, which will form part of the pre-selection recruitment process:
- Cover letter and CV
- Technical proposal, outlining the methodologies and tools be used and Budget Summary.
- Proven samples of the previous similar work with INGOs and UN agencies preferably Baidoa.
- Reference – At least two credible referees
Time frame for the activities will be:
24th December 2018 – 20th January 2019
5A. TIMEFRAME AND ACTIVITIES
The evaluation should be completed over a period of approximately 26 days from 24thDecember, 2018 to 20th January 2019. Detailed time table for the evaluation is illustrated below.
Desk Review – Inception Report
|Review of background documents and consultation with relevant SC staff||(3 days)||24th -26th December 2018|
|Presentation of Inception Report||1 day||27th December 2018|
|Design of assessment methodology and tools for data collection||2 day||28th -29th December 2018|
|Review and feedback on inception report and data collection tools
|1||30th December 2018|
Field data Collection
|Primary data collection from project beneficiaries||10 days||1st -10th January 2019|
Data Analysis and Report writing
|Data analysis and Draft preliminary report with recommendations
|(5 days)||11th-15th January 2019|
|Save the Children Staff review and feedback on the report||2days||16th-17th January 2019|
|Finalize report (following review/vetting) and presentation)||(2 days)||January 18th to 20th 2019|
5.1. DURATION OF ACTIVITIES
The duration of the evaluation exercise in the field shall be 26days from a mutually agreed date no later than 20th January 2019
The evaluation will follow the key phases:
Phase I – Desk study: Review of documentation and elaboration of field
Study [7 days]
The lead consultant/evaluation team will review relevant documentation from section 5 above (Reference material). Based on this review, they will produce an inception report which will include an elaborate plan, methodology and sampling strategy of the data collection for evaluation study. The evaluation will only proceed to the next stage upon approval of this inception report. An appropriate inception report format will be provided to the team later on.
Phase II: Field Data Collection [10 days]
This phase of the evaluation will seek to collect primary data on the key evaluation questions explained under evaluation criteria. The team will use the agreed plan, methodology and sampling strategy from phase-1 to conduct the field work.
Phase III – Data analysis and production of evaluation report [9days]
The team will draw out key issues in relation to evaluation questions and produce a comprehensive report.
6.0 OUTPUT AND DELIVERABLES
The following specific outputs are expected –
(i) An Inception report, produced by notlater than 29th December 2018
(ii) A draft evaluation report produced by notlater than 15th January 2019
(iii) A Final evaluation report including country case studies and recommendations for Country Programme for future use of the project design and/or alternative solutions to achieve the same objectives produced by 20th January 2019
(iv) Data sets (SPSS, Excel, Stata – for all collected data (quantitative and qualitative). Qualitative data should be transcribed for future use by Save the Children Country Programmes. The data sets should be in an appropriate format (SPSS, Excel, and Word) and will be submitted together with the final evaluation report 20th January 2019
(v) PowerPoint presentation, summarizing the key findings from the evaluation submitted together with the final evaluation report on 20th January 2019
- Ethics and child safeguarding
The consultant is obliged to conduct the research in an ethical manner making sure children are safe at all times. The consultant should seek the views of various stakeholders, including children. Efforts should be made to make the research process child-centred and sensitive to gender and inclusion. The consultant must respect the rights and dignity of participants as well as comply with relevant ethical standards and SC’s Child Safeguarding Policy and Code of Conduct. The research must ensure a voluntary, safe and non-discriminatory participation and a process of free and un-coerced consent. Informed consent of each person (including children) participating in data collection should be documented.
Send your application to CSS.LOGISTICS@savethechildren.org