Afghanistan: Consultant - Final Evaluation Operationalizing the Comprehensive School Safety Framework (CSSF)

1. Background

Save the Children is an international independent non-governmental, non- profit, organization founded in 1919. We work in over 120 countries worldwide. We are the world’s leading independent organization for Children. Our vision is a world in which every child attains the right to survival, protection, development and participation. Our mission is to inspire breakthroughs in the way the world treats children and to achieve immediate and lasting change in their lives.

Afghanistan is a country that suffers from recurrent natural disasters including flash floods, drought, extreme cold, avalanches, earthquakes and sandstorms. Natural disasters in Afghanistan have historically caused displacement of local populations, severe delays in the provision of humanitarian assistance to those in need and the disruption of essential services. Natural disasters also exacerbate the vulnerability of communities who have been affected by long-term food insecurity and protracted conflict. Recent earthquake has had huge impact on schools which made the MoE to realize the need for school safety mechanism to be put in place. The communities that are more vulnerable are those living in disaster prone areas or having very low coping capability. School children are one of the most vulnerable groups during disasters, since many schools do not have proper buildings or are poorly built without considering disaster resistance features. Furthermore, many schools across Afghanistan do not have the school safety plans[1].

Operationalizing the Comprehensive School Safety Framework (CSSF) project was being implemented by Save the Children in partnership with the Ministry of Education and in coordination and collaboration with the Afghanistan National Disaster Management Authority (ANDMA), Ministry of Rehabilitation and Rural Development (MRRD). This project was funded by ECHO. The proposed project had a budget of EURO 400,317 for a period of eighteen (18) months (started in April 2015 will be ending in September 2016). As per the plans Save the Children (SC) was supported the curriculum department of the Ministry of Education to review the school curriculum and revise textbooks for Grades 1-6 to include DRR/DRM messages in three subjects: Language, Sciences and Social Sciences. These subjects were best allowed inclusion of DRR messages into the standard school curriculum. Under the General Education Department (GED) SC was worked with the Disaster Education Unit and Education in Emergencies (EiE) Coordination Group to build their capacity by supporting the introduction of necessary guidelines and plans, including the Comprehensive School Safety Framework (CSSF) and with the support of MoE to train its Master Trainers to replicate trainings at Provincial, District and School levels. 18,770 children and 10,391 adults were benefited by this action.

The principle objective of the project was to increase the resilience of schools in the face of natural hazards in order to ensure uninterrupted access to education and school safety for children

The specific objectives of the project was to support the Afghan government to develop and implement curricula and systems to prepare for and respond to risks and hazards affecting schools throughout the country. This project focuses on three main results:

Result 1: Pillar 3 of the Comprehensive School Safety Framework (CSSF) – Risk Reduction and Resilience education is operationalized and tested in partnership with MoE in Afghanistan through the integration of DRR into school text books and training materials.

Result 2: Pillar 2 of the Comprehensive School Safety Framework – School Disaster Management – is operationalized and tested in partnership with the Ministry of Education in Afghanistan through training and capacity building of MoE Officials at national and province level to train School Management Committees (SMCs).

Result 3: Effective guidelines and practices for the promotion and coordination of DRR within the education sector are approved by the Ministry of Education for country wide implementation.

2. Purpose of the Evaluation

The purposes of this evaluation are to assess the project’s impact and to document lessons learned best practices and gaps and make concrete recommendations for future improvements.

Specific objectives of the evaluations are to;

2.1. Assess the outcomes and impact of the Operationalizing the Comprehensive School Safety Framework project at government and school levels

· To assess what extend the resilience of schools in the face of natural hazards was increased related with the principle objective?

· How has the project improved/helped for MoE at national and provincial levels on school safety?

· What was the impact of the intervention, if any, on adult educational authorities and children’s empowerment and participation?

· How useful is the Comprehensive School Safety Framework (CSSF) and what areas would need further improvement?

· How useful is the inclusion of DRR key messages into the school curriculum/text books?

· How far are the groups, organizations and initiatives supported by the project sustainable beyond the project duration?

· Have there been any unintended consequences, positive or negative, at any level as a result of the project?

2.2. Assess effectiveness, appropriateness and efficiency of implementation processes and coordination, coherence mechanisms

· Was the project implemented in a timely manner to best meet the needs? If any, what factors limited the timeliness of implementation and what was the impact of any delays?

· What is the perception of the beneficiaries and other stakeholders towards the quality of materials produced and type of trainings, school based DRR initiatives and other support that they have received?

· Which activities have been the most effective and why?

· Was the collaboration with stakeholders (Ministry of Education at national level) including the national level ANDMA and the Provincial and District Education Department effective? What added value did the collaboration bring to the project?

· Were the activities appropriate to the culture and context (including security situation) of the areas we are working in?

· Did we promote accountability to beneficiaries to ensure meaningful participation in our activities? Are they any Complaint Feedback Mechanism (CFM established and how it’s functioning?

· To what extent has this project been implemented in harmony with other SCI projects in the same provinces?

2.3. Identify lessons learned and best practices of the project

· What are the key factors and best practices contributing to the project’s successes or failures and what are recommendations on ways to improve the programme in a follow-on project?

· What are the major lessons learned of the project?

3. Documents to be consulted / reviewed by the consultant:

· Approved Project proposal- Operationalizing the Comprehensive School Safety Framework to indicate what activities or components of the action were given most importance

· SCI MEAL Framework -Complaint Feedback Mechanism (CFM)

· Monthly, Quarterly and Annual Project Plans (DIPs)

· Baseline Survey Report

· Monthly, Quarterly, and Interim Project Reports

· Comprehensive School Safety Framework (CSSF)

· Revised School Textbooks and Teacher Guidelines

· DRR Key messages workshop report and project related final report and documents

· MoE-Tabeyat Quarterly Magazines (four quarterly magazines)

· Child Sensitive School Based Disaster Preparedness Plans

· Project training record and database

4. Evaluation Design and Methodology

The evaluation begins with a review of documents in Kabul and Jawzjan and necessary preparation for the field visit. While designing the methodology the consultant will work closely with MEAL and ECHO DRR departments of Save the Children. Between 28^th August and 17th September 2016, guided by an approved TOR and evaluation work plan, the consultant will interview (1) the SC Kabul, and Jawzjan provinces staff (2) project beneficiaries (MoE Departments) and (3) Provincial government officials.

Stakeholders to be interviewed include MoE (Curriculum Department, General Education Directorate, Education for All Department, EiE Coordination Unit, Teacher Education Department), ANDMA, PED, DED, SMCs, CDCs, Afghan Resilience Consortium (ARC), community leaders, children at school, activists, service providers and direct beneficiaries, individually or in groups, in Kabul and Jawzjan provinces.

The consultant will work with the project staff to develop an appropriate research methodology which needs to be signed off by the International Project Manager-SBDRM. It is expected that the consultant will use a combination of quantitative and qualitative methods involving all actors. A particular focus will be placed on meaningful participation of children in the evaluation.

The consultant should also design an appropriate sampling method to be used during evaluation which also needs to be signed off by the International Project Manager-SBDRM

Qualitative and quantitative tools should be prepared in collaboration with the ECHO DRR Project team and require sign off by the International Project Manager-SBDRM.

5. Timeline

The evaluation will be commenced from 28th August 2016 should be completed by 17^th September 2016 (21 days). The tentative schedule of the evaluation will be as follows (TBC upon appointment):

• Review of literature 2 days


• Field trip for interview, discussion and FGD: 8 days


• Report writing 4 days


• Sharing of evaluation findings 1 day


• Feedback incorporate and prepare final report 6 days

The report needs to be finalized by 17^th September 2016.

6. Expected Deliverables and Responsibilities

6.1 Save the Children Responsibilities

· SC Afghanistan MEAL team will design tools and collect the quantitative data, it will be entered into the database and be ready for consultant analysis

· Provide all necessary support to consultant to ensure timely completion and compliance with international consultancy standards

· SC Afghanistan will provide the transportation and accommodation and will facilitate visa for international consultants visits in Kabul and Jawzjan Province

6.2 Consultant Responsibilities

· Review of the available relevant project documents i.e. project proposals, log frame, donor reports and conduct an in-depth desk review of the relevant secondary data

· Prepare and finalize the evaluation questions for qualitative data collection at different levels and design the methodology

· Attend for qualitative discussions, FGDs and KII sin Kabul and Jawzjan Provinces

· Analyze the quantitative data collected by SC MEAL department

· Analyze the qualitative information collected by consultant

· Produce the Draft the report

6.3 Final Evaluation Report by Consultant:

The final evaluation report should be not more than 30 pages in total excluding annexure in English including but not limited to:

• Introduction


• One page summary report according to SCI requirement


• Major achievements of the project and anticipated short and long term impacts


• Efficiency and effectiveness of the project implementation and appropriateness of the project design

· Coordination / Coherence

• Sustainability of the project


• Major learning and best practices


• Key recommendations action plan and conclusion

The report should be clear and concise, with an executive summary not exceeding five pages. Annexes should include the TORs, a list of individuals interviewed and consulted, evaluation tools a bibliography, a description of methods employed a summary of survey results (if appropriate) and any other relevant materials. Additionally the consultant should support for finalizing the report if any feedback received from SC member and Technical Advisors before processing the payment.

Preliminary findings workshop with project staff: Upon return from the field the consultant will present preliminary findings to project staff. This will be an opportunity for the consultant to raise question and to obtain feedback from project staff before embarking on the formal write up of findings.

Debrief with project and management staff: A power point presentation that includes a summary of the evaluation report (including process, findings and recommendations) delivered to the ECHO DRR project staff and SCI management.

7. Budget and Payment Schedule:

· Consultancy payment to be proposed by the consultant on the basis of daily rate which includes tax.

· Agreed fees will be transferred to the consultant’s bank account on official acceptance of the deliverables and on submission of invoice duly signed by the consultant.

· Transportation, accommodation, food allowance, incidental costs, insurance, expenses during stay in Kabul and Jawzjan and travel to the project sites will be covered as per the SCI Afghanistan country office policy.

· Save the Children will cover the one flight cost and visa processing. Accommodation will be provided at the shared guest house for international consultant at Kabul level and in the province level accommodation will be at the hotel.

Payments will be released based on following schedule and submission

Schedule

40% After Designing Tools and Methodology

40% Draft Report

20% Final Report

All costs related to the work (visa, travel, accommodation and insurance) including consultancy fee will be covered by Save the Children.

8. Personnel Specification:

· Master Degree in Social Sciences, Development Studies, Disaster Management field or equivalent significant experience in Education, Child Protection and Disaster Risk Reduction

· At least 3-5 years’ experience working in the NGO/DRR and Education sector

· Understanding of the challenges and best practices related to international development programmes for children in difficult circumstances, particularly those living in disaster prone environments.

• Technical expertise in School Safety, EiE and DRR programming in humanitarian and development contexts


• Previous experience of M&E in DRR and Education programming including conducting end of project evaluations for large-scale DRR projects


• Experience and knowledge of School Safety programming and evaluations in the context of Afghanistan


• Very strong written and spoken communication and facilitation skills


• Experience of working directly with children and commitment to meaningful child participation


• Experience of using participatory techniques in data collection and child-friendly participatory techniques (preferred)


• Fluency in English is essential