TERMS OF REFERENCE
FOR EXTERNAL EVALUATION OF GANJA FAMILY STRENGHENING PROJECT (GANJA FSP)

DOWNLOAD | Offer form | (*.doc file)

1. Background

From the roots of the organisation, SOS Children’s Villages pioneered family-based child care for children who have already lost the care of their own family, through development of the SOS family childcare model.  More recently, we have been developing programmes to address the situation of those children who are at risk of losing the care of their biological family.  This prevention work has taken shape in the development of family strengthening programmes, which aim to prevent children from losing the care of their family.  Within SOS Children’s Villages and family strengthening programmes, we work together towards our vision of a world where “every child belongs to a family, and grows with love, respect and security”.

Ganja Family Strengthening Project (FSP) has been part of the organisation’s strategic plan since 2007 and the organisation is striving to further develop the family strengthening approach to prevent children from losing the care of their biological family. Ganja FSP started to operate from 2008, from 2010 to 2012 it was funded by the Ministries of Foreign Affairs of Norway and Luxembourg. In 2013 Ganja FSP continues its operation from the approved budget for no-cost extension  from the Ministry of Foreign Affairs of Norway  based on saving during implementation three years project. 

Goal of Ganja Family Strengthening Project is to contribute in the implementation of De-Institutionalization reform in Ganja by preventing family separation, reunifying children with their biological families and protecting them from repeated out-of-home placements.

The objectives of Family strengthening Project  are as follows::
At least 160 biological parents/caregivers capacitated and supported to keep families intact and provide adequate care for children.
At least 400 children’s healthy and harmonious development is guaranteed.
Awareness among communities and stakeholders is raised and capacity is built to address the needs of children at risk.

The target group of Ganja FSP comprises of:
- Children who are at risk of losing parental care when their basic material, emotional, health and educational needs are being neglected or when they are abused because their caregivers lack the capacity or commitment to adequately care for them;

- Children who already lost parental care but there is potential that they can be reintegrated into their families of origin because their caregivers demonstrate commitment and improved their capacity to adequately care for their children so that they can enjoy a caring environment, with love, respect and security.

2. Purpose of the external evaluation

SOS Azerbaijan calls for periodic (3-5 years) independent external evaluation of implemented on local level Family Strengthening Project in Ganja which has been operating since 2008. Project target areas are Ganja, Samukh and Goygol.

The evaluation will answer to the following overarching questions:

1. What changes has the project made in the lives of the participating children within target group, their families and their communities?
2. How relevant, effective, efficient, sustainable and participatory are the project interventions?
3. What lessons can be drawn from the project that can be taken to further develop the project? 

As a forward-looking exercise the evaluation should identify the key lessons learned that can be applied to improve the relevance; efficiency; effectiveness; sustainability and impact of the programme interventions. The findings of the evaluation will be used as a basis for strategic planning on national level; for improvement of programme design and for future development of family strengthening approach on project and national levels.

Key users of evaluation results are:
a. Project level: Family strengthening project team; implementing partners
b. Management level: FS Project Coordinator; NFSA; NPDD; ND; Board
c. Regional/Continental level: RO and CO counterparts

3. Scope and focus of the external evaluation

The external evaluation should analyse the progress that programme has made against the agreed set of global minimum indicators, as a minimum and in addition to any programme specific indicators. The reference frame for these minimum indicators is the FSP Manual (manual, working paper, Jan 2007) the FSP Global Log Frame (May 2009) and Ganja FSP Log Frame (2010) approved by the Ministry of Foreign Affairs of Norway and Luxembourg.  Log Frame Indicators:

• Children who are at risk of losing the care of their family are enabled to grow within a caring family environment.
• % of child participants within the specified target group
• % of the families who have exited the programme due to self-reliance
• % of children separated from their family by the time they exit from the FSP
• % of reduction of incidence of child separation at the programme location
• % of children eligible for reunification and currently living in the orphanage were reunified with their families of origin.

• Children of participating families realise their developmental needs and rights by accessing essential services.
• % of children with stable and adequate shelter
• % of children with birth certificates and residence registration
• % of children received immunization as per national standards
• % of school-going age children regularly attend formal or informal education
• The families of child participants have the capacity and commitment to take care of their children.
• % of primary caregivers who ensure that the child gets 3 meals a day
• % of primary caregivers who ensure the child’s access to health care and treatment as required
• % of caregivers supported in fulfilment of their parental responsibilities. 
• % caregivers improved their psycho-social well-being
• % of caregivers have knowledge and skills in claiming their social, economic and legal rights
• % of families achieved their self-reliance.
• % of caregivers improved living conditions promoting healthy child development.
• Communities and/or local authorities/governments’ commitment and/or capacities are strengthened to promote and protect the rights of children to grow within a caring family environment.
• Capacity of coordination team on international practices in child protection is built through regular trainings and study tours
• Communities increased their awareness of situation of children and families and know how to respond to their situation
• Child Rights are promoted regularly in media
• % of resources (financial, human, material) allocated from the government and other stakeholders to the project
• An active network of stakeholders contributes to ensure that all children can grow within a caring family environment.
• A coordination team is established and operational
• Range of services is available to address the situation of children and families;

Moreover, the external evaluation should look at the following:

• Relevance
• To what extent is the programme focussed on target group – children most at risk of losing the care of their family?
• To what extent the project interventions respond to the needs and priorities of the project’s participants?
• To what extent project design address locally defined needs and priorities?

• Effectiveness
• To what extent are the objectives of the project being attained?
• To what extent have children remained in their families instead of losing their care? To what extent has their quality of life improved?
• How many children and families successfully left the project since the beginning and became self-reliant?
• To what extent have beneficiaries been satisfied with the project interventions?
• To what extent community capacity to protect rights of children has been strengthened?
• To what extent does the approach of following a structured family development planning process contribute to families becoming self-reliant?
• Is replication or removal of certain approaches required to increase effectiveness?

• Efficiency
• Is the relation between input of resources and results achieved appropriate and justifiable (cost-benefit ratio)? What are the social benefits for children/families/communities compared with resources spent for activities?
• What are the annual total running costs and the average running costs per child per month? And how do these costs compare with the similar initiatives in the country?
• Have resources been used most economically? (e.g. tenders for the purchase of goods/services)
• Are there any alternatives for reaching the same result with less input? Are there feasible options for cost reduction and cost saving while realizing the same level of results and quality? Could the outcomes and results have been achieved at lower cost through applying a different approach?

• Sustainability
• Has the capacity of the implementation partner been developed?  If so, in what areas and how?
• Is the project supported by local/national institutions? Do these institutions demonstrate commitment and capacity to continue programme activities or replicate it?
• To what extent can activities, results and effects be expected to continue after SOS involvement has ended? Do partners have financial capacity to maintain programme?
• Is there progress towards SOS withdrawing from its direct involvement and handing over the full responsibility to run the project to an implementation partner?

• Participation
• To what extent are stakeholders (participant families, particular children, partners, local authority) involved in the design and implementation of the project? In particular, to what extent are stakeholders involved in the family development planning process?
• To what extent is the project designed to develop the necessary local institutional (governmental and/or non-governmental) capacity to respond to the problem?

4. Key areas for the external evaluation

Within the Ganja Family Strengthening Project, children, families, and communities have been supported in various areas, which the external evaluation exercise should look at. The evaluation will focus on following key areas and illustrative set of questions.  The evaluation team needs to decide in cooperation with the national programme team and in consultation with CO counterparts which of the below mentioned areas and in what extent apply to the particular programme and set of questions will be further elaborated during preparatory phase. Nevertheless, main sections 1-6, which look at the definition of the target group as well as the selection process of participating families; project’s interventions supporting children, families and communities; main approaches; project partners; allocated resources should apply for all FSP.  

Key areas	Illustrative evaluation questions
Target group	How the target group has been defined How participant families are identified, verified and selected (criteria and process)
Programme interventions supporting children and families
2.1. Nutritional support	Number of children and caregivers receiving the support; Type and appropriateness of nutritional support provided for children by the project directly or through referral (i.e. food package, food vouchers); identify the impact and if there is something to be added, reduced or removed. Any other support caregivers get from the project with regard to nutrition of their children (e.g. cooking lessons) Type of meals (composition), quantity, and frequency per day How long families receive food support; is food support gradually reduced or are there signs of dependency? If caregivers and/or project staff feel that children have weight/height according to age; any symptoms of malnutrition observed by mothers and/or project staff; find out the reasons. Level of satisfaction  of children and caregivers regarding this support
2.2. Health support	Number of children and caregivers receiving the support – if possible in proportion (number should be proportionate to the total of those who have been identified as needing such support Type and appropriateness of health support provided to children and/or care givers by the project directly or through referral; any health education provided; identify the impact and if there is something to be added, reduced or removed. Access to medical care by children and care givers; Frequency of health checks for children as well as for caregivers; constraints for visiting and undertaking regular health check-ups for children and caregivers; Capacity of the project to respond to medical problems commonly experienced by the target group (e.g. related to HIV/AIDS, tuberculosis, drug addiction) How the health conditions of children and caregivers in the project are; did they improve, decrease or remain the same since they joined the project? What difference this support has made in the participants’ lives Level of satisfaction of children and caregivers regarding this support
2.3. Educational support	Number of children supported to access pre-school, formal basic (primary and secondary) education as well as  non-formal/ informal education – if possible in proportion (number should be proportionate to the total of those who have been identified as needing such support) Type and appropriateness of educational support provided to children by the programme directly or through referral; identify the impact and if there is something to be added, reduced or removed. How is the track record concerning children’s attendance, as well as staying enrolled in the educational system Level of educational performance of children against their past performance and against national standards/averages (explain the parameters used for evaluating this) Level of satisfaction of caregivers and children regarding this support
Psychosocial support	Number of children and caregivers being supported to access counselling services – if possible in proportion (number should be proportionate to the total of those who have been identified as needing such support) Type and appropriateness of psycho-social support provided to children and/or care givers by the project directly or through referral; identify the impact and if there is something to be added, reduced or removed. How children and caregivers have been effectively supported to address relevant issues Participation of children in seminars/workshops on life skills (if possible provide their number and frequency and type of participation) How the project staff  and other external specialists view children’s psychological development – (if possible provide number of children who have improved in their psychosocial development after the intervention) Frequency of counselling sessions for children and care-givers and the impact of counselling Frequency of family home visits by the programme Participation of caregivers in community life Number of caregivers who have recovered from drug and alcohol addiction Level of satisfaction of children and caregivers regarding this support
2.5.Living conditions	Number of families/children supported with shelter and other items related to child’s living conditions Type and appropriateness of support provided by the project directly or through referral (e.g. shelter, clothes, and blankets); identify the impact and if there is something to be added, reduced or removed. How safe families’ living conditions are, and how they feel about it Are living conditions adequate relative to acceptable local standards? (relates to such things as number of rooms, especially sleeping rooms; size of house; sanitation; source of light and heat; water supply, etc.) Level of satisfaction of caregivers and children regarding this support
Promotion of child rights/parenting skills	Number of children and caregivers taking part in education workshops in child development and child rights e.g. parental education programmes/trainings Type and appropriateness of activities provided to children and/or caregivers by the programme directly or through referral; identify the impact and if there is something to be added, reduced or removed. To what extent children, caregivers  and communities have knowledge on children’s rights Any example where children have started to exercise their rights If caregivers have improved their parental skills (provide specific information on the parameters used for improved parental skills) Any changes or improvements in parent and child relationship and child care practices (if possible provide number of cases known)
Legal support	Type and appropriateness of legal support provided by the programme directly or through referral; identify the impact and if there is something to be added, reduced or removed. Number of families who have got the necessary legal documents, and indicate the documents obtained. Number of families who have secured their property and assets through proper documentation (e.g. wills, ownership documents)
Economic support	Type and appropriateness of economic support provided by the programme directly or through referral (e.g. skills training, income generation activities, debt payment, short term cash support; referral to employment service); identify the impact and if there is something to be added, reduced or removed. The extent to which this support has changed the lives of families; caregivers have jobs or other income generation activities providing “stable” and “sufficient” income Role of economic support in reducing dependency, stress and building confidence; Appropriateness of the economic support responding to the needs and expectations of participants If loans are provided, how much and at what rate, and to what extent repayment system is working
Other types of support	What other types of support project provided to target children and families? The extent of appropriateness to child/family needs; The extent to which this support has changed the lives of families
Main approaches applied by project contributing to families becoming self-reliant
3.1. Family Development Planning	Number of families who have an up-to-date family development plan (in proportion to the total number of participating families) To what extent does the approach of following a structured family development planning process contribute to families becoming self-reliant What are particular strengths and weaknesses of following the family development planning process – where are areas of improvement To what extent families take responsibility (ownership) in developing and realising the goals set in FDP What constraints they encounter to take such responsibility Level of satisfaction and participation of caregivers and children regarding the FDP process Analysis of drop-outs: Number of families who have exited the project – broken down into: how many families successfully achieved goals in their FDP; how many were dropped from the project due to lack of commitment; and how many simply moved out of the project for other reasons, e.g. changed place of living.
3.2. Other approaches leading to family self-reliance	What other kind of approaches does the project implement to support family self-reliance? Types and appropriateness; To what extent applied approaches are effective and efficient?
4.Project interventions supported community  structures	Type of community based structures (e.g. family committees, community groups, self help groups, volunteer groups) the project collaborates with Type and number of community structures and activities established with the support of the project to support families to prevent child abandonment Kind and number of community based structures or activities which existed, but have been strengthened by the project (specify the type of capacity building work carried out) Type and number of community based implementation partners, and what interventions they carry out. Is the capacity of community structures built in a way that leads to sustainability? Is it foreseeable that families will be supported by community structures without SOS involvement? Satisfaction level by the families of the services provided to them by community based structures
Project partners	Type and number of  local implementation partners (e.g. local government, local NGOs, international NGOs) Is the capacity of the implementation partner strengthened in a way that is sustainable? Is it foreseeable that families will be supported by partners also without SOS involvement? Type of partnerships with other service providers with whom the project collaborates; type of services they each provide Kind and number of local organisations that have their capacity strengthened through the project; in what way The trend in number of families supported by other service providers collaborating with the project (number in the increase or decrease) Satisfaction level by the participants/clients of the quality of services provided
Resources
6.1.Management/administration system	How well the project is structured regarding management and staff is supervised To what extent the project is supported by an appropriate management system?  Is this system geared towards sustainability (with takeover of implementation partner in mind)?
6.2. Monitoring and evaluation system	What are main tools//mechanism for monitoring and evaluation of project progress? What are main tools//mechanism for monitoring and evaluation of project interventions?   To what extent the project is supported by an appropriate monitoring and evaluation system?
6.3. Human resources	Adequate human resources in project? If not, where/how? Relationship between staff members and team work Relationship between staff and implementation partner (including volunteers) Training of staff and volunteers; Any training needs? Clear division of roles and responsibilities between the staff and between FS project and its partners
6.4. Infrastructure/ programme budget	To what extent infrastructure is appropriate to project activities? Adequate budget for projet development?

5. Existing informational sources:

5.1. Key reference documents:

• FSP Manual (manual, working paper, Jan 2007)
• FSP Global Log Frame (May 2009). 
• FSP Framework of Monitoring & Evaluation Indicators (February 2009)
• Continental guidelines Financial and in-kind assistance for families of origin
• Continental paper: Core approach with working with families of origin in CEE/CIS/Baltics
• NA strategic plan and annual plans;
• Ganja FSP Log Frame (2010 - 2012)
• Ganja FSP annual plans;
• Project progress reports; annual reports; self-evaluation reports; SWOT analyses, annual budget;
• Databases; case files; any other collected data by project team
• Previous monitoring/audit/evaluation reports

5.2. Key persons:

• Children and families participating in project; families left the project.
• Project team
• Project partners; community members
• Project management on national level

6. Evaluation Approach, Process and Methods:

6.1 Evaluation Approach

• As a minimum requirement, this evaluation exercise looks at the changes that have happened in participating family’s lives due to their participation in the project. At this stage, it is not a requirement to work with a control group and we are aware of the fact, that therefore the external factors that impact on family’s lives are only marginally considered.
• Pre-existing information sources should be used for the external evaluation.
• The evaluation should use participatory methodologies to involve project participants (families currently being on the project as well as families that have already left the project) and wherever possible children should be involved in the design and implementation of the evaluation. Evaluation team must respect participants’ rights regarding voluntary participation in the evaluation, including their right to withdraw such participation at any stage if they so decide.
• Evaluation team must respect participating communities’ culture, social norms, values and behaviour; and maintain appropriate relationships with participants to this evaluation.
• The evaluation process must ensure triangulation: findings and results must be cross-checked between the external evaluator, the project-, national- and the region/continental- level staff members. 
• The evaluation process must ensure that results are reported back to the participants.
• As this external evaluation is an opportunity for learning for the project staff, it is critical that the project staff participates throughout the evaluation process: evaluation design and planning; information collection; development of findings; evaluation reporting and results dissemination.
• Evaluation team must respect confidentiality of information gathered from respondents. The evaluators should observe standards relevant to researches in child protection field and to uphold standards of behaviour that are acceptable to respective SOS National Association. External consultants should sign the Code of Conduct before starting the evaluation.

6.2 Evaluation process/Main responsibilities of external evaluator:
6.2.1 Evaluation plan

External evaluator should plan the evaluation design in coordination with project staff on local/national and regional/continental level. That includes:

• Propose composition of a survey team;
• Plan and design data collection process:

• Identification of the major stakeholders who are associated with the project to be interviewed, such as the direct participants of the project (children, care-givers) implementation partners, key local government representatives, and other service providers
• Selection of representative sample of communities or groups to be interviewed on the basis of agreed criteria
• Agreeing on the type of information to be collected
• Preparation of checklists and other tools for data collection
• Develop a methodological tools for data collection and consult with project staff on project/national/continental level

6.2.2 Data collection and field mission

The evaluation will include a site visit to project location to meet participants (children; families); project team and project stakeholders; and to collect information in accordance with the requirements stipulated in the evaluation design. This visit is expected to be no longer than two weeks in duration. Project team is to be consult on visit period. Main tasks include:

• Review the project documentation and other sources of information
• Undertake a SWOT exercise with project  staff
• Collecting of data
• Facilitate learning during the data collection exercise

6.2.3. Data analysis and elaboration of evaluation report

External evaluator will analyse collected data and will prepare an evaluation report that describes the main evaluator’s findings, recommendations and lessons learned.The final report should follow the structure and content as outlined in the terms of references.
That stage includes:

• Analysis of the data

• Preparation of a draft report
• Present the findings to the respective project staff on local/national and regional/continental level to ensure triangulation 
• Finalise the report after inputs from various stakeholders. 

6.3. Methodology

• The evaluation should use quantitative (e.g. surveys) and qualitative data collection methods such as semi-structured interviews (for example with focus groups, key informant, large groups, individual interviews, etc.) as necessary.
• The methodology of evaluation will include the following:

• Document review including analysis on key reference documents listed in Terms of Reference;
•  Case study of randomly selected beneficiaries files (present and those who exited the programme)
• Interviews (structured and/or semi-structured; in person and/or by telephone) with key informants listed in Terms of Reference
• Focus groups with selected key informants – children, care-givers, partners, community members;
• Other methods relevant to evaluation objectives and scope

7.  Expected deliverables:

External evaluation team will commit to the following key deliverables:
7.1. Evaluation design – contains the evaluation framework; detailed evaluation methodology; work plan and budget
7.2. Developed evaluation tools
7.3. Draft evaluation report – Draft report will be prepared in line with the proposed structure bellow and should be submitted to National Family Strengthening Advisor, electronically via e-mail, in English,
7.4. Final evaluation report - The findings of the external evaluation shall be presented in a written report following the proposed outline. Attachments – Templates of applied evaluation tools (questionnaires; main areas for focus groups and etc.). Final evaluation report should be submitted to National director in both national language and English, in electronic format.

8. Profile of the external consultant

The consultant (e.g. research institutions, university, independent freelancers, etc.) must have:

• proven competency and experience in monitoring, evaluation and impact assessment
• a social science background
• a good understanding of development work and partnerships
• good facilitation and interpersonal skills
• proven experience or participatory evaluation and data collection methods
• strong skills in coordinating teamwork
• strong analytical and conceptual skills
• excellent writing and communication skills in English and local language.
• ability to transfer complex concepts / ideas into practical and simple language
• ideally experience in organising research processes with/for SOS Children’s Villages

All interested candidates should submit their CV and filled out offer form (attached to the advertisement) to Anar Khalafov, National Family Strengthening Adviser at [email protected] by May 24 2013. Research-oriented organizations can also apply and submit their organizational profile and CVs of their experts.