CALL FOR APPLICATIONS – TERMS OF REFERENCE FOR EXTERNAL EVALUATION

Educational Centre PRO DIDACTICA (CEPD) in partnership with National Association of Private ICT Companies (ATIC) have implemented the project „Advancing quality in Information and Communication Technology (ICT) vocational education in Moldova“ since December 2012. Among problems and challenges of the VET sector, one of the most important ones is the problem of quality of knowledge and practical skills of the young ICT specialists.

Project title: Advancing quality in ICT vocational education in Moldova
Country: Republic of Moldova
Project number: 8295-00/2012
Project duration: 01.12.2012 – 31.05.2015

Background

For any economic system it is important to link its education system with the labour market. The ideal ratio between these two systems implies that the labour demands at all levels are in close correspondence with the offers of the educational system at all levels. In practice, achieving such a goal is difficult, because each system has its own dynamics and logic of evolution. Achieving a balance between IT labour market and IT education becomes even more problematic because of the complexity and the internal dynamics of this area. In Moldova, as in other countries from South-East Europe transformations in the current IT labour market are much faster than the capacity of the education system to respond quickly to labour market needs, and the absence of a monitoring mechanism of employing graduates in the workforce, makes it extremely difficult to estimate the proportion of those who are committed to work after graduation.

The overall objective of the project is to contribute to the harmonization of ICT education in Moldova with labour market demands. The project purpose is to support the qualitative improvement of content and process of ICT professional education in 6 institutions (colleges and vocational schools), thus fostering the improvement of quality of knowledge and practical skills of the young ICT specialists.

The expected results of the project are:

Improved, updated curricula for ICT specializations, relevant for market needs developed in beneficiary colleges with involvement of ICT companies’ representatives;
Improved practical skills of IT students updated to labour market demands through enrolment of trained students in private sector
Awareness about the possibility to get a Career in ICT at a VET institution raised

The project aims at improving several aspects in the sphere of vocational education and training in Moldova by creating/improving:

a demand from the side of the industry for VET graduates as well as a demand for the choice of technical studies at VETs (ICT Career Orientation and inclusion of VET in the map)
opportunities for VET students
the gender balance presence in the ICT field by attracting females to technical specialties
a market driven curricula for ICT and BPO companies
sustainable cooperation models with the private sector to decrease the shortage in the ICT and BPO field

As a target group for the envisioned project, were selected 6 institutions – 4 colleges (specialty: Informatics and Computers) and 2 vocational schools (specialty: Operator for technical support of computer) which represent geographically all the country (North, South, Centre). Some of main expected results from the project are: improved, updated curricula for ICT specializations, relevant for market needs developed in beneficiary colleges with involvement of ICT companies’ representatives; ICT specialists from beneficiary colleges trained in methodology of active learning and IT topics; trained English Language specialists in modern approaches of foreign language teaching; upgraded computer laboratories in each beneficiary institution etc.

These will be a benefit for ICT young students, through creation the opportunities for them and the fortification of the institutions, access to modern laboratories (and example would be the innovation labs aiming at boosting creative ideas, already proving to be a successful model in many European countries), combined with clear profiles and programs for the students to be capable to integrate easily into the working field.

Purpose of evaluation

The evaluation will focus on assessment the level of compliance of activities, achievement of results and objectives, and identify processes in which the project has contributed, with a view to the evaluation of qualitative improvement of content and process of ICT professional education in 6 institutions.

The evaluation is intended to contribute to an improvement of policies, processes and methods.
The evaluation makes recommendations on how this or a similar project or programme could be continued.

The evaluation contributes to the accountability towards the stakeholders and taxpayers, including clear statements if the objective and the results have been achieved.

Purpose of evaluation

The evaluation is intended to contribute to an improvement of policies, processes and methods.
The evaluation makes recommendations on how this or a similar project or programme could be continued.

The evaluation contributes to the accountability towards the stakeholders and taxpayers, including clear statements if the objective and the results have been achieved.

Objectives

The evaluation’s main goal is to document project achievements towards the initial project proposal and project developments. Furthermore, the evaluation should review and assess the relevance of the project activities, the occurrence of the expected results and their progress towards sustainability. It should include a reflection about lessons learnt and make recommendations for the project counterparts.

The specifically objectives of evaluation are:

Analysis of materials and other relevant documents prepared by the project team;
Analysis and evaluation of results foreseen in the logframe and achieved in reality;
Identify the immediate and potential longer-term impact of project activities in the 6 beneficiary institutions;
Analysis of compliance of project activities with the specific needs of ICT companies and VET system;
Determining the degree of correspondence of the results with respect to anticipated progress indicators intention stage project;
Identifying potential risks or constraints that may affect the effective implementation of the project;
Provide recommendations on how to further improve the project implementation for the next 4 months to maximize the impact of the project

Identify potential development issues of labour market oriented VET beyond the end of this project.

Subject and focus (scope)

The subject of the evaluation is the project “Advancing quality in ICT vocational education in Moldova”.

The evaluator(s) will review and summarise the available evidence of the quality, accountability and impact of the project activities and will raise and address all relevant issues that may emerge during the evaluation.

The evaluation will consist of both desk and field study and will take place four months before the end of the project in order to finish the report in time before the final project closure.

ADA’s cross-cutting issues: Gender and Environment have to be taken into consideration throughout the evaluation and reporting process.

Subject and focus (scope)

The subject of the evaluation is the project “Advancing quality in ICT vocational education in Moldova”.

The evaluation will consist of both desk and field study and will take place four months before the end of the project in order to finish the report in time before the final project closure.

ADA’s cross-cutting issues: Gender and Environment have to be taken into consideration throughout the evaluation and reporting process.

Main evaluation questions

Relevance:

To what extent is the establishment of the new occupation „Operator for technical support of computer” relevant for labour market in Republic of Moldova?
To what extent do the development and adoption of curricula at specialisation „Information Technology” and „Computers” supported the process of harmonization of ICT education in Moldova with labour market demands?
To what extent is the development and implementation of English Language Program for ICT specialists relevant?
To what extent does the upgraded computer laboratories in each beneficiary institution according to international and market standards was necessary?
Were the objectives set out in the project document relevant, attainable and realistic?
To what extent does the project cater for the target group’s needs? Were they taken into consideration by CE PRO DIDACTICA and ATIC in their work and how?
To what extent was the bottom-up approach helpful for the assessment of market needs developed in beneficiary colleges with involvement of ICT companies’ representatives?
To which extent is this project relevant for the Moldovan Government?
Will the newly developed and piloted curriculum be applied nationwide?
What is the benefit of the conducted Baseline Studies on Needs Assessment in beneficiary institutions, labour market and students?
To what extent are the stakeholders (Working group, Ministry of Education, Ministry of Information Technology, etc.) involved in the project’s decision-making processes? To what extent have stakeholders participated in the project activities?
To what extent the project would require an extension?

Efficiency:

Is financial spending efficient per planned activity and in line with the costs on local level?
Is the equipment procured according to market prices in Moldova?
Are the items procured used as planned and are they all still functioning? Are they also maintained properly, and their maintenance foreseen in the budgets and workplans of the schools?
How economically have resources and means been used?

Effectiveness:

To what extent are the achieved results in line with the planned results (reference is made to the project document)?
Were the capacities of institutions relevant (duty bearers) for the needs assessment of the labour market and pilot institutions?
Were the capacities of institutions relevant (duty bearers) for updated curricula for ICT specializations according to ICT labour market needs?
To which extent are these achievements anchored in the different target groups?
What is the benefit for the piloting institutions? What is the benefit for the Ministry of Education and Information Technology? What is the benefit for ICT labour market? What is the benefit for the whole of the ICT education?
To what extent can the implemented activities serve as a basis for future achievements and results in the field of VET system?
Were the planned resources (human) sufficient, deficient or as planned to achieve the expected results?
Was the curriculum pre- tested in all 6 institutions as planned? Which issues have emerged, if any?
Was it possible for all 250 students to conduct training in companies already? Which issues have emerged, if any?
Have the different stakeholders (name different institutions here) worked together as planned? Are there any issues that need to be solved? If yes, which ones?
How likely is it that the project will achieve its results

Impact:

To what extent is the project contributing to a long-term positive effect on the development of VET educational system? To what extent can the development and adoption of curricula contribute to ICT labour market in Republic of Moldova?
Did the project contribute to an improved reputation and attractiveness of the six VET schools in IT for students?

Sustainability:

To what extent does the Ministry of Education express their willingness and commitment to support the project in the country? To what extent do the schools recognize the importance of their involvement in this process?
Does the Ministry of Education ensure continuity of project results beyond the project end?
Can the project be replicated in other VET sectors?
To what extent did the organization of the internships contribute to the process of transfer of necessary competencies to students in order to make them attractive for the labour market?
To what extent did the organisation of internships contribute to students finding jobs in real companies?
What kind of activities and results can be envisaged beyond the project end by the project partners?

Evaluation approach and methods

The indicative number of working days foreseen is 15:

0,5 days briefing
2,5 days document review
1 day inception report
0,5 days debriefing on inception report
1 day mission preparation
4 days field mission (2 field trips)
3 days report drafting
2 days report finalization
0,5 days debriefing

Please comment on the indicative distribution of working days in your suggestions regarding the evaluation methodology.

A combination of primary data collection and secondary data review is expected during the evaluation. Primary data shall be collected through a variety of methods, including field observations, key stakeholder interviews and focus group discussions. Secondary data shall be collected from statistics and other relevant data at various levels of aggregation.

The ADA Evaluation Quality Standards are to be applied and compliance needs to be comprehensible in the evaluation.

Timetable

External Evaluator Ad: 19 December
Closing date and time: 14 January, 2015 4 PM
Public Opening: 15 January, 2015 4 PM
Signature of contract: 20 January, 2015

Evaluation Tasks and Timeline: The evaluation will start at January 21 and will take around 3 weeks. The deadline for the 1st draft of the evaluation report is 20th February 2014, with the final report by the end of February.

Tendering:
Tenders provide an outline of the proposed evaluation methodology (max. 3 pages), CV(s) of expert(s) proposed for the assignment, statement on availability of the proposed expert(s) during the planned evaluation period, information on fee rate in EUR, calculation of number of work days, as well as information on travel costs.

Briefing:
Half-day meetings with the CEPD, ATIC and the Austrian Development Agency.

Document review (desk study):
Project proposal, narrative reports, project publications, ADA Guidelines for Evaluation

Inception report:
The evaluator will produce an inception report which will be shared before starting the field studies. The format for the report can be found athttp://www.entwicklung.at/en/evaluation/

Field study:
The evaluation team (consisting of a senior and a junior evaluator) will undertake one field mission to Republic of Moldova to consult with all relevant actors (incl. Ministry of Education, Ministry of Informational Technology, ICT companies, piloting schools, other relevant stakeholders).

Report drafting:
The evaluator will draft a report in English, which should contain at most 25 pages without annexes. The draft report will be provided within 10 days after the field study. Comments will be provided by ADA. The format for the report can be found athttp://www.entwicklung.at/en/evaluation/

Report finalization:
The evaluation team will finalize the report according to the comments received by the ADA offices. The final version should not require any additional editing.

Debriefing:
Half-day meeting with the CEPD, ATIC and ADA.

Evaluation team

The evaluation will be done by an evaluator / evaluation team (senior and junior expert/evaluator), who need to have experience in evaluating projects. They will further have experience with organisational development and capacity building.

The selection criteria are the following:

For the senior evaluator:
– Minimum of 7 years experience of evaluation of development projects;
– Sound knowledge of the evaluating projects;
– Fluency in English;
– Knowledge of the regional languages is an asset.

for the junior evaluator:
– Minimum of 2 years experience of evaluation of development projects;
– Sound knowledge of the evaluating projects;
– Fluency in English;
– Knowledge of the regional languages is an asset.

CVs, reference evaluations and evaluation methodology need to be provided.

Reports

The evaluator(s) will produce three reports (inception report, draft report, final report based on the draft report and feedback). The reports shall be in English and between 15 and 25 pages in length (excluding annexes).

The draft evaluation report and final report need to be structured according to the ADA criteria and the evaluation questions. The quality of the reports will be assessed according to the evaluation quality criteria of the ADA:

Were the terms of reference fulfilled and is this reflected in the report?
Does the report contain a comprehensive and clear summary?
Is the report structured according to the ADA criteria and the evaluation questions?
Are cross-cutting issues (e.g. governance, gender) indicated in the report separately?
Does the report describe and assess the intervention logic (e.g. logframe)?
Are the conclusions and recommendations based on findings clearly stated in the report, and are they derivable from the latter?
Does the report clearly differentiate between conclusions, recommendations and lessons learnt?
Is it comprehensible how the evaluators have achieved their findings?
Are the recommendations and lessons learnt realistic and is it clearly expressed to whom the recommendations are addressed to?
Are the methods and processes of the evaluation sufficiently documented in the evaluation report?
Were the most significant stakeholders involved consulted?
Were the most important documents taken into consideration, and is the content of the latter reflected in the report?
Does the report present the information contained in a presentable and clearly arranged form?
Is the report well formatted and free from spelling mistakes and unclear linguistic formulations?
Can the report be distributed in the delivered form?

Coordination/Responsibility

The evaluator should work in close cooperation with ADA, CEPD and ATIC during the preparatory and implementation phase.

Logistical support will be offered to the evaluators by the CEPD and ATIC, e.g. assistance in arranging interviews and project visits. Ms. Rima Bezede, Ms. Octombrina Moraru (CEPD) and Ms. Ana Chiriţa (ATIC) will be the contact persons for the implementation phase.

The costs for the local travel in Moldova will be born by CEPD. Costs for international travel, accommodation and per diems, as well as translation services have to be covered by the evaluator and should therefore be included in the evaluator’s offer.

Additional Documents to be delivered upon the assignment of the task:

Project proposal
Logical Framework
Annexes will be provided as requested or needed
Other project documentation as requested or needed

CONTACT AND APPLICATION PROCEDURES:

The applications (CV(s) of expert(s) proposed for the assignment, reference evaluations, statement on availability of the proposed expert(s) during the planned evaluation period, description of the proposed methodology, information on fee rate in EUR, calculation of number of work days, as well as information on travel costs) have to be delivered to Educational Centre PRO DIDACTICA no later than 12 January, either by snail mail (at the risk of applicant) or hand delivered. The applications have to be delivered in English language.

Any application received after the deadline will not be considered.

Contracting Authority:                                  Educational Centre PRO DIDACTICA
Contact person:                                           Ms. Octombrina Moraru
Address:                                                      Educational Centre PRO DIDACTICA
13, Armeneasca
2012 Chisinau
Republic of Moldova
E-mail:                                                        omoraru@prodidactica.md
Tel:                                                              +373 69 535228

„Operator for technical support of computer“ is name of new speciality of VET IT specialization which is piloting now in 2 vocational schools, appeared after needs assessment of labour market.