TERM OF REFERENCE (TOR)
DIOCESE ACCOMPANIMENT EXTERNAL EVALUATION
I. BACKGROUND
Since May 17th, 2006, KARINA (Caritas Indonesia) was established as socio pastoral body of KWI. Karina grew up as one of the humanitarian organization in Indonesia. Established with the aims to become national coordinator, animator and facilitator of all Diocese Caritas organizations, Karina realized that the frontline of the socio pastoral work is based within the dioceses that should be stronger and more professional in order to provide better services to the "poorest of the poor". Considering the basic context of the dioceses and the future challenges, Karina in 2010 decided to accompany Diocese Caritas by developing an initiative called Diocese Accompaniment (DA). DA has aim to"Develop and strengthen Karina as socio pastoral body of Indonesian Bishop Conference (KWI), capable of working in professional, sustainable, accountable and committed way at the service of Caritas network in order to serve the poorest of the poor".
After the first years of working with Diocese Caritas, it was highlighted the need for a more comprehensive and holistic accompaniment in order to strengthen the Caritas network in Indonesia which is a precious tool for a wider articulation of the social pastoral work of the church. The reflection has widely acknowledged the important role that Karina and the Caritas network could play, as an articulation of the social pastoral services of the Catholic Church of Indonesia.
Inspiration of Diocese Accompaniment came out from many sharing and deep exchanges, as well as the results of Midterm evaluation of Karina KWI strategic plan in 2010 and from feed-backs by Diocese Caritas during the implementation of Global Proposal project and its final evaluation. Some of the feed-backs were:
Ø During the first years, there wasn't a complete follow up after Karina KWI activities in/with the Diocese Caritas [Some reasons were offered in relation to the achievements of these activities as a result either of no follow-up after the trainings or activities, or lack of understanding of Caritas].
Ø Karina used to propose "one standardized approach" which is not flexible: the Diocese Caritas hoped that: "both trainings and workshops conducted by Karina will be not standard for all Diocese Caritas".
Ø Most of the Diocese Caritas think that it's important that Karina would start to listen to Diocese Caritas needs and they're willing to face the issues in their structure with the help of Karina;
Ø There is a needs of the Diocese Caritas to play a more active role within Caritas services.
Diocese Accompaniment aims to answer to these needs expressed by Diocese Caritas:
Ø Avoiding the risk to give to Diocese Caritas the same answers without acknowledging the uniqueness of each Diocese Caritas (each Diocese Caritas has different situations which should be first understood and then developed together);
Ø Accompanying the Diocese Caritas in order to give a concrete follow-up after trainings and activities implemented during the first years;
Ø Listening to the Diocese needs directly from them, trying to spend more time with them and understand the context where they live and operate;
Ø Accompanying them in order to build their capacity to let them become a more sustainable Diocese Caritas.
Karina decided to start Diocese Accompaniment with a pilot experience called "Diocese Accompaniment start up activities"which was run for five months, starting on September 2010 until January 2011. This experience was very important to build the Diocese Accompaniment approach and to write the proposal for 2011 – 2013. In some of the involved Diocese Caritas (Jayapura, Tanjung Selor, Makassar) several activities were started during this pilot period. For this reason in thepresent report some of these steps are also described, although officially out of the period of this report.
Upon request the report of "Diocese Accompaniment start up activities Sep 2010-Jan 2011" is available.
Karina Kwi (Caritas Indonesia) with Diocese Accompaniment initiative is working in order to promote the strengthening of the 3 dimensions of a Diocese Caritas: Identity/Values, Structure, Context Analysis. In particular the aim is:
Ø to strengthen Diocese Caritas Basic Identity,
Ø to strengthen Diocese Caritas Basic structure,
Ø to accompany Diocese Caritas in pursuing locally defined objectives based on a better knowledge of the territory.
These main dimensions are integrated and there are important links and connections to be acknowledged. The final aim is to provide better services to the poorest of the poor at local/Diocese level.
II. QUESTIONER FOR ALL DIOCESES CARITAS
What are the different between Diocesan caritas that had DA and the other onesthat had no DA? We can have a survey among all the diocesan caritas (with some questions) to analyse what are the differences:Is DA too intensive?Why is possible to do DA in some places and no in other?
Here are some Draft questions:
1. How did you feel supported by Karina? (capacity development, Funds, Accompaniment, partnership, friendship, knowledge)
2. What kind of relationship do you have with Karina
3. What kind of learning did you get from Karina
4. Have you ever heard about the threebasic dimensions (Identity, Structure and Context)?
5. In your opinion, is it very important for Diocese Caritas to know and understand as well making activities plan related to the three basic dimensions? Why is it important?
III. EXTERNAL EVALUATION PURPOSES AND OBJECTIVES
The intent andpurpose of the external evaluation is tolook at the RELEVANCE, EFFECTIVENESS, EFFICIENCY, IMPACT and SUSTAINABILITY of the implementation of the Diocese Accompaniment Program which was run by Karina-KWI together with DioceseCaritas.
IV. MAIN POINTS OF EVALUATION
4.1. Relevance
The key questions
1. Whether the result of the Diocese Accompaniment Program in accordance with its objectives?
2. Was this Diocese Accompaniment Program has been able to address the needs of Caritas-Caritas Diocese?
3. Whether during the implementation of the Diocese Accompaniment Program has been a change of context, so that the implementation of the program have been unable to answer the context changes? What has changed from the context?
4.2.Effectiveness
Includes all effortsor actions toachieve the objective or goal.
The key questions:
1. Was mentoring activities have been carried out with fairly realistic?
2. Were all accompaniment activities can already be run easily?
3. Were all the accompanimet activities already planned measurable and beneficial to the Caritas Diocese?
4.3.Efficiency
The accuracy of the methodand the ability to do a good job. In other words, minimum resources toachieve optimal results.
The key questions:
1. Was the implementation of mentoring programsrun on time?
2. Was staff capacity according to need?
3. What has been done to achieve optimal efficiency?
4.4. Impact
Positive or negativeinfluence and the changes experienced by Diocese Caritas as a result of the implementation of the Diocese Accompanimenwhich has been implemented by Karina-KWI is for this.
The key questions:
1. Extent to which the implementation of the Diocese Accompaniment program has been able to empower all Diocese Caritas? The empowerment can be assessed through the increase of:
1) Knowledge of Diocese Caritas in relation to the three dimensions of institutional policy; The identity, structure and context and how the third dimension applied to the real service every day.
2) Knowledge and skills of Diocese Caritas to makemapping or profilingeach diocese Context.
3) Knowledge and skills of Diocese Caritas in designing programs that are SMART (Specific, Measurable, Attainable, Realistic, and Timebond).
4) The ability of Diocese Caritas in managing the program in a professional, transparent and accountable.
2. The extent to which Diocese Caritas has been able to manage the strategic methods andthe important steps in order to avoidfriction or conflictwith the commissions in each diocese? What Methods and steps have beentaken to synergizebetween Caritas withcommissions, in eachdiocese?
3. The extent to which DioceseCaritas has been able to apply knowledge and skillsin designing service programs. These capabilities can be assessed through:
1) Logframe design
2) Proposals that have been made
3) Methods andstrategies of program implementation
4. Are there impactsdo not become goalsof the Diocese Accompaniment (DA) but occurredduring the implementation of the accompaniment?
4.5. Sustainibility
Diocese Caritas ability to continue andmanage the benefits which is the goal of the Diocese Accompaniment after the implementation of the program ended in June 2014.
The key questions:
1) The extent to which the Bishopgave support to Diocese Caritas?
2) The extent to which the commissions in Dioceses provide support to Diocese Caritas?
3) The extent to which external parties of the church (government and non-government agencies) provide support to Caritas Diocese?
4) Is Caritas Diocese has had personnel (staff or volunteer) who are committed to service work of the Caritas Diocese?
5) Is Caritas Diocese already has a fund raising strategy as an effort in the provision of funds for his own services?
6) The extent to which the commitment of Diocese Caritas in updating the action plan of the three basic dimensions and profiling?
V. EVALUATION TARGETS
The target of this external evaluation are all Dioceses Caritas which accompanied by Karina-KWI throughDiocese Accompaniment program. The Dioceses Caritas in question are; Diocese Caritas of Amboina, Diocese Caritas of Ruteng, Diocese Caritas of Sintang, Diocese Caritas of Tanjung Selor, Diocese Caritas of Jayapura and Diocese Caritas of Ketapang.
VI. EVALUATION STAGES
6.1. Preparation
Includes a preparatory meeting with the evaluator-Karina KWI(DA team) and review documents related tothe planning, implementation and evaluation of the Diocese Accompaniment program.
6.2. Field
Include socialization purposes and objectives ofthe evaluation to The Caritas-The Caritas Diocese, data collection and informationrelating to the impact and sustainability of the Diocese Accompaniment Program.
6.3. Final Report
Includes analysis of the data andinformation collected relating to the Relevance, Effectiveness, Efficiency, impact and sustainability of the Diocese Accompaniment Programas well as the preparation of the final report.
VII. EVALUATION METHOD
1. Reviewing Karina's documents relating to the planning, implementation and evaluation of the DioceseAccompaniment Program.
2. Collecting data and information relating to the Relevance, Effectiveness, Efficiency, impact and sustainability of the Diocese Accompaniment Program at Diocese Caritas through interviews of key figures, FGD and observation.
3. Analyze data and information to be able to provide an assessment of the Relevance, Effectiveness, Efficiency, Impact and Sustainability of the Diocese Accompaniment Program.
VIII. EVALUATOR RESPONSIBILITY
In general, the responsibility of the evaluator arethe following:
1. Conform to established procedures, regarding funding andemployment solutions.
2. Do a review of the documents related to Karina KWI-Planning, implementation and evaluation of the Diocese Accompaniment Program.
3. Develop appropriate methods and tools to gather data and information relating Relevance, Effectiveness, Efficiency, Impact and Sustainability of the Diocese Accompaniment Program.
4. coordinate the overall activitiesand provide an update on the development ofevaluation to Karina-KWI.
5. Discuss the draft finalevaluation report with Karina-KWI.
6. Submit a finalevaluation report in Indonesian andEnglish to Karina-KWI timely manner.
IX. EVALUATION SCHEDULE AND EVALUATOR CAPACITYREQUIRED
9.1. Implementation schedule
This external evaluationactivities will be conducted for 21 (twenty one) calendar days, commencingAugust 25 - 13 September 2014.
9.2. Evaluator
The evaluationrequires 2 evaluators. One comes from the Indonesian Caritas networkthat understands the context of the DA services and the others are from external. To obtain optimal results, the followingarethe expected capacityoftheevaluators:
Intenal evaluator:
1) Have experience, knowledge andskills to evaluate the empowerment program.
2) Know and understand about the Diocese Accompaniment Program.
External Evaluator:
1) Have experience, knowledge andskills to evaluate the empowerment program.
2) At least Know and understand the peculiarities of Caritas services.
X. EVALUATION PRODUCT
The final results are expected to be prepared by evaluator arefinal report written in Indonesian andEnglish containing detailed findings and conclusions on the evaluation accompaniedwith realistic recommendations that useful for future learning for Karina. The report should include both quantitative and qualitative information include following structure.
Text Box: 1. Cover page (1 page) 2. Abbreviation (1 page) 3. Acknowledgement (1 page) 4. Executive summary (1-2 pages) 5. List of content (1 page) 6. List of figures (1 page) 7. Introduction: (1-2 pages) 8. Project setting (1-2 pages) 9. Evaluation objective (1 page) 10. Evaluation methodology (1 page) 11. Evaluation findings (max 20 pages) a. Relevance b. Implementation standard c. Effectiveness of input delivered d. Efficiency e. Impacts f. Sustainability 12. Recommendation (1-2 pages) 13. Evaluator's biography Annexes
XI. PERSON IN CHARGE
The person in charge of theactivities of this external evaluation is Karina-KWI.
XII. PLAN FEES AND PAYMENT SYSTEM
Payment of the cost of the work is based on achievement or progress of the workat each stage, namely:
1) the amount of funds for evaluators, RP100,000,000 (One Hundred Million)
2) Term I: As much as 20% (twenty percent) afterthe MoU was signed
3) Term II: As much as80% (eighty percent) after the final report is received.
XIII. CONCLUDING
Karina-KWI request that evaluators submit complete bidding documents that include curriculum vitae, offer evaluation methodologies and alsooffers the cost ofthe evaluation address sdmkarina@gmail.com at the latest 24 Aug 2014
No comments:
Post a Comment