Monitoring and evaluation
Monitoring is a process of carefully observing and generating data in order to assess the successes and drawbacks of the implementation phase and use this data for assessing project’s effectiveness, which is also called project evaluation.
The evaluation particularly focuses on the achievement of defined objectives, effectiveness, impact and sustainability of the action, based on specific explicit or implicit indicators.
The overall Monitoring and Evaluation phases of a project often tend to be given less significance than the actual implementation phase, mainly due to a tendency to perceive monitoring and evaluation as mere ways to check, to prove and highlight how successful (or how much of a failure) a project is. Often this perception is a result of more top-down approaches that come from donor requirements, however a culture of peace approach to these important processes in observing, tracking and following up as well as evaluating whether an action has been relevant and resulted in necessary changes would mean that the project team regards this process as a process of accountability to the community/ies they work with. What this means is that the project implementers feel a sense of responsibility and hold respect in high regard toward the communities they work with and thereby are committed to learning from the overall process of implementing projects that aim to transform/change a given context for the benefit of given communities. Understanding the benefit and usefulness for these processes is key, but it requires shifting from a donor-driven mentality to a community-centering one.
The main tools fo incorporating culture of peace in these cycles of the project is to maintain transparent and honest dialogue with community/ies where projects are taking place and maintain opportunities for those communities to give feedback, to decide what should be monitored and how it should be monitored, evaluated and learned. In this sense a common theme throughout the entire project cycle starting with the planning phase and especially with regards to ensuring a collaborative theory of change is active engagement of the community/ies in planning and deciding what is relevant to achieve and how to evaluate results of actions. This process must be inclusive of diverse perspectives, transparent and committed to continuing trust building. As a result, projects and their results can be more sustainable in the long run because communities will be empowered to bring about the change they seek and become co-creators of that change along with a wider network of actors collabortating toward a common goal.
Here again it is important to be transparent, open and flexible with monitoring and evaluating a project. For example, if failures/mistakes are openly and transparently acknowledged, then there is a CoP approach to a project, because it allows for flexibility to shift dynamics/assumptions that are not working. Also addressing challenges need to be openly discussed with communities, rather than withholding information and coming up with solutions within a set project team. On the other hand, sometimes a project can have successes that go unnoticed because one may take such successes for granted when not taking a particular context into account. This is often the case with more difficult, closed and conflicted contexts where even a small-scale change is in fact a significant success given the circumstances. A culture of peace approach to evaluation would look deeper at the extent to which a particular change is significant when placed against the background of a particular context. Therefore, context, conflict and cultural sensitivity must be taken into account for evaluation in order to respect the extent of change that occurs given these sensitivities. Consequently, a qualitative assessment can often contribute to this process and provide much more learning than a quantitative assessment. This is not to say that evaluating based on quantitative measurement is insignificant, however any quantitative measurement by itself is not enough for a culture of peace approach to evaluation because it does not give the full picture of what the numbers mean in real life.
While preparing a results-based framework (also called logical framework/matrix), as well as a monitoring and evaluation plan, certain indicators can be taken into account in order to integrate culture of peace into the results framework as well as monitoring plan. These indicators can be both qualitative and quantitative.
Some things to keep in mind when considering indicators in monitoring plans:
- keeping context and cultural sensitivity in mind; being aware and mindful of context;
- building and maintaining trust with and among communities where projects are implemented;
- promoting non-violent modes of relating to others and respecting diversity during the monitoring process;
- ensuring effective communication where the purpose of monitoring is transparent to all involved in a project, and
- where observations are shared with relevant actors for assessment and evaluation;
- reflection meetings as a tool to make the monitoring and evaluation process more participatory and horizontal;
- staying open to changes that are not planned for and/or any results or lack of results that can be deemed as a failure
- instead of a success – these are important to notice and document for learning; the main idea is to learn from
- mistakes and keep an open mind about what may be considered as failures
Doing a baseline study initially can improve the final evaluation process of a project by giving a baseline for measuring change in a particu- lar context once the implementation phase is complete. The CoP way of doing this is value based and includes respect toward communi- ties by keeping communities infromed and seeking their feedback with trust, understanding that information and knowledge is non-linear and that what may be provided as research by leading institutions such as the UNDP, academic institutions, etc are not any more relevant than what comes from the grassroots. If a baseline study involves interviews with individuals and/or focus groups, then it must be clear for what purpose the study is being conducted for the interviewees to ensure a high ethical standard. Furthermore, in the process of monitor- ing, collecting data and tracking changes it is crucial to maintain confidentiality with sensitive data and in most contexts of the South Caucasus, it is crucial to save information on encrypted files in order to maintain safety of people involved in any given project. This ensures respect to individuals involved in a given project and a committment to do no harm as ways to practice monitoring in a culture of peace sensitive way.
What to ask yourself (and wishing your team) to ensure a culture of peace approach to monitoring:
- Have you and your team planned for reflection meetings inclusive of all relevant actors involved in a project?
- To what extent are you and your team engaging the community in the monitoring pro- cess and creating space for feedback?
- How flexible are you and your team to changing activities where necessary in order to ensure the process flows organically and to avoid the process stagnating?
- Are you and your team acknowledging and reacting accordingly to any unexpected progress in your project that goes against the original expected results?
- Are you and your team staying sensitive to confidentiality of data and encrypting all saved files?
Depending on the extent of the trust and transparent communication within the project team, it could be most conducive to a culture of peace approach when conducting an internal evaluation for a project and involving participants as co-evaluators, instead of beneficiaries of the activities and merely questioning them about how those activities impacted them. This would mean that the platform for gathering the community set up from the planning stage of the project is maintained throughout the project cycle. Given that the project team knows best the process through which a given project has been planned, implemented, monitored and followed up, it can be useful to evaluate the project with the support of any self-assessment tool and possibly external support. Such a process can build even more trust within the group and support the learning process of the team and participants of a project overall as they navigate the challenges faced, the way challenges have been addressed, the shortcomings of any given component of the project cycle and any successes that may have gone unnoticed. At the same time, an external evaluation can also contribute to a culture of peace approach to evaluating a project by allowing for more distance from the project and internal dynamics of a project team. Such an approach to evaluation is usually best done when there are challenges within the team that have not yet been overcome and a need for an outside perspective can assess the project successes and challenges with a fresh eye.
Both short-term and long-term evaluation should contribute to ongoing and future assessment of results. What this is means is that any observation through monitoring once evaluated as not conducive to the overall objective of the project should be considered as a motiva- tion to change what is not working. For instance, conducting daily evaluations among participants in trainings allows for feedback with which it is possible to improve the program for the next day. It is important to get feedback from participants on a regular basis, taking into account their perspectives and planning reflection sessions to have better insights what worked well and what did not during project implementation. While evaluating project’s results several points should be taken into consideration:
- Timing and design for monitoring and evaluation should be relevant and should not disadvantage any group participating in the project;
- The evaluator/s should know context in which the project occurred, have sensitivity towards the local context and sufficientlanguage skills to find out all relevant details;
- Analysis of assessed results during an evaluation process is as important as the results themselves. Often this work does not get done and the most important component of the learning process gets lost. Analysis contributes to great learning by getting to the bottom of the “how’s” of results, regardless of whether the changes observed are positive or negative. If one does not learn why and how some- thing works or does not work, one cannot have sustainable programs and projects cannot evolve in an accountable and respectful manner toward all involved.
In a culture of peace approach participants of a project are perceived and accepted as co-creators/implementers, which means that they are seen as human beings who have agency over an outcome. During the monitoring and evaluation process it is important to move away from simply perceiving participants as recipients of benefits or target groups and involve them more in all processes of the entire project cycle. This means, as mentioned previously, that with any reflection of observations, evaluations, etc. participants are also consulted. Such an approach will not only create more trust between project implementers and participants, but will also make the project stronger by ensuring that the process stays relevant and meaningful for all involved. Furthermore, such a process will involve beneficiaries and target groups in the learning process and thereby ensure the position of co-creators for continued action for social change even after a particular project ends
What to ask yourself (and wishing your team) to ensure a culture of peace approach to evaluation:
- Is your evaluation planned in an inclusive and participatory manner? Do project par- ticipants get to decide what should be measured and how?
- Is there an openness, willingness and focus on learning from challenges and successes of the project?
- Is there sufficient effort put into analysis of evaluation results for further learning?
- Is feedback from project beneficiaries taken into account for next steps?
- Are evaluation results presented to the community as well as donors and other relevant actors such as other NGOs or groups working with similar issues that can learn from the experience of your team?