An evaluation of a school leadership development programme - Jansen, Rethar (Institute for Monitoring and Evaluation, 2015)
This study set-out as an evaluation of Symphonia's Partners for Possibility leadership development programme for school principals in South Africa. Broadly, the goal of the programme is to improve the educational outcomes of under-resourced schools. The primary audience of this evaluation is the Partners for Possibility Operations Manager. During initial engagement with the programme stakeholders the feasibility of different evaluation types were considered. These deliberations centred on one of, or combination of, the following evaluation types: theory evaluation, process evaluation and outcomes evaluation. At first it appeared that an outcomes evaluation was entirely possible. After examining if an outcomes evaluation is feasible it became clear it was impossible to do this. As a result the focus shifted from outcome evaluation to what other kind of evaluation was feasible and acceptable to programme staff. In order to determine this, the evaluability assessment approach was utilised in order to conduct a theory evaluation and to determine if it was possible to obtain, at reasonable cost, relevant performance data. The revised programme theory behind the Partners for Possibility's programme was derived via semi-structured interviews with the programme Stakeholder Engagement Consultant and Operations Manager. A thorough analysis of programme documentation and the programme website also informed the development of the revised programme theory. The plausibility of the revised programme theory was assessed by conducting a literature review of similar evaluations and related literature. The literature review suggests that the revised programme theory is plausible. The revised programme theory can be summarised as follows: if the programme incorporates activities associated with both the instructional and transformational leadership development models then principals will be able to address the barriers that prevent their school from improving its educational outcomes. The results of the investigation into, is it possible to obtain relevant performance monitoring data, found that this was not possible. This investigation did determine that the programme has designed a monitoring system that effectively maps onto the process theory that was developed as part of the theory evaluation. However, semi-structured interviews with the Operations Manager and Stakeholder Engagement Consultant revealed that this monitoring system was not systematically maintained and that data was not available. This finding is what compromised the ability for an outcomes evaluation to proceed. It is strongly recommended that the programme staff implement and make use of their monitoring system. If the programme staff do this then a process evaluation is recommended for the programme in order to determine if the programme is implemented with fidelity.