Third-party Program Evaluation by the Salam Institute for Peace and Justice

The Salam Institute recently released its final report detailing the findings of its evaluation, which will be used to inform the future development of the project. The following represent some of the key findings of this report:

To assess the effectiveness of ICRD’s efforts to facilitate reform of the madrasas of Pakistan, the Salam Institute for Peace and Justice was engaged by the Smith Richardson Foundation to conduct a comprehensive third-party evaluation of the project. The Salam Institute, which has a history of conducting successful evaluations of programs involving themes such as democracy, nonviolence, pluralism, human rights, and peacebuilding as they relate to Islamic societies, conducted a program evaluation of the ICRD Pakistan Madrasa Project in May 2008.

In addition to multiple interviews and participant observation, the Salam Institute’s on-the-ground evaluation activities included seven focus groups comprising a total of 62 madrasa administrators, teachers, and university professors who have participated in the ICRD programs, as well as surveys to which a total of 57 responses were received.

The Salam Institute recently released its final report detailing the findings of its evaluation, which will be used to inform the future development of the project. The following represent some of the key findings of this report.

Necessity and Effectiveness of ICRD’s Madrasa Programs

  • ICRD’s Pakistan Madrasa Project “came at an excellent time in a context and process of change and is very relevant as it addresses an urgent need in Pakistan.” In addition to the improvement of teaching methods (pedagogical aspects of the training), the ICRD project is one of the very few madrasa programs “that directly focuses on themes of human rights, democracy, women rights, inter- and intra-faith dialogue, and conflict resolution [sic]. . . . [The project] is absolutely relevant to the existing needs and wants of the madrasa leaders.”
  • Focus group participants noted that there are “no safe spaces” in Pakistani society to explore differences and interact with people from other sects to learn more about their views and beliefs. The ICRD project provided a safe space for participants from different sects to come together and discuss various issues, thus fulfilling an important function in Pakistani society.
  • Focus group participants noted that there are “no safe spaces” in Pakistani society to explore differences and interact with people from other sects to learn more about their views and beliefs. The ICRD project provided a safe space for participants from different sects to come together and discuss various issues, thus fulfilling an important function in Pakistani society.
  • The majority of survey participants were satisfied with ICRD, as 63.2% found the program extremely useful and 96.5% found ICRD qualified to carry out these activities.
  • Ninety-one percent (91%) of survey participants felt that their abilities as administrators/faculty were enhanced; 78.8% felt that the ICRD project provided them with critical skills for carrying out their administrative/faculty responsibilities.
  • All of the teachers who attended the focus groups confirmed their need for faster change and more intense programs to reform their system of teaching.

Curricular Enhancement

  • The majority of focus group participants said the ICRD training programs and seminars have reduced their fear and concerns about the inclusion of sciences and secular subjects.
  • A total of 92.9% of survey respondents felt that they better understand the importance of incorporating social and scientific disciplines into the madrasa curriculum, while 61.4% have worked in teaching and encouraging others in their madrasa to teach more social and scientific disciplines.
  • A primary contribution of the ICRD programs, according to many of the teachers, is that they have allowed them to discuss reform openly with outside experts and have provided an outlet for their concerns and needs regarding how to integrate their schools into the Pakistani educational system.
  • Many of the participants felt that ICRD was effective in introducing the necessity of integrating principles of peace tolerance and dialogue into their curriculums and in introducing conflict resolution skills. Some participants shared how they have applied these skills.
  • Many of the teachers in the focus groups asserted that they have already accepted the need to incorporate secular disciplines such as mathematics, sciences, and computers. However, they continue to focus mainly on religious themes and lack the means to fully integrate these new subjects. The madrasas lack resources such as textbooks, manuals, and computers. One teacher stated, “We don’t even have soap to give to our students so they could take a proper bath.”

Pedagogical Enhancement

  • The ICRD project was effective in introducing new pedagogical styles and approaches, including critical thinking and problem-solving skills, to madrasa administrators and teachers. Seventy point two percent (70.2%) of the survey respondents felt that identifying and employing new teaching materials and styles was the most effective outcome of the program.
  • According to the surveys, 63.2% of respondents felt that the most effective feature of the ICRD program was the interactive and experiential teaching style.
  • Additionally, 98.2% of the participants considered interactive and hands-on learning and teaching methods as important (68.4% considered these very important) and 92% considered critical thinking skills as an important concept of the program (with 49.1% viewing it as very important.
  • Fifty-six point one percent (56.1%) of the participants stated that they increasingly employ interactive and experimental teaching styles.
  • Many of the focus group participants and interviewees stressed that ICRD was successful in the reduction of harsh punishment and the improvement of teaching methods for madrasa teachers. Reducing the use of corporal punishment in madrasas has been one of the key behavioral changes that has occurred as a result of participation in the ICRD programs. On various occasions, teachers stated that they have stopped relying on physical punishment and have begun opposing the use of “extreme forms of teaching” when they heard the opinions of experts. Quotes from participating teachers are indicative:
    • “I learned that a teacher must treat his students the way he treats his own children. And now by learning this each time I try to punish students, my children’s face appears in front of my eyes, and then I try to be polite and loving towards my students. ”
    • “I learned that when we teach we shouldn’t behave like dictators and we, the teachers, need to create an environment where students become friends with the teachers. I applied this learning and found it really wonderful as now my students frequently ask me questions”.
  • According to the surveys, 98.3% of the participants regarded understanding the psychology of the education process as one of the most important concepts that they were exposed to during the training.
  • Teachers in the focus groups provided examples of how they began utilizing new teaching methodologies as a result of the ICRD workshops. Some of the new approaches included the use of blackboards, taking breaks, using poetry to energize and engage students, and developing students’ writing capabilities (as the traditional focus has been only on reading). Participants stated:
    • “Our teaching methods became different, for example only one hour and then take a break so students can be fresh and can focus; learn to change the subject when students are bored or disengaged, then they became active.”
    • “Students in classes were encouraged to talk and express their views.”
    • “I learned through this program that a teacher should be very well-prepared before giving a lecture in his/her class, so to teach the best of the best in the classroom. Also, I learned how to focus on students who aren’t that good in studies, and that a teacher should give them extra time too.”

Promoting Tolerance, Peace, Human Rights—Attitudinal Changes

  • Ninety-eight point three percent (98.3%) of survey respondents agreed that they better understand the role of Islam in promoting religious tolerance and dialogue as a result of the ICRD program, with 66.7% strongly agreeing.
  • All of the survey respondents stated that promoting religious tolerance and dialogue was the most important concept they were introduced to during the course of the project. Many of the participants stated that now:
    • they are more open to meeting with others,
    • their perception about other sects and religious communities has changed, and
    • they have a better understanding of the West and the United States.
  • Additionally, 82.5 % of the survey respondents felt that the most effective feature of the ICRD program was the teaching of conflict resolution skills. In terms of outcomes, 78.9% felt that inclusion of concepts of peace, tolerance and human rights into the curriculum was the most effective; while 64.9% emphasized listening and working together with non-Muslims to create a more peaceful society and 52.6% emphasized the change in negative perceptions about non-Muslims and Western society and culture. (Note: The rating scale permitted multiple responses.)
  • According to survey results, 78.9% of the participants felt women’s rights to be one of the most important concepts they were exposed to during the program, while 57.9% felt that change in perceptions about the education and role of women in Muslim society was the most effective outcome of their participation in the ICRD Program.
  • Seventy point two percent (70.2%) of the participants stated that if there were an opportunity to do follow-up activities and apply their learning from the training, they would choose to work on teaching and encouraging others to teach Islamic principles and practices of interreligious/intercultural dialogue, human rights, democracy, women’s issues, and conflict resolution. Forty-nine point one percent (49.1%) of the participants indicated that if they had the opportunity, they would organize seminars and invite other scholars and non-Muslim members of their community to talk about peace, tolerance, and coexistence.
  • Forty-two point one percent (42.1%) of participants said they had “frequently” visited members of non-Muslim communities as a result of the ICRD program; 19.3% said they had “sometimes” visited these communities.
  • Participant quotes are indicative of these trends:
    • “I learned that when we are teaching we shouldn’t be criticizing other religions and we need to teach more about inter-religious understanding and harmony.”
    • “[American attacks on Afghanistan and Iraq] caused hatred against the US among the local people. Religious madrasas and religious people were misinterpreted and [the] wrong image was given. [The] situation was worse and tourists were attacked (but they were stopped by local Muslims). But now [after the ICRD program] they are against agitation and they want to educate people to record their protest peacefully and to remove the misperceptions about them in the world.”
    • One of the contributing factors to these changes of perception was the understanding that “Christians do care about Muslims”, that there are mosques in the United States, and that Muslims are free to practice their religion. According to one focus group participant: “Our misperceptions about Europe and America were corrected. Before we did not know that they care for Muslims and have mosques or freedom. This made our hearts soft and we thought if they can do that, why we cannot do the same here… we also corrected their perception that not everyone with a turban is a terrorist [sic].”
  • Now 61.4% of the participants feel that they are committed to staying involved with issues of dialogue and tolerance, pointing to the fact that they have traveled such long distances, under extremely difficult conditions and threats to their lives, as a strong indicator of this commitment.
  • According to the surveys, at least 40% of the participants are interested in participating in exchange programs where they can meet with scholars and religious leaders from different parts of the world; they were particularly interested in meeting Muslim scholars and teachers from the United States, as they were surprised to hear that Muslims could freely practice their religion and teach in America. This finding was supported by focus group participants, who showed an eagerness to join such programs and to learn about different religious traditions and cultures. While many of these teachers are isolated, they are also anxious to interact with people from other sects and faiths. “There is now a momentum and a desire to interact with the outside world which creates a unique window of opportunity for developing long-term constructive approaches to engaging with these institutions.” One participant stated:
    • “The trained people of different countries should get a chance to visit one another. It will give them help in developing a better understanding of each other and relevant issues. Those educators who are running madrasas in the US should visit and have interaction with their counterparts/colleagues in Pakistan which would help them in better understanding of their pedagogies.

Impact on Behavioral Changes

  • Fifty-nine point six percent (59.6%) of the survey participants indicated that as a result of their participation in the ICRD programs, they have started teaching and have encouraged others to teach Islamic principles and practices of inter-religious and inter-cultural dialogue.
  • Forty-three point nine percent (43.9%) of the respondents indicated that they have organized seminars and invited other scholars and non-Muslim members of the community to talk about co-existence and peace among others. Some of the participants organized meetings and workshops with other sects.
  • Participants shared that after their experience in the ICRD training, they “stopped referring to the other sects as zallin [those who went astray].” The well-known head of one madrasa stated that “after the training I issued a statement to all faculty in the madrasa to stop teaching extreme narratives about the other sects.”
  • One interviewee related the following story:
    • “When parliament issued new rules on women rights, the madrasas began shouting, but in training we proposed that they draft a more scientific and well-researched response and discuss it. They also sent it to the parliament and it was discussed and gained significant attention from politicians. They gave the advantages and disadvantages. Your role as madrasa is to perform such duty of giving a learned Islamic response to such issues. Parliament members do not know religion.”
  • Survey results indicate that participants feel they have had some success in making others aware of the themes they have been exposed to in the ICRD trainings—42.1% of the participants felt that they have made others “very aware”, 35% “moderately aware”, and 21.1% “somewhat aware.” Focus group participants also stated that they have recommended the ICRD program to other teachers and briefed their principals about it.
  • In addition to teaching and encouraging others to teach these themes, 66.7% of the participants stated that they have incorporated these themes into Friday sermons and other lectures. Such Friday sermons are one of the most common means of influencing the congregations of these religious leaders.
  • The ICRD program appears to have had the largest impact on the individual level. For example, one teacher called the ICRD program “the most influential event in his life and it changed his entire views about madrasas and ways of teaching and working with his students [sic].” One focus group participant stated:
    • “I used to ignore my wife and not interact with my children. They were afraid of me. They never told me about their lives. After these programs it changed.”
  • It is less clear how these changes are being implemented in the madrasas as a system or structure. According to participants, application of these new conflict resolution and teaching skills, as well as incorporation of new sciences and others disciplines into the madrasas, has not been easy due to lack of funding, books, manuals, or lecture outlines. Participants asked for specific lesson plans, textbooks, manuals, and other tools that they could take with them to apply in their schools.

Recommendations

  • ICRD should continue with its exposure programs, but focus on localized regional training that can produce more institutional effect. Investing in fewer madrasas, but with a higher and more significant number of participants from each madrassa, will produce more effective long-term outcomes. Madrasa participants should also play a greater role in the design of the programs to ensure that they are tailored to their specific needs.
  • In recruiting training participants, greater focus should be directed to “strategic targeting” of influential individuals, decision-makers, and heads or principals of madrasas who both can have a larger and wider impact in disseminating these ideas and also have the authority to implement them, including initiating changes in the curriculum.
  • Workshop participants should be provided with more training resources that they can take back to use in applying what they’ve learned in their schools.
  • ICRD should develop a systematic mechanism for monitoring whether the workshop participants have been able to apply skills and integrate themes learned in the trainings into their classes, use these skills in solving problems in their community, and continue to work with members of other sects or religions, as well as the impact on students. A more structured mechanism for monitoring the workshops conducted by master training program graduates should also be developed.
  • ICRD should initiate follow-up activities for workshop participants, including meetings where participants can meet other participants and discuss how the program impacted them, what kinds of activities they have been engaged in, what challenges they have faced, and how they addressed them. Mechanisms to provide support for those who are facing difficulties trying to apply these newly-acquired skills to their schools or communities should also be developed.
  • Exchange programs should be organized where selected scholars from Pakistan, the United States, and other countries can visit each other and participate in seminars, conferences, and workshops. This would create an opportunity for madrasa leaders to meet other religious leaders in their own contexts, clarify misunderstandings and contribute to changing negative stereotypes, and provide an opportunity for participants to exchange ideas and learn from each other.
  • It is now necessary for the Pakistan Madrasa Project to move to its second phase—institutionalization. In the difficult and volatile environment of Pakistan, institutionalization is necessary to sustain the project and expand its impact. To institutionalize the project, government institutions, local NGOs, and universities must be engaged and partnerships must be enhanced. As lack of resources invested in NGOs by the Pakistani government was cited as a limitation on the training programs by focus group participants, this is an area that should be explored. ICRD should also open a local office in Pakistan to enhance its capacity on the ground.
  • At the policy level: “The evaluation process demonstrates that change of perceptions of the teachers and madrasas administrators through this project contributes to a more pluralistic and democratic environment in Pakistan and especially among the communities which are under constant pressure form [sic] the militant Muslim groups to fight. As such, the [ICRD Pakistan Madrasa] Project provides clear evidence to US foreign policy experts that engaging with madrasas through educational reform and training is far more effective in changing perception than the approach of calling to demolish or abandon this school system in Pakistan or the Muslim world more generally.”
  • These recommendations confirm ICRD’s own assessment that we have reached the point of critical need and opportunity for “scaling up” the Pakistan Madrasa Project. The Madrasa Teacher Training Institute currently being developed to provide systematic training and eventual certification for madrasa teachers is a key first step in this regard. What is further needed is the commitment of significant resources by Pakistan and the international community to follow through with the rest of the enhancements necessary to take the project to scale.

—-

Click here to download a PDF version of the report.

Comments are closed.