This section discusses tools designed to evaluate the implementation and outcomes of Listening to One Another to Grow Strong (LTOA). Some of our evaluation tools were developed to help coordinators and facilitators track program deliveries, while others are designed to be sent back to our McGill University-based research team, where data will be analyzed to determine the program's effectiveness and provide feedback to help improve future material.
Team Report
We designed the Team Report as a tool for coordinators to track and evaluate program deliveries throughout—before, during and after—the implementation process. Coordinators can fill out the Team Report incrementally, as the deliveries in their region progress. After completion, coordinators send the Team Report back to McGill University, where aspects of the implementation can be compared with other years, and other communities. The Team Report can be printed, or filled-out online.
Facilitator Contributions to Evaluation
Aside from coordinators, facilitators also contribute to the evaluation process by filling out progress tracking forms. These forms are embedded into the first page of the description of each session. In addition to completing progress tracking forms, facilitators take part in brief, weekly conference calls with their coordinator and other facilitators in their region. These calls bring coordinators up to speed on the program progress in each community, while offering facilitators an opportunity to seek advice or offer recommendations to one another.
Questionnaires
Research is a fundamental component in the developmental process of LTOA. All communities that deliver our program are also research partners by virtue of their involvement with our project. Participation in our research process is straight-forward for youth and adults: before the first session and after the last session, all participants fill out an anonymous survey. We designed the survey to measure changes in different aspects of participants social, emotional and psychological wellbeing from the first to the last session, as well as to explore the quality of participants' experiences in the program. Research data is then aggregated and analyzed at our research unit at McGill University. We keep all of our data securely stored and uphold high standards of privacy. Before publishing our findings, we engage our collaborators in a consultation process to ensure that all parties feel accurately represented by our interpretations of the data.
Adult and Youth versions of our questionnaire, as well as consent forms and instructions for questionnaire administration, can be obtained by contacting the program coordinator: Michelle Kehoe at ltoanationalcoordinator [at] gmail.com
Our research team is at the service our partnered communities and offers assistance with data analysis to any community that would like to explore specific hypotheses about their data.
Regional and National Meetings
Beyond the process of collecting data, we frequently invite program collaborators—facilitators, coordinators and elders—to regional or national meetings to collect qualitative information on their experiences as collaborators on the project. In conversation with our partners, our research team develops an understanding of the aspects the program that are particularly effective and enjoyable, and the components of the program that are in need of improvement.
