Policy 126: Periodic Program Review of Graduate and Undergraduate Programs

Periodic program review (PPR) serves to ensure that programs strive to achieve the highest possible standards of academic quality, maintain a culture of continuous improvement, and continue to satisfy societal need. All undergraduate and graduate programs are required to undertake a periodic program review on a cycle not to exceed eight years.

Periodic program review is part of the Toronto Metropolitan University Institutional Quality Assurance Process (IQAP) which includes the following policies:

Policy 110: Institutional Quality Assurance Process
Policy 112: Development of New Graduate and Undergraduate Programs
Policy 126: Periodic Program Review of Graduate and Undergraduate Programs
Policy 127: Curriculum Modifications: Graduate and Undergraduate Programs

Together, the policies that constitute the IQAP serve to promote a culture of continuous improvement, striving to achieve the highest possible standards of academic quality.

1. Purpose

This policy governs the review of undergraduate and graduate programs that have been approved by Ontario Universities Council on Quality Assurance (Quality Council).

2. Scope

This policy includes all undergraduate and graduate programs¹ , both full and part-time, offered solely by Toronto Metropolitan University (the University) or in partnership with any other post-secondary institutions, including multi-disciplinary and interdisciplinary programs, offered across all modes of delivery. Programs offered jointly with other post-secondary institutions will be subject to the periodic program review policies of all the institutions. Programs which have been closed or for which admission has been suspended are out of scope for a PPR.

3. Definitions

3.1.Refer to Policy 110 for definitions related to this policy.

3.2.Refer to Policy 110 for Degree Level Expectations for Undergraduate and Graduate Programs.

4. External Authority and Responsibility

4.1.Ontario Universities Council on Quality Assurance (Quality Council)

4.1.1. The Quality Council reviews PPR Final Assessment Reports (FARs) on an annual basis.

4.1.2. The Quality Council audits the quality assurance process for PPRs on an eight-year cycle and determines whether the University has acted in compliance with the provisions of its IQAP.

5. Internal Authority and Responsibility

5.1.Senate

5.1.1. Senate has the final authority for the approval of PPRs of all Toronto Metropolitan University programs.

5.1.2. Senate has the final internal authority for the approval of all new and revised academic policies.

5.2.Standing Committees and Governance Council of Senate

5.2.1. Academic Standards Committee (ASC): A Standing Committee of Senate that assesses and provides recommendations to Senate for approval of undergraduate PPRs and assesses PPR follow-up reports as an information item for Senate. An additional update and course of action by a specified date may be requested of the program if ASC believes that there has not been sufficient progress.

5.2.2. Yeates School of Graduate Studies Council (YSGSC): A Governance Council of Senate that assesses and provides recommendations to Senate for approval of graduate program PPRs, and assesses PPR follow-up reports as an information item for Senate. An additional update and course of action by a specified date may be requested of the program if the YSGSC believes that there has not been sufficient progress.

5.2.2.1. YSGS Programs and Planning Committee (PPC): A committee of the YSGSC that reviews the PPR self-studies and appendices of graduate programs for completeness and determines if there are any issues prior to submission to a peer review team. Assesses complete graduate PPRs and provides recommendations to YSGSC.

5.3. Provost and Vice-President Academic

5.3.1. Following Senate approval, reports the outcomes of a PPR to the Board of Governors.

5.3.2. Submits FARs, including Implementation Plans and Executive Summaries, for all undergraduate and graduate PPRs to Quality Council annually, as per Quality Council’s required process.

5.3.3. Is responsible for the University’s participation in the Quality Council cyclical audit process.

5.4.Deputy Provost and Vice-Provost University Planning

5.4.1. Provides institutional data for PPRs.

5.5. Vice-Provost Academic

5.5.1. Has authority for PPRs of all undergraduate degree programs.

5.5.2. Is responsible for the undergraduate PPR schedule, for informing programs in written format of their forthcoming review, including the specific program or programs that will be reviewed and identifying, where there is more than one mode or site involved in delivering a specific program, the distinct versions of each program that are to be reviewed.

5.5.3. Is responsible for providing an orientation to PPR.

5.5.4. Is responsible for advising and monitoring throughout the PPR process.

5.5.5. Assesses PPR self-studies and appendices for completeness and determines if there are any issues prior to submission to a Peer Review Team (PRT).

5.5.6. Forwards complete PPRs to the ASC for their review and recommendation for approval to Senate.

5.5.7. Ensures that there is a FAR, Implementation Plan, and Executive Summary for each PPR.

5.5.8. Submits an undergraduate program FAR, including recommendations from ASC, for assessment and approval by Senate.

5.5.9. Forwards mandated follow-up reports to the ASC for their information, assessment, and report to Senate, then forwards to Senate for information.

5.5.10. Develops a manual that details the process and supports the preparation of the PPR of undergraduate degree programs.

5.6.Vice-Provost and Dean, YSGS

5.6.1. Has authority for PPRs of all graduate programs.

5.6.2. Is responsible for the graduate PPR schedule, for informing graduate programs in written format of their forthcoming review, including the specific program or programs that will be reviewed and identifying, where there is more than one mode or site involved in delivering a specific program, the distinct versions of each program that are to be reviewed.

5.6.3. Is responsible for providing an orientation to PPR.

5.6.4. Is responsible for advising and monitoring throughout the PPR process.

5.6.5. Appoints Peer Review Teams (PRT) for graduate programs.

5.6.6. Responds to the PRT Report as well as to the Program Response and the Faculty Dean’s Response to the PRT Report for graduate programs.

5.6.7. Ensures that there is a FAR, Implementation Plan, and Executive Summary for each graduate PPR.

5.6.8. Submits graduate program FARs, including recommendations, to Senate for assessment and approval.

5.6.9. Forwards mandated follow-up reports to YSGSC for its information, assessment, and report to Senate, then forwards to Senate for information.

5.6.10. Develops a manual that details the process and supports the preparation of the PPR of graduate degree programs.

5.7. Faculty Dean or Dean of Record^{2 3}

5.7.1. Reviews the undergraduate PPR self-study and appendices prior to submission to Department/School/Program/Faculty Council(s) and endorses the self-study and appendices following Council endorsement.

5.7.2. Appoints Peer Review Teams (PRT) for undergraduate programs.

5.7.3. Provides consultation to the Vice-Provost and Dean, YSGS regarding the appointment of PRTs for graduate programs.

5.7.4. Receives the PRT report for initial review. Where appropriate, requests further input or clarification from the PRT if the PRT Report does not address the requirements as outlined in the IQAP. Distributes to the program for response.

5.7.5. Responds to the PRT Report as well as to the Program Response to the PRT Report for undergraduate and graduate programs.

5.7.6. For undergraduate programs, reviews mandated follow-up reports to ensure progress with the recommendations from ASC and ensure that the implementation plan is effectively accomplished in a timely manner. If it is believed that there has not been sufficient progress, an additional update and course of action by a specified date may be required.

5.7.7. For graduate programs, reviews mandated follow-up reports to ensure that the implementation plan is effectively accomplished in a timely manner. If it is believed that there has not been sufficient progress, an additional update and course of action by a specified date may be required.

5.8. Chair/Director

5.8.1. Undergraduate Chair/Director of Department/School

5.8.1.1. Oversees the preparation of the undergraduate program self-study and appendices within the appropriate timelines.

5.8.1.2. Actively engages faculty, staff and students in the periodic program review process, and ensures their views are considered during the process of completing the self-study.

5.8.1.3. Presents a completed PPR self-study and appendices to the Faculty Dean or Dean of Record for initial review prior to presentation to Department/School/Program and/or Faculty Councils, as appropriate.

5.8.1.4. Prepares a response to the PRT Report.

5.8.1.5. Prepares the mandated PPR follow-up report for submission to the Faculty Dean or Dean of Record, and the Vice-Provost Academic by the specified date, normally within one year of Senate approval of the program review.

5.8.1.6. Administers the implementation plan to ensure that it is effectively accomplished in a timely manner.

5.8.2. Graduate Program Director

5.8.2.1. Oversees the preparation of the graduate program self-study and appendices within the appropriate timelines.

5.8.2.2. Actively engages Chairs/Directors, faculty, staff and students in the periodic program review process, and ensures their views are considered during the process of completing the self-study.

5.8.2.3. Presents a completed PPR self-study and appendices to the Faculty Dean or Dean of Record for graduate programs for initial review prior to presentation to Program Council.

5.8.2.4. Prepares a response to the PRT Report.

5.8.2.5. Prepares the mandated PPR follow-up report for submission to the Faculty Dean or Dean of Record, and the Vice-Provost and Dean YSGS by the specified date, normally within one year of Senate approval of the review.

5.8.2.6. Administers the implementation plan to ensure that it is effectively accomplished in a timely manner.

5.9.Department/School/Program and Faculty Council (where applicable)

5.9.1. Endorses the undergraduate or graduate self-study and appendices prior to submission to the Faculty Dean or Dean of Record.

6. Review of IQAP Policy and Procedures

6.1. The review of Toronto Metropolitan University IQAP policies will follow the procedures set out in the University’s IQAP Policy 110.

Back to top

Procedures

This document outlines the sequential stages of the Periodic Program Review (PPR) including the self-study report, the peer review and report, responses to the Peer Review Team (PRT) Report, assessments, endorsements, and approvals of undergraduate and graduate PPRs and implementation of recommendations. The key outcome from a PPR is the Final Assessment Report and associated Implementation Plan, which become the basis of a continuous improvement process through monitoring of key performance indicators.

1. The Self-Study Report

The self-study has descriptive, explanatory, evaluative and formative functions. It provides an opportunity for programs to assess academic quality and societal need, and plan for continuous improvement. It is essential that the self-study is reflective, self-critical, analytical, forward looking, and that it actively involves faculty, students, and staff in the process.

The self-study clearly identifies which program(s) is/are the subject of review, includes a description of how the self-study was written (i.e. the process), and explains how input was received from faculty, staff and students. The Director, Curriculum Quality Assurance and the YSGS Associate Dean, Programs, as appropriate, will advise programs throughout the review process on matters of content and format and to ensure that policy requirements are met.

Views of employers and/or professional associations incorporated into the self-study process will be solicited via methods deemed relevant and meaningful by the program and made available to the PRT committee. Some examples include surveys, interviews, or focus groups.

Self-Study Report details (Quality Council requirements are italicized)

1.1.Program Objectives

1.1.1. Consistency of the program’s objectives with the University’s mission and academic plans;

1.1.2. Program addresses societal need.

1.2.Program requirements

1.2.1. Appropriateness of the program’s structure and the requirements to meet its objectives and the program-level learning outcomes;

1.2.2. Appropriateness of the program’s structure, requirements and program-level learning outcomes in meeting the institution’s undergraduate or graduate Degree Level Expectations;

1.2.3. Discussion of the way(s) in which an EDI/anti-racism lens has been applied to the program;

1.2.4. Evidence of any significant innovation or creativity in the content and/or delivery of the program, including experiential learning opportunities;

1.2.5. Appropriateness and effectiveness of the mode(s) of delivery to facilitate students’ successful completion of the program-level learning outcomes; and

1.2.6. Ways in which the curriculum reflects the current state of the discipline or area of study. For Graduate Programs only:

1.2.7. Clear rationale for program length that ensures students can complete the program-level learning outcomes and requirements within the time required;

1.2.8. Evidence that each graduate student in the program is required to take a minimum of two-thirds of the course requirements from among graduate level courses; and

1.2.9. For research-focused graduate programs, clear indication of the nature and suitability of the major research requirements for degree completion.

1.3. Assessment of teaching and learning

1.3.1. Appropriateness and effectiveness of the methods for assessing student achievement of the program-level learning outcomes and degree level expectations;

1.3.2. Appropriateness and effectiveness of the plans to monitor and assess:

i) The overall quality of the program;
ii) Whether the program is achieving in practice its proposed objectives;
iii) Whether its students are achieving the program-level learning outcomes;
iv) How the resulting information will be documented and subsequently used to inform continuous program improvement; and

1.3.3. Grading, academic continuance, and graduation requirements, if variant from the University’s graduate or undergraduate policies.

1.4. Admission requirements

1.4.1. Admission requirements are appropriately aligned with the program objectives and program-level learning outcomes; and

1.4.2. Alternative requirements, if applicable, for admission into a graduate, second-entry or undergraduate program, such as minimum grade point average, additional languages or portfolios, and how the program recognizes prior work or learning experience.

1.5. Resources

Given the program’s class sizes and cohorts, as well as its program-level learning outcomes:

1.5.1. Participation of a sufficient number of qualified core faculty who are competent to teach and/or supervise in and achieve the goals of the program and foster the appropriate academic environment;

1.5.2. Discussion of the role and approximate percentage of adjunct and part-time faculty/limited term appointments used in the delivery of the program and the associated plans to ensure the sustainability of the program and quality of the student experience.

1.5.3. Supervision of experiential learning opportunities (if required);

1.5.4. Adequacy of the administrative units’ planned utilization of existing human, physical and financial resources;

1.5.5. Evidence that there are adequate resources to sustain the quality of scholarship, and research, and creative activities produced by students, including library support, information technology support, and laboratory access;

1.5.6. Identify areas that the program’s faculty, staff and/or students have identified as requiring improvement, or as holding promise for enhancement and/or opportunities for curricular change

For Graduate Programs only:

1.5.7. Evidence that faculty have the recent research or professional/clinical expertise needed to foster an appropriate intellectual climate, sustain the program, and promote innovation;

1.5.8. Where appropriate to the program, evidence that financial assistance for students is sufficient to ensure adequate quality and numbers of students; and

1.5.9. Evidence of how supervisory loads are distributed, in light of qualifications and appointment status of the faculty.

1.6.Quality and other indicators

1.6.1. Faculty: e.g. qualifications, funding, honours, awards, innovation, scholarly, research and creative (SRC) record, appropriateness of collective faculty expertise to contribute substantively to the program, commitment to student mentoring, class sizes; percentage of classes taught by permanent or non-permanent (contractual) faculty; numbers, assignments and qualifications of contractual faculty; other evidence that the program and faculty ensure the intellectual quality of the student experience;

1.6.2. Students: e.g. applications and registrations; grade-level for admission, retention rates; time-to-completion; final-year academic achievement; academic awards; scholarly output, success rates in provincial and national scholarships, competitions, professional and transferable skills, student feedback on their program and learning experiences;

1.6.3. Graduates: rates of graduation, employment six months and two years after graduation, post-graduate study, "skills match", employer and alumni feedback on program quality.

1.7. Quality Enhancement

1.7.1. Initiatives taken to enhance the quality of the program and the associated learning and teaching environment.

1.8. Appendices

1.8.1. Appendix I: Reports and data supporting the self-study, as outlined in PPR Manuals.

1.8.2. Appendix II: Concerns and recommendations raised in previous reviews: document and address. New programs undertaking their first program review will, in lieu, incorporate any steps taken to address issues or items flagged in the interim monitoring report for follow-up, and/or items identified for follow-up by the Quality Council.

1.8.3. Appendix III: Faculty Curriculum Vitae, containing abbreviated CVs with any personal information removed and relevant undergraduate and graduate program teaching included, as outlined in the PPR manuals.

1.8.4. Appendix IV: For undergraduate programs, Courses Outlines for all core required and core elective program courses and for graduate programs, Course Outlines for all courses offered by the program.

1.8.5. Appendix V: Summary of the self-study completion process, together with documentation of approvals and related communications⁴ .

Detailed guidelines for the Self-Study and Appendices are in PPR Manuals, provided by the Office of the Vice-Provost Academic and the Yeates School of Graduate Studies.

2. Protocol for Concurrent Undergraduate and Graduate Periodic Program Reviews

2.1.Where there are concurrent undergraduate and graduate PPRs, separate self- studies and appendices, with evaluation criteria and quality indicators for each discrete program being reviewed, are required.

2.2.External peer reviews of both undergraduate and graduate programs may be coordinated if the Department/School chooses to do so; however, separate PRT reports are required.

3. Protocol for Joint Programs

3.1. The self-study clearly identifies which program(s) is/are the subject of review, and explains how input was received from faculty, staff and students at each partner institution. There will be a single self-study, initiated by the Vice-Provost Academic (for undergraduate joint programs) or by the Vice-Provost and Dean, YSGS (for graduate joint programs), in consultation with the partner institution.

3.2. Selection of the reviewers involves participation by each partner institution.

3.2.1. Where applicable, selection of the internal reviewer requires joint input

3.2.2. The selection of the peer reviewer could include one internal to represent all partners; and

3.2.3. The selection could give preference to an internal reviewer who is from another joint program, preferably with the same partner institution.

3.3. The site visit involves all partner institutions and preferably at all sites.

3.3.1. Reviewers consult faculty, staff and students at each partner institution, preferably in person.

3.4.Feedback on the reviewers’ report is solicited from participating units at each partner institution, including the Deans or Dean of Record.

3.5.Preparation of a FAR, including Implementation Plan and Executive Summary, requires input from each partner.

3.5.1. There is one FAR, including Implementation Plan and Executive Summary, that is subject to the appropriate governance processes at each partner institution;

3.5.2. The FAR, including Implementation Plan and Executive Summary is posted on the university website of each partner;

3.5.3. Partner institutions agree on an appropriate monitoring process for the Implementation Plan; and

3.5.4. The FAR, including Implementation Plan and Executive Summary should be submitted to the Quality Council by all partners.

4. Protocol for Interdisciplinary and Multidisciplinary Programs

4.1. For multidisciplinary and interdisciplinary programs, the Faculty Dean of Record will oversee the periodic program review.

4.2. The self-study clearly explains how input was received from faculty, staff and students of the program. There will be a single self-study and site visit.

5. Protocol for Accredited Programs

5.1. With approval of the Vice-Provost Academic or the Vice-Provost and Dean, YSGS, as applicable, PPRs may be coordinated with any professional accreditation review; however, a self-study and appendices (with all the evaluation criteria listed in Section 1), separate from an accreditation review, are required.

5.2. In the case of accredited programs, at their discretion, the Vice-Provost Academic or the Vice-Provost and Dean, YSGS, as applicable, may require a separate Peer Review Team when the accrediting body’s assessment does not fully cover all the areas required by the University’s PPR process. The Peer Review Team Report must be a separate document from the Accreditation PRT Report.

6. Reviews and Endorsements Prior to Submission to an External Peer Review Team

6.1.Initial review by Faculty Dean or Dean of Record

6.1.1. The Faculty Dean or Dean of Record will review the undergraduate self- study and appendices for completeness and to determine if there are any issues prior to a review and endorsement by the Department/School/Program/Faculty Council.

6.2.Department/School/Program Council; Faculty Council

6.2.1. Following the review of the self-study and appendices by the Faculty Dean or Dean of Record, the Department/School/Program Council and Faculty Council, as appropriate, will review and endorse the self-study and appendices. A record will be kept of the date(s) of the relevant Council meeting(s), along with any qualifications or limitations placed by the Council(s) on the endorsement.

6.3.Program Advisory Council (for Undergraduate Programs)

6.3.1. Consultation with the Program Advisory Council (PAC), established in accordance with Senate Policy 158, is an integral part of the review process. The timing and nature of the PAC consultations can vary depending on the program and its specific requirements. In some instances, it may be advantageous to seek input from the PAC earlier in the process and incorporate the feedback into the self-study report. In other cases, the Faculty Dean or Dean of Record may present the endorsed self-study report and its appendices, along with any qualifications or limitations, to the Program Advisory Council (PAC) for its review and comments. In all instances, a record will be kept of the date(s), minutes, and members attending the meeting(s). A response to the comments of the PAC may be included in the Peer Review Team (PRT) Report (see Section 7.6) and/or the responses to the PRT Report (see Section 8).

6.4.Faculty Dean or Dean of Record

6.4.1. Following endorsement of the self-study and appendices by the Department/School/Program Council and Faculty Council, as appropriate, and a review by the PAC (for undergraduate programs), the Faculty Dean or Dean of Record will endorse the self-study and appendices for preliminary submission to the Vice-Provost Academic for undergraduate PPRs, or to the Vice-Provost and Dean, YSGS for graduate PPRs.

6.5.Vice-Provost Academic

6.5.1. The Vice-Provost Academic will review the undergraduate self-study and appendices for completeness and to determine if there are any issues prior to submission to a Peer Review Team.

6.6. YSGS Programs and Planning Committee (PPC)

6.6.1. The YSGS PPC will review the graduate self-study and appendices for completeness and to determine if there are any issues prior to submission to a Peer Review Team.

7. Peer Review

Peer Review Teams are required for program reviews for all undergraduate and graduate degree programs, and graduate diploma programs.

As soon as possible after the self-study and appendices have been reviewed for completeness by the Vice-Provost Academic, for undergraduate programs, or the YSGS PPC, for graduate programs, it will undergo review by a Peer Review Team (PRT), as described below.

7.1.Selection of Peer Review Team (PRT) Members

7.1.1. All members of the PRT will be at arm’s length⁵ from the program under review. The Dean of Record or Vice-Provost and Dean YSGS, as appropriate, is responsible for verifying members of the PRT meet this criterion.

7.1.2. The external and internal reviewers will be active and respected in their field, and normally associate or full professors with program management experience, including an appreciation of pedagogy and learning outcomes.

7.1.3. If graduate and undergraduate program reviews are done concurrently, the Faculty Dean or Dean of Record, and the Vice-Provost Academic and the Vice-Provost and Dean, YSGS may authorize a combined PRT, if appropriate. However, separate PRT reports are required.

7.1.4. PRT for Undergraduate Periodic Program Reviews The PRT for undergraduate program reviews will consist of:

7.1.4.1. Two external reviewers qualified by discipline and experience to review the program(s); and

7.1.4.2. The option of one further internal reviewer from within the university, but from outside the discipline (or interdisciplinary group). Internal reviewers are not members of the program under review. Internal reviewers will provide external reviewers with an institutional perspective on related policies and processes.

7.1.4.3. The PRT composition is the same for undergraduate programs taught in collaboration with colleges or institutions outside of Ontario. In a joint program with other Ontario universities, unless one internal reviewer is agreed upon by all participating institutions, if applicable, one internal reviewer will be appointed from each participating institution.

7.1.4.4. External review of undergraduate periodic program reviews will normally be conducted on-site, but may be conducted by desk review, virtual site visit or an equivalent method if the external reviewers are satisfied that the off-site option is acceptable. The Provost (or designate) will also provide a clear justification for the decision to use these alternatives.

7.1.5. PRT for Graduate Periodic Program Reviews The PRT for graduate program reviews will consist of:

7.1.5.1. Two external reviewers qualified by discipline and experience to review the program(s); and

7.1.5.2. One further external reviewer, or an internal reviewer from within the university, but from outside the discipline (or interdisciplinary group). Internal reviewers are not members of the program under review. Internal reviewers will provide external reviewers with an institutional perspective on related policies and processes.

7.1.5.3. The PRT composition is the same for graduate programs taught in collaboration with colleges or institutions outside of Ontario. In a joint program with other Ontario universities, unless one internal reviewer is agreed upon by all participating institutions, if applicable, one internal reviewer will be appointed from each participating institution.

7.1.5.4. External review of a doctoral program must incorporate an on-site visit.

7.1.5.5. Certain master’s programs (e.g., professional master’s programs, fully online) may be conducted by desk review, virtual site visit or an equivalent method if both the Provost (or designate) and external reviewers are satisfied that the off-site option is acceptable. An on-site visit is required for all other master’s programs.

7.1.6. PRT for Concurrent Periodic Program Reviews The PRT for the concurrent review of an undergraduate and graduate program will consist of at least:

7.1.6.1. Two external reviewers qualified by discipline and experience to review the programs; and

7.1.6.2. One further external reviewer, or an internal reviewer from within the university, but from outside the discipline (or interdisciplinary group). Internal reviewers are not members of the program under review. Internal reviewers will provide external reviewers with an institutional perspective on related policies and processes.

7.2. Appointment of Peer Review Team (PRT) Members

7.2.1. Undergraduate

7.2.1.1. The membership of the undergraduate PRT will be determined and appointed by the Faculty Dean or Dean of Record based on written information provided by the program.

7.2.1.2. The program will provide the Faculty Dean or Dean of Record with names and brief biographies of four or more faculty external to Toronto Metropolitan University and two or more faculty internal to the University (if applicable).

7.2.1.3. Initial communications to the reviewers, such as interest, availability, and invitation to serve on a PRT, will come only from the Faculty Dean or Dean of Record.

7.2.1.4. The Faculty Dean or Dean of Record will invite one of the external reviewers to act as Chair of the PRT.

7.2.2. Graduate

7.2.2.1. The membership of the graduate PRT will be determined by the Vice-Provost and Dean, YSGS in consultation with the Faculty Dean or Dean of Record and the program.

7.2.2.2. The program will provide the Vice-Provost and Dean, YSGS with names and brief biographies of four or more faculty external to Toronto Metropolitan University and two or more faculty internal to the University (if applicable).

7.2.2.3. Initial communications to the reviewers, such as interest, availability, and invitation to serve on a PRT, will come only from the Vice-Provost and Dean, YSGS.

7.2.2.4. The Vice-Provost and Dean, YSGS, in consultation with the Faculty Dean or Dean of Record for graduate programs, will invite one of the external reviewers to act as Chair of the PRT.

7.3. The Mandate of the Peer Review Team (PRT)

The general mandate of the PRT is to evaluate and report in writing on the academic quality of the program and the capacity of the School or Department to deliver it in an appropriate manner. Recommendations on significant resource issues, such as faculty complement and/or space requirements, that are within the purview of the university’s budgetary decision-making processes, must be tied directly to issues of program quality or sustainability.

The PRT will submit a joint report, based on the template provided by the University, that addresses all of the following:

7.3.1. commentary on the substance of the self-study as outlined in Section 1 above;

7.3.2. identification and commendation of the program’s notably strong and creative attributes;

7.3.3. description of the program’s strengths, areas for improvement, and opportunities for enhancement;

7.3.4. commentary about the way(s) in which an EDI/anti-racism lens has been applied to the program;

7.3.5. evidence of any significant innovation or creativity in the content and/or delivery of the program relative to other such programs;

7.3.6. at least three recommendations for specific steps to be taken that will lead to the continuous improvement of the program, distinguishing between those the program can itself take and those that require external action; and

7.3.7. if appropriate, identify the distinctive attributes of each discrete program documented in the self-study, where more than one program/program level, program mode, and/or program location has been simultaneously reviewed.

7.4.Information Provided to the Peer Review Team Before the Site Visit

7.4.1. Undergraduate

7.4.1.1. The PRT will be provided with a Letter of Invitation from the Faculty Dean or Dean of Record, the PRT’s mandate, information on the University, and its mission and Academic Plan. Once confirmed, the Dean or Dean of Record will provide to the PRT a site visit agenda, and the self-study with all appendices. This communication will remind the PRT of the confidentiality of the documents presented and all aspects of the review process.

7.4.2. Graduate

7.4.2.1. The PRT will be provided with a Letter of Invitation from the Vice-Provost and Dean, YSGS, the PRT’s mandate, information on the University, and its mission and Academic Plan. Once confirmed, the Vice-Provost and Dean, YSGS will provide to the PRT a site visit agenda, and the self-study with all appendices. This communication will remind the PRT of the confidentiality of the documents presented and all aspects of the review process.

7.5. The Peer Review Team (PRT) Site Visit

7.5.1. The PRT will be provided with:

7.5.1.1. Access to program administrators, staff, and faculty (including representatives from joint or collaborative Ontario institutions), administrators of related departments and librarians, and students (including representatives from joint or collaborative Ontario institutions), as appropriate. Access can be via group discussions, one-on-one meetings, tours, or other methods deemed relevant by the program, and as indicated in the PRT site visit agenda.

7.5.1.2. Coordination of site visits to Ontario institutions offering joint programs (excluding college collaborative programs), where appropriate, and any additional information that may be needed to support a thorough review.

7.5.2. Undergraduate

7.5.2.1. At the opening of the site visit the Vice-Provost Academic will review the PRT mandate, outline the role and obligations of the PRT, the format for the PRT Report as outlined in the template guidelines, and the timeline for completion of the PRT report. Recognition of the university’s autonomy to determine resource priorities will also be clearly indicated.

7.5.2.2. At the close of the site visit the PRT will hold a debriefing involving the Provost and Vice-President Academic, the Vice-Provost Academic, the Faculty Dean or Dean of Record, and any others who may be invited by the Faculty Dean or PRT.

7.5.3. Graduate

7.5.3.1. At the opening of the site visit the Vice-Provost and Dean, YSGS will review the PRT mandate, outline the role and obligations of the PRT, the format for the PRT Report as outlined in the template guidelines, and the timeline for completion of the PRT report. Recognition of the university’s autonomy to determine resource priorities will also be clearly indicated.

7.5.3.2. At the close of the site visit, the PRT will hold a debriefing involving the Provost and Vice-President Academic, the Vice-Provost and Dean, YSGS, the Faculty Dean, and any others who may be invited by the Faculty Dean or PRT.

7.5.4. Concurrent

7.5.4.1. At the opening of the site visit the Vice-Provost Academic and the Vice-Provost and Dean, YSGS will review the PRT mandate, the format for the PRT Reports as outlined in the template guidelines, and the timeline for completion of the PRT Reports.

At the close of the site visit the PRT will hold a debriefing involving the Provost and Vice-President Academic, the Vice-Provost Academic, the Vice-Provost and Dean, YSGS, the Faculty Dean and any others who may be invited by the Faculty Dean or the PRT.

7.6. Peer Review Team (PRT) Report

7.6.1. Undergraduate

7.6.1.1. Within four weeks of the completion of the site visit, the PRT for an undergraduate program will submit its written report to the Faculty Dean or Dean of Record and the Vice-Provost Academic. The Faculty Dean or Dean of Record will forward this report to the Chair/Director of the program.

7.6.1.2. The Faculty Dean or Dean of Record may request further input or clarification from the PRT if the PRT report does not meet the requirements of the IQAP.

7.6.2. Graduate

7.6.2.1. Within four weeks of the completion of the site visit, the PRT for a graduate program will submit its written report to the Vice- Provost and Dean, YSGS. The Vice-Provost and Dean, YSGS will forward this report to the Chair/Director of the program and to the Faculty Dean or Dean of Record.

7.6.2.2. The Faculty Dean or Dean of Record may request further input or clarification from the PRT if the PRT report does not meet the requirements of the IQAP

8. Responses to the Peer Review Team (PRT) Report

8.1.Program Response

8.1.1. Undergraduate

8.1.1.1. Within four weeks of receipt of the PRT Report, the program will submit a written response to the PRT Report to the Faculty Dean or Dean of Record. The written response will include:

• Comments, corrections and/or clarifications of items raised in the PRT Report;
• An implementation plan that identifies and prioritizes recommendations. The implementation plan should take into consideration the recommendations from the PRT as well as the self-study. A template for the Implementation Plan is provided in the undergraduate PPR manual; and
• Where relevant, an explanation of why recommendations of the PRT will not be acted upon.

8.1.2. Graduate

8.1.2.1. Within four weeks of receipt of the PRT Report, the program will submit a written response to the PRT Report to the Vice-Provost and Dean, YSGS and to the Faculty Dean. The written response will include:

• Comments, corrections and/or clarifications of items raised in the PRT Report;
• An implementation plan that identifies and prioritizes recommendations. The implementation plan should take into consideration the recommendations from the PRT as well as the self-study. A template for the implementation plan is provided in the graduate PPR manual; and
• Where relevant, an explanation of why recommendations of the PRT will not be acted upon.

8.2. Faculty Dean’s or Dean of Record’s Response

8.2.1. For undergraduate and graduate programs, within four weeks a written response must be provided by the Faculty Dean or Dean of Record. The response will address:

• The recommendations proposed in the self-study report;
• Further recommendations of the PRT;
• The Program Response to the PRT Report;
• Any changes in organization, policy or governance required to meet the recommendations;
• The resources that would be provided to support the implementation of selected recommendations; and 
• A proposed timeline for the implementation of any of those recommendations.

8.2.1.1. If the self-study report or the implementation plan is revised following, or, as a result of, the PRT review, the original and the revised documents must be resubmitted through the Faculty Dean or Dean of Record to the Vice-Provost Academic or the Vice-Provost and Dean, YSGS. If the Faculty Dean or Dean of Record and the Vice-Provost Academic or the Vice-Provost and Dean, YSGS believe that this document differs substantially from the original, it must be resubmitted to the Department/School/Program Council(s) and Faculty Councils, if appropriate, for further endorsement followed by decanal endorsement.

8.3. Vice-Provost and Dean, YSGS’s Response

8.3.1. For graduate programs, within four weeks a written response must be provided by the Vice-Provost and Dean, YSGS. The response will address:

• The recommendations proposed in the self-study report;
• Further recommendations of the PRT;
• The Program Response to the PRT Report;
• The Faculty Dean’s Response to the PRT Report;
• Any changes in organization, policy or governance required to meet the recommendations;
• The resources that would be provided to support the implementation of selected recommendations; and
• A proposed timeline for the implementation of any of those recommendations.

8.3.1.1. If the self-study report or the implementation plan is revised following, or as a result of, the PRT review, the original and the revised documents must be resubmitted through the Faculty Dean or Dean of Record to the Vice-Provost and Dean, YSGS. If the Faculty Dean or Dean of Record and the Vice-Provost and Dean, YSGS believe that this document differs substantially from the original, it must be resubmitted to the Department/School/Program Council(s) and Faculty Councils, if appropriate, for further endorsement followed by endorsement by the Faculty Dean or Dean of Record and the Vice-Provost and Dean, YSGS.

9. Assessment and Recommendations by ASC or YSGS Council

9.1. Academic Standards Committee (ASC)

9.1.1. For undergraduate programs, the PPR, which includes the Self-Study Report and Appendices (Section 1), with revisions if required, the PRT Report, the Program Response, and the Faculty Dean or Dean of Record’s Response is submitted to the Vice-Provost Academic for submission to the ASC for assessment.

9.1.2. The ASC will then make one of the following recommendations:

9.1.2.1. Senate approve the PPR, with a mandated follow-up report(s).

9.1.2.2. Senate approve the PPR with conditions, as specified, and with a mandated follow-up report(s).

9.1.2.3. The PPR be referred to the Faculty Dean or Dean of Record for further action in response to specified weaknesses and/or deficiencies.

9.1.2.4. The PPR, as submitted, be rejected.

9.2. Yeates School of Graduate Studies (YSGS)

9.2.1. For graduate programs, the PPR, which includes the Self-Study Report and Appendices (Section 1), with revisions if required, the PRT Report, the Program Response, the Faculty Dean or Dean of Record’s Response, and the Vice-Provost and Dean, YSGS’s Response is submitted to the YSGS Programs and Planning Committee (PPC).

9.2.1.1. The PPC will assess the PPR and make one the following recommendations:

9.2.1.1.1. That the PPR be sent to the YSGS Council with or without qualification;

9.2.1.1.2. That the PPR be returned to the program for further revision.

9.2.2. Upon approval by the YSGS PPC, the YSGS Council will assess the report and make one of the following recommendations:

9.2.2.1. Senate approve the PPR, with a mandated follow-up report(s).

9.2.2.2. Senate approve the PPR with conditions, as specified, and with a mandated follow-up report(s).

9.2.2.3. The PPR be referred to the Faculty Dean or Dean of Record for further action in response to specified weaknesses and/or deficiencies.

9.2.2.4. The PPR, as submitted, be rejected.

10. Final Assessment Report (FAR)

10.1. For undergraduate programs, the Office of the Vice-Provost Academic, or for graduate programs, the Office of the Vice-Provost and Dean, YSGS will prepare for Senate a Final Assessment Report (FAR)⁶ which provides an institutional synthesis of the peer review team report and strategies for continuous improvement. The FAR:

10.1.1. identifies significant strengths of the program;

10.1.2. identifies opportunities for further program improvement and enhancement with a view towards continuous improvement;

10.1.3. lists all recommendations of the external reviewers and the associated separate internal responses and assessments from the unit and from the Dean(s);

10.1.4. explains why any external reviewers’ recommendations not selected for further action in the Implementation Plan have not been prioritized;

10.1.5. includes any additional recommendations that the unit, the Dean(s) and/or the university may have identified as requiring action as a result of the program’s review;

10.1.6. identifies who will be responsible for approving the recommendations set out in the FAR.

10.2. The FAR must include an executive summary suitable for posting on the university website, excluding any confidential information; and

10.3. The FAR must also include the Implementation Plan as per Sections 8.1.1.1 (undergraduate) and 8.1.2.1 (graduate) that identifies and prioritizes program recommendations for implementation, who will be responsible for providing resources needed to address the recommendations, as well as who will be acting on those recommendations, and timelines for acting on and monitoring the implementation of those recommendations.

11. Senate Approval

11.1. The Vice-Provost Academic and/or the Vice-Provost and Dean, YSGS, as appropriate, will submit a PPR Report to Senate which includes the FAR and the requirements of a mandated Follow-up Report(s).

11.2. Senate has the final academic authority to approve the PPR Report to Senate, which includes the FAR and the mandated follow-up report(s).

12. Follow-up Report

12.1. The PPR Report to Senate will include a date, within one year of Senate approval of the PPR, for a mandated follow-up report to be submitted to the Faculty Dean or Dean of Record and the Vice-Provost Academic or the Vice-Provost and Dean, YSGS, as appropriate, on the progress of the implementation plan and any further recommendations. The PPR Report to Senate may also include a date(s) for subsequent follow-up reports.

12.2. The Chair/Director and Faculty Dean or Dean of Record, and the Vice-Provost and Dean, YSGS, if applicable, are responsible for requesting any additional resources identified in the PPR through the annual academic planning process. The relevant Faculty Dean or Dean of Record, or the Vice-Provost and Dean, YSGS, if applicable, is responsible for providing the identified resources, if feasible, and the Provost and Vice-President Academic is responsible for final approval of requests for extraordinary funding. Requests should normally be addressed, with a decision to either fund or not fund, within two budget years of the Senate approval of the PPR.

The follow-up report will include an indication of any resources that have been provided at the time of the report.

12.3. The follow-up report(s) will be reviewed by the Faculty Dean or Dean of Record and ASC or YSGS Council, as appropriate. If it is believed that there has not been sufficient progress on the implementation plan, an additional update and course of action by a specified date may be required.

12.4. The follow-up report will be forwarded to Senate as an information item following review by the ASC or YSGS Council, as appropriate.

13. Distribution Requirements

13.1. Under the direction of the Vice-Provost Academic and the Vice-Provost and Dean, YSGS, the Office of the Vice-Provost Academic shall publish the Executive Summary, the FAR (excluding any confidential information), and the action of Senate for each approved PPR on the University’s Curriculum Quality Assurance website with links to the Senate website and the Provost and Vice-President Academic’s website, all of which are publicly-accessible.

13.2. Complete PPR documentation, respecting the provisions of FIPPA, will be made available through the Office of the Vice-Provost Academic and Office of the Vice-Provost and Dean, YSGS.

13.3. The approved FAR, Executive Summary and Implementation Plan (excluding any confidential information) will be provided to the program Department/School to act on, as appropriate.

13.4. The Provost and Vice-President Academic will submit annually the FARs (excluding any confidential information) of all approved PPRs to the Ontario Universities Council on Quality Assurance (Quality Council), as per the required process.

13.5. The Provost and Vice-President Academic is responsible for the presentation of the PPR Executive Summary and its associated implementation plan to the Board of Governors for its information.

14. Selection for Cyclical Audit

The Cyclical Review of undergraduate and/or graduate programs that were undertaken within the period since the conduct of the previous Audit are eligible for selection for the university’s next Cyclical Audit.

Back to top

Appendix I

Choosing Arm’s Length Reviewers

Best practice in quality assurance ensures that reviewers are at arm’s length from the program under review. This means that reviewers/consultants are not close friends, current or recent collaborators, former supervisor, advisor or colleague.

Arm’s length does not mean that the reviewer must never have met or even heard of a single member of the program. It does mean that reviewers should not be chosen who are likely, or perceived to be likely, to be predisposed, positively or negatively, about the program.

Examples of what may not violate the arm’s length requirement:

• Appeared on a panel at a conference with a member of the program
• Served on a granting council selection panel with a member of the program
• Author of an article in a journal edited by a member of the program, or of a chapter in a book edited by a member of the program
• External examiner of a dissertation by a doctoral student in the program
• Presented a paper at a conference held at the university where the program is located
• Invited a member of the program to present a paper at a conference organized by the reviewer, or to write a chapter in a book edited by the reviewer
• Received a bachelor’s degree from the university (especially if in another program)
• Co-author or research collaborator with a member of the program more than seven years ago
• Presented a guest lecture at the university
• Reviewed for publication a manuscript written by a member of the program

Examples of what may violate the arm’s length requirement:

• A previous member of the program or department under review (including being a visiting professor)
• Received a graduate degree from the program under review
• A regular co-author and research collaborator with a member of the program, within the past seven years, and especially if that collaboration is ongoing
• Close family/friend relationship with a member of the program
• A regular or repeated external examiner of dissertations by doctoral students in the program
• A recent doctoral supervisor (within the past seven years) of one or more members of the program

Additional Advice for Choosing External Reviewers

External reviewers should have a strong track record as academic scholars and ideally should also have had academic administrative experience in such roles as undergraduate or graduate program coordinators, department chair, dean, graduate dean or associated positions. This combination of experience allows a reviewer to provide the most valuable feedback on program proposals and reviews.

Source: Ontario Universities Council on Quality Assurance (Quality Council)

Back to top

¹ For the purpose of the IQAP, program refers to the credential(s) under review, including undergraduate degree, graduate degree, professional master’s degree, or graduate diploma.

² The Dean of Record for interdisciplinary graduate programs that cross faculty lines is the Vice-Provost and Dean, YSGS (Policy 45)

³ See Toronto Metropolitan University, Senate Policy 110 for definition.

⁴ Reviews, endorsements, approvals and related communications must be documented and retained at every stage of the PPR process. The documentation (1.11.5. Appendix V) accompanies the complete PPR that is submitted to the ASC or YSGS Council (Section 9.0)

⁵ See Appendix I for information on arm’s length selection of PRT members.

⁶ See Toronto Metropolitan University, Senate Policy 110 for a definition