Academic Program Reviews, Emerson College

Index

Purpose
Guiding Principles
Procedures
Timeline and Useful Information
Self-Study Outline
External Evaluators Report

Purpose

The purpose of academic program reviews at Emerson College is to put into place a system of continuous assessment, planning, and improvement, which is intended to help achieve the academic mission and commitments of the college. Program review is an evaluative, not a descriptive process; it results in judgments about the quality of a program and the adequacy of its resources. The process of program review should be a cooperative exercise between the program faculty, chair or director, and administration that results in meaningful recommendations for program improvement

Guiding Principles

The academic review process is a shared responsibility of the faculty, staff, and administration.
Program reviews should be directed toward the continued development and improvement of programs.
The evaluation of programs should be based on academic or professional criteria, within the context of the current and available institutional resources.
All relevant College constituencies should be involved in the process.
Program reviews should be independent from other reviews, such as professional accreditation, although the data collected should be shared as much as possible to avoid unnecessary extra work for the units involved.
The review process should result in tangible actions.

Procedures

All programs will be reviewed every five to seven years
All program reviews will be coordinated through the Office of Academic Affairs. A member of that office will be designated as coordinator of program reviews and will be responsible for coordinating timetables, the collection of institutional data, etc., with the School Dean, Department Chair, Academic unit Director, etc
Academic program reviews will include an internal self-study by the department or unit under review
Academic program reviews will also include external review by a team of evaluators from outside of Emerson
The self-study report is a narrative discussion of the topics specified in the “Self-Study Outline” (below), to be used by both the external reviewers and internal audiences to better understand the academic unit. The report will be organized according to a prescribed and approved outline. Recognizing that each department and unit is different, the self-study outline should be customized to address the specific and unique topics and issues relevant to that program or unit
A department Chair or unit Director may assign a faculty or staff member to serve as coordinator for the self-study report. (Note: Normally, the department Chair should not serve as the author of the self-study report)
The external review team will be comprised of two to four evaluators from outside of Emerson who are acknowledged experts in the academic discipline or disciplines, or professional areas, represented by the department/unit under review. The School Dean and Department Chair or Unit Director will select the external evaluators. When possible, external reviewers will be at the department chair, dean, or director level to ensure broad experience in the discipline and some understanding of higher education procedures and processes. The Dean or Director, and the Provost, must approve the external evaluators
The role of the external team is two-fold. First, it will evaluate the department/unit based on the self-study; meet with undergraduate and graduate students, faculty, staff, administrators, and the Department Chair or unit Director; and review department/unit documents such as student files, student theses and projects, syllabi, faculty curricula vitae, and department/unit policies, practices and programs
Second, it will provide recommendations for strengthening the department/unit. The external evaluators will be provided with a copy of the self-study in advance of their site visit. They will conduct a two- or three-day campus visit to meet with key constituencies. The external review team will discuss its initial findings with the Dean and Chair or Administrative Director, Assistant Provost for Academic Programs, and the Provost and then submit to the Provost, Assistant Provost for Academic Programs, and Dean or Director a draft program review for commentary, fact-checking and clarification
The external evaluator’s final report will be shared with the department/unit. The department faculty or unit staff will be asked to respond to the report and its recommendations. Based on this discussion, the Chair and Dean, or Director, will write a departmental response to the recommendations that details the department/unit goals and the responsibilities of the department/unit and the methods for addressing the recommendations
The Provost, Assistant Provost for Academic Programs, Graduate Dean (if appropriate), School Dean, Department Chair or unit Director, and one faculty member from the department or staff member from the unit will meet to discuss the self-study, the report, and the department/unit response. The agreed-upon final recommendations will be disseminated to and discussed with the department’s faculty or unit staff, and should be integrated into the department/unit long-range planning and budgeting

Guiding Principles

The academic review process is a shared responsibility of the faculty, staff, and administration.
Program reviews should be directed toward the continued development and improvement of programs.
The evaluation of programs should be based on academic or professional criteria, within the context of the current and available institutional resources.
All relevant College constituencies should be involved in the process.
Program reviews should be independent from other reviews, such as professional accreditation, although the data collected should be shared as much as possible to avoid unnecessary extra work for the units involved.
The review process should result in tangible actions.

Procedures

All programs will be reviewed every five to seven years
All program reviews will be coordinated through the Office of Academic Affairs. A member of that office will be designated as coordinator of program reviews and will be responsible for coordinating timetables, the collection of institutional data, etc., with the School Dean, Department Chair, Academic unit Director, etc
Academic program reviews will include an internal self-study by the department or unit under review
Academic program reviews will also include external review by a team of evaluators from outside of Emerson
The self-study report is a narrative discussion of the topics specified in the “Self-Study Outline” (below), to be used by both the external reviewers and internal audiences to better understand the academic unit. The report will be organized according to a prescribed and approved outline. Recognizing that each department and unit is different, the self-study outline should be customized to address the specific and unique topics and issues relevant to that program or unit
A department Chair or unit Director may assign a faculty or staff member to serve as coordinator for the self-study report. (Note: Normally, the department Chair should not serve as the author of the self-study report)
The external review team will be comprised of two to four evaluators from outside of Emerson who are acknowledged experts in the academic discipline or disciplines, or professional areas, represented by the department/unit under review. The School Dean and Department Chair or Unit Director will select the external evaluators. When possible, external reviewers will be at the department chair, dean, or director level to ensure broad experience in the discipline and some understanding of higher education procedures and processes. The Dean or Director, and the Provost, must approve the external evaluators
The role of the external team is two-fold. First, it will evaluate the department/unit based on the self-study; meet with undergraduate and graduate students, faculty, staff, administrators, and the Department Chair or unit Director; and review department/unit documents such as student files, student theses and projects, syllabi, faculty curricula vitae, and department/unit policies, practices and programs
Second, it will provide recommendations for strengthening the department/unit. The external evaluators will be provided with a copy of the self-study in advance of their site visit. They will conduct a two- or three-day campus visit to meet with key constituencies. The external review team will discuss its initial findings with the Dean and Chair or Administrative Director, Assistant Provost for Academic Programs, and the Provost and then submit to the Provost, Assistant Provost for Academic Programs, and Dean or Director a draft program review for commentary, fact-checking and clarification
The external evaluator’s final report will be shared with the department/unit. The department faculty or unit staff will be asked to respond to the report and its recommendations. Based on this discussion, the Chair and Dean, or Director, will write a departmental response to the recommendations that details the department/unit goals and the responsibilities of the department/unit and the methods for addressing the recommendations
The Provost, Assistant Provost for Academic Programs, Graduate Dean (if appropriate), School Dean, Department Chair or unit Director, and one faculty member from the department or staff member from the unit will meet to discuss the self-study, the report, and the department/unit response. The agreed-upon final recommendations will be disseminated to and discussed with the department’s faculty or unit staff, and should be integrated into the department/unit long-range planning and budgeting

Timeline

Before the Review
1. Preparation: In the Spring semester of the academic year before the external review, the Assistant Provost meets with the Chair of the department, or unit Director, the Associate Chair, and the faculty, or, in the case of other units, the staff member designated to lead the Self-Study committee. At this meeting, we will:
  - share the Academic Program Review Resources Handbook;
  - discuss the purpose, guiding principles and procedures of reviews at Emerson College;
  - confirm the calendar for the department or unit self-study and dates of external reviewer visit;
  - organize the members of faculty and staff who will form the self-study committee.
2. Reviewer Selection:
  1. In May, the Chair or unit Director:
    - provides the names and contact information of possible external reviewers to the Dean, who will also add their suggestions. The list should have at least eight names and their contact information.
  2. In early June, when the list is confirmed by the Chair and the Dean:
    - the Chair sends the list to the Assistant Provost who will contact those listed to invite them to be an external reviewer.
  3. No later than late August:
    - the external reviewer team is confirmed and the Assistant Provost shares the confirmed reviewer names with the Chair, or unit Director.
3. The Schedule
  1. The department Chair, working with staff, plans the schedule of the visit, which must include:
    - An introduction meeting with the Provost and Assistant Provost when the reviewers arrive;
    - Meetings with the Dean, the Chair, faculty (full time and affiliated), undergraduate students, graduate students (where applicable), staff;
    - An exit meeting with the Provost, Assistant Provost, the Dean, and the Chair before the reviewers leave.
  2. The completed schedule is shared by the Chair with the Assistant Provost and the Senior Executive Administrator to the Provost.
4. The Self Study
  1. The Associate Chair, or, in the case of other units, the staff member designated to put together the draft of the department or unit Self-Study:
    - gathers from the members of the Self-Study committee, each one representing an area of the department or unit, a draft of their area responses to the elements of the self-study outline assigned to them;
    - ensures that all data is included in the self-study;
    - drafts the full Self-Study from the components provided by the Self-Study committee members;
    - shares the draft with the department and with the Assistant Provost for feedback and edits;
    - the Self-Study should be no more than 50 pages, excluding the Appendices sections.
The Reviewer Visit
- At least two weeks before the visit, the Chair or unit Director shares the Self-Study with the reviewers, the Dean, the Assistant Provost, and the Provost
- At the exit meeting, the Chair takes notes on the initial observations of the reviewers, answers and asks questions, and clarifies any factual errors
After the Visit
1. Department or Unit Engagement and Chair’s Response
  1. When the external reviewers send their review, the Chair:
    - shares it with the Provost and Assistant Provost;
    - corrects factual errors and sends it back to the reviewers to update;
    - shares the updated review with the Provost, Assistant Provost, and Senior Executive Administrator to the Provost
    - shares the review with the department for discussion and prioritization of the action items from the recommendations;
    - writes the Chair’s response to the review and sends it to the Provost, Assistant Provost, and Dean;
    - fills in, in bullet point form, the reviewer recommendations and department action items and their prioritization in the APR Recs Action Item Tracking excel spreadsheet on Drive
Closing the Loop
- A final meeting of the Provost, Assistant Provost, Dean, Chair, Associate Chair, and any other key faculty or staff is scheduled by Academic Affairs to discuss the insights, next steps, department action items from the review.
Before the Next Review
- As they are implemented, the Chair updates the completed action items from the review recommendations in preparation for the next review and Self-Study.

Self-Study Outline

This self-study outline is a template for all departments; academic support units should address all that are relevant. Insert links of the required data to support the narrative. Any supporting data that cannot be included as a link should be added in the Appendices. NOTE: The Data Requests Sources doc, in the APR Resources Drive folder, contains information on where to acquire data and who to contact.

The Self-Study should be no more than 50 pages, excluding the Appendices

Introduction
1. Describe (not to exceed one page) the departmental review process and faculty/staff participation in developing the self-study, including numbers of faculty and/or staff in the unit and composition of the self-study committee.
2. The Department Chair’s or unit Director’s overall summary of the department/unit (not to exceed 5 pages) that discusses program/unit strengths, any major changes that have taken place since the last review, persistent challenges and concerns, and recommendations, strategies and priorities for improvement. Include (in a link) the department’s, or unit’s, current strategic plan (reviewed and updated each January) and provide information on the status of the goals outlined in the plan. Please include how the department/unit’s strategic plan aligns with the College’s Strategic Plan.
Department/Unit History, Organization, and Mission Statement
1. Brief history of the department/unit formation and organization. Please include mission statement, goals, and objectives of the program as well as reflection of and contribution to the institution’s mission.
2. Local, regional, and national, reputation, and accomplishments of the program; national or discipline ranking (if applicable)
Curriculum
1. Degrees offered, including undergraduate degree programs, majors and minors, graduate degree programs, and certificate programs (if applicable).
2. Requirements for each degree housed in the department
3. Program structure for each degree, including recommended course plan from year one to completion
4. Current undergraduate, graduate, and combined courses; please note frequency of course offerings
5. Major changes in curriculum in the last five years
6. Representative course syllabi (in the Appendices)
Equity, Innovation, Curiosity, and Inclusion
Describe the ways in which the program enacts Emerson’s commitment to the values of curiosity, equity, innovation, and inclusion as defined in Emerson’s Strategic Plan. Academic programs should address all of the following areas; academic support programs should address all that are relevant.
1. Shifts in program goals based on identified opportunities, challenges or barriers to equity. Name the specific opportunity, challenge or barrier that these shifts addressed.
2. Shifts in program operations, academic offerings, or launch of any special projects or initiatives to be responsive to student feedback on these issues
3. Pedagogical innovation that promotes curiosity, inclusion, authentic cultural expression, or greater deliberative dialogue across backgrounds, experiences, identities, and beliefs
4. Curricular development (creation of new courses, elimination or substantial revision of existing courses) to foster representation in the curriculum
5. Recent faculty development efforts to equip program faculty with knowledge and tools to advance an inclusive learning environment built from a strengths-based approach. This could be department-led efforts or efforts done in coordination with equity resources across the College.
6. Innovative efforts to retain and support an inclusive faculty
7. Innovative efforts to promote access and accessibility. This could be to increase physical access, language access, socio-economic access, or access in terms of neurodiversity
8. Name two emerging or urgent changes in the industries related to this program and describe how the program is preparing students to be more informed and career-ready
9. Other areas/approaches/actions not listed above
Faculty and/or Staff
1. Total number of faculty (full-time, affiliated, visiting, tenure/non-tenure track); please note any new faculty and retiring full-time faculty; please also include percent tenured; terminal degrees; fields of specialization represented and a demographic breakdown according to race, nationality, legal sex, and gender.
2. Faculty recruitment and retention efforts since the last program review
3. Short faculty bios (include description of faculty research, scholarship, and creative activities; special projects; professional awards and recognition; grants awarded)
4. Role, contribution, significance, and participation of affiliated faculty in the department
5. Faculty Development: include resources available to support faculty development in teaching, research, or creative activities; faculty involvement in development activities such as teaching seminars and training to update skills or knowledge.
6. Number and type of staff; description of how faculty and professional staff work together to meet curricular needs; evaluation of adequacy of support, information on staff responsibilities and lines of reporting; staff development activities; brief staff bios.
Faculty Workload (if applicable)
1. Describe faculty workloads, including: average course load; average class size; average advising load per faculty member; average thesis load per faculty member; faculty/student ratio (undergraduate and graduate).
2. Describe service obligations, special assignments, time spent on scholarly and creative activities as part of overall workload.
Students (or student constituents)
1. Enrollments for the last five years (undergraduate, graduate)
2. Narrative description of current enrollment breakdowns, strategies and recent trends
3. Number and type of degrees awarded over the last five years; degree completion rates; average time to degree completion.
4. Student admissions profile and criteria; number of applicants and admitted students; part-time/full-time status; transfer credits; number of international students
5. Department recruitment, advising, and retention efforts.
6. Financial Assistance data, including, for undergraduates, institutional funds; the percent of students on financial aid; average level of support; for graduate students, the ratio of grant-to-loan funds; number of teaching, advising, and/or administrative assistantships and selection process
7. Student productivity: number of theses, Master’s projects, and dissertations produced in the last five years; number of student publications, exhibitions, productions, and professional presentations (both undergraduate and graduate); financial resources to support student work
8. Career Outcomes: a sampling of areas and professions where alumni are employed; any data indicating career placement 1, 5, and 10 years out.
Assessment of Student Learning
1. For Departments
  1. Complete the “Assessment of Student Learning Overview - Departments” table
  2. Briefly describe how assessment of student learning is discussed within the department and provide two to three key examples of substantial changes made as a result of student learning outcomes assessment since the previous program review (student learning outcomes, curriculum, pedagogical changes, etc.)
  3. Describe current partnerships to enhance and assess key co-curricular learning (e.g. Iwasaki Library Information Literacy Outcomes, Information Technology Group, NACE Career Development Outcomes)
  4. External reviewers should be invited to observe a class and should meet with students to elicit their feedback
2. For Offices & Programs
  1. Complete the “Assessment of Student Learning Overview - Offices & Programs” table.
  2. Briefly describe the process for collaborating with departments to assess student learning and provide two to three key examples of how assessment has been used to improve services and programming since the previous program review.
  3. Describe any additional partnerships to enhance and assess key co-curricular learning (e.g. Iwasaki Library Information Literacy Outcomes, Information Technology, NACE Career Development Outcomes).
Stakeholder Satisfaction and Program Climate
Stakeholders include students, faculty, staff, alumni, and community partners. Gather and summarize information on stakeholder satisfaction; quality of the scholarly community; teaching; academic advisement; and diversity of faculty and students as it defines community activities related to promoting diversity among students and faculty.
Community Outreach and Partnership
Describe any community outreach activities that the faculty and the department engage in and how they contribute to the department and college mission. Describe any partnerships with organizations and the benefits realized by the college and the organization.
Facilities
Description of number, type, and size of dedicated facilities and assigned facilities including: classrooms, computer labs, studios, and offices. Describe any inadequacies and needs.
Budget
Payroll and Operational Budget sheet annual summaries. Instructional and general expense budget; portion of budget used for undergraduate and for graduate programs(s). Identify any amount of external funding (externally funded research, gifts, contracts, special programs). Describe any major changes, shifts, trends in budget over the last five years, and the impact of those changes.
Department/Unit Policies and Practices
Department policies and procedures: admission policies, attendance policy, grading policy or practices, academic standards, teaching evaluations and evaluation practices, department handbooks, student handbooks, etc.
Appendices (provided by the department/unit)
Include data that you have not added as links in the narrative

External Evaluators Report

Following the site visit, the External Evaluators should consult with each other to discuss and formulate their final report and recommendations. There will be time during the site visit to meet and begin formulating the final report and to discuss initial findings with the Dean or Director, the Assistant Provost for Academic Programs and the Provost
The team should submit to the Provost, Assistant Provost for Academic Programs, and Dean or Staff Director a draft program review for commentary, fact checking and clarification. The draft report is usually due 14 to 21 days after the site visit. The final report, incorporating facts and feedback, is usually due 30-60 days after the site visit
The External Evaluators should select a site team coordinator whose responsibility will be to compile the member’s contributions into the report. The coordinator will be responsible for submitting the final report by the deadline
Both the draft report and the final report should be submitted to the Assistant Provost for Academic Programs electronically as a Word document so it can be formatted into the program review document
The External Evaluators should address the following issues in their final report:
- Program/Unit Strengths
- Program/Unit Concerns and Persistent Challenges
- Recommendations for Improvement
  - Please describe recommendations for program improvement. If possible, please prioritize your recommendations and include a brief rationalization for your recommendations