The ETLC Responds to Problems with Student Evaluations of Teaching

Join the Excellence in Teaching and Learning Committee for our 9th annual conference!

Friday, January 25
9:00 a.m. to 5:00 p.m.
Bovee University Center, Terrace Rooms


Conference Schedule

The CLASS Excellence in Teaching and Learning Committee Winter Conference will continue our discussion from our Fall 2018 workshops on addressing Student Opinion Surveys (SOS) at CMU. Topics will include discussions of the problems identified in the research on SETs and possible solutions to some of these problems - both in classroom teaching and in interpreting and responding to SOS data as it is used in personnel decisions. Our conference panels include students discussing the SOS and their perceptions of its usefulness, and faculty and administrators reflecting on options we might consider going forward.


Registration in the Terrace Rooms
(Breakfast and beverages provided)


Richard Rothaus, Dean of the College of Liberal Arts and Social Sciences
Welcome Conference Participants

Chris Davoli (PSY)
Introduction to Conference
"A Brief Review of Problematic Findings from Research on Student Evaluations of Teaching (SET)"
Resources on Issues Related to Student Evaluations of Teaching


Amanda Scherr (APA)
Digging Deeper Into the SOS from the Perspective of Survey Design
The goal of the presentation is to dig deeper into SOS scores, investigate if there are unique differences based on student factors, and if so, discuss how these differences relate to survey design. The presentation will cover how SOS questionnaires function as surveys and the implications of using them as a measure of faculty performance.

Jim Felton (FIN) & Scott de Brestian (ART)
A Method for Adjusting the SOS for Rigorous Courses


Lunch Break in the Terrace Rooms
(Bring your lunch or purchase it in the U.C., and then join colleagues in the Terrace Rooms. Beverages provided.)


Anja Mueller (CHM)
A Better Tool for Examining Teaching Effectiveness: Teaching Portfolios with Peer Evaluations, Self-Evaluations, and Student Evaluations
The SOS alone does not effectively evaluate teaching.  This presentation examines several validated methods of documenting teaching effectiveness and student learning which can be part of a teaching portfolio for RPT.  After a discussion of these strategies, participants will work together in groups to develop RTP-style teaching portfolios. 

Jessica Wicks (CIS)
Strategies & Tools for Soliciting Early, Informal Student Feedback on Teaching Effectiveness
In this session, participants will gain familiarity with early feedback strategies for teaching, reflect on feedback question types and content, identify opportunities for informal feedback in their classes, and consider response strategies for student feedback.


David Kinney (SASW), Student Panel Facilitator
What Do Students Think about the SOS? A Panel of Students Will Tell You!
While there exists a large body of research on the existence of biases in students' evaluation of teaching, little attention has been given to how college students perceive the rationale and process of evaluating their professors. On this panel, a group of CMU students from diverse backgrounds will share their views and experiences with Student Opinion Surveys.  The students will address the following questions: (1) What is their level of understanding regarding the purpose and use of the SOS? (2) To what extent do they understand what the specific items are designed to measure? (3) What factors (e.g., their expected grade, how much effort they put into the course) shape their answers to the SOS questions? (4) What recommendations do students have to improve the process and focus of their evaluations of teaching at CMU?


Tracy Collins (ENG), "Action" Panel Facilitator
Considering Problems with SET/SOS: What Might Be Done? What Can Be Done? What Is to Be Done?
Panelists from Faculty and Administration: Stephanie Bechtel (Director of Faculty Support, CIS); Mike Gealt (Provost); Pete Ross (Vice Provost/Academic Development); Richard Rothaus (Dean, CLASS); and Mary Senter (SASW, Director, CARRS)

This panel consists of administrators and faculty members who have relevant expertise or experience with the SOS and its use at CMU. For example, Mary Senter (SASW) was on the committee that designed the SOS as we use it now and can offer a historical perspective. Panelists will discuss the concerns they see as paramount in evaluating teaching and learning, particularly with respect to the use of SOS. They will also offer suggestions for consideration, and invite attendees to do the same, as we discuss our current practices and future options in evaluating teaching effectiveness.

Conference Resources/Links

About ETLC

In 2018, the Teaching and Learning Collective merged with the college's Excellence in Teaching Committee to form the Excellence in Teaching & Learning Committee. The TLC was a grassroots, college-based initiative aimed at looking at ways in which we could promote effective strategies for improved teaching and learning. It quickly received university-wide attention, and became—with the support of the Provost’s Office and CETL—a university endeavor led by the TLC community within our college and across campus.

Past Events Related to this Conference

Evaluating Students' Evaluations of Teaching: Bias and Beyond

Friday, November 2, 2018
3:30 to 5:00 p.m.
Bovee U.C. Mackinaw Room

The CLASS Excellence in Teaching and Learning Committee (ETLC) hosted an interactive workshop on student evaluations of teaching facilitated by Chris Davoli (PSY) and David Kinney (SASW).

Bias manifests in many ways in student evaluations of teaching (SETs). For instance, research shows that students evaluate their professors differently based on factors like gender and ethnicity. Moreover, many surveys used in SETs fail to protect against these types of implicit responder biases because of how they are worded. Complicating matters further is the fact that low vs. high ratings mean different things to students, faculty, and administrators. With this workshop we aimed to 1) inform attendees about some of the research that has revealed the existence of biases in SETs, 2) engage an interdisciplinary discussion about how bias presents problems for SETs at CMU, specifically, and 3) identify possible solutions for reducing and/or eliminating biases in our SETs.

Does the SOS Evaluate Teaching and Learning? 
If Not, What Can We Do About It?

Facilitated by Rachael Barron-Duncan (ART) and Merlyn Mowrey (PHL/REL, retired)
Friday, October 12

3:30 - 5:00 p.m.
Bovee U.C. Mackinaw Room

What do students reveal about their learning and their teachers’ abilities in their responses to the SOS? The CLASS Excellence in Teaching and Learning Committee (ETLC) hosted an interactive workshop on student evaluations of teaching. Attendees identified the things they most want to learn from student course evaluations, and then identified the things they’re most likely to learn. In response, a variety of options to modify/add to the SOS were presented, with recommendations for alternative questions or other strategies to get the info that faculty value most.