Below are the prepared remarks given at the meeting of the Faculty of Arts and Sciences on Tuesday, November 4 concerning the attendance study.
Prepared Remarks from Professor and Vice Provost Peter Bol:
A year ago, on taking the position as Vice Provost for Advances in Learning, part of my brief was to support the growing interest in improving teaching and learning across the campus. I began to wonder if there was a growing disconnect between how students were choosing to spend time and the expectations teachers had of their students.
Over the years I had heard colleagues assert that students in increasing numbers were skipping class, that the amount of work done outside of class (with some very notable exceptions) was decreasing, and that there was less rigorous note taking. Such anecdotes raised questions about the effectiveness of lectures as a way of helping students learn and suggested that there might be some value in exploring how new media and pedagogical techniques might be used by faculty to turn the lecture into something that was more interactive and engaging rather than simply an exercise in listening.
However it turned out that we did not have any data to support the anecdotes. I thus looked for a way of getting data on attendance, because that seemed to be the only thing that could be measured in a straightforward way that did not rely on self-reporting. I am told that there are no published multiple-course results on objectively-measured attendance to rely on.
But in designing such a study there were some very important considerations. We did not want to bias the sample. We did not want individual students to be tracked or in any way identified. And we did not want the results to be used for the purpose of evaluating the teachers. We wanted to know if we could get valid evidence on attendance, and we wanted to see if there were any patterns in the data that might support conclusions about whether or not we should care.
The protocol was sent to the Committee on the Use of Human Subjects in Research – this is the Institutional Review Board, the group responsible for deciding if research uses human subjects and reviewing that use to make sure that they are line with regulations -- which concluded that the study did not constitute human subjects research. It thus did not go to the full committee for review. The protocol was to install a camera that snapped an image of the audience in a lecture hall every minute. The images were processed through a program that counted whether seats were empty or filled. The quantities were calculated for each lecture. Once the data were in hand I made appointments, beginning in August, with course heads (two are still outstanding) to tell them what had been done and to show them generalized numerical data on their respective classes. At that time I ordered that images of students be destroyed. The course heads were asked to decide what should happen next. The course head could choose to have the numerical data removed from the study and deleted permanently. The data could be maintained without identifying the course. The data could be maintained with the identity of the course. There could be discussions with the researcher to better understand the data and consider ways of improving outcomes if so desired.
I can report that every single person I met with thought the data were interesting and potentially useful, agreed to the use of the data keeping the identity of the course, and was interested in learning more about the research. Faculty do care about their classes and their students.
The analysis did reveal patterns in the data (patterns that made sense once they were found). The results of the analysis are being shared with course heads. The aggregated data, without identifying courses, have been presented at Harvard to people interested in teaching and learning issues. Here I will only note that there was great variability in attendance.
I do understand the concern with faculty control, but ultimately course heads did have control over the data on students in their classes. Yet this has certainly raised questions about studies involving students that might not be set up to avoid identifying students. For that reason the Committee on the Use of Human Subjects in Research will automatically contact the Assistant Dean for Undergraduate Education in regard to studies that involve undergraduate students.