“Student evaluations of teaching… have been shown again and again to be subject to student biases” – Flaherty (2019)
With the National Student Survey (NSS) results published on 6 July, the higher education (HE) sector will once again be sharpening its focus on student satisfaction. For many higher education institutions (HEIs), this sharpening of focus and the action plans that will inevitably follow results day will vie with other priorities for focus, including equality, diversity and inclusion (EDI). However, to see the two as competing is to create a false dichotomy. EDI and NSS success are interrelated and cannot be disentangled.
Question 27 of the survey is about ‘overall satisfaction’ and asks students to rank the following statement on a sliding scale from ‘strongly agree’ to ‘strongly disagree’: ‘Overall, I am satisfied with the quality of the course’. This final question is disproportionately important in the survey as it is the question that is referred to most by those compiling university league tables. However, HEIs have understandably struggled to work out how to address ‘overall satisfaction’. Fortunately, there have been some robust longitudinal statistical studies that have identified that the strongest predictors of ‘overall satisfaction’ have consistently been shown to be ‘organisation and management’, addressed through questions 15–17 (Scale 5 NSS), and ‘teaching quality’, addressed through questions 1–4 (Scale 1 NSS).
“Overall student satisfaction is achieved by supporting teaching staff so that they can teach to the best of their ability” – Burgess, Senior & Moores (2018)
Scale 5 NSS (organisation and management) is heavily impacted by staff workloads and their engagement in CPD. In April 2021, Misra et al published research evidencing that workloads within HE in the US were unequal on grounds of gender and race, and that this was more pronounced at the intersection of the two, ‘with women of color facing particularly high workload burdens’. Research, including work done in the UK by the University and College Union (UCU), demonstrates that similar inequalities persist within the UK. As academic staff become more diverse, HEIs must look to workloads to ensure that all staff have the time to organise and manage their teaching to the best of their abilities.
Women and academics of colour also suffer in student teaching quality assessments because of students’ ‘unconscious’ perceptions of the subject expert being a white man.
In some subject areas, classism also has an impact here. HEIs will therefore suffer for the diversity of their teaching staff unless they work to address the ‘unconscious’ misperceptions of students and staff and ensure that teaching staff from minoritised backgrounds are equally as supported and venerated as their white, middle-class, male counterparts.
Access to CPD – which most HEIs argue is necessary in supporting their staff in both their organisation and management (Scale 5 NSS) and ensuring teaching quality (Scale 1 NSS) – is also gendered and racialised. At some HEIs, accessing CPD is difficult for staff on casualised contracts and getting paid for that time often impossible. At others, there is a low uptake of staff on casualised contracts of CPD opportunities. As staff on casualised contracts are more likely to be women, people of colour, or both, then again these opportunities that contribute to NSS success are unequally distributed.
Thus far, a cynic might argue that all these points make a case for rowing back on equality gains and returning to a ‘male, pale and stale’ body of teaching staff. Even if we ignore the gross injustice of this approach and the myriad advantages diversity brings to HEIs, it is still true that EDI and NSS success are necessarily related.
‘Learning opportunities’, addressed through questions 5–7 (Scale 2 NSS), assess for effective, participative learning throughout programmes of study. Students are asked about opportunities throughout their programmes. Further, question 8 asks students to assess the statement: ‘The criteria used in marking have been clear in advance.’ HEIs are being evaluated on the learning at programme level and on the clarity of assessment criteria, which theoretically should be well aligned. To do well in questions 5–8, there need to be clear objectives, opportunities that address these objectives, and assessments that evaluate these objectives. Programme-level learning outcomes that map onto module-level outcomes and assessment tasks in a clear and transparent way are central to this. Running programme-level rather than module-level design events means that opportunities to bring information and ideas together from different topics (assessed in question 6) are identified. Involving students at appropriate points within the design process allows for sense-checking and ensures that the student voice (assessed in questions 23–25, Scale 8 NSS) is incorporated into the programme structure. This not only advances NSS success but also EDI, as the clearer we are about expectations and how we assess them, the more we surface the ‘hidden curriculum’ which unfairly advantages students with intergenerational experience of HE and who share the lived experiences of teaching staff. Clarity and transparency therefore disproportionately benefit students from disadvantaged backgrounds.
Further, designing learning and teaching opportunities for learners to reflect on their experiences and appreciate how they have explored concepts in depth is directly addressed in question 5 (‘My course has provided me with opportunities to explore ideas or concepts in depth’). Applying what students have learned to their own circumstances is addressed in question 7 (‘My course has provided me with opportunities to apply what I have learnt’). Critical reflexivity, which includes reflecting on one’s positionality (beyond a white, middle-class perspective), is therefore key to NSS success. We have argued previously (2020), in the long tradition of critical pedagogists, that critical reflexivity contributes to creating a more liberatory and inclusive curriculum.
Finally, for NSS success, students need to feel that ‘we’ listen to them. The ‘student voice’ is evaluated through Scale 8 NSS. Part of staff reflection has to be listening and responding to the student voice. In this (and other areas), poor scores disproportionately pull down overall scores: as most students who are happy score highly, a few dissatisfied students are a big factor between HEIs that by and large score similarly.
These dissatisfied voices are often from students who feel marginalised, and even where they are a very small number of students, this can sink a score. Seeking out marginalised voices, as we did in the ‘Living Black at University’ project, and responding to them, is vital. A representative body of student advocates (see for example Barefoot & Boons, 2019) is one example of an intervention that would achieve this, improving both NSS scores and inclusivity.
‘Teleola Cartwright and Dr Nick Cartwright are Consulting Fellows at Halpin. Halpin has a wealth of experience in working with universities to enhance the student experience, including in the areas of how to improve equality, diversity and inclusion. Working with an external partner is often what is needed to test and challenge existing ways of working and produce different results. Contact us to find out more about how Halpin and our team of HE experts can support you.
Barefoot, H & Boons, C (2019) ‘Developing a BME Student Advocate Programme’, Compass: Journal of Learning and Teaching 12(1). Available at https://journals.gre.ac.uk/index.php/compass/article/view/936
Burgess, A, Senior, C & Moores, E (2018) ‘A 10-year case study on the changing determinants of university student satisfaction in the UK’, PLoS ONE 13(2): e0192976. https://doi.org/10.1371/journal.pone.0192976
Cartwright, N & Cartwright, T (2020) ‘“Why is it my problem if ‘they’ won’t take part?” The (non)role of white academics in decolonising the law school’, The Law Teacher 54(4), 532–46.
Flaherty, C (2019) ‘Teaching Evals: Bias and Tenure’, Inside Higher Ed [online], available at: https://www.insidehighered.com/news/2019/05/20/fighting-gender-bias-student-evaluations-teaching-and-tenures-effect-instruction (accessed 24 June 2022).
Misra, J et al (2021) ‘Gendered and Racialized Perceptions of Faculty Workloads’, Gender & Society 35(3), 358–94.
Office for Students (2020) National Student Survey [online], available at: https://www.officeforstudents.org.uk/advice-and-guidance/student-information-and-data/national-student-survey-nss/