Introduction: Covid 19 – emergency examinations in higher education
In his poignant reflection on the disaster that befell the space shuttle Challenger, Latour (2004: 233–234) talks of how the shuttle’s journeys, which had become routine and unremarkable (an object), had suddenly disintegrated, literally, and transformed into a thing that became widely discussed around the world:
What else would you call this sudden transformation of a completely mastered, perfectly understood, quite forgotten by the media, taken-for-granted, matter-of-factual projectile into a sudden shower of debris falling on the United States, which thousands of people tried to salvage in the mud and rain and collect in a huge hall to serve as so many clues in a judicial scientific investigation? Here, suddenly, in a stroke, an object had become a thing, a matter of fact was considered as a matter of great concern.
In much the same way, Covid-19 has prompted, disruptively and almost violently, a re-arrangement of the well-rehearsed practices of school and university examinations. Williamson et al. (2020: 108) argue that as a result of the Covid-19 pandemic, the beginning of 2020 marked the start of a distinctive approach to pedagogy, characterized by the extensive use of distance education, remote teaching and online instruction. ‘Pandemic pedagogies’, they state, are the specific response to the field of education suddenly becoming an ‘emergency matter’. We argue that not only has a specific approach to pedagogy emerged, but equally, all over the world, specific forms of emergency examinations have been created. The impossibility (in many cases) of bringing students physically to an examination hall during the (almost worldwide) lockdown meant that the routine and ‘black-boxed’ practices of examinations, previously taken for granted as a necessity, were called into question. While examination processes always have been subject to glitches, interruptions and workarounds, the scale (i.e. the extent and global spread) at which the lockdown sent the processes into disarray was unprecedented, turning them into a matter of public concern. To use Latour’s trope: examinations transformed from matters of fact into matters of concern (Latour, 2004). Almost all educational institutions found themselves having to innovate on the fly, figuring out new alternatives to the time-tested examination system, with little or no time for thorough deliberation or rehearsal. Examination performances were re-arranged, and in some cases, postponed or even suspended. In this paper, we argue that the virus induced the configuration of new choreographies of examinations, where diverse social and material actors, with differing ontologies or logics, were required to re-arrange, reformulate and redistribute themselves, even as the established and familiar spaces and times of these choreographies were (partly) relocated and redefined (Cussins, 1996; Decuypere and Simons, 2016).
Based on examination cases from six countries (Australia, Belgium, Chile, India, Sweden and the UK), we present analytical stories of new, Covid-19-induced, examination choreographies. Our cases refer mainly to university-level examinations, while one of the cases deals with a college admission examination. The analytical stories illustrate measures taken in the different countries and also how different actors (not least different digital technologies), came to the fore as they confronted the realities of new, hastily constructed practices.1
We start this paper by situating examinations as part of formal assessment more generally, and argue that conventional forms of examinations have, throughout history, functioned as a sacred ritual. We frame this sacred nature of the examination as an obligatory point of passage that students necessarily have to pass through to progress their educational journey (or to exit the educational system by graduating). In order to get a grasp on the changing practices of examinations in the higher education sector, we use the notion of ‘choreography’ as an analytical heuristic. After outlining our methodological approach, we present our data and choreographic analysis. Our case studies enquire into examples of the adaptations and novel practices that arose in the effort to preserve the sacredness of examinations. We reflect on what changed and what remained of conventional examinations and conclude that the improvisations were focused on maintaining the sacredness of the ritual of examinations at a heavy cost to students, particularly in the Global South, and that universities by and large did not take up the opportunity to rethink examinations.
Examinations: Obligatory points of passage, sacred rituals and choreographies
In many respects, the examination constitutes one of the core parts of education. For decades, and despite national and regional variations, the examination has been the place and time where students and teachers, knowledge and content, pedagogy and didactics, laws and policies, individual dreams and societal visions come together. It can be seen as one of education’s most central and sacred ‘obligatory passage points’ (Callon, 1986) for gaining formal qualifications, for admittance to further education, for entry into a profession, and so on. Obligatory passage points – a term stemming from actor-network theory (ANT) – function as ‘central assemblages through which all relations in the network must flow at some time’ (Fenwick and Edwards, 2011: 9). Obligatory passage points require that other actors, or assemblages of actors, pass through them in order to form their own networks and in order to pursue their own interests (Fenwick and Edwards, 2011). In that sense, obligatory passage points turn themselves into indispensable actor(-network)s in a particular assemblage (Latour, 1987). Moreover, because they are obligatory, such passage points are often easily overlooked, taken for granted or simply not actively noticed as taking up a central, or perhaps better, an agential, position: much like the Challenger, they become part of a ‘black-boxed system’ (Latour, 1987). However, because of their indispensability, obligatory passage points play a crucial role in any setting, since they exclude actors that cannot (or will not) pass through them. It is precisely when practices break down in an emergency such as a pandemic that the agential capacity of such actors and assemblages is rendered very clearly visible (cf. Alirezabeigi et al., 2020). Using different terms and phrasings, many prominent thinkers such as Durkheim (1938/1977), Marx (1977), Foucault (1977) and Bourdieu and Passeron (1977) have also pointed to such obligatory features and characteristics, describing examinations as sacred rituals within educational systems.
In order to understand the sacred nature of examinations, we need to look at the broader functions of educational assessment in which they play a role. Broadly, from the very first examinations to what is nowadays sometimes called ‘assessment for learning’, three overall functions or purposes of assessment have remained intact. First, assessments work as a technology of control of individuals and of systems; second, assessments assign merit; and third, assessments fulfil a function in learning (Tveit, 2019). There are, of course, great variations in content, format and tradition between different countries and subjects (Kellaghan and Greaney, 2019), but it is precisely because assessments fulfil all these functions at once that they contribute to the legitimacy of the educational system.
Examinations, as the oldest and most formal type of assess
ment in education (e.g. Lundahl, 2018), play an important role in validating and generating societal trust in educational knowledge. Examinations follow a kind of ‘script’ that stems from best practice developed over centuries (Kellaghan and Greaney, 2019; Stray, 2001) and also from more recent psychometric research (American Educational Research Association et al., 2014). The script regulates the construction and scoring of the test, the fairness of the test, the implementation of the test, and its consequences, that is, issues of reliability, validity and interpretation (American Educational Research Association et al., 2014). Despite their status as a global institution, examinations refer to specific assessment cultures, which in turn are determined locally, nationally and/or regionally (Alarcón and Lawn, 2018). Because of their sacred nature, examinations (particularly those that lead to professional qualifications) tend to be heavily standardized and regulated. Examinations are thus not neutral tools: rather, they act as selective mechanisms for including and excluding, for sorting, valuing and gatekeeping (Berg and Timmermans, 2000; Broadfoot et al., 2000). They are imbued both with strictly delineated institutional rules and with (inter-)national and regional laws and regulatory requirements (e.g. the European Credit Transfer System [ECTS] (Simons, 2020)). Such rules, laws and regulations result in examination practices that are rigid, formal and difficult to alter, seemingly prioritizing reliability rather than validity in the exam process (cf. Brookhart, 2015).
The ability to conduct examinations that lead to the conferral of particular awards is also a matter of status for the examining institution; and as such, the quality and integrity of the examination – both content and process – become a matter of institutional pride. The certificates and degrees obtained through examinations are also crucial for the achievement of status positions in the context of individual biographies. We argue that part of the sacredness – part of the obligatory features and qualifications – of examinations is related to the naturalization of a scripted and regulated performance carried out by a complex network of actors. Although there has been a rise of alternative forms of examination in past decades, the width, speed and the range of subjects that the pandemic caused to change, moving even high-stake examinations into digital, remote and open-book formats, are unprecedented (Boursicot et al., 2020; Sam et al., 2020; Tan et al., 2021).
In normal parlance, ‘choreography’ refers to the careful and arranged set of movements or dances that involve bodies, energy, space and time. The term also refers to dimensions such as contact, distance, continuity, interruptions, suspensions, acceleration and deceleration (Foster, 2011). Echoing this everyday use of the term, the term is regularly invoked in the social science literature in order to comprehend social life in terms of movements, rhythm and changes, rather than in terms of fixed structures (e.g. Aronsson, 1998; Cussins, 1996; Goffman, 1959). Used in the context of examinations, it sensitizes us to the design of movement sequences and the actual performance of the examination ‘dance’ on stage. Through the metaphor of choreography, we become aware of and sensitive to such aspects as the relationality which links the whole system together and the ways in which actors take cues from each other and adjust their movements to accommodate other actors, including the material affordances of the stage and the setting (Decuypere and Simons, 2016). The metaphor also makes visible the disruptions of movement that can be caused by human as well as non-human actors that do not follow the script or adjust to the flow of the dance. Importantly, even though we use the concept ‘choreography’ as a metaphor and as a heuristic tool in order to make sense of this paper’s empirical material, the general term – as well as associated terms such as movements and relationality – also served as a conceptual framework. That is to say, we used the terms in such a way that they conceptually sensitized us for analyzing our data in a relational way, looking for movements and processes of flow, or precisely for moments in which relational movements were interrupted and stabilized.
Cases, methodology and analytic approach
The idea for this paper arose from a series of trans-national conversations between the authors as a way of coping with the pandemic-instigated changes to our lives and our work all through 2020. Observing developments in Australia, Belgium, Chile, India, Sweden and the UK, we exchanged thoughts and notes about how our universities as well as the universities that were our sites of research were coping with the pandemic, how our work and the experiences of students were changing, how news media reported on matters, and so on. The conversations led us to focus specifically on Covid-19-responses to examinations as a pivotal part of education in all these countries, as we witnessed how these responses varied in their vision and were halting in the ways they were realized, with new disruptions and improvisations caused by the virus and by other actors.
Our collective writing in this paper is, in and of itself, to be considered as a method of inquiry (Richardson and St Pierre, 2005). Initially, during the spring and summer of 2020 we exchanged stories about the events in our sites of research, and we began to see the possibility of more systematically generating data in our respective sites. We differed in the methods employed, with the methods available largely driven by the research limitations imposed, ironically, by the pandemic itself. It was not possible for us to travel anywhere for fieldwork, nor speak to respondents face to face. Instead, we used data sources available to us safely and remotely. Our data consists of interview transcripts, with students (Sweden), media interviews (India), teachers, IT specialists and student support advisors (Australia, Chile, the UK), directors and administrators (Sweden, Australia); ethnographic fieldnotes and pictures (Belgium); questionnaires to students, teachers and directors (Chile, Sweden); media coverage (India and the UK); and our own quarantined lived realities of being mothersfathersdaughterssonsscholarscolleaguescitizensfriends at different places (realities) in the pandemic.
Initially, as a way to examine, illustrate and share our stories, we each wrote a vignette describing our own ‘case’ based on our research site, adding our stories into a shared online document. While juxtapositioning our ‘vignettes’ provided some grounding and a structure for organizing our data, we found it impossible to keep our ‘national’ stories discreet and intact. Our data kept shifting even as we wrote, with new developments occurring practically every day. Moreover, there were wide disparities, we knew, between universities in the same nation, and, at the same time, many similarities across nations. Eschewing analytical nationalism, we let our shared commitment to such foundational STS (Science and Technology Studies) concepts as relationality, emergence and multiplicity guide our discussions and our writing. As we examined the staging of the examination in different locations, we were struck by the determination displayed everywhere that the ‘show must go on’. The metaphor of ‘choreography’ appeared apt to understand these efforts to coordinate and orchestrate. It functioned as an ‘analytical prism’ that made particular insights crystallize (Ellingson, 2008). So, although we present analytical stories from six different countries, our aim is not to compare them in order to determine ‘best practices’, or to seek ‘melioration through imitation’ (Jasanoff, 2004: 15). Rather, we believe that each case is situated and sensible only within its own unique context (Arnove et al., 2013). Nor
is it our intention to systematically generalize our findings on each case to the entire country context. The varied cache of analytical stories located in different continents with different educational, cultural, historical and political conditions provided us with a rich set of practices from which we could account for multiple versions of Covid-19-driven emergency examinations.
The Australian case is located in the state of Victoria, which declared a state of emergency on 16 March 2020 in response to Covid-19. Although at this time universities were not mandated to go to an off-campus mode, all Victorian universities were preparing to do so in anticipation of campus closures (Murray-Atfield, 2020) and university instruction moved to online mode almost overnight soon after. No one knew how long the lockdown would last, and whether on-campus examinations could be held as usual. But as the weeks went on, it became clear that the knotty problem of examinations needed to be tackled. A variety of solutions were used across different subjects: abandoning examinations altogether and instead grading assignments; holding online examinations using the regular learning management system; using an online ‘lockdown environment’ software that used electronic trackers to observe students; and a proctored online examination system. Interviews were conducted with six university officials from one Australian university between August and October 2020.
In the context of a wider government-led lockdown, Belgium decided to adopt a wide variety of examination options in higher education, where both online examinations and offline variants were allowed. For the offline/physical variants, proper sanitary measures needed to be put in place and the principle of social distancing needed to be maximally maintained. We focus on how this was done in one university which partly shifted from the usual venues of examinations (i.e. lecture halls) to new ones. These new venues included football stadiums, large student restaurants, and, in this case, a large, off-campus, exhibition hall that is normally used for organizing big fairs, and that permitted maximal regulation of the principle of social distancing. During a participatory observation by one of the authors in June 2020, field notes and pictures were taken and came to serve as the empirical material for the Belgian case.
The Chilean case refers to two public universities, one located in the capital Santiago and the other 50 km south of that city. On 18 March, the government declared a ‘state of catastrophe’ due to Covid-19. Only a few days earlier, in mid-March, the universities had decided, as a preventive measure, to go into off-campus mode. However, since October 2019, Chilean society and the universities had already become scenes of a massive and unprecedented protest movement that questioned the foundations of the prevailing ‘neoliberal’ socioeconomic model. The shift to online classes during the pandemic evidenced a sharp digital divide among students, preventing many students from participating regularly in classes or making it difficult for them to do so. Student anxiety was further heightened by long, strict lockdowns and few government initiatives to support families and workers. The data was collected between May and August 2020 and consists of media articles, interviews with a teacher and a university director and questionnaires from 13 students, 14 teachers and two university directors.
In India, during the early stages of the pandemic, the University Grants Commission (the central body controlling higher education) recommended against nation-wide online examinations, on the grounds of the digital divide and the potential for cheating. Nevertheless, Delhi University, a central university located in the capital city, went ahead with its plan of online, open-book examinations for 225,000 students, fuelling massive protests from students and teacher unions, and an appeal to the High Court by students to cancel the examinations. Delhi University draws students of varying socioeconomic profiles from all corners of the country many of whom had returned to their hometowns when the pandemic struck. When the examinations were held, students were expected to participate from states experiencing devastating floods and internet blackouts. Some wrote their examinations from quarantine centres using mobile data. The university pushed through with the examinations, arguing that this protected the future and health of the students. Subsequently, the Delhi High Court gave an order that students who had been unable to complete the online examination would be given the opportunity to appear for an offline version later. The data in the India case consists of news media reports and media interviews.
Since mid-March 2020, all Swedish universities have for the most part practised distance teaching instead of teaching on campus. This has led to universities increasingly using open-book, online examinations, which in many cases required them to monitor examination-taking practices in students’ homes. This was a new and experimental use of surveillance in hitherto private spaces. The student perspective on this is documented here, in interviews with two students in May 2020, for a case that makes visible the changing choreography of exams in the formations and re-arrangements of actors, and in redistribution of agency. At the same time, we can see from a second Swedish case that some educational programmes in Sweden were well prepared, since they had been working with digital examinations and alternative examinations prior to Covid-19 as part of their higher education development programme. The data was collected in interviews with university directors and a questionnaire.
On 20 March 2020 the UK government closed schools in England. It required higher education to adopt social distancing measures in all aspects of teaching and learning but – unlike schools – did not force them to close. Universities were left to make their own arrangements for assessing student achievement during and at the end of courses, in consultation with employers and professional associations. This led to a widespread move away from examinations and toward coursework and tutor assessment which was carried out without media fanfare despite considerable anxiety and upset among both students and staff. The most visible disruption, documented in our case through media accounts (see Adams, 2020), erupted in a different place on the stage: the improvised arrangements for alternative, machine-based assessment of school performance. This assessment was designed to replace the exams used to allocate entrance to higher education but it failed spectacularly. The failure had knock-on effects not only for students and teachers but for admissions systems in the universities (Hubble and Bolton, 2020) and it damaged trust in government policy. While the assessments in this case were pre-entry, they were central to the sorting and ordering processes involved in admitting students to universities and are therefore entirely relevant to this paper.
We have collaged our different empirical stories in order for them to hang together. Our aim is to say something about the parts and, even more, about the combined picture of the multiple realities that have been enacted. When data from the different countries had been composed, we analyzed our material with the help of a matrix that was organized by our two primary research questions: How were the choreographies (roles, stage, movements) of examinations changed and challenged by Covid-19? How did these changes and re-arrangements affect the sacredness of examinations? We present our analysis of the Covid-19-inspired emergency examinations in the next section.
Emergency examinations: Improvising in times of disaster
The new examination choreographies that were created as a result of the pandemic inaugurated a new arrangement of roles of pre-exist
ing actors as well as the emergence of new actors: lecturers, students, exam invigilators, university and unit heads, college administrators, technical managers, IT support teams, internet providers, national and provincial governments as well as objects such as computers (tablets, laptops, etc.), mobile phones, internet connections, hand sanitizer and more.
Scrambled roles
In the normal course of events, the examination process runs almost on autopilot, with fixed roles assigned to those who prepare the examinations, organize the administrative and logistical details, invigilate, sit the examinations and mark them. Over the course of years, these practices become routine – everyone knows their role. There is little need for meetings and consultation. In our Australian case, faculty members prepare their examination and submit it to one of the administrative support departments, which does a basic check to see that the examination is formatted correctly, the marks add up to the required maximum, and the instructions are clear. Then they organize the venue, furniture, invigilator and so on. But when the pandemic hit, everything changed:
We had to do a lot of things on the fly while figuring out how to do them, and for students who were [also] coming up with a lot of things on the fly, figuring out what their job’s going to be like [the lockdown meant widespread unemployment and students were hit hard] and how do I do the Job Keeper [a programme to support those who were out of work due to Covid-19]. (Interview, Australian case study)
The ‘emergency examinations’ prompted by Covid-19 changed the assigned functions of many actors. A range of actors – the learning support team members, the cloud environment managers, the student experience officers – hitherto in the background and only supporting the work of academics got involved in questioning whether examinations were required at all, how to make the examination meaningful (for example, not asking simple definitional questions which students could look up easily, since online non-invigilated examinations were necessarily ‘open book’), what constituted ‘authentic assessment’ and how the requirements of online examinations might impact on students’ wellbeing. Meetings were set up across many more departments and faculties, and officials high up on the university ladder, such as the Deputy Vice-Chancellor came to be involved. Solicitors were called in to check the legality of requiring students to have computers with certain capabilities, such as a camera, to facilitate invigilated examinations. Each of these new roles tried to maintain what was considered the most critical and sacred aspects of examinations. This story offers a window particularly on the ‘backstage’ administrative actors who played a key role in re-choreographing the examinations – but not before questions were raised about whether examinations were required at all, and if so, why.
In Chile, the transition to online courses and examinations during a lockdown lasting several months, not only made a new actor visible but transformed it into a new kind of protagonist: the internet provider. Emergency examinations necessarily depended on internet providers to ensure the flow and speed of internet traffic without interruptions (see also Williamson and Hogan, 2020).
Similarly, this moment was ripe for an increase in reliance on private companies in Delhi University, because the academics had little experience of conducting examinations online. One professor called it ‘an invitation for commercial enterprises to flourish’, arguing:
There’s nothing wrong in online classes but we are not in a stage where we can hold exams. Only commercial enterprises are going to benefit from this, this is not just happening in India but throughout the world, this is an opportunity for them to make money. . .. Many teachers themselves are not trained in holding open book exams. We don’t have experience or expertise. And by handing over the responsibility of setting the paper and evaluation to someone else, we lose out on the credibility of the process. (Media interview, Indian case study; Deeksha, 2020)
At one Swedish university, actions were taken to safeguard the examinations immediately after the government recommended remote teaching. On 18 March 2020, a group of people, four university lawyers and three university investigators, published a new regulation for examinations at a distance where they stated that:
The principle is that normal regulations apply as far as possible. Deviations from the normal regulations regarding examinations must be documented, and all other deviations should be documented. It is important to focus on disadvantaging the students as little as possible, while at the same time maintaining legal certainty regarding e.g., transparency, proportionality and equal treatment. (Excerpt from university website, Swedish case study)
Instead of examinations in exam halls, it was now recommended to offer take-home examinations (which require some adaptation of the actual exam questions), or to postpone the examinations. In order to help teachers change their examinations, a link was provided to a website to the Centre for Academic Development [Högskolepedagogiskt Centrum] at the university. Here, one could find suggestions for rephrasing questions written for on-campus examinations to make them more suitable for open-book examinations, such as (a) putting the questions more clearly in a context, (b) making the questions more individual-related or (c) making the questions more complex.
In Chile, online courses and examinations in some contexts were disrupted by weak or slow internet connections, or directly by the digital divide and the complex family/social realities of students. In fact, the Chilean case shows how the digital divide produced a new digital student hierarchy based on access to digital equipment and the infrastructural and emotional conditions necessary for online learning. Such hierarchization and concomitant subjectivation is not only linked to pre-existing class and gender hierarchies (including LGBTQIA+)2, but even enhances them: the transition from campus to home intensifies previously existing inequalities and tensions. One student stated:
The home is often a space full of violence (especially towards women and the LGBTQIA+ community). Domestic tasks are not distributed equitably. The care of others often falls to the women of the household. There are many questions such as ‘Why am I studying while people are hungry’ and many people have said that they prefer to take a leave [from their studies], but do not have money to pay for another year. On top of that, the supposed economic recession has many people doubting whether to continue with their studies. (Questionnaire, Chilean case study)
This domino effect – where the interconnections between different practices bring to view the range of hidden actors (the ‘elephants in the room’) echo Latour’s (2004) examples of how ‘the thinging of things’ comes to view when matters of fact are shattered to become matters of concern. As more and more actors came to be involved – private companies, student advisors, computers with cameras, regulations, policies and so on – the ‘stage’ of this exam choreography, formerly contained within large university halls, came to be distributed across many sites.
The scattered stage
One of the most central features of conventional examinations is that of proctoring, as noted not least by Foucault (1977). The strict staging of an examination is a measure taken to spatially hold students apart (and to feel normal about this). The reason for this is to produce knowledge about the individual that can be used in comp
arisons (Meadmore, 1993). Just like a dance stage, the exam halls provide a clear framing for the movements of the actors involved, restricting and enabling them and determining what can be seen and monitored.
However, it turns out that as long as certain key aspects of their ‘sacredness’ could be guaranteed, examinations could be conducted anywhere. Students did not have to sit together in the same hall, under the same conditions, at similar desks, observed by the same invigilators. Distance or online examinations relocated and distributed the ‘stage’ of the performance from the classical lecture hall on campus to the homes of the students or to other public spaces like exhibition halls.
In order to maximize social distancing, examinations at a university in Belgium were moved out to big rooms and halls available in the vicinity of the university city (including football stadiums, banquet halls and museums). In the Belgian case presented here, the examination was staged in a huge building that is normally used for commercial exhibition fairs. In the massive building, there is room for 200 students – even when they are spread out. Although the students are brought together into the same space, the effect of this grand stage is not to connect the students – it is, rather, to separate them. It is to isolate them, not only from each other, but from teachers and most of all from the virus. The stage of the examinations had to radically shift to offer students an examination practice that emulated as closely as possible a ‘traditional’ examination format.
Elsewhere, digital solutions distributed the examinations to much smaller environments – into the studies, kitchen tables or beds – wherever in their homes students could work without too much disturbance. Students scattered, if possible, their partners, children, pets and other ‘disturbances’ out of the way for the duration of the examination. In India, students without internet connections in their homes went to community centres where they could access their examinations. They were permitted to submit the completed examinations over WhatsApp on whatever devices they had available. In Chile, as a strong internet connection was far from guaranteed, some students had to climb onto their roof each day in search of a cell signal strong enough to transmit their online classes (Reuters, 2020). In Sweden, students took examinations in their homes as well. One student, Tyra, tells us that she was instructed to turn on her computer camera and place a second camera from an angle behind her, so that the camera could show the entire computer screen and the desk.
We had to log in to the same Zoom room, first from the computer, and you had to place that so they could see the nearest surroundings. And then, from an angle behind, we had to log in from a mobile or I-Pad or whatever you had and then I used my mobile. I used the camera to log into the same room. (Interview, Swedish case study)
Another student, Rachel, said:
Well [the instruction] was like, it said that in order to minimize the risk of cheating, it is absolutely not ok to use any kind of material even if you’re at home. That’s why it is going to be surveilled via zoom, and that’s why it’s so important for them to see the table and such. So you don’t have a note under the computer. (Interview, Swedish case study)
And although they have to orchestrate it themselves, with their own devices, the students seem to consider that the surveillance apparatus was the actor that made the examination ‘as realistic as possible’.
These new, scattered stages can carry many students writing their examinations at the same time. Each stage includes a room with certain props: a table or desk, a computer (or a cell phone) and a camera. In some contexts, the whole space has to be ‘synoptic’ for the supervisor via computer and mobile phone cameras. Digital tools connect the students with the test itself, with the supervisor(s) and the other students dispersed in their homes. Much of the work is already in place – the students are presumed to have the required technology, the same test can be used as before, the technology used (Zoom or similar; examination software) is already in use. The small becomes connected and thus becomes a huge stage.
Not all students welcomed the prying eyes entering their homes, and some students in Australia expressed concern that the camera made visible their living arrangements, which they would have preferred to keep private. The images and the identity information and other data scattered the students and the university across various databases. None of the people we interviewed appeared to know the protocols for where this data was stored, how secure it was, or how long the data could be held by the companies or the universities before it was purged.
The scattering of the stage has created new inequities and difficulties. Special accommodations needed to be made for international students in Australian universities who were stuck overseas or in remote parts of Australia. Students living in the country sometimes had shaky internet connections, adding to their anxiety and uncertainty. Making provision for online teaching and examination was not easy, particularly for students residing in China. As one interviewee said:
It’s China, you know. Things get blocked, things get delayed, and the censorship issue was a big thing . . . Synchronous tools like BlackBoard were not working in China so we implemented Zoom working closely with [the IT department] in all of this. (Interview, Australian case study)
In India, the internet had been shut down earlier by the central government in the entire state of Kashmir over political issues. Subsequently, when it was reinstated, only 2G internet was permitted. Students from Kashmir who attended Delhi University and had returned during lockdown were disenfranchised by the move to online examinations. Maintaining protocols for Covid-19 which required the relocation of out-station students to quarantine centres created further impediments, since these centres lacked internet connections and were subject to frequent power cuts. One student explains how technical malfunctioning and the lack of information made this process among the ‘most harrowing experiences’ of her life:
The exam started at 11.30 am and was to go on till 3.30 pm. After writing the exam, it took me around 15–20 minutes to scan my paper and I started uploading the paper in the last 15 minutes of the examination. Due to an overload of traffic on the portal, my files refused to get uploaded. It kept showing me an ‘internal server error’, forcing me to reload the website multiple times. The moment the clock struck 3:30 pm sharp portal closed. I was unable to upload my answers. The sheer amount of fear and apprehension I faced in those last 10 minutes is unimaginable, because all said and done, these are my final semester marks. Finally, I sent my answers to an alternate email address provided by Delhi University and received a confirmation. A lot of students faced similar problems and many have sent emails but not received confirmation. Students have got wrong question papers, server errors, and some have also been given two question papers at the same time. This also begs the question – how are papers going to be segregated and corrected? Some of our papers could easily be missed out. There are no detailed guidelines or transparency as to how our answers are going to be evaluated. Will they be digitally assessed, or will they be downloaded by the examiners? (Media interview, The Quint (2020), Indian case study)
The new choreography challenges the conventional staging of examinations where it is less clear where to stand in order to see, and what, for that matter, to monitor. In some ways the new staging allows for a wider dynamic and a more open space, which in turn allows examinees to choose t
o take the examination in a ‘secure’ environment. At the same time, this also works like the mirrors in a dance hall, where everyone can see everyone regardless of where you stand. Of course, this raises questions of integrity and privacy. In other cases, the stage of the examinations and therefore the performance of the choreography could not be guaranteed for all students, neither in private nor in public space. This also raises the question of fairness.
Rhythms and flows
For a choreographed performance to function well, especially if it involves a large number of actors, complex technologies and props need to perform their assigned roles as per direction. Discipline, precision and adherence are key. Unless all the dancers – including technology – play their part, the performance will be chaotic. The spotlight will fall at the wrong spot, the sound will be out of sync, and dancers will get in each other’s way. Compliance must be coaxed out of even the most unruly divas for the choreography to flow.
Examinations – both off- and online – demand similar compliance on the part of the actors. A key aspect of the examination stage is its ‘visibility’ – students are seated in ways that make them visible to the invigilator, to prevent cheating. The scattering of students across the globe – and in some cases into little towns and farms where the internet was unavailable or unreliable – made maintaining this visibility a challenge. It was especially challenging in Chile and India, where students could only participate by keeping their cameras off during Zoom classes. The reasons for doing so were diverse: unstable internet connections, often precarious and/or overcrowded spaces, or students who had to take care of children, parents or grandparents. These challenges were met in different ways. In some cases, as in the UK, the examination was simply abandoned and other means – such as using the performance on prior assessment tasks – were used to calculate grades. Some, like Chile, adapted the paper–pencil examination to open-book examinations (online with one day or more to be solved) – in many cases, universities already had learning management systems that permitted this. Yet others went in for invigilated online examinations – a practice that was already beginning to be explored pre-Covid-19.
The invigilated exam uses technology to permit humans or artificial intelligence (AI) to observe and/or record students as they sit their examinations, using cameras on laptops and desktops. Some programmes worked in such a way that when you entered the examination portal, it locked you out of the possibility of opening any other screen. This was used in some subjects in the Australian case:
When you go home and do your exam, you have to go through a particular portal. And that portal can track when you copy and paste, how much you’ve copied and pasted, how long you’ve spent on a document. It can analyse your [keyboard] strokes. It’s being used also in a way that is formative, it is not only anti contract-cheating . . . for example, if you copy and paste a piece of text, it will say ‘You have copied a large piece of text. Make sure if you do use this in your assignment, you paraphrase it and you reference it’, and there will be some instructions on how to paraphrase and reference etc. (Interview, Australian case study)
Like the invigilators in examination halls, the online invigilator also imposed strict protocols. Any breach, and you might be marked as having behaved in a suspicious manner, drawing the attention of the relevant university personnel. Not all students were comfortable with that kind of surveillance. Medical student Rachel in Sweden, who is in her fourth term, had two major written examinations which were supervised by cameras – props set up by herself – both in front and from behind. When asked how she experienced this new way of exam supervision she says:
It felt a bit like, you know, like Big Brother, to put up the mobile phone for them to observe you, so, I was a bit afraid that my phone for some reason would stop working, or that I would jump out from Zoom in some way from the mobile, and that then they would think that I cheated. So, I had to sit and take quick glances back at the mobile phone all the time to see that everything was working and that nothing weird had happened. I was a bit worried that something might have gone wrong like ten minutes ago, and that they would think that ‘Rachel has left – well ok – flunk her then’. (Interview, Swedish case study)
Rachel also says that one of the biggest differences between taking the examination in an examination hall and doing it in this new digital way was that they had much less time to write it in this new situation. Previously, they had six hours at their disposal, now they only had four and a half hours. She says:
I think they wanted to cut down the time to reduce the risk of cheating, so you won’t have time to google stuff or browse through your notes or anything. And of course, that becomes a bit problematic for those who have difficulties in reading. (Interview, Swedish case study)
A director of studies at a Swedish university confirmed that limiting the time was a strategy to discourage cheating – the students would not have enough time to cheat.
The traditional examination in Delhi University presumes reliance on memory to answer questions, but in the context of online examinations, it becomes impossible to regulate whether students refer to reference material in answering the questions. One possible response, to continue the exercise, was to re-script it in such a way as to legitimize those actions that cannot be restricted. The form shifts from a closed-door examination to an open-book examination. In the new dance, the previously illegitimate moves are not only permitted, they constitute its defining feature.
Whether it is tutors watching on Zoom or AI tracking keyboard strokes and eye movements, or a human invigilator in some distant part of the world using commercial proctoring software, the defining feature of invigilated examinations is its presumption that, unsupervised, students will cheat. This assumption was, in itself, stressful to students, and compounded the anxiety around whether the programme would work and whether the gods of technology would cooperate. Being watched was also stressful, as one Australian interviewee explained:
The invigilated exam is predicated on the notion that students are trying to cheat. Because otherwise, why would you not just have the assessment done virtually and returned back to the academic, the unit chair etc. I suppose putting that notion on the student, and then on top of that, during the time of Covid-19, students were already under a lot of financial and emotional stress, and so I think the timing of adding this new software system . . . added some undue burden. (Interview, Australian case study)
Now that online examinations have been trialled, some universities expect the practice to continue after Covid-19. Students will know when they join that they will be required to have a device with a camera, and that they will be surveilled electronically during their examinations.
Ironically, while universities are monitoring students to ensure compliance, perhaps to gain reputation as institutions where examination integrity is not compromised, the act of bringing in such surveillance tools could bring universities using them under scrutiny. Many issues regarding the use of online proctoring have not been worked out as yet. As our Australian interviewee pointed out:
There are a lot of issues that come with that. How long are you going to keep the recordings? As an institution, we have not had an integrity case come up yet, however, it is highly probable we will in this trimester
[since the number of students using invigilated tests has gone up from around 90 to about 4000]. It is precedence for the University if an integrity case comes out of that. We have never shown a video as part of an allegation case. [In these panels], everybody has to see the evidence, and there is discussion about the evidence. I am not saying precedence is a bad thing in an institution, but I think we just have to be mindful about scaling and how it’s received by students as well and what comes out of that. (Interview, Australian case study)
A much better system, this interviewee says, is the honour system, in which, upon being accepted into the university, you sign an ‘honour code’ ‘and the expectation is that you are always meeting that standard’. This way, the focus is not on catching students out doing something wrong.
Tensions and compliance
Normally there is a great compliance with the authority of conventional examinations, at least among teachers and students, whereas scholars in the field of assessment stress the need for more alternative measures (e.g. Boud, 2007; Eisner, 1991). When the choreography of examinations changed, in some cases, the flow of examinations was retained, but sometimes there were tensions and resistance. Resistance is an inherent part of any form of movement or dance. It refers to the force that acts to stop the progress of bodies through the air, slowing it down or even interrupting it. At the same time, no movement is possible without resistance (this is why movement is difficult without gravity). Besides these physical dimensions of resistance, resistance may also refer in sociological terms to a political act of fighting against something. There are dances, like the Brazilian Frevo dance, that embody this political dimension by defining themselves as a dance of resistance against racial injustice. Everyday acts of political resistance may be hidden to outside view (Johansson and Vinthagen, 2016) but we were aware that the emergency improvisations did not always go down well with the audience or the actors in our case study sites: in at least three of them (India, the UK, and Chile) examination choreographies and performances were confronted with visible resistance movements as the actors fell out of step.
In the UK example, a well organized protest, supported by students, teachers and school and college administrators focused on ‘the algorithm’ which was used to grade student performance when the examination was abandoned. Accidentally made visible, the algorithm became a maligned, hated actor in the network. Many people seemed not to be aware that a version of algorithmic, normalized grading is always in place in the UK for nationally recognized qualifications since the individual is always judged within the context of others’ performance and related institutions. Political considerations and inequalities are baked into this process (see McArthur, 2020). By changing the conditions for the assessment process, the virus un-black-boxed it. Arguably, it was aided in this unravelling by a government and a qualifications authority (plus their corporate partners) who acted incompetently and without transparency or consultation with others. Resistance erupted when the ‘realism’ of the grading system broke down. In addition, the timing of decisions about examinations in schools that regulate admission to higher education was an important focus of concern and anger among both students and university staff in the UK (Hubble and Bolton, 2020). Decision-making considered ‘jerky’ and ‘U-turns’ at the last minute were destabilizing to different parts of the network and the government was criticized as much for the pace of decisions as for their content. The quality of movement of the new choreography was compromised. Synchronization was lost.
In the Chilean case, the online classes and the home study of many students were interrupted by the digital divide and by their complex family-social reality. Students responded to these interruptions by organizing online strikes. The dominant arguments of the student agenda were technical and emotional: the digital divide and the health, social and economic effects of the pandemic make learning and examinations unviable or at least difficult due to high levels of anxiety, stress and panic. The response of directors and professors was twofold: on the one hand, the content of the emergency examinations was reduced, and the requirements made flexible, by, for instance, extending the deadlines. Some examinations were even cancelled. Additionally, other formats were introduced such as formative, reflective and integrative written assignments or so called “Reflective logbooks” (bitácora reflexiva) or reflective and integrative open-book examinations (on- and offline). One university director even considered changing the grading scale from 1 to 7 to a binary pass/fail scale. Nevertheless, the idea was dropped, as the binary logic was not in line with the logic of academic promotion based on a predetermined grading scale. Furthermore, a week of institutional rest was established in both universities, demonstrating emotional and care logics: in one university this was explicitly called the ‘mental health week’. However, the most radical consequence of the strikes was the introduction of a kind of Covid-19 grading bonus in the final examinations.
In the Indian example, the resistance to this shift has come from both students and professors (Nandini, 2020). The university’s teacher unions, apart from making an empathetic claim about the students who will not be able to participate in such an exercise due to the digital divide, also argue that cheating will abound, weakening the integrity and sanctity of the examination. They view examinations as too important (for the future of the student) for them to support a compromised version of the examination. A media survey noted that 85{0841e0d75c8d746db04d650b1305ad3fcafc778b501ea82c6d7687ee4903b11a} of Delhi University (DU) students were calling for cancellation of the online examinations. The student unions mirrored the access critique, pointing out that students with disability required scribes, that many did not have access to study material from their homes, and that some were finding it impossible to study for examinations while thrown into a situation of hunger and material precarity (The Hindu, 2020). Student unions have coordinated online protests (Twitter storms) and appealed to the courts.
The resistance we see in the India, Chile and the UK examples are directed towards injustices within the emergency examination, or within the idea of examinations – where assessment becomes biased and even racist (for example, in the UK) or when examination practices reveal unfair preconditions, such as the digital divide (India and Chile).
These examples show how certain actors publicly resisted the emergency examinations. The degree, strength and emotionality of these resistance movements can be explained above all by the fact that examinations are not – and have never been – neutral practices, since they are directly related to the distribution of status and the ordering of society. Furthermore, these examples demonstrate how, in the context of emergency examinations, the newly emerged dancer, ‘digital technology’, revealed injustices or unfair preconditions (the ‘malign’ algorithm or the digital divide). Moreover, the examples illustrate how these resistance movements have led to new movements and thus, as in the case of Chile, to a re-choreography of the emergency examinations, with the aim of maintaining the legitimacy (fairness) and sacredness of the examination process.
Conclusion: The show must go on – the ‘thinging’ of examinations
This paper has been concerned with the sacred n
ature of examinations, a concept we have closely tied to the STS notion of ‘obligatory passage points’ (OPPs) (Callon, 1986), as well as to the analytical heuristic of ‘choreography’ (Cussins, 1996). The pandemic of 2020 opened the black box of conventional examinations and made us notice practices that we did not easily see in the flow of things before it was disrupted. It turns out that the examination is, at the same time, both a fixed set of ideal practices and a fluid set of practices. In our analytical stories we see the interplay between the maintenance of fixed and sacred aspects of examinations and the fluidity of improvisations aimed at meeting the threat of spreading Covid-19. Indeed, it is the fluidity that permitted the maintenance of the fixed notion of sacredness (cf. De Laet and Mol, 2000). The measures taken in response to the pandemic forced the complex network of examinations both to reinforce conventional actors and to assemble new actors and stages, even going so far as to devise radically new choreographies.
The stage itself has been ‘upstaged’, with the development and shuffling of new spaces of examinations (Decuypere and Simons, 2016). Actors suddenly found themselves taking on roles they had not anticipated, with some of the backstage actors – student advisors, moderators, administrative and technical support personnel – thrust into the front stage spotlight. Power shifted between actors as new ones entered and existing ones gained or lost strength: the virus itself; digital technologies and their allies; ed-tech corporations; university boards; and new government policies. The stage itself was scattered: the performance could no longer take place in the familiar lecture theatres or large classrooms, but instead it played out in homes, in massive fair halls, in distant community centres and on roof tops.
The metaphor of choreography has allowed us to focus on these movements and flows, to explore the continuous re-ordering of performances as actors improvised in real time, on new stages, taking on new and sometimes unexpected roles (Pickering, 2010). It also highlighted the inevitable emergence of tensions and resistances among actors as their relationships stretched and changed. It became clear as we used the metaphor in our analysis, that it is impossible to pin down one choreographer in the various cases. Power did not inhere solely within any actor – whether institutions or the practice of examinations. Instead, it had to be dynamically maintained through creative and enterprising action by various intersecting actors. We see a distributed and contingent agency, in line with the STS standpoint we have put forward in the paper and with modern concepts of choreography that emphasize close collaboration with the dancers, and the emergence of different actors taking the lead in the creation of the performance.
Countries have responded by adjusting examinations in different ways, and the re-working of choreographies is ongoing. They often include continuous elements of (more or less thought-through) improvisation. Constant adjustments and workarounds are needed as plans meet practices in the shaping of emergency examination realities (see also Boursicot et al., 2020; Tan et al., 2021). Responses have ranged from temporarily abandoning examinations in favour of teacher assessments, to retaining conventional face-to-face examinations by ‘controlling’ the risk of Covid-19 through the introduction of elaborate protocols around distancing and sanitization. Some countries adopted a position of tempered risk avoidance, holding online examinations for those able to access it, with a contingency offline option for those unable to. In affluent countries where the question of the digital divide was less prominent, online examinations with heavy digital proctoring tools were employed. Interestingly, within our own analytic stories, the economic resources of the country did not appear to be the deciding factor behind which approach a country would take. Nor did these predict the tensions and resistances that would arise as examinations flowed into new forms as the partly similar experiences of the British, Chilean and Indian cases have shown. The online option allowed risk avoidance in terms of Covid-19, but potentially disenfranchised a large part of the student population in the Global South from accessing examinations; and it was unable to adequately proctor the process given universities’ digital infrastructural constraints. Although students have been kept safe from the spread of the virus as they take the examinations from home, other actors, such as bad internet connections, have contributed to the (dis)placement of students’ bodies and work. In that sense both cases of the Global South revealed shared challenges and problems related to the digital divide. Where students or teachers were able to assert their rights, the balance of power changed.
Our examples show that the most sacred aspect of examinations has been their obligatory nature. Universities worldwide wanted to preserve the sorting and ordering function of examinations in one way or another. Huge efforts have been made to ensure the validity of examinations, keeping them ‘authentic’ and ‘reliable’ with the help of supervision delegated to technology and the willingness and adaptivity of the students, as they agreed to set up surveillance devices themselves on themselves, within their homes and private spheres, and also submitted to the new rationales of open-book examinations instead of, for example, multiple choice tests. Other features of the sacred examination such as privacy and the principle of equal opportunity, have been easily abandoned. Examinations posed particular challenges for students with disabilities and special educational needs even in pre-Covid-19 times. In the rush to adapt examinations to Covid-19, the needs of these students have been taken even less seriously than before. These students have had to comply with the measures taken to reduce the risk of students cheating, in order to maintain the flow of the emergency choreography. Preventing cheating seems to be more sacred than equal opportunity.
This has not been a time in which universities have reflected much upon how examinations could develop or evolve, drawing on alternative pedagogic visions or ideals. The changes made have been about the form and execution of examinations, in order to preserve examinations as something reliable and authentic. Although the black box of examinations was opened and the contents revealed and some parts scattered and even lost, it seems as if it is the box itself that is kept sacred. The show must go on – but not necessarily develop into an entirely new spectacle. What we see in our examples is the emergency pull-through of the ideal of the examination and its varied enactments in different contexts. In a way, this pull-through has been legitimized by the stance that when this is over, the status quo will return, all will be normal again, so we just have to juggle things until that happens. This could lead one to surmise that things have not irrevocably changed at all (unlike the Challenger with which we started the paper).
The movement from the university (public space) to the students’ homes (private space) made visible different and sometimes conflicting logics: the private family logic, in which the families of many students in the context of the lockdown were submerged in health, spatial (overcrowding) and economic deprivation; the logic of online examinations which presupposed the existence of specific technological equipment and infrastructural conditions; and the university logic, which oscillated between enhancing the education process, ensuring certification and obtaining student fees for self-financing. Such different logics, especially in less affluent communities, were fertile grounds for tension and resistance, making actors out of step and improvisation chaotic. Interestingly, these resistance movements themselve
s adopted new modalities in the form of online protests.
A central actor within the improvised choreographies has been technology, as both a matter-of-fact object and a matter-of-concern thing (Latour, 2004). It has functioned as a hero of the dance as well as a trickster – proposing itself as both a solution and a problem. Examinations as OPPs have relied on technology as a mandated requirement for maintaining Covid-19 safety, and at the same time technologies have undermined putative solutions and raised new problems and ethical issues – for example surveillance, unequal access and biased decision-making through algorithms.
In most cases, as we have argued above, technologies were not being deployed to innovate assessment, but to mimic the pre-Covid-19 format where bodies are disciplined by being ‘spaced-out-in-a-lecture-theatre-with-someone-watching-you’. As it took over the task of maintaining the integrity of the examination system, technology has created new forms of compliance, allowing things to go on as usual, and moving examinations to “safe” places.
The disruption caused by Covid-19 brings something important to education research. It has made visible the processes and practices, the valuations and values, the history, the policies, and the politics of examinations that had all become so deeply embedded into the routines of examinations that they were invisible. The pandemic has exposed the vulnerabilities and injustices in the existing setup of examinations and the strong emotions invested in it. From an assessment studies perspective, the illumination of the complex network of examinations might help to understand why this assessment practice is so resistant to change and to the acceptance of alternative approaches such as the assessment for learning movement.
Finally, we are fully aware that this paper is part of wider academic efforts to seize the experience of living with and through the Covid-19 pandemic, consciously making sense of it as human actors entangled in its everyday disruptions. Our STS perspective permits us to deepen our analysis of this changing assemblage and the re-inventions it makes possible. It is gripping to see new orders in the making and to participate in trying to stabilize the meanings of this history-making set of events.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
ORCID iDs
Mathias Decuypère 
https://orcid.org/0000-0002-0983-738X
Elin Sundström Sjödin https://orcid.org/0000-0003-3732-2022
References
|
Adams, K (2020) The U.K. used an algorithm to estimate exam results. The calculations favored elites. Washington Post, UK News, 17 August. Available at: https://www.washingtonpost.com/world/europe/the-uk-used-an-algorithm-to-estimate-exam-results-the-calculations-favored-elites/2020/08/17/2b116d48-e091-11ea-82d8-5e55d47e90ca_story.html (accessed 25 January 2021). Google Scholar |
|
|
Alarcón, C, Lawn, M (eds) (2018) Assessment Cultures: Historical perspectives. Studia Educationis Historica. Berlin, Germany: Peter Lang. Google Scholar |
|
|
Alirezabeigi, S, Masschelein, J, Decuypere, M (2020) Investigating digital doings through breakdowns: a sociomaterial ethnography of a Bring Your Own Device school. Learning, Media and Technology 45(2): 193–207. Google Scholar |
|
|
American Educational Research Association, American Psychological Association, National Council on Measurement in Education , et al. (2014) Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association. Google Scholar |
|
|
Arnove, RF, Torres, CA, Franz, S (2013) Comparative Education: The Dialectic of the Global and the Local. Lanham: Rowman & Littlefield. Google Scholar |
|
|
Aronsson, K (1998) Identity-in-interaction and social choreography. Research on Language and Social Interaction 31(1): 75–89. Google Scholar | Crossref | ISI |
|
|
Berg, M, Timmermans, S (2000) Orders and their others: On the constitution of universalities in medical work. Configurations 8(1): 31–61. Google Scholar | Crossref | ISI |
|
|
Boud, D (2007) Reframing assessment as if learning was important. In: Boud, D, Falchikov, N (eds) Rethinking Assessment for Higher Education: Learning for the Longer Term. London: Routledge, pp. 14–25. Available at: https://www.researchgate.net/publication/305060897_Reframing_assessment_as_if_learning_were_important (accessed 25 September 2020). Google Scholar |
|
|
Bourdieu, P, Passeron, J (1977) Reproduction in Education, Society and Culture. London: Sage. Google Scholar |
|
|
Boursicot, K, Kemp, S, Ong, TH, et al. (2020) Conducting a high-stakes OSCE in a COVID-19 environment. MedEdPublish 9. DOI: 10.15694/mep.2020.000054.1. Google Scholar | Crossref |
|
|
Broadfoot, P, Osborn, M, Sharpe, K, et al. (2000) National assessments: Underlying cultural values revealed by comparing English and French national tests. European Journal of Education 35(3): 361–374. Google Scholar | Crossref |
|
|
Brookhart, SM (2015) Graded achievement, tested achievement, and validity. Educational Assessment 20(4): 268–296. Google Scholar | Crossref |
|
|
Callon, M (1986) Some elements of a sociology of translation: Domestication of the scallops and the fishermen of St Brieuc Bay. In: Law, J (ed.) Power, Action, Belief: A New Sociology of Knowledge. London: Routledge, pp. 196–233. Google Scholar |
|
|
Cussins, C (1996) Ontological choreography: Agency through objectification in infertility clinics. Social Studies of Science 26(3): 575–610. Google Scholar | SAGE Journals | ISI |
|
|
Decuypere, M, Simons, M (2016) What screens do: The role(s) of the screen in academic work. European Educational Research Journal 15(1): 132–151. Google Scholar | SAGE Journals | ISI |
|
|
Deeksha, J (2020) Teachers not trained to conduct open book exams: Why professors, students are dead set against online exams in India. Edexlive. 22 June 2020. Available at: https://www.edexlive.com/news/2020/jun/22/teachers-not-trained-to-conduct-open-book-exams-why-professors-students-are-dead-set-against-onlin-12799.html (accessed 13 November 2020). Google Scholar |
|
|
De Laet, M, Mol, A (2000) The Zimbabwe Bush Pump: Mechanics of a fluid technology. Social Studies of Science 30(2): 225–263. Google Scholar | SAGE Journals | ISI |
|
|
Durkheim, É (1938/1977) The Evolution of Educational Thought. Lectures on the Formation and Development of Secondary Education in France. London, Henley and Boston: Routledge & Kegan Paul. Google Scholar |
|
|
Eisner, EW (1991) The Enlightened Eye: Qualitative Inquiry and the Enhancement of Educational Practice. New York: Macmillan. Google Scholar |
|
|
Ellingson, LL (2008) Engaging Crystallization in Qualitative Research. Thousand Oaks: SAGE. Google Scholar |
|
|
Fenwick, T, Edwards, R (2011) Introduction: Reclaiming and renewing actor network theory for educational research. Educational Philosophy and Theory 43(1): 1–14. Google Scholar | Crossref |
|
|
Foster, S (2011) Choreographing Empathy: Kinesthesia in Performance. Oxford and New York: Routledge. Google Scholar |
|
|
Foucault, M (1977) Discipline and Punish: The Birth of the Prison. New York: Pantheon Books. Google Scholar |
|
|
Goffman, E (1959) The Presentation of Self in Everyday Life. New York: Anchor Books. Google Scholar |
|
|
Hubble, S, Bolton, P (2020) A level results in England and the impact on university admissions in 2020–21. House of Commons Research Briefing paper. Available at: https://commonslibrary.parliament.uk/research-briefings/cbp-8989/ (accessed 20 September 2020). Google Scholar |
|
|
Jasanoff, S (2004) Designs on Nature: Science and Democracy in Europe and the United States. Princeton: Princeton University Press. Google Scholar |
|
|
Johansson, A, Vinthagen, S (2016) Dimensions of everyday resistance: An analytical framework. Critical Sociology 42(3): 417–435. Google Scholar | SAGE Journals | ISI |
|
|
Kellaghan, T, Greaney, V (2019) Public Examinations Examined. Washington, DC: World Bank. Available at: https://openknowledge.worldbank.org/handle/10986/32352 License: CC BY 3.0 IGO. (accessed 15 June 2020). Google Scholar |
|
|
Latour, B (1987) Science in Action: How to Follow Scientists and Engineers through Society. Cambridge, MA: Harvard University Press. Google Scholar |
|
|
Latour, B (2004) Why has critique run out of steam? From matters of fact to matters of concern. Critical Inquiry 30(2): 225–248. Google Scholar | Crossref | ISI |
|
|
Lundahl, C (2018) The Organising Principles of Disciple Assessment in the Swedish School Ordinances 1561–1724. In: Alarcón, C, Lawn, M (eds) Assessment Cultures: Historical perspectives. Studia Educationis Historica. Berlin, Germany: Peter Lang. Google Scholar |
|
|
McArthur, J (2020) Scotland’s exam result crisis: Assessment and social justice in a time of COVID-19. In: Lancaster University Education Research. Available at: https://www.lancaster.ac.uk/educational-research/blogs/scotlands-exam-result-crisis-assessment-and-social-justice-in-a-time-of-covid-19 (accessed 20 September 2020). Google Scholar |
|
|
Marx, K (1977) Critique of Hegel’s philosophy of right. CUP Archive. Available at: https://issuu.com/shangoking/docs/marx—critique-of-hegel-s-philosophy-of-right (accessed 15 June 2020). Google Scholar |
|
|
Meadmore, D (1993) The production of individuality through examination. British Journal of Sociology of Education 14(1): 59–73. Google Scholar | Crossref |
|
|
Murray-Atfield, Y (2020) Victoria has declared a state of emergency because of coronavirus COVID-19. Here’s what that means. ABC News, 16 March. Available at: https://www.abc.net.au/news/2020-03-16/victoria-coronavirus-state-of-emergency-powers-explained/12059194 (accessed 20 May 2020). Google Scholar |
|
|
Nandini (2020) DU teachers, student bodies against ‘open-book’ online exam decision, call it ‘discriminatory’. Hindustan Times, 15 May. Available at: https://www.hindustantimes.com/education/du-teachers-student-bodies-against-open-book-online-exam-decision-call-it-discriminatory/story-NDJA9KB5Jzne6LCdKFbvSP.html (accessed 25 November 2020). Google Scholar |
|
|
Pickering, A (2010) Material culture and the dance of agency. In: Hicks, D, Beaudry, M (eds) The Oxford Handbook of Material Culture Studies. Oxford: Oxford University Press, pp. 191–208. Google Scholar |
|
|
Reuters (2020) Chilean student hits the roof for online classes during pandemic. Reuters, 29 May. Available at: https://fr.reuters.com/article/us-healht-coronavirus-chile-education-idUSKBN2352Y9 (accessed 15 July 2020). Google Scholar |
|
|
Richardson, L, St Pierre, EA (2005) Writing: A method of inquiry. In: Denzin, NK, Lincoln, YS (eds) The SAGE Handbook of Qualitative Research (3rd ed.). Thousand Oaks: SAGE Publications, pp. 959–978. Google Scholar |
|
|
Sam, AH, Reid, MD, Amin, A (2020) High-stakes, remote-access, open-book examinations. Medical Education 54(8): 767–768. Google Scholar |
|
|
Simons, M (2020) The figure of the independent learner: On governing by personalization and debt. Discourse: Studies in the Cultural Politics of Education. DOI: 10.1080/01596306.2020.1732302. Google Scholar | Crossref |
|
|
Stray, C (2001) The shift from oral to written examination: Cambridge and Oxford 1700–1900. Assessment in Education: Principles, Policy & Practice 8(1): 33–50. Google Scholar | Crossref |
|
|
Tan, CK, Chua, WL, Vu, CKF, et al. (2021) High-stakes examinations during the COVID-19 pandemic: To proceed or not to proceed, that is the question. Postgraduate Medical Journal. Published Online First: 15 January 2021. DOI: 10.1136/postgradmedj-2020-139241. Google Scholar | Crossref |
|
|
The Hindu (2020) Most Delhi University students not in favour of open-book exams at home. The Hindu, 27 May. Available at: https://www.thehindu.com/news/cities/Delhi/most-delhi-university-students-not-in-favour-of-open-book-exams-at-home/article31682421.ece (accessed 13 November 2020). Google Scholar |
|
|
The Quint (2020). Glitches, errors in open book exam: Being prepared not DU’s forte! The Quint, 13 August. Available at: https://www.thequint.com/my-report/delhi-university-open-book-exam-student-experience-technical-glitches#read-more (accessed 17 November 2020). Google Scholar |
|
|
Tveit, S (2019) Policy Legitimation of Educational Assessment Reforms. The Cases of Norway and Sweden. PhD dissertation, Oslo University, Norway. Google Scholar |
|
|
Williamson, B, Hogan, A (2020) Commercialisation and Privatisation in/of Education in the Context of Covid-19. Brussels: Education International. Google Scholar |
|
|
Williamson, B, Eynon, R, Potter, J (2020) Pandemic politics, pedagogies and practices: Digital technologies and distance education during the coronavirus emergency. Learning, Media and Technology 45(2): 107–114. Google Scholar | Crossref |
Cristina Alarcón López, PhD, is Postdoctoral Researcher and Lecturer at University of Vienna (Comparative and International Education). Her research focuses on educational transfer, educational policy, global reforms, assessment, privatization and social movements.
Mathias Decuypère, PhD, is Assistant Professor in Education at KU Leuven, Belgium (Methodology of Educational Sciences Research Group). His research focuses on digital education platforms, data practices, open education, qualitative research methods, sociomateriality and social topology.
Joyeeta Dey is an independent researcher, with seven years of work experience in the education development space in India. Currently she is working as a research associate for the University of Melbourne.
Radhika Gorur, PhD, is Associate Professor in Education at Deakin University, Australia, and a Director of the Laboratory of International Assessment Studies. Her research focuses on education policy, global reforms, and science, technology and society.
Mary Hamilton is Professor Emerita of Adult Learning and Literacy at Lancaster University, UK. She is associate director of the Lancaster Literacy Research Centre and a founding member of the Research and Practice in Adult Literacy group. Her research focuses on literacy policy and governance, academic literacies, digital technologies and change.
Christian Lundahl is Professor of Education at Örebro University. Lundahl is specialized in the history of assessments, evaluation and of Swedish educational research. Lundahl is presently involved in research projects concerning the production and internationalization of data in education systems.
Elin Sundström Sjödin, PhD, is a researcher and lecturer at Mälardalen University. Her research interests are literary didactics, critical literacy, and science- and technology studies. At present, she conducts research on valuations of reading, reading robots, and reading as radical aesthetics.