Strategies for Exploring Technology as a Proctoring Solution

Tay T. McEdwards
Oregon State University
Ecampus

Author Notes

Tay McEdwards is the proctoring manager for Ecampus at Oregon State University. He holds an M.S.Ed. in Information Technology and is a doctoral student in the Adult and Higher Education program at Oregon State University. Tay has worked in higher education for over a decade with a focus on accessibility and online learning. His areas of expertise include testing center operations, technology integration, accessibility, and online proctoring. He served on the board of directors for the Oregon Association of Higher Education and Disability and co-chairs a distance learning group for the National College Testing Association.

Abstract

Until recently, when the massive shift from on-campus to remote learning occurred, there was little interest in online exam proctoring and limited technology available. Additionally, few resources were available on how to explore that technology as a potential proctoring solution. This commentary draws on the author’s extensive experience in the online exam proctoring space to help the reader identify the goals of their institution and to provide a framework for analyzing technology as a potential proctoring solution. The framework provided in this article (P.A.W.S) can help readers consider how to explore technology as a proctoring solution for online exams.

Keywords: Online proctoring, exam proctoring, proctoring technology, exam invigilator

Over the past ten years, I have worked with students, faculty, vendors, and administrators on exam proctoring in higher education. Much of that time was spent specializing in disability-related testing accommodations for both in-person and online exam environments. Currently, I work as the proctoring manager for Oregon State University’s Ecampus, which, as an established online program, has provided students with at least one online exam proctoring option for over a decade. Throughout my time as proctoring manager, I have negotiated with proctoring vendors, led our transition to new proctoring practices, and consistently prioritized accessibility. Most recently, my focus has been on developing equitable online exam proctoring for diverse learners located around the world, facilitating approximately 45,000 exams in academic year 2021-2022; the exams we proctor have grown year over year to an estimated 50,000 exams in academic year 2022-2023.

When I ventured into the field of exam proctoring, there were best practices for in-person exam proctoring but very few resources available for online exam proctoring. Little to no information was available about how to determine what type of proctoring would work best for online learners. Based on years of experience, I created a framework for the process of examining technology as a potential proctoring solution. Until recently, when the massive shift from on-campus to remote learning occurred, there was little interest in online exam proctoring. However, online learning is here to stay (Capranos et al., 2022) and the needs of institutions will vary. Therefore, I urge you to consider the needs of your students as you read these suggestions. This article will help you identify the goals of your institution and provide a framework for analyzing proctoring tools.

Types of Proctoring

To begin, let us consider the most common types of proctoring for online exams and how technology is often used with each of them. The examples below provide a brief description of each, however, this is not an exhaustive list.

Automated proctoring

• Students download a web browser extension to their device.
• The device’s camera and microphone records audio and video during an exam session.
• The instructor may review the recording after it is filtered through automated proctoring or algorithmic software to determine and flag anomalies.

Online proctoring

• Students may need to log into a specific site where a live proctor at a different location monitors the exam session through their device’s camera and microphone.
• The exam session is recorded and may be reviewed by the instructor.

In-person proctoring

• Students go to a testing center in a local city where a person at that location monitors the exam.
• A computer is provided at the testing center for the student to use.

Each type of proctoring could potentially use a lockdown browser, which is a technology that restricts a test-taker’s ability to access other programs or navigate outside of the exam window during an exam session by ending an exam if that occurs.

Know your Goals

Before diving into the world of technology solutions for exam proctoring, consider these questions that can help you remain focused on the needs of your institution:

• What is the goal of using technology for proctoring? (e.g., reduce costs, increase efficiency or testing capacity)
• Why is technology needed to accomplish this?
• How do you want your staff, students, and instructors to interact with the technology?

Your responses to these three questions can guide you as you begin to explore all of the technology available for proctoring online exams. Establishing your responses as early as possible will help you navigate through technology options and allow you to quickly narrow your search to specific products of interest when exploring the capabilities of each. For example, if you determine your major goal is to reduce costs for your students and increase affordability, you can quickly weed out options that are only student pay or over budget.

P.A.W.S.

When exploring technology as a proctoring solution, some of the most important things to consider can be summarized as P.A.W.S. (Pricing models, Accessibility, Workflow/integration, Support/customer service).

Pricing Models

Pricing models are often separated into two main categories: institution pay and student pay. There is not one model that is guaranteed to work best for everyone, so determining what will work best for your students and your institution is key.

• Institution pay is often categorized as institutions paying for proctoring per exam, per user, or through a site license. It is important to consider the number of students who will be using the technology and the number of exams it will be used for to determine what would make sense for your institution. For example, if you have a low number of students in courses with a high number of exams, comparing the cost per student versus per exam may be helpful. If you have many students taking a low number of exams, then a per-exam pricing model may work best. However, if you have many students taking a high number of exams, a site license might be the most economical solution.
• Student pay is often categorized as students paying for proctoring per exam or with a flat fee. Consider the parameters for a flat fee rate to determine whether it is per term or yearly. This option is less likely to be beneficial for students unless you are certain they will remain enrolled during the entire time frame (e.g., the academic term). Additionally, you should consider the average number of exams in a course.

Keep in mind that using a student pay model increases the additional costs of education for students. If affordability is a top priority, I caution against using a student pay model without a thorough examination of how it could impact your students financially. It is important to consider the diverse needs of online learners who are typically older (Venable, 2022a), balancing work, their education, and family responsibilities (Aslanian et al., 2022; Venable, 2022b). Like technologies, pricing models are not one size fits all, so explore all available options. One solution could be to provide two options, a free proctoring technology option and a student pay option, so students can choose which approved method will work best for them.

Accessibility

Many vendors will indicate that a product is accessible; however, the definition of accessibility differs across companies, geographic locations, and industry standards. Additionally, one company’s idea of an accessible product might not align with your institution’s commitment to accessibility for students. When examining the accessibility of technology, using established guidelines and standards for accessibility in your institution and country can be a good starting point, but should be considered the bare minimum. The following questions can be helpful to evaluate the accessibility of technology beyond minimum expectations.

• What makes it accessible? Ask vendors how a technology meets the minimum guidelines and standards required to be considered accessible. Keep in mind that meeting these requirements should be considered the baseline. During a demonstration, it can be helpful to request the vendor show how a technology exceeds these standards. A good way to approach exploring the accessibility of a technology is understanding that even an accessible technology can become inaccessible in certain situations given the variability of technology, learner ability, and instructor exam settings.
• Does it work with the assistive technology your students might use? It is extremely important to consider how proctoring technology will interact with assistive technology used by your students. Is the technology compatible or would there be barriers for use? Are there specific settings or methods of use that would determine whether it would be compatible? For example, does the technology allow a student to use their computer screen reader or does it depend on what settings are used for the exam? Consider how using technology in a certain way or with specific settings could impact its accessibility to better understand how your students would interact with the product.
• Does it create any barriers for your students? If the technology works with computer screen readers, ensure it works with multiple types of computer screen readers and those available at your institution. Otherwise, you may inadvertently be creating a barrier to an accessible product.
• Are there multiple ways to get support? Ensure support is available to students through multiple methods. Consider the various ways students communicate and prefer to receive assistance (e.g., email, 24/7 live chat, telephone). A greater number of support options might meet the needs of more students; however, be sure those options are accessible.

Workflow/Integration

Workflow/Integration for new technology can have both short-term and long-term effects on test-takers, faculty, and staff, requiring a thorough examination when considering technology for proctoring solutions.

• How would integrating the technology impact workflows? Does the technology integrate with your current learning management system (LMS)? If it does, to what extent? Would it streamline workflows by using exam information already in the LMS or shift workloads by requiring an individual to input exam information into a separate platform? Identify the potential benefits and barriers of implementing any new technology and how all users would interact with it.
• What training would be needed? Be sure to consider the training needs of all who would use the technology, including students, faculty, and staff. Do you have the capacity to deliver this training? Does the technology provider offer trainings or resources that could help address these needs?
• Where are exam sessions stored? Privacy and security of data collection have become priority topics in the field of online proctoring. Technology providers should be able to tell you how they meet or exceed federal guidelines and industry standards for privacy and data security. Consider what data is collected, where exam sessions are stored, how long they are retained, and who can access them.

Support/Customer Service

Consider the type of support each technology provider offers test-takers, faculty members, and staff members. Would it meet their needs? For example, if you have asynchronous exams, support may be needed at different times than it would be for synchronous exams. Your test-takers and faculty members may both need 24/7 access to support.

• Customer satisfaction ratings should be critically examined to understand how a proctoring vendor balances customer service with exam integrity. As mentioned in the forthcoming piece by McEdwards & Underhill, online proctoring vendors are not held to academic regulations and conventions for transparency in collecting, analyzing, and reporting data. For example, some vendors might collect data about student views only at the end of exams, missing valuable insight from students who did not complete their exam.

This framework is a starting point to assist you with analyzing the proctoring needs at your institution and assessing technology as a potential proctoring solution. I hope you will find these suggestions helpful as you consider how to explore technology as a proctoring solution for online exams.

Final Considerations

Go beyond the sales pitch

Remember, the goal of vendors is to sell their product. Regardless of how a product is marketed, there is not a one-size-fits-all technology solution for proctoring online exams. By acknowledging this limitation, institutions will be better positioned to critically examine the potential benefits and barriers of each technology. The benefits and barriers often depend upon your institution’s operations, needs, and current technologies. The three main what, why, and how questions should be used to guide your explorations of potential technology solutions. Understanding how technology currently used at your institution would interact with potential proctoring solutions will help to quickly narrow your search.

Always request a demonstration

After years of working with various vendors, I have seen the benefit of “show me” versus “tell me.” Experiencing a demonstration can result in generating useful questions. If a vendor is unable to answer your questions during a demonstration, this could signal a potential issue. Do they follow up with a response within a reasonable amount of time? This can provide helpful insights into the type of customer service you would experience if buying that product.

The amount of technology available for proctoring online exams is rapidly increasing. Consider your goals using what, why, and how questions to streamline the process of reviewing potential proctoring solutions. Additionally, using P.A.W.S. (Pricing model, Accessibility, Workflow/integration, Support/customer service) can help you navigate available options to determine which would work best for your students and your institution.

References

Aslanian, C. B., Fischer, S., & Kitchell, R. (2022). Online college students report 2022. EducationDynamics. https://insights.educationdynamics.com/2022OnlineCollegeStudentsReport.html

Capranos, D., Dyers, L., & Magda, A. J. (2022). Voice of the online learner 2022: Shifting preferences in post-pandemic online learning. Wiley University Services.
https://universityservices.wiley.com/voice-of-the-online-learner-2022/

McEdwards, T. & Underhill, G.R. (2024). Online students’ preferences for features and types of exam proctoring [Manuscript in preparation]. Ecampus, Oregon State University

Venable, M. A. (2022a). 2022 online education trends report. BestColleges.com.
https://www.bestcolleges.com/research/annual-trends-in-online-education/

Venable, M. A. (2022b). 2022 trends in online student demographics. BestColleges.com.
https://www.bestcolleges.com/research/online-student-demographics