Multiple Choice Marking
New! D.I.Y. Multiple Choice Forms.
Not surprisingly, interest in using the School's Multiple Choice
marking facility is increasing. This Web Page is intended to provide
information about, and promote the use of, this system.
If you have a large class and can reformulate all or part of your
assessment procedures in multiple choice form, then the efficiency
gains can be spectacular. In First Semester 1999 approximately 2400
script hours of examination material were processed in this way.
Other advantages include:
It seems likely that other useful applications will be found. For
example Humphrey Gastineau-Hills has developed a method to speed entry
of marks from first year assignments.
Elimination of manual entry of marks.
Current usage and checking procedures appear to give extremely
high accuracy, almost certainly better than manual marking.
Ability to vary number of questions and number of choices.
Facilities for processing non-homogenous sets of answer forms,
such as come from a single class using several different
sets of questions (to reduce copying, for example).
Note that the current software cannot handle the difficult problem
of reliable recognition of handwritten characters. Instead it is
based on simple and reliable Optical Mark Recognition
The completed forms are first scanned using a high-speed, batch-feed
scanner. The scanner processes around 1 page per second, and the
images are stored on the hard disk of the controlling PC. It can also
scan both sides of the sheet simultaneously, although this facility
has not so far been used.
The project initially used a low-cost flatbed scanner/sheet feeder
combination. This was much slower, and prone to misfeeds, but still
much faster than manual marking. The present machine seems largely
immune to paper misfeeds.
Here is a typical completed multiple choice form, designed in TeX by
Roger Eyland (and with student name erased):
Click here for a bigger image
The main features of the form are:
Spaces to write name and SID.
OMR area to enter SID.
OMR area for answers.
Limited space for questions, if required. For more than a few questions
most people put the questions on a separate sheet.
The OMR software can reliably handle rectangular fill-in arrays of
of any size, and interpret the results in various natural ways.
In addition to OMR arrays, the software can also read barcodes
and capture images from sections of the page. In this example, the
OMR process on this image generates the data
The first item in this list is the SID. The second item is the pathname
of a small image containing the handwritten SID (used for later
checking). The remaining items show the selected answers. It is
easy to match these against the correct answers with a short
python or perl script, or by reading the
data into a spreadsheet.
The forms can be further parametrised by adding `office-use only'
data in extra OMR fields, or barcodes.
Checking the Results
There are various sources of error which need to be monitored.
Students commonly enter their SID incorrectly. The most common problem
is the choice of the first row of the SID array to represent the
digit 1. There are several ways to check for this.
If the forms are
sorted alphabetically by name, the SIDs can be matched against a correct
list. An out-of-order or unmatched SID signals an error.
Alternatively images of the handwritten SIDs can be captured,
decomposed and sorted according to the corresponding OMR
selection. Then errors can be found by visual inspection.
Some students complete the forms in ink, and then cross out, or
use whiteout, to make corrections. In the former case the software
flags a MULTIPLE selection. In the second case it typically
rejects the whole array.
If the form is completed in hard pencil, the mark may not be dark
enough to trigger recognition, and the software flags a BLANK.
Fortunately the software is quite conservative and prefers to flag
errors rather than make guesses. It also provides a reasonably
friendly user interface for checking and correcting these types
of recognition failure.
The success of the operation is quite sensitive to some features of
the form design. The forms currently in use are the result of some
experimentation and tuning.
Note that there is now a web interface
for customising and printing OMR forms for individual lecturers
If you are interested in this, or have other suggestions for future
developments, please let me know.