Multiple Choice Marking

New! D.I.Y. Multiple Choice Forms.

Not surprisingly, interest in using the School's Multiple Choice marking facility is increasing. This Web Page is intended to provide information about, and promote the use of, this system.

If you have a large class and can reformulate all or part of your assessment procedures in multiple choice form, then the efficiency gains can be spectacular. In First Semester 1999 approximately 2400 script hours of examination material were processed in this way.

Other advantages include:

It seems likely that other useful applications will be found. For example Humphrey Gastineau-Hills has developed a method to speed entry of marks from first year assignments.

Note that the current software cannot handle the difficult problem of reliable recognition of handwritten characters. Instead it is based on simple and reliable Optical Mark Recognition (OMR) technology.

Operational Details

The completed forms are first scanned using a high-speed, batch-feed scanner. The scanner processes around 1 page per second, and the images are stored on the hard disk of the controlling PC. It can also scan both sides of the sheet simultaneously, although this facility has not so far been used.

The project initially used a low-cost flatbed scanner/sheet feeder combination. This was much slower, and prone to misfeeds, but still much faster than manual marking. The present machine seems largely immune to paper misfeeds.

Here is a typical completed multiple choice form, designed in TeX by Roger Eyland (and with student name erased):


Click here for a bigger image
The main features of the form are:
  • Spaces to write name and SID.
  • OMR area to enter SID.
  • OMR area for answers.
  • Limited space for questions, if required. For more than a few questions most people put the questions on a separate sheet.

The OMR software can reliably handle rectangular fill-in arrays of of any size, and interpret the results in various natural ways. In addition to OMR arrays, the software can also read barcodes and capture images from sections of the page. In this example, the OMR process on this image generates the data

9821105,c:\images\2002\sid0033.PCX,3,1,4,4,2,2,4,5,4,1

The first item in this list is the SID. The second item is the pathname of a small image containing the handwritten SID (used for later checking). The remaining items show the selected answers. It is easy to match these against the correct answers with a short python or perl script, or by reading the data into a spreadsheet.

The forms can be further parametrised by adding `office-use only' data in extra OMR fields, or barcodes.

Checking the Results

There are various sources of error which need to be monitored.

Fortunately the software is quite conservative and prefers to flag errors rather than make guesses. It also provides a reasonably friendly user interface for checking and correcting these types of recognition failure.

Future Developments

The success of the operation is quite sensitive to some features of the form design. The forms currently in use are the result of some experimentation and tuning. Note that there is now a web interface for customising and printing OMR forms for individual lecturers and courses.

If you are interested in this, or have other suggestions for future developments, please let me know.

Nigel O'Brian
nigel@maths.usyd.edu.au
4/8/99