GRADING IN-HOUSE BUBBLE EXAMS

Physics Computing Services

May, 1999

This document describes use of the Physics Scanning Workstation, OMR software, and interface to the Physics "class" web-server system to electronically grade quizzes and exams. These processes are part of the larger electronic web-based course management system in use in the Physics Department.

Physics Scanning Workstation:

Hostname: kitt.physics.ucsb.edu

IP Address: 128.111.8.109

Processor: 350 Mhz Pentium II

Scanner: Fujitsu Scan Partner 600c

With Auto Document Feeder

Operating System: NT Workstation 4.0

Scanning Software: Remark Office OMR 4.0

User Authentication: PCS Domain (samba)

 

Procedure Overview:
    1. Create A Scanned Template
    2. Process Student Bubble Forms creating answer file
    3. Upload answer file to "class" system and process (a couple different methods are available, depending on how the quiz/exam was created)

Step 1: Create a Scanned Template

Login to the Scanning Workstation

use your Physics (email) username and password

PCS Domain

Your Unix home directory will be automatically mounted from our Physics file-server, "spacecase"

Create a working directory in H:

Double-click on "My Computer":

click H:[\\SPACECASE\username] (your Unix Home directory)

select "File" then "New" then "Directory"

type a meaningful name: eg: Phys6C

Launch Remark Office Software

From "Start" Taskbar,

Select "Apps" Menu, Select Remark Office 4.0

From within Remark Office Software:

Select "File" from top menu bar

Select "Edit/Create Template"

When prompted to specify a location for the template,

Change drive to H:[\\SPACECASE\username]

Directory: H:\Phys6C (directory you just created)

Filename: Quiz_1.omr (omr filetype is required)

(tip: name should be relevant for later reference)

In "Template Options" dialog box:

Accept defaults,

<edit>

Place unmarked template page in scanner, face down, head first

<OK>

Check the image. Is it good?

If not, then <cancel>, and try again.

Once you’ve got a good image, move on…

Select and Define OMR fields:

Press <Select OMR> button

Use mouse to define a bubble region

Click and hold on upper left of bubble region,

Drag to lower right corner of region & release.

Defined area is enclosed in red box, and

should only include bubbles, not numbers.

Be sure to include a decent margin to improve accuracy)

Edit OMR field: (eg: student’s 7-digit PERM number)

Field Name: perm (give it a relevant name)

Field Size: Colunms: 7

Rows: 10

Data Type: Numeric Values

Is the data a single response, or multiple response?

Select "single"

Select "Each column is one character of answer"

Output Labels: Choose 0-9 (the fields are numeric: 0-9)

<OK> to finish this field

Define another fields

(eg: a block of 10 multiple choice questions)

Press <Select OMR> button

Use mouse to define a bubble region

Edit OMR field:

Field Name: mult (give it a relevant name)

Field Size: Colunms: 5

Rows: 10

Data Type: Text Labels

Is the data a single response, or multiple response?

Select "multiple"

Select "Each row is a separate question"

Output Labels: Choose A-E (possible answers to questions)

Do the answers have advanced attributes?

(for example, permitting multiple answers, or blanks)

If so, select <Advanced> button,

Go to "Blank/Multiples" Tab

Answer questions appropriately

<OK> to exit Advanced

<OK> to finish field

Continue until all fields are defined.

(Note: Only define those fields that you expect to have responses in.

The default behaviour for defined fields is that <BLANK> responses

are flagged as errors.)

Save your Template:

From "Template" Menu, Select "Save"

Confirm Save "Current page as default image for page 1"?

<OK> to confirm

Step 2: Process Student Bubble Forms:

Press <READ> button

Dialog box: Read Pages from Scanner

Change file location to your working directory

H:\\SPACECASE\username\:Phys6C

Base Name: 6C_Quiz1 (or whatever)

Check "Store Images" box

When prompted, place grade sheets in scanner document feeder, (face-down, head-first, and fanned-out, 50-sheets maximum)

<OK>

ReMark Software May Prompt you For Corrections:

During the scanning process, you may be presented with a scanned image, and asked to confirm or correct an entry. For whatever reason, the software is unable to discern the student’s answer. You can examine the scanned image, and make corrections using the dialog box.

Often if there is a problem with one answer in a block of multiple choice, there are problems with more. Sometimes the software will not recognize that there are additional problems, so it is very important for you to manually check all answers in any column that the scanner seems to have problems with.

Scanner will process the first set of up to 50 documents.

When prompted,

Continue with the next set of 50,

Reload scanner, and press <Continue> (or)

Click <Finished> when done

Correct Errors:

Not all errors are flagged for dynamic correction during scanning. After all documents are scanned, you need to process the entire response file to correct errors:

From the "Commands" menu, select "Correct Errors".

From the Dialog Box, select the types of errors you would like to consider:

(typically Form, No Response, and Multiple Response)

Correct errors as they are presented to you, as above.

Save your final response file:

From "File" menu, select "Save Data As":

Choose your working directory:

H:[\\SPACECASE\username]\Phys6C

Check "Save Headers" box

Choose Type: ASCII[tabs] [*.ASC]

Name: 6C_Quiz1.asc

Exit ReMark Software:

From "File" menu, select "Exit"

When asked whether to delete temporary image files, consider that these files can quickly eat up lots of disk space. However, it’s probably advisable to retain them for as long as the responses might be needed for review.

 

 

Step 3: Upload Answer File to "Class" Web System

What you do with the scanned bubble sheet ascii file now depends on how the exam was defined in the class web system. There are two main ways to do this: the "Front Door" and the "Back Door".

"Front Door": (can be accomplished by grader independently)

A) The questions are entered into the class web system, but the students aren't allowed to take them on-line. The correct answers are indicated in the normal course of creating the exam " grade items".

B) The instructor can use the class web system to print out the exam with randomized forms.

C) The bubble answer sheets are processed as noted in Steps 1 and 2 to

produce an ascii file.

D) The instructor can then choose the "Enter bubble answer sheets" to upload the answers into the class web system. Detailed steps for

doing this follow:

Log Into the Class Web System:

Using Netscape, go to class.physics.ucsb.edu

With your Login Name and Password

Stage 1: file upload

Go to Admin Index, Select Courses

Select your course from: View Class [Phys6C]

Select the graded item from: View Item [Quiz1]

Enter bubble answer results

Upload raw file

Browse H:[\\SPACECASE\username]

Select: All files

Select file: 6C_Quiz1

Click <send it>

Stage 2: Verify or make changes <Continue>

Stage 3: Press <Continue>

Stage 4: Save File

From "File" menu, select "Save As"

Enter Filename:

H:[\\SPACECASE\username]:6C_Quiz1

Select: Grade entry

Click <Save>

"Back Door": (requires assistance from Webmaster)

A) The exam questions are not entered into the class web system.

B) The bubble answer sheets are processed as noted in Steps 1 & 2 to produce an ascii file. The ascii file is mailed to webmaster@physics.ucsb.edu

C) An additional key file is sent to webmaster@physics.ucsb.edu with the following:
    1. Information identifying which grade item this is in the class system:

sgi_oid = #####

class_oid = #####

ii) The number of multiple choice questions that you want scored and the number of raw scores that you want included.

nummul = ##

numraw = ##

iii) an answer key in the form:

mulkey[115,1] = "D";

mulkey[115,2] = "D";

mulkey[115,3] = "A";

mulkey[115,4] = "C";

where 115 is the form number and 1,2,3,4 are the question numbers, and

D, D, A, C are the respective answers. If two answers are correct, put them both e.g. "AB" (note that this is NOT all-that-apply, either A or B will be accepted for full credit).

D) Webmaster takes the answer key and edits the awk script backdoor.awk

template (currently in ~emm) to apply to the specific course.

E) Webmaster runs the shell script:

awk -f backdoor.awk bubble-answer-ascii-file > upload-commands.sql

and reviews output for errors in perm numbers or form numbers. If someone gets a small number of the answers correct, they may have miss-bubbled the answer form.
  1. Webmaster runs on class: psql classweb < upload-commands.sql >& errors.dat

review errors.dat for incorrectly entered perm numbers, indicated by :

INSERT 0 0

and emails errors from awk processing to faculty member