Paper and Pencil Test

INTRODUCTION

Billions of examinations and assessments are administered every year across educational institutions, industries and around the globe. And what are these tests for? To us, it is to accurately – and so scientifically – measure the knowledge and competence of an individual. By delivering a test, you are aiming to use the information provided by the test in making a decision about the individuals taking the test. Are they competent to practice a certain profession? o they have the basic knowledge and skills needed to succeed at university? o they understand the road rules to safely drive a car? Tests have been traditionally administered through the paper and pencil Tests !""T# method but at the advent advent of the twent twentiet ieth h century century,, resear researche chers rs sought sought to find find a more more or rather rather conven convenien ientt means means of  administering tests. Thus the introduction of the $omputer based test !$BT#, the $BT continues to impress and has seen more and more institutions adopt it as a reliable means of test administration. %n &igeria, the 'oint Admissions and (atriculation Board, )niversity of %lorin, &ational *pen )niversity amongst others, have adopted this method and plans are on the way for other educational institutions and +xam bodies to follow suit. %n this paper we would try to compare and contrast the "aper and "encil Tests Tests !""T# and $omputer Based test !$BT#. PAPER AND PENCIL TEST

"aperandpencil test refer to a general group of assessment tools in which candidates read -uestions and respond in writing. This includes tests, such as knowledge and ability tests, and inventories, such as  personality and interest interest inventories. inventories. "aperandpencil tests can be used to assess subect/$ourse, obrelated knowledge and ability or skill -ualifications. Because many candidates can be assessed at the same time with a paperandpencil test, such tests are an efficient method of assessment. Developing paper-and-pencil tests: 4 asic steps!

%n developing a ""T all assessment methods must provide information that is relevant to the $ourse or  0ubect being assessed. The following 1 steps ensure that paperandpencil tests provide this information. "# Listing topic areas

2or each knowledge area that will be assessed by the test, list the topic areas to be covered. $heck off  any critical topic areas that are particularly important to the course or subect. 2or example, the topic areas that will be covered for us to pass this course include "sychological tests,  psychometrics, test construction, test standardi3ation, standardi3ation, psychological constructs and their  measurements and applications to computer programming. $# Speci%&ing t'e response %or(at# n)(er o% *)estions# t'e ti(e li(it and di%%ic)lt& level

"rior to writing the -uestions for your test, you should decide on such things as the response format, the number of -uestions, the time limit and the difficulty level.

+'at t&pe o% response %or(at s'o)ld I c'oose, T'e t'ree (ost co((on response %or(ats are: •

!a# multiplechoice4 multiplechoice4

•

!b# short answer4 and

•

!c# essay.

a. /)ltiple-c'oice

5ith a multiplechoice response format, a large number of different topic areas can be covered within the same test and the -uestions are easy to score. 6owever, because all potential answers must be chosen by some students, it is timeconsuming to write good -uestions.  . S'ort-ans0er

5ith a shortanswer response format, a large number of different topic areas can be covered within the same test and these -uestions are easy to score. %n addition, less time is re-uired to write these -uestions compared to multiplechoice ones. c. Essa&

5ith an essay response format, only a few topic areas can be covered due to the amount of time it takes to answer -uestions4 however, the content can be covered in greater detail. +ssay -uestions re-uire little time to write but they are very timeconsuming to score. Although at first glance a multiplechoice format may seem a relatively easy and logical choice if   breadth of coverage is emphasi3ed, don7t be fooled. %t is hard to write good multiplechoice -uestions and you should only choose this type of response format if you are willing to devote a lot of time to editing, reviewing, and revising the -uestions. %f depth of coverage is emphasi3ed, use an essay response format. 5hile preparing a paper and pen test, the number of -uestions you need depends on the breadth and depth of coverage re-uired and the importance of each topic area. 8enerally, the more important a topic area, the more -uestions you should have. 9ou 9ou should initially write several -uestions from each topic area so that you can choose the best ones for the final version of the test. 5ith a multiplechoice response format, :; -uestions are often sufficient. %f you are using a shortanswer or essay format, fewer -uestions will be re-uired. r e-uired. )nless speed is being assessed, the time limit should be set to allow the maority of students to finish within the allotted time. 5hile developing -uestions one should one should design the tests to have a range of difficulty levels. 1# +riting t'e *)estions and developing t'e scoring g)ide!

All -uestions should tap meaningful information. Also, the level of language used for the -uestions should be appropriate for the students to fully understand. The -uestions do not have to always be expressed verbally. verbally. iagrams, graphs, or tables may be incorporated into a -uestion where useful. 5hen you are writing each -uestion, you should prepare the answers, designate the marks to be allotted to each item, and decide on the rules for scoring. This ensures that there is a clearcut answer  for each -uestion. %t also allows you to indicate the value of each -uestion so that candidates can decide for themselves the amount of time they should spend on each -uestion. The marks assigned to each -uestion should reflect the relative importance of the -uestion.

a. /)ltiple-c'oice

The scoring guide for multiplechoice -uestions must include a scoring key indicating the correct answer and it may also include a rationale for or explanation of the correct answer. %f marks are to be deducted for guessing, this must be determined and stated in the instructions to students. students. . S'ort-ans0er

The scoring guide for shortanswer -uestions should include predetermined scoring procedures and mark allocations. +ach re-uired point in the answer should be listed with its r elative mark allocation. c. Essa&

The scoring guide for essay -uestions should include predetermined scoring procedures and mark  allocations. The maor points of the answer should be listed with their relative mark allocation. %f  marks are to be deducted for incorrect grammar, spelling and punctuation, this must be stated in the instructions to students. The scoring guides for shortanswer and essay -uestions should be clear enough so that scorers can  udge whether or not marks marks should be given to to a variation of the answer. answer. 4# Revie0ing *)estions and scoring g)ide!

6ave the -uestions and scoring guide reviewed by senior colleague or any other teacher familiar with the $ourse or 0ubect. 6ave them confirm that the -uestions are answerable and not too difficult or  not too easy, and that it conforms to the course outline provided at the beginning of the programme. Also have them make necessary grammatical corrections and edit any ambiguous, inappropriate or  mislea misleadin ding g conten content. t. The The review reviewer er should should also also assess assess the approp appropria riaten teness ess of the the time time limit limit and difficulty level of the -uestions. 9ou could as well have them take the test in testing conditions. 9ou should should only only includ includee in the final final versio version n of the paper paperand andpe penci ncill test test those those -uesti -uestions ons that that your  your  reviewers endorsed. E2a(ple ":

Below is an example of a multiplechoice -uestion and scoring guide !with accompanying rationale# designed to assess the ability to solve numerical problems. 3)estion:

<. 9ou are responsible for purchasing e-uipment and furniture. 6ow many filing cabinets can you  purchase with a budget of &=,;;;, if >; filing cabinets cost &<;,;;;? $hoose $hoose the correct answer from among the four choices provided. &o marks will be deducted for guessing. !"lease note that the use of  calculators is not permitted.# <. <> . ; :. > 1. :; Scoring g)ide rationale:

d!1

 &<;,;;;/>;@&=,;;;/  &<;,;;;/>;@&=,;;;/ @&:;;,;;;/<;,;;; @:; E2a(ple $:

Below is an example of a short answer -uestion and scoring guide designed to assess the knowledge of "sychological tests. 3)estion:

<. 5hat are the 1 maor characteristics of a "sychological test? !1 marks# •

eliability

•

Calidity

•

•

 &orms elative "ower 

Scoring g)ide: " point eac' Advantages o% Paper-And-Pencil Test

<. . :. 1. >.

A large large number number of differen differentt topic areas areas can be covered covered within within the the same test. test. The -uesti -uestion onss are are easy easy to to sco score. re. "aper and and pencil pencil are are easily easily portable portable and can can be used used in any any settin setting. g. %t is manual manual in nature nature i.e.you i.e.you don7t don7t need electric electricity ity or batterie batteriess to use paper paper and pencil. pencil. 9ou 9ou donDt need need any computer computer skills skills to write a paper pencil pencil test. The only skills skills needed needed is reading reading and writing skills. =. %t is very flexib flexible le and and cheap cheap to set set up. up. Disadvantageo% Paper-And-Pencil Test

<. . :. 1.

%t is is timeco timeconsum nsuming ing to write write good -uestion -uestions. s. %t tak takes es tim timee to eval evalua uate te There is high high chances chances of malprac malpractice tice in in this this type type of test test There is possi possible ble human human error error in the grading grading system system

Co(p)ter 5ased Test

A $omputerBased Test, also known as $omputerBased Assessment, eexam, computeri3ed testing and comput computer eradm admini iniste stered red testin testing, g, is a method method of admini administe sterin ring g tests tests in which which the respon responses ses are electronically recorded, assessed, or both. As the name implies, $omputerBased Assessment makes use of a computer or an e-uivalent electronic device !i.e. handheld computer#. $omputerBased Assessment enables educators and trainers to author, schedule, deliver, and report on surveys, -ui33es, tests and exams. 0ince the early ;;;s, much has occurred in $BT. $BT seems to have advantages over paper and  pencil testing, both for states that run the assessment programs and for the students who participate in them. There currently is strong interest in $BT and advocates have identified many positive merits of this approach to assessment includingE efficient administration, student preference, selfselection options for  students students,, improved improved writing writing performanc performance, e, builtin builtin accommodati accommodations, ons, immediate immediate results, results, efficient efficient item

development, increased authenticity, and the potential to shift focus from assessment to instruction !e.g., Becker, ;;=4 0alend, ;;F4 Thompson et al., ;;#. $BT also allows new ways of assessing students that move beyond the traditional multiple choice and constructed response items. 2or example, innovative assessments are now being developed that enable students to manipulate data and role play. 9et, as states move forward with $BT they are discovering that it is important to consider not only the positive  benefits, but also potential negative unintended conse-uences. These include, for example, the possibility that that additi additiona onall traini training ng will will be needed needed for studen students ts with with disabi disabilit lities ies to intera interact ct succes successfu sfully lly with with computers and the challenges of determining the best way to present some accommodations such as screen readers. espite the fairly dramatic increase in attention to $BT, accessibility challenges continue to have the  potential to reduce the validity of the assessment results and to exclude some groups of students from assessmen assessmentt participa participation tion.. %n the early years of $BT many fairly basic basic design design issues baffled testing testing companies and states as they sought to transfer paper and pencil tests onto a computerbased platform !Thompson, Guenemoen, H Thurlow, Thurlow, ;;=#. Conte2t)al Iss)es Related to Co(p)ter-ased Co(p)ter-ased Testing

The implementation of $BT occurs within a context that both supports and limits its use. %n this  presentation, we would try to briefly address two of the contextual factors that surround $BT includingE !a# the technological capacity of schools to support $BT, !b# universal design applied to $BT, Tec'nological Capacit& in Sc'ools

Access to computers and %nternet capabilities have for some time been a stumbling block for the push to widesprea widespread d use of computer computerbase based d and online assessmen assessments. ts. 2or example, example, Becker Becker !;;=# !;;=# -uestion -uestioned ed Idigital e-uityJ in computer access, computer use, and statelevel technology policies. 0tudents in ural areas and $ertain )rban area schools are likely not to have access to computers. And And even if they do have access most students arenDt properly trained on how to utili3e the devices. %n most schools, computers available are ust used for instruction purposes and most times the < computer could be used to teach a class of over :; students. Universal Design Applied to C5T

The term Iuniversal designJ applied to assessments in general has been defined in several ways !$A0T, ;;F4 ;;F4 Thompson, Thompson, Thurlow, Thurlow, H (alouf, (alouf, ;;1#. ;;1#. )niversal )niversal design of assessment assessment generally means an approach that involves developing assessments for the widest range of students from the beginning while maintaini maintaining ng the validity of results results from the assessment. assessment. )niversal )niversal design also sometimes sometimes refers to multiple means of representation, action/expression, and engagement. olan olan et al. !;;F# !;;F# prepared prepared a set of guideline guideliness specifica specifically lly for computer computerbase based d assessmen assessments. ts. The  principles address test delivery considerations, considerations, item content and delivery considerations, considerations, and component content and delivery considerations. A variety of topics relevant to computerbased testing and universal design is addressed in the component content and delivery considerations section of the guidelines !e.g., text, images, audio, video#, with each organi3ed according to categories of processing that students apply during testing. As the application of universal design principles to $BT has been considered, there also has been increased attention to various assistive technology re-uirements. Assistive Assistive technology devices can include such things as speech recognition or text to speech software, as well as sophisticated technology such as refreshable braille displays or sip and puff technology !which allows individuals unable to use a mouse or  speechtotext technology to send signals to a computer via a straw device using air pressure by sipping

and puffing#. *ne of the challenges of $BT has been to ensure that the assistive technology that is needed  by students with disabilities is available and that the students know how to use it. ussell !;;F# has considered this challenge and developed a set of <> capabilities that should be incorporated into the computer or online platform !e.g., allowing all text appearing within each test item and on all interactive areas of the screen, including menus and navigation buttons, to be read aloud using a human voice and synthesi3ed voices, etc.#. Advantage o% Co(p)ter 5ased Test %or St)dents

<. %t gives gives you you instan instantt feedba feedback, ck, unlike unlike paper paper examin examinati ations ons in a tradit tradition ional al class classroo room m learni learning ng session. . %t gives gives you the the option option of taking taking practi practice ce tests wheneve wheneverr you you want want as some assessm assessment entss are %nternetbased, and this allows students to take the test at home or anywhere they want. Advantage Advantagess o% Co(p)ter Co(p)ter 5ased Test Test %or t'e Teac'er eac'er## Sc'ool Sc'ool Ad(inist Ad(inistrator ratorss and E2a(inatio E2a(ination n odies!

<. Teache acherr can can dist distri ribu bute te mult multip iple le vers versio ions ns of the the exam examss and and assi assign gnme ment ntss with withou outt havi having ng to manually monitor which students got which tests. . %t allows allows the teache teacherr to -uickly -uickly evaluat evaluatee the perform performance ance of the the group. group. :. %t take takess up up less less time time and and effor effort. t. 1. All data data can can be stor stored ed on a sing single le serve server. r. >. Teacher Teacherss can mix and match the -uestion -uestion styles styles on exams, exams, includin including g graphics graphics more interactive than paper exams.

and make them

=. +limi +liminat nates es huma human n error error in grad grading ing.. K. This This test test is less less pron pronee to exam exam malpr malpract actice ice.. L. "urchasing "urchasing or print printing ing of answer answer sheet sheetss is reduc reduced. ed. F. "rint "rinting ing of test test booklets, booklets, in some some cases altern alternate ate forms forms of the same test test is reduced reduced or completely completely stopped. <;. $ost of "ackaging and shipping shipping of booklets booklets and answer sheets to testing testing locations is saved. <<. $ost of "ackaging and shipping shipping for return of answer sheets sheets and booklets from testing testing locations is saved. <. $ost of 5are arehous housing ing of book booklets lets and answer sheets, both before and after test administrati administration on is saved. <:. $ost of estruction estruction of booklets booklets and answer sheets after after they have served their purpose purpose is also saved. Disadvantage o% Co(p)ter 5ased Test Test %or St)dents

<. &ot all students students can use use the the computer computer very well. well.

. %t doesnDt doesnDt give give teachers teachers the option optionss to see your your line of thinkin thinking g to get to your your answer. answer. Disadvantage o% Co(p)ter 5ased Test %or Teac'ers

<. Techno Technology logy isn7t isn7t always always reliable. reliable. %nformatio %nformation n can be lost if a system breaks breaks down. down. . %n some some cases, teache teachers rs need some technic technical al expertis expertisee to create create exams. exams. :. The cost to set set up an electronic electronic assessm assessment ent system system in a learning learning instituti institution on or business business trainin training g environment is very expensive. 1. Testing Testing online online is not not suitable for essay writing writing and analysis or or cognitive thinking testing. Co(parison et0een Co(p)ter 5ased Test C5T. and Paper 5ased Test PPT.

<. CalidityE CalidityE Both $omputer Based Test Test !$BT# and "aper "aper "encil Test Test !""T# can be classified classified as valid valid if they both measure what they are set out to measure. 2or example in an aptitude test for a ob,  both $BT and ""T can be used to measure a test takers potentials for learning and ability to  perform in a new ob or situation. situation. . eliabili eliabilityE tyE both $omput $omputer er Based Test Test !$BT# !$BT# and "aper "encil "encil Test Test !""T# can be classifie classified d as  being reliable i.e. the test test results are consistent. consistent. :. Topic Topic areaE areaE Both $omputer $omputer based test test and paper paper pencil pencil focus on specific specific topic topic area and task task to cover. 1. Both $omputer $omputer Based Based Test Test !$BT# and "aper "aper "encil Tes Testt !""T# could be in (ulti (ulti $hoice format format or in short answer format. Di%%erence et0een Co(p)ter 5ased Test C5T. and Paper 5ased Test PPT.

S6N

Paper Pencil Test

Co(p)t p)ter 5ase Test

<

"aperandpencil in instruments re refer to to a genera generall group group of assess assessmen mentt tools tools in which which candid candidate atess read read -uesti -uestions ons and respond in writing

$omputeradministered testing, is a method of  administering tests in which the responses are electronically recorded, assessed, or both.



"aper pencil test is very ea e asy an a nd flex flexib ible le.. %t can can be cond conduc ucte ted d in any situation

$omputer based test are very expensive to set up and cannot be used in every situation.

:

e-uires on only th the sk skills of of re reading an and writing skills to prepare a paper pencil test

Asid Asidee from from read readin ing g and and writ writin ing g skil skills ls the the teacher and students needs computer skills to be able to use the computer based test

1.

it is prone to human grading error

%t eliminates human grading errors

>

%t takes time to evaluate the paper pencil test

%t saves time to evaluate test scores

=

%t ta takes ti time be before th the st students kn know the outcome of the test

%t gives instant feedback immediately after the test

K

%t is highly prone to exam malpractice

%t is less prone to exam malpractice

L

"aper and pencil test is very suitable for   Testing online is not suitable for essay writing essay writing and analysis or cognitive and analysis or cognitive thinking testing. thinking testing.

$onclusionE %n conclusion, can one really say one type of test administration is better than the other? *r which of  these assessment modes more accurately reveals the students7 actual knowledge? Bugbee !
References: Bugbee, A. $. 'r. !
28 !:#,

LFF.

Bunderson, $. C., %nouye, . M., H *lsen, '.B. !
2ederico, ". A. A. !
114,

11F1>L.

"arsha "arshall, ll, $. 8., H Mromre Mromrey y, '. . !
11  !<#,

<: