ࡱ> UWT@ WbjbjFF 4L,,W       |4#2FFFFF!"C Os#u#u#u#u#u#u#$&%Rx'# :!!::#  FF#~~~: F Fs#~:s#~~V!@  !F: \xa:W! "#0#c!x2(.F2(!    2( !W"y~WWW##DF Dt F Confidence Based Marking - think it through Try improving your student resources this summer with Confidence Based Marking (CBM). New software ( at www.ucl.ac.uk/lapt ) allows you to experiment with your own and others' material. Even if you work within a VLE, students can jump to CBM, returning their grades automatically. CBM marks a student according to confidence (or 'degree of certainty') 1 in each answer. The scheme in use at UCL is: Degree of Certainty :C=1 (low)C=2 (mid)C=3 (high)No ReplyMark if correct :123(0)Penalty if wrong :0- 2- 6(0) This rewards students who think things through, to the point that they can either:- (a) justify the confidence to risk a serious penalty if wrong, or (b) see reasons for reservation, lower confidence and reduced risk. Either way, they gain by thinking more deeply and correctly reporting high or low confidence2. A student who distinguishes reliable from uncertain knowledge does better than one with the same number of correct answers who cannot judge this correctly. Surprisingly, CBM is little used, despite many research studies showing its merits. There seem to be no implementations yet from major vendors, but CBM is actually easier to use than you may think. You don't need new question styles. CBM motivates students to think more carefully about any objective question (TF, MCQ, EMQ, numerical, etc.). Existing question banks are fine. You can access the programs from UCL with exercises on your local computer or server. Adaptation and authoring tools and experienced help are available. Alternatively, you can copy and maintain everything within your institution. Our experience shows that students rapidly take to the marking scheme and appreciate its merits without even instruction (yes, really!). What issues do staff and developers raise? I have pushed people hard at meetings, to try to understand reservations about CBM. The responses below are substantially confirmed by evaluation interviews recently conducted for our HEFCE funded project by JA Longstaffe and JWB Bradfield. The idea of CBM is attractive to most teachers in a wide range of institutions and disciplines. Few doubt the student satisfaction within UCL and Imperial (the main users), or the benefit of increased exam reliability. A common idea is that CBM may somehow reward particular personalities and exhibit gender bias. Our data clearly refute gender bias 3. If individuals are initially diffident or self-confident, a predominance of 1s or 6s soon seems to sets this right - recorded data show good calibration, using CBM in a near optimal way 3. Some suggest that what matters is just getting right answers, and the degree of certainty is a side issue - useful in formative work but irrelevant when measuring knowledge. Without entering philosophical debate, some points are clear. Lucky guesses are not knowledge, and confident errors are worse than acknowledged ignorance. Thirdly, even if 'number correct' is one's criterion of merit, predictions of this from scores on separate questions are better when using CBM than number-correct scores 3. Quite often people say "I see the point in medicine, where confident errors can be serious, but it seems less relevant to other subjects". Medical decisions certainly highlight the issues, but expression of the reliability of conclusions is a universally valued skill, seldom taught or assessed. Some staff and institutions are wary of objective testing in any form. Used without care, objective testing can encourage superficial learning. But it can equally test thinking skills 4. Objective testing actually has the potential to relieve us of core assessments, allowing teachers to concentrate on challenging tasks that only they can accomplish. Examiners worry about summative CBM, and passmarks. Nobody advocates CBM in exams without prior use for study and revision. CBM in formative work stimulates reflective learning and provides constructive feedback. When CBM was used only formatively at UCL and Imperial, students suggested use in exams, as implemented at UCL in 2000. For passmark setting, CBM data includes the 'number correct' information on which conventional pass standards are based; so immediate comparisons are available to inform the process fully. People worry that adoption of CBM may be time consuming, lacking support, or going against institutional strategies. At UCL we adopt a policy of immediate response to student and staff problems. Unsurprisingly, this has ensured that systems work smoothly with little support. Ease of use and flexibility are prime considerations. As for institutional strategies, these rarely constrain resources for study. To trial formal tests, UCL (with Speedwell Computing Services) offers Optical Mark Reader processing to avoid setup costs. Everyone worries about the RAE. Remember, efficient work that students enjoy on their own (or better, discuss in pairs) is a saving in contact hours. The website (www.ucl.ac.uk/lapt) is the place to explore exercises, publications and development tools. Encourage your students to try it. There are many thousands of questions, mostly good and some no doubt bad. Use the context driven comment system to tell us! Implementing CBM for your students may be easier and more rewarding than you think. Tony Gardner-Medwin Physiology, UCL a.gardner-medwin@ucl.ac.uk Footnotes 1. Darwin Hunt, one of the pioneers of CBM, prefers 'degree of certainty' because it is less tainted by association with generalised self-confidence. 2. Several schemes abound in the literature, not all properly motivating in the sense set out here. Our scheme is among the simplest, with also a theoretical foundation in information theory. 3. Gardner-Medwin AR & Gahan M (2003) Proc. 7th Int. CAA Conference, 147-155, available at www.ucl.ac.uk/%7Eucgbarg/tea/caa03.doc 4. Try practising for the Biomedical Admissions Test devised by UCLES: www.ucl.ac.uk/lapt?bmat1 +,-017@Rc  ƸƪƸzzƸzl[z hMRzh7zOJQJ^JmH sH hxOJQJ^JmH sH h7zOJQJ^JmH sH  hMRzh~OJQJ^JmH sH  hMRzh`9OJQJ^JmH sH h~OJQJ^JmH sH h`9OJQJ^JmH sH  hMRzhorOJQJ^JmH sH hMRzhPmH sH hdmH sH hMRzhmH sH hMRzhormH sH !,-G H ($@&IfgdY[L $@&IfgdY[Lxgdorgd)f W   / 0 7 D E   ȷȗȷxk]kLk]!hMRzhorB*OJQJ^JphhMRzhorOJQJ\^JhMRzhorOJQJ^Jh~OJQJ^JmH sH  hMRzhOJQJ^JmH sH hO2OJQJ^JmH sH #h)fh)fH*OJQJ^JmH sH  hMRzhorOJQJ^JmH sH h`9OJQJ^JmH sH hHOJQJ^JmH sH hUOJQJ^JmH sH hO+dOJQJ^JmH sH        PEEEEE $@&IfgdY[Lkd$$IflrT 6444U4lap2   1 3 ; ? A F [ p    ) Y z {  ; ij{iR,hMRzhB*OJQJ\^JnHphtH#hO2hm-2H*OJQJ^JmH sH hO+dOJQJ^JmH sH hm-2OJQJ^JmH sH h`9OJQJ^JmH sH hdOJQJ^JmH sH  hMRzhorOJQJ^JmH sH h8OJQJ^JmH sH hMRzhorOJQJ^JhMRzhorOJQJ\^J$hMRzhorB*OJQJ\^Jph  1 3 7 ; ? PC8888 $@&IfgdY[L $@&IfgdY[Lkd$$IflrT 6444U4lap2? @ A PJE8 ^`gd~gdor@&gdorkd6$$IflrT 6444U4lap2   ,y->4]r & Fxgd & FxgdMRzxgd2yMgdMRz & Fxgd2yM & Fxgdxgdxgdorx^`gd~; C H Q Y k o s   N \ h p q *,9;CU`nn邭魂ՙ&hY[LB*OJQJ\^JnHphtH,hMRzhB*OJQJ\^JnHphtH&hm-2B*OJQJ\^JnHphtH&hwr%B*OJQJ\^JnHphtH&h~B*OJQJ\^JnHphtH&hO+dB*OJQJ\^JnHphtH,hMRzh2yMB*OJQJ\^JnHphtH+`xyz +,-/;GRTW쪟t`L``&hUB*OJQJ\^JnHphtH&h!B*OJQJ\^JnHphtHhMRzhnHtHhUnHtHhMRznHtHh_nHtHhSnHtHhMRzh2yMnHtH,hMRzh2yMB*OJQJ\^JnHphtH&h<B*OJQJ\^JnHphtH,hMRzhB*OJQJ\^JnHphtH&hwr%B*OJQJ\^JnHphtH +/M}nWJ9!hMRzhMRzB*OJQJ^JphhMRzhMRzOJQJ^J,hMRzh2yMB*OJQJ\^JnHphtH&hUB*OJQJ\^JnHphtH&hMRzB*OJQJ\^JnHphtH,hMRzhdgB*OJQJ\^JnHphtH&h!B*OJQJ\^JnHphtH&h_B*OJQJ\^JnHphtH&hSB*OJQJ\^JnHphtH,hMRzhMRzB*OJQJ\^JnHphtH}~<?LWY^kp|}~̰̙̾}̾ooooaN$hVhVB*H*OJQJ^JphhVB*OJQJ^JphhUB*OJQJ^Jphh_B*OJQJ^JphhO+dB*OJQJ^Jph,hhB*OJQJ\^JnHphtHhSB*OJQJ^Jphh!B*OJQJ^JphhB*OJQJ^Jph,hhMRzB*OJQJ\^JnHphtHhMRzB*OJQJ^Jph;<=>CK *㮚rr^r^J^&h*B*OJQJ\^JnHphtH&h`B*OJQJ\^JnHphtH&hEB*OJQJ\^JnHphtH&h!rB*OJQJ\^JnHphtH&hSB*OJQJ\^JnHphtH,h!rh!rB*OJQJ\^JnHphtHhVB*H*OJQJ^JphhUB*OJQJ^JphhVB*OJQJ^Jphh!rB*OJQJ^Jph*+o|!"01234AGJV\inذĄp\\p&h5hB*OJQJ\^JnHphtH&hVB*OJQJ\^JnHphtH&h*B*OJQJ\^JnHphtH/h)fh)fB*H*OJQJ\^JnHphtH&h)fB*OJQJ\^JnHphtH&h<B*OJQJ\^JnHphtH&h`B*OJQJ\^JnHphtH&hEB*OJQJ\^JnHphtH!#(HO\]b*4ǰt`L4t/h@%h@%B*H*OJQJ\^JnHphtH&h@%B*OJQJ\^JnHphtH&h B*OJQJ\^JnHphtH&hY[LB*OJQJ\^JnHphtH&h_B*OJQJ\^JnHphtH&h5hB*OJQJ\^JnHphtH,hVhVB*OJQJ\^JnHphtHhVB*OJQJ^JphhUB*OJQJ^Jphh*B*OJQJ^Jphh5hB*OJQJ^Jph4fr{}!:PYZ[n춨~pp~~pbh<B*OJQJ^JphhUB*OJQJ^Jphh@%B*OJQJ^JphhzB*OJQJ^Jphh5hB*OJQJ^JphhUB*OJQJ^Jphh B*OJQJ^Jph&h<B*OJQJ\^JnHphtH&h5hB*OJQJ\^JnHphtH&hY[LB*OJQJ\^JnHphtH# !%*6G#=ǹ㢎zfzfR&hjB*OJQJ\^JnHphtH&h]B*OJQJ\^JnHphtH&hQ7BB*OJQJ\^JnHphtH&hO.B*OJQJ\^JnHphtH,hO.hO.B*OJQJ\^JnHphtHh B*OJQJ^JphhzB*OJQJ^JphhUB*OJQJ^Jphh@%B*OJQJ^JphhO.B*OJQJ^Jph169S]_ostv{|qrsİذĜ؜؜؜ĜİąqZ,hMRzhMRzB*OJQJ\^JnHphtH&hGB*OJQJ\^JnHphtH,hUh_B*OJQJ\^JnHphtH&h<B*OJQJ\^JnHphtH&h]B*OJQJ\^JnHphtH&hQ7BB*OJQJ\^JnHphtH&hjB*OJQJ\^JnHphtH&hUB*OJQJ\^JnHphtHs"$FGFİĜt`tXD-,h)fh)fB*OJQJ\^JnHphtH&h)fB*OJQJ\^JnHphtHh)fnHtH&h7zB*OJQJ\^JnHphtH&h8B*OJQJ\^JnHphtH&hO2B*OJQJ\^JnHphtH&hdB*OJQJ\^JnHphtH&hjB*OJQJ\^JnHphtH&h'B*OJQJ\^JnHphtH&hQ7BB*OJQJ\^JnHphtH&h B*OJQJ\^JnHphtH sWxgd)fgd)fxgd F6BsvVWխՙn,h)fhdB*OJQJ\^JnHphtH&h@%B*OJQJ\^JnHphtH&h7zB*OJQJ\^JnHphtH&hdB*OJQJ\^JnHphtH&h)fB*OJQJ\^JnHphtH,h)fh)fB*OJQJ\^JnHphtH&hO2B*OJQJ\^JnHphtH21h:p j/ =!"#$% $$If!vh5 5555#v #v#v#v#v:V l6,5 55/ /  / 444Uap2 $$If!vh5 5555#v #v#v#v#v:V l6,5 55/ / / /  444Uap2 $$If!vh5 5555#v #v#v#v#v:V l6,5 55/ / / /  444Uap2@@@ NormalCJ_HaJmH sH tH V@V MRz Heading 3$<@&5CJOJQJ\^JaJDAD Default Paragraph FontRi@R  Table Normal4 l4a (k(No List>> x Footnote TextCJaJ@&@ xFootnote ReferenceH*WL137;?@AY@0 @0@0 @0@0 @0@0 @0 @00@0 @0 @0 @0 @0 @00@0 @0 @0 @0 @0 @00@00 },-GH137;?@A,y-> 4 ] r sY(0000000 00 00 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (00@0@0@0@00@ 0@ 0@ 0@ 0@0(00 0 0 0 0 0 0 0 0 0000`0`0`0`(00000,-GH137;?@A,y-> 4 ] r sY*000000 00 00 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (00J0J0J0J0J 0J 0J 0J 0J*00 0 0H 0 0 0 0 0 0 00000000(00000  ; `}*4sFW !"#%  ? W$W8@0(  B S  ? _Ref104640665 Y; Y \ wwY~~Y8*urn:schemas-microsoft-com:office:smarttagsCity9*urn:schemas-microsoft-com:office:smarttagsplace +[p [n<<=69JJS]_o{{q$FGppVVYY333301A swYY*F%vrH)^Vb kG2eh^`OJQJo(hHh^`OJQJ^Jo(hHohpp^p`OJQJo(hHh@ @ ^@ `OJQJo(hHh^`OJQJ^Jo(hHoh^`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohPP^P`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohpp^p`OJQJo(hHh@ @ ^@ `OJQJo(hHh^`OJQJ^Jo(hHoh^`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohPP^P`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohpp^p`OJQJo(hHh@ @ ^@ `OJQJo(hHh^`OJQJ^Jo(hHoh^`OJQJo(hHh^`OJQJo(hHh^`OJQJ^Jo(hHohPP^P`OJQJo(hHb kG*F%rH)                           76O.7z ET)fH!@%wr%'*;(m-2`9<PO?Q7BY[L2yMSVOYO+ddg5h jos_3zMRzzUGorO2PU!r*]q?dj8# x\W~_`P137;?@Y@16{WP@UnknownGz Times New Roman5Symbol3& z Arial?5 z Courier New;Wingdings"1h&˕f•F { *{ *$24dLL 3qH)?_3z@Confidence Based Marking - it's easier and better than you thinkargmargm   Oh+'0 4 DP l x  AConfidence Based Marking - it's easier and better than you thinkronfargmdenrgmrgm Normal.dotBargml.d13mMicrosoft Word 10.0@]H@cL`@Ɯ_@a{՜.+,0, hp   UCLn* L AConfidence Based Marking - it's easier and better than you think Title  !"#$%&()*+,-.0123456789:;<=>?@ABCEFGHIJKMNOPQRSVRoot Entry F`?aXData '1Table/J(WordDocument4LSummaryInformation(DDocumentSummaryInformation8LCompObjj  FMicrosoft Word Document MSWordDocWord.Document.89q