CAMBRIDGE CPE 2013



An Analysis of Paired Speaking Tasks inThe Cambridge Certificate of Proficiency in EnglishSusan MayerThe Monterey Institute of International StudiesMA TESOL ProgramDr. Kathi Bailey4 October 2013I was lucky enough to have a summer internship with a language program in the alpine paradise of Garmisch, Germany. One of the seminars I observed was hosted by the Bureau for International Language Coordination (BILC), an organization tasked to work with NATO and NATO-partner countries to organize English language teaching and testing around a common set of standards, called the STANNAG 6001 (BILC 2013). My role was to observe the instructors and learn about this newly-developed seminar “Language Standards and Assessment Seminar”. Worried that it might be tedious, I was instead pleased to discover the complexity of the task of language testing, as the two weeks flew by. Thirteen people represented thirteen countries, spending their time learning strategies to undertake this standardization process when they returned to their roles as leaders in their country’s English program. Often during this seminar, I heard the term ‘CPE’ used, with a reverent tone, suggesting it might be the gold standard in the testing industry. Since my dream job is to work with language learners at the diplomat level, I realized that the Cambridge Certificate of Proficiency in English (CPE) is the type of test they are likely to take in order to demonstrate their professional-level English proficiency.In order to provide context for the reader, I will begin with a brief history of the test, and move into a description of the individual parts of the test, looking more closely at the paired oral test format. Finally, I will provide an analysis of the test, using frameworks developed by Wesche (1983) and Swain (1984). History of the CPEAs the first of Cambridge University’s tests of English for non-native speakers, the Certificate of Proficiency in English (CPE) is comprised of a series of papers, each of which tests discrete skills. In 2013, the CPE passed the 100-year mark of its existence. In the hundred years since it was first given to three candidates, the test has gone through several major revisions on the way to the most recent (2013 UCLES) version. On the occasion of the launch of the 2002 revisions, Weir (2002) published a summary of how the CPE has been transformed over time , connecting the rationale for each new revision to the state of language teaching and testing at the time. According to O’Sullivan (2011), the early CPE was based upon the premise of using the language, surprisingly parallel to its current permutation. However, we do see a link to the behaviorist/cognitive paradigm (Myles p. 320), dominant at the time of the early CPE tests, in the required tasks. In fact we see little change in the test suite during the years between 1913 and 1945, when language learning was organized around language as a cognitive process, and teaching focused on translations (Myles p. 320) and memorization of vocabulary. Early CPE ‘papers’ as the test sections are called (UCLES 2013) contained British literary or cultural references, allowing test-takers options to choose from, among those presented. This remained as the format of the CPE until 1975, when it was recognized that a growing number of candidates came from countries outside the United Kingdom (Weir 2002) and might not have the sociocultural frame necessary to respond appropriately. According to Weir (2002), the 1975 revision to the CPE represented a turning point in the Cambridge testing family, reflecting a growing trend in language teaching toward ‘language in use’ (Weir p. 5) which favored communicative language teaching and the use of relevant materials in both teaching and testing. Further revisions have resulted in the CPE being offered both on paper at one of the thousands of testing centers worldwide, and online for the first time in March of 2013. Currently the test is administered to approximately 45,000 people in 130 countries (UCLES 2013). ?Description of the TestGeneral OverviewAccording to Stoynoff & Chapelle (2005, ch. 3), the guidelines for a large-scale test should provide the best source of information about the test, and in fact “test practices are enhanced when developers and users embrace the same professional standards.” To that end, much of the information gained in learning about the test comes from the materials provided by the University of Cambridge Local Examinations Syndicate (UCLES) on their website [] or written materials available to the public. The website provided a wealth of information beyond what a test-taker needs to know about this test; the site also has extensive information about the background of test-writing, statistical information about scoring, evidence of reliability, and links to hundreds of researched journal articles which touch upon every aspect of the test. It would be impossible to include links to each relevant article or web page within the site for this summary, but the reader is invited and encouraged to explore further. The Cambridge Certificate of Proficiency in English, or CPE, is a criterion-referenced test which assesses candidates’ English abilities based on all of the four skills; reading, writing, speaking and listening, using ‘realistic tasks and situations’ (UCLES 2013). It represents UCLES’ highest level test, rating candidates based on level C2 on the Common European Framework of Reference (CEFR). At this level, the learner is approaching the “linguistic competence of an educated native speaker, and is able to use the language in a range of culturally appropriate ways” (UCLES 2013). Success on the CPE is an indicator that the person is capable of undertaking graduate-level work in English at a university setting. Employers also use the CPE as a qualifying standard for employment if English use at a near-native level is a requirement of the position. According to Leung & Lewkowicz (2006), the job of a test is to ‘encourage the display of the target qualities through some kind of performance’ (p. 220). The information provided to the English learner on the CPE website explicitly states their commitment to excellent testing overall, through constant evaluation of test materials and oversight of the marking and grading of each section (UCLES 2013). Information about the cost of the CPE on the UCLES site is elusive; however O’Sullivan (2011) alludes to the pricing system in his videotaped lecture, which has a baseline price plus ‘local costs’, which can vary widely. Without signing up for the test on the UCLES’ website, it is sufficient to say that for most Western European countries, the cost will be approximately ?150 (O’Sullivan 2011). One might guess that for countries where the economy does not align with Western economic standards, the local costs (O’Sullivan 2011) are likely to be much lower than their more affluent counterparts. Test InformationReading & Use of English. Candidates will read and demonstrate their understanding of meaning from a variety of sources, both fiction and non-fiction; such as novels, journals, newspapers, magazines (UCLES 2013). This test is divided into sections, each of which matches a reading passage with questions in various formats, including Cloze, modified Cloze, multiple choice, Gapped text, and matching. Skills assessed in this section include idioms, collocations, complementation, phrasal verbs, grammar awareness and control, affixation, understanding of text structure, cohesion, global meanings, opinion vs. fact, attitude and detail. Sample questions with answer key can be found in APPENDIX B. Writing. This section assesses the candidates’ skills in writing a variety of texts, such as articles, letters, and essays. Skills include summarizing, evaluating, synthesizing, and reformulating ideas in their own words to form a cohesive argument. Responses are generally between 250 and 500 words (UCLES 2013). See APPENDIX B.Listening. To demonstrate listening skills, candidates are given four separate listening tasks and asked to listen for things such as gist, detail, function, attitude, opinion, topic, specific information, inference, and interpret context or discover the main idea. The listening passages comes with its own performative task. In the Sample Test for 2013 [See APPENDIX C] we can see that the candidate will be asked to answer multiple choice questions (parts 1 and 3), fill in the blanks (part 2), and match (part 4).The Paired Speaking Task. Unique in the English language proficiency testing world, the Cambridge family of exams pairs two candidates together for the speaking portion of the exam. Two candidates are in the room with two examiners. One of the examiner will be seated at a table opposite the two candidates, acting as the interlocutor, while the other examiner is seated at a table slightly removed, using a checklist (O’Sullivan) to tabulate scoring. The 16-minute test begins with a 2-minute individual interview, in which the candidate is asked to give information about him/herself, and express an opinion about a topic introduced by the examiner. Part 2 is a 4-minute task where the examiner asks both candidates to look at a set of photographs [see APPENDIX D], first answering a question, and then interacting with the other candidate in a decision-making task (UCLES 2013). In the final section, the interlocutor will present a card with a question and some ideas. Each candidate will speak for two minutes individually, after which the interlocutor will ask both candidates a question, opening a discussion among the two candidates. Scoring SystemEach section of the CPE is scored independently and represents 20% of the total score, with the Reading & Use of English sections combining to account for 40% of the total. For each Paper, the grade boundaries for A, B, C, D, and E, are set based upon statistics for the overall candidature, the performance of the individual candidates, recommendations by the Examiners and comparison with previous years’ scores (UCLES 1995:14). Grades of A, B, and C are considered passing grades, and a candidate’s overall score is based upon the aggregate of the test papers. Once passed, the CPE award does not expire. AnalysisThe CPE can be analyzed on two levels. First, taking the test as a whole, we must consider Wesche’s Framework, which asks us to consider four things in regard to a test: Stimulus, Task, Response, and Scoring Criteria. Figure 1. Wesche’s Structural Analysis Framework (1983)StimulusTaskResponseScoring CriteriaCPE Test ComponentReading & Use of EnglishThree short Use of English textsThree longer reading textsRead texts about interesting things in the real world.Read general, academic, or work-related texts.Cloze or C-testForm a new wordRe-writing sentencesMultiple Choice Gapped ParagraphsMultiple MatchingTime: 1.5 hours40% of total scoreWritingTwo short textsSummarize main idea from textsChoose task from possible optionsWrite an article, report or reviewTime: 1.5 hours20% of total scoreScoring is based upon four criteria: Content – how well the candidate has done what was asked of him/herCommunicative Achievement – is the writing appropriate for the task in terms of registerOrganization – do the ideas flow logically and orderlyLanguage – is the vocabulary and grammar appropriate to this taskListeningFour parts. Short extractsLong discussionListen to work-related, university, or on-the-street discourseMultiple choice Matching TestsInterpret LanguageTime: 40 minutes20% of total score?As seen in Figure 1, each skill is based upon real-world tasks that are engaging and relevant to the test-taker. According to UCLES 2013, the best preparation strategy for success on the CPE is to use English in a variety of ways, every day. ?????????? ?Another way of looking at the test as a whole is to view it using Swain’s framework (1984). Swain argued that test development should follow four principles: Start from somewhere, Concentrate on content, Bias for best, and Work for Washback. Bailey (1998, p. 154-5) provides context for these frames. Of particular interest and value is the concept of washback. Bailey tells us about the meaning of washback, which can be either positive or negative, depending on the effect the test has on teaching methods. UCLES addresses washback explicitly, acknowledging that test design should align with teaching goals, for positive test washback, The less preferred outcome, that of negative washback, is seen when teaching is adjusted in order to ‘teach to the test’. In language learning, this is especially undesirable if we are seeking to encourage language usage in real-life circumstances. This is again evidenced by the extensive research on washback reported on the UCLES 2013 website under the heading of Research Notes. Figure 2. Swain’s Theoretical Analysis FrameworkStart from Somewhere Concentrate on ContentBias for BestWork for WashbackCPE Test ComponentReading & Use of EnglishPreparing for the reading portion of the test can enhance learners’ real-life reading skills. Read texts about interesting things in the real world.Read general, academic, or work-related texts.CPE designers and researchers are engaged in the ongoing process of production and pretesting of question papers. Each test must meet the internationally recognized ISO 9001:2008 standard for quality, supporting five essential principles: Validity, Reliability, Impact, Practicality and Quality. (UCLES 2013).Firmly connected to authenticity, the tasks in each section of the CPE are extracted from real-life sources which encourage positive learning experiences and a positive impact on teaching as well. (UCLES 2013).WritingCPE seeks to create tasks that the learner would use in normal life, at work, or at school. Summarize ideas from real-world materialsWrite essays on current, relevant topics ListeningLearners enhance this by engaging in everyday conversation, listening to the radio, watching tv Listen to work-related, university, or on-the-street discourse, radio broadcasts. SpeakingCPE seeks to evaluate co-constructed meaning through the use of interesting materials.Learners give information about themselves. Express opinions, exchange ideas, making suggestion and speculations, evaluating, negotiating outcomes. See aboveSee aboveIn a high-stakes test such as the CPE and its sister tests, Cambridge re-iterates UCLES’ (2013) firm belief in remaining current by using relevant, real-life situations and materials at every test level and for every skill. O’Sullivan (2011) alludes to what he terms ‘the Atlantic divide’ between testing philosophies in Great Britain and the United States. Since relevance has explicitly remained an ongoing guiding principle in the UCLES products over time, the shift to more authentic testing materials by testing companies in the United States is viewed by some as catching up to what the Cambridge Tests have done all along. The final point of analysis of the CPE is based upon the unique use of paired speaking tests. Critics of this strategy point to issues that need to be considered. Macqueen & Harding (2009 p. 470) argue that by introducing the second candidate into the testing session, the door is opened to a wide range of factors which ‘may pose threats’ to the test’s validity. Among these were listed age, social class and personality, as well as the potential effect if the co-candidates happen to be acquaintances. Macqueen & Harding point us to the test’s own website (UCLES 2013) for further information on this issue. To that end, we find several revealing studies among Cambridge’s Research Notes. One is an extensive analysis undertaken by O’Sullivan et. al., (2002), who conclude that in regard to potential skewing of scores among two interlocutors in the Speaking portion of the CPE, the role of the examiner is to ensure that each person is judged on their own merit (O’Sullivan et. al., 2002). UCLES is explicit in providing information about examiner training, allowing public access to the examiner’s checklist which these authors developed to address potential scoring inconsistencies. All of the Cambridge research points to the same conclusion on the point of testing speaking in pairs: conversation in life is always linguistically co-constructed. None of the research demonstrated significant difference between groups of people who tested individually versus those tested in pairs (Nakatsuhara 2006, p. 20) using 24 undergraduate participants at Essex University, England, to prove or disprove the hypothesis that there is in fact a difference between the two groups. They constructed careful procedures and compared results using ANOVA. The data did not reveal statistically significant differences among the groups tested. In addition, as Macqueen & Harding (2009, p. 471) remind us, speaking in conversation requires de facto listening skills in the co-construction of the responses elicited during this part of the examination. To prepare themselves for success in this part of the CPE, candidates must spend time on both speaking and listening skills, which we may recall is the kind of positive washback Swain (1984) would have preferred. Further StudyIn order to grasp the depth and breadth of planning that has gone into the CPE over the past 100 years, one would want to study further the volumes of Research Notes available on the UCLES website. From the earliest version (1913) of the Certificate of Proficiency in English test until the present time, there is strong evidence of a continual commitment to creating an instrument which measures the linguistic competence of the candidate against criterion which are internationally accepted and codified in the Common European Framework of Reference (CEFR 2011). A comparison of the evolution of testing in the United States, alluded to by O’Sullivan (2001) in his lecture on the past and future of testing, would also prove useful, in order to gain a greater understanding of the implications of these various tests in terms of work and educational opportunities that may be made available, or not, as a result of a score or achievement level from these tests. Additionally, it might be of value to conduct research to determine if there is validity to O’Sullivan’s (2011) suggestion that the Atlantic Divide has significance in the matter of washback upon language teaching and learning. If we are to accept that the Cambridge tests are carefully constructed to reveal language in use, then it would be hoped that norm-referenced tests used to make educational and employment decisions would yield similar results. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download