A Developmental Writing Scale - Educational Testing Service
Research Report
A Developmental Writing Scale
Yigal Attali Don Powers
April 2008
ETS RR-08-19
Listening. Learning. Leading.?
A Developmental Writing Scale Yigal Attali and Don Powers ETS, Princeton, NJ
April 2008
As part of its educational and social mission and in fulfilling the organization's nonprofit charter and bylaws, ETS has and continues to learn from and also to lead research that furthers educational and measurement research to advance quality and equity in education and assessment for all users of the organization's products and services.
ETS Research Reports provide preliminary and limited dissemination of ETS research prior to publication. To obtain a PDF or a print copy of a report, please visit:
Copyright ? 2008 by Educational Testing Service. All rights reserved. E-RATER, ETS, the ETS logo, LISTENING. LEARNING.
LEADING., and TOEFL are registered trademarks of Educational Testing Service (ETS). CRITERION and TEST OF ENGLISH AS
A FOREIGN LANGUAGE are trademarks of ETS.
Abstract This report describes the development of grade norms for timed-writing performance in two modes of writing: persuasive and descriptive. These norms are based on objective and automatically computed measures of writing quality in grammar, usage, mechanics, style, vocabulary, organization, and development. These measures are also used in the automated essay scoring system e-rater? V.2. Norms were developed through a large-scale data collection effort that involved a national sample of 170 schools, more than 500 classes from 4th, 6th, 8th, 10th, and 12th grades and more than 12,000 students. Personal and school background information was also collected. These students wrote (in 30-minute sessions) up to 4 essays (2 in each mode of writing) on topics selected from a pool of 20 topics.
The data allowed us to explore a range of questions about the development and nature of writing proficiency. Specifically, this paper provides a description of the trajectory of development in writing performance from 4th grade to 12th grade. The validity of a single developmental writing scale is examined through a human scoring experiment and a longitudinal study. The validity of the single scale is further explored through a factor analysis (exploratory and confirmatory) of the internal structure of writing performance and changes in this structure from 4th grade to 12th grade. The paper also explores important factors affecting performance, including prompt difficulty, writing mode, and student background (gender, ethnicity, and English language background). Key words: Writing proficiency, Automated essay scoring, e-rater
i
Table of Contents Introduction..................................................................................................................................... 1
Essay Writing Assessments ..................................................................................................... 1 Automated Essay Scoring........................................................................................................ 3 The Present Study .................................................................................................................... 5 Method ............................................................................................................................................ 9 School Sampling Plan.............................................................................................................. 9 Topic Selection and Allocation to Classes ............................................................................ 10 School Invitation.................................................................................................................... 11 Procedures.............................................................................................................................. 12 Background Questionnaire .................................................................................................... 13 Results on the Participation of Classes and Students.................................................................... 13 Creating Essay Writing Scores ..................................................................................................... 18 The Writing Scale and Grade Norms............................................................................................ 20 Cross-Classified Random Modeling for Student and Topic Effects............................................. 22 Score Reliability............................................................................................................................ 30 Human Scoring Experiment.......................................................................................................... 32 Longitudinal Study........................................................................................................................ 36 Factor Analysis ............................................................................................................................. 39 Discussion ..................................................................................................................................... 48 Summary of Results............................................................................................................... 48 Possible Applications............................................................................................................. 50 Limitations............................................................................................................................. 51 References..................................................................................................................................... 52 Appendix....................................................................................................................................... 56
ii
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- big 1 dr8 11 utah education network
- writing prompts student rubrics and sample
- a developmental writing scale educational testing service
- milestone messenger cobb county school district
- georgia grade 5 writing assessment 2013
- examples of curriculum based measurement probes
- math journal prompts about attitudes and dispositions
- grade 4 first prompt departments
- 4th grade prompt 4 departments
- end of course writing prompts
Related searches
- certificate in educational testing dubai
- what is a professional writing sample
- what is a developmental disability
- developmental stages of writing examples
- developmental stages of writing preschool
- ged testing service transcript
- foreign service pay scale 2020
- water testing service near me
- susan ferguson child family school and community a developmental perspective
- a developmental psychologist might use
- what does a developmental psychologist do
- developmental reading and writing syllabus