An Application of Generalizability Theory on Writing Assessment: Effects of Marking Components Weighting

Literaturnachweis - Detailanzeige

Trefferliste

Autor/in	Lam, Ling Chi Tenny
Titel	An Application of Generalizability Theory on Writing Assessment: Effects of Marking Components Weighting
Quelle	(2010), (126 Seiten) PDF als Volltext Verfügbarkeit Ed.D. Dissertation, The Chinese University of Hong Kong (Hong Kong)
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
ISBN	978-1-2670-0875-6
Schlagwörter	Hochschulschrift; Dissertation; Foreign Countries; Grading; Scoring Rubrics; Educational Assessment; Educational Policy; Public Agencies; Elementary School Students; Secondary School Students; Language Tests; Mathematics Tests; Training; Measurement; Writing Tests; Writing Evaluation; Standardized Tests; Raw Scores; Generalizability Theory; Chinese; English (Second Language); Weighted Scores; Reliability; Psychometrics; Hong Kong + Suchen Sie Ihr Suchwort? Thesis; Dissertations; Academic thesis; Ausland; Notengebung; Schulnote; Scoring formulas; Auswertungsbogen; Education; assessment; Bewertungssystem; Politics of education; Bildungspolitik; Öffentliche Einrichtung; Sekundarschüler; Language test; Sprachtest; Ausbildung; Messverfahren; Writing test; Schreibtest; Standadised tests; Standardisierter Test; China; Chinesen; English as second language; English; Second Language; Englisch als Zweitsprache; Reliabilität; Psychometry; Psychometrie; Hongkong
Abstract	In writing assessment, there are quite a number of factors influencing the marking stability and the reliability of the assessment such as the attitude towards marking and consistency of markers, the physical environment, the design of the items, and marking rubrics. Even the methods to train markers have effects on the reliability of the assessment. Generalizability Theory was used in this research to analyze the Chinese writing assessment of the Territory-wide System Assessment (TSA) so as to improve the reliability of the assessment. TSA is a standardized test administered centrally by the Hong Kong Examinations and Assessment Authority every year. The target groups are students from Primary 3, Primary 6 and Secondary 3. TSA focuses on assessing students' basic competency on the three core subjects, Chinese Language, English Language and Mathematics. In contrast to the traditional Chinese writing assessment, there was no requirement on the minimum number of words produced by the student. An analytical approach was adopted to assess students' writing tasks. As a result of this measure, students who did well in some particular marking criteria would end up with a good overall performance. This study was a post-mortem analysis of the raw scores from a sample of 6,000 students who participated in TSA 2006. As there were three sub-papers, the sample consisted of 2,000 students from each sub-paper. Brennan's GENOVA program (1983) was used to calculate the reliabilities of the assessments. In the assessment, each student was marked by two raters, assigned at random to the student from a pool of 200 raters. These raters had undergone a series of instructional programs and training prior to the job. Each of the two raters gave seven scores to the script. As there was no minimum number of words as required in the writing assessment, a general belief would be generated that if there was insufficient content (as evidenced by a low score in "content") and poor organization (low score in "organization"), then the student would have written so few words that the chance of making mistakes in "vocabulary" (the 6th score) and in "punctuation" (the 7th score) would be relatively small. In order to rectify the deficiency in marking, this study used three different methods to apply weights on the "vocabulary" score and on the "punctuation" score. For each method, the GENOVA program was used to calculate the reliability of the assessments. After due comparison, it was found that each of the methods used was able to raise the reliabilities of the assessments under investigation, and the most recommended method was to use students' scores in "content and in structure" as weights. On the one hand, the study has examined the present mode of marking of the writing assessment in the TSA. This gives opportunity for improving the item-setting and the script-marking procedures of the assessment with a view to raising its reliability and giving valuable feedbacks to teaching and learning. On the other hand, the favourable results of applying weights to sub-scores will serve to provide a good example on improving marking rubrics in large-scale standardized tests of writing assessment in Chinese Language. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.] (As Provided).
Anmerkungen	ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10