New and not so new methods for assessing oral communication
Main Article Content
Abstract
Author/s
Gary J. Ockey
Zhi Li
Iowa State University, USA
ABSTRACT
The assessment of oral communication has continued to evolve over the past few decades. The construct being assessed has broadened to include interactional competence, and technology has played a role in the types of tasks that are currently popular. In this paper, we discuss the factors that affect the process of oral communication assessment, current conceptualizations of the construct to be assessed, and five tasks that are used to assess this construct. These tasks include oral proficiency interviews, paired/group oral discussion tasks, simulated tasks, integrated oral communication tasks, and elicited imitation tasks. We evaluate these tasks based on current conceptualizations of the construct of oral communication, and conclude that they do not assess a broad construct of oral communication equally. Based on our evaluation, we advise test developers to consider the aspects of oral communication that they aim to include or exclude in their assessment when they select one of these task types.
Downloads
Article Details
References
Abeywickrama, P. 2013. “Why not non-native varieties of English as listening comprehension test input?” RELC Journal 44(1), 59-74.
ACTFL. 2012. ACTFL Proficiency Guidelines 2012 – Speaking. Retrieved from http://www.actfl.org/publications/guidelines-and-manuals/actfl-proficiency- guidelines-2012/english/speaking
Alastuey, M. C. B. 2011. “Perceived benefits and drawbacks of synchronous voice- based computer-mediated communication in the foreign language classroom”. Computer Assisted Language Learning, 24(5), 419–432.
Bachman, L. F. 2001. Speaking as a realization of communicative competence. Paper presented at the meeting of the American Association of Applied Linguistics – International Language Testing Association (AAAL-ILTA) Symposium. St. Louis, Missouri.
Barkaoui, K., Brooks, L., Swain, M. and Lapkin, S. 2012. “Test-takers’ strategic behaviors in independent and integrated speaking tasks”. Applied Linguistics, 34(3), 304–324.
Batty, A. 2006. “An analysis of the relationships between vocabulary learning strategies, A Word Associates Test, and the KEPT”. Studies in Linguistics and Language Education: Research Institute of Language Studies and Language Education, 17, 1-22.
Bernstein, J. C. 2013. “Computer scoring of spoken responses”. In C. A. Chapelle (Ed.), The Encyclopedia of Applied Linguistics. Wiley-Blackwell.
Bonk, W. and Ockey, G. J. 2003. “A many-facet Rasch analysis of the second language group oral discussion task”. Language Testing, 20(1), 89–110.
Bosker, H. R., Pinget, A.-F., Quene, H., Sanders, T. and de Jong, N. H. 2012. “What makes speech sound fluent? The contributions of pauses, speed and repairs”. Language Testing, 30(2), 159–175.
Brooks, L. 2009. “Interacting in pairs in a test of oral proficiency: Co-constructing a better performance”. Language Testing, 26(3), 341–366.
Brooks, L. and Swain, M. 2015. “Students’ voices: The challenge of measuring speaking for academic contexts”. In B. Spolsky, O. Inbar, and M. Tannenbaum (Eds.), Challenges for language education and policy: Making space for people (pp. 65–80). New York: Routledge.
Brown, A. 2003. “Interviewer variation and the co-construction of speaking proficiency”. Language Testing, 20(1), 1–25.
Carey, M. D., Mannel, R. H. and Dunn, P. K. 2011. “Does a rater’s familiarity with a candidate's pronunciation affect the rating in oral proficiency interviews?” Language Testing, 28(2), 201–219.
Chun, C. W. 2006. “Commentary: An analysis of a language test for employment: The authenticity of the PhonePass Test”. Language Assessment Quarterly, 3(3), 295– 306.
Elder, C. and L. Harding. 2008. “Language testing and English and an international language: Constraints and contributions” in Sharifian, F. and M. Clyne (eds.): Australian Review of Applied Linguistics (special forum issue) 31(3), 34.1– 34.11.
Fulcher, G. 2003. Testing second language speaking. London and New York: Longman.
Ginther, A., Dimova, S. and Yang, R. 2010. “Conceptual and empirical relationships between temporal measures of fluency and oral English proficiency with implications for automated scoring”. Language Testing, 27(3), 379–399.
Harris, D. P. 1969. Testing English as a Second Language. New York: McGraw-Hill Book Company.
He, L. and Dai, Y. 2006. “A corpus-based investigation into the validity of the CET-SET group discussion”. Language Testing, 23(3), 370–401.
Hunston, S., Francis, G. and Manning, E. 1997. “Grammar and vocabulary: Showing the connections”. ELT Journal, 51(3), 208-216.
Jamieson, J., Eignor, D. R., Grabe, W. and Kunnan, A. J. 2008. “Frameworks for a new TOEFL”. In C. A. Chapelle, J. Jamieson, and M. K. Enright (Eds.), Building a validity argument for the Test of English as a Foreign Language (pp. 55–95). New York: Routledge.
Johnson, M. and Tyler, A. 1998. “Re-analyzing the OPI: How much does it look like natural conversation?”. In R. F. Young and A. W. He (Eds.), Talking and testing: Discourse approaches to the assessment of oral proficiency (pp. 27–52). Amsterdam and New York: Benjamins.
Kenyon, D. M. 1992. Development and use of rating scales in language testing. Paperpresented at the annual meeting of the Language Testing Research Colloquium. Vancouver, Canada.
Leaper, D. A. and Riazi, M. 2014. “The influence of prompt on group oral tests”. Language Testing, 31(2), 177–204.
Lin, H. 2014. “Computer-mediated communication (CMC) in L2 oral proficiency development: A meta-analysis”. ReCALL, 1–27.
Liskin-Gasparro, J. E. 2003. “The ACTFL Proficiency Guidelines and the Oral Proficiency Interview: A brief history and analysis of their survival”. Foreign Language Annals, 36(4), 483–490.
McNamara, T. 1996. Measuring second language performance. London: Longman. Nakatsuhara, F. 2011. “Effects of test taker characteristics and the number of participants in group oral tests”. Language Testing, 28(4), 483–508.
Nation, I.S.P. 1990. Teaching and learning vocabulary. Boston, MA: Heinle & Heinle.
O’Sullivan, B. 2002. “Learner acquaintanceship and oral proficiency test pair-task performance”. Language Testing 19(3), 277-295.
O’Sullivan, B. 2013. “Assessing speaking”. In A. J. Kunnan (Ed.), The Companion to Language Assessment (pp. 156–171). Wiley.
Ockey, G. J. 2009. The effects of group members’ personalities on a test taker's L2 group oral discussion test scores. Language Testing, 26(2), 161–186.
Ockey, G. J., & French, R. 2014. From one to multiple accents on a test of L2 listening comprehension. Applied Linguistics. doi: 10.1093/applin/amu060.
Ockey, G. J., Koyama, D., Setoguchi, E., and Sun, A. 2015. The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students. Language Testing, 32(1), 39–62.
Ockey, G. J., Koyama, D., & Setoguchi, E. 2013. Stakeholder input and test design: A case study on changing the interlocutor familiarity facet of the group oral discussion test. Language Assessment Quarterly, 10(1), 1–17.
Papajohn, D. 1999. The effect of topic variation in performance testing: The case of the chemistry TEACH test for international teaching assistants. Language Testing, 16(1), 52–81.
Park, M. 2015. Task selection and simulation for authentic aviation English assessment. Paper to be presented at the invited colloquium “Task-based language assessment in practice: Justification and realizations” at the 6th International Conference on Task-Based Language Teaching. Leuven, Belgium. September 16- 18, 2015
Plakans, B. S. and Abraham, R. G. 1990. The testing and evaluation of international teaching assistants. In D. Douglas (Ed.), English language testing in U.S. colleges and universities (pp. 68–81). Washington, DC: NAFSA.
Sato, M. 2014. Exploring the construct of interactional oral fluency: Second Language Acquisition and Language Testing approaches. System, 45, 79–91.
Skehan, P. (1998). Processing perspectives to second language development, instruction, performance, and assessment. Thames Valley Working papers in Applied Linguistics, 4, 70-88.
Smith, L. and Bisazza, J. 1982. The comprehensibility of three varieties of English for college students in seven countries. Language Learning, 32(2), 259-269. Swender, E. 2003. Oral Proficiency Testing in the Real World: Answers to Frequently Asked Questions. Foreign Language Annals, 36(4), 520–526.
Sydorenko, T., Maynard, C. and Guntly, E. 2014. Rater behaviour when judging language learners’ pragmatic appropriateness in extended discourse. TESL Canada Journal, 32(1), 19–41.
Turner, C. 2009. Examining washback in second language education contexts: A high stakes provincial exam and the teacher factor in classroom practice in Quebec secondary schools. International Journal of Pedagogies and Learning, 5, 103-123.
Van Lier, L. 1989. Reeling, writhing, drawling, stretching, and fainting in coils: oral proficiency interviews as conversation. TESOL Quarterly, 23, 489–508.
Winke, P. 2013. The effectiveness of interactive group orals for placement testing. In K. McDonough and A. Mackey (Eds.), Second Language Interaction in Diverse Educational Contexts (pp. 247–268). Amsterdam/Philadelphia: John Benjamin Publishing Co.
Wu, S.-L. and Ortega, L. 2013. Measuring global oral proficiency in SLA research: A new elicited imitation test of L2 Chinese. Foreign Language Annals, 46(4), 680– 704.
Xi, X., Higgins, D., Zechner, K. and Williamson, D. 2012. A comparison of two scoring methods for an automated speech scoring system. Language Testing, 29(3), 371–394.
Ye, F. 2014. Validity, reliability, and concordance of the Duolingo English Test. Pittsburgh, PA. Retrieved from https://s3.amazonaws.com/duolingo-certifications- data/CorrelationStudy.pdf
Zheng, Y. and Cheng, L. 2008. Test review: College English Test (CET) in China. Language Testing, 25, 408–417.