THE RECURRING ISSUE OF AVIATION ENGLISH TEST VALIDITY: ECHOES FROM TEST-TAKERS AND ASSESSORS OF THE ENGLISH FOR AVIATION LANGUAGE TESTING SYSTEM IN ALGERIA
Vol.9, Issue 2, 2023, pp. 239-269 Full text
DOI: https://doi.org/10.33919/esnbu.23.2.6
Web of Science: 001131896300006
Authors:
1 Tarek Assassi https://orcid.org/0000-0002-3514-1941
2 Tarek Ghodbane https://orcid.org/0000-0002-4517-7438
Affiliation:
1 Mohamed Khider University, Biskra, Algeria 05fr5y859
2 University of Ghardaia, Ghardaia, Algeria 02ck5yd04
Contributor roles
Conceptualization: T.A. (lead);
Data curation: T.A, T.G. (equal);
Investigation: T.A. (lead), T.B. (supporting);
Methodology: T.A. (lead), T.G. (supporting);
Writing – original draft: T.A. (lead);
Writing – review and editing: T.A. (lead), T.G. (supporting)
Abstract
The English for Aviation Language Testing System (EALTS) is one of the international tests for pilots wishing to operate in international airspace. This test presents a wide range of difficulties for Algerian pilots, but little research has been conducted to estimate its validity. This paper seeks to answer two questions: (1) What are test-takers perspectives on the EALTS test construct? and (2) What is the assessors' perception of the testing procedure and test validity? A descriptive study was conducted using qualitative data from a semi-structured questionnaire for ten pilots and a semi-structured interview for three certified assessors and four university researchers. The authors used 'first-hand' data from the targeted sample to cross-check results' validity through triangulation. The results show specific difficulties from an affective dimension, such as stress and anxiety caused by inefficient preparation and unfamiliarity with test tasks. A different interpretation of ICAO descriptions using the rating scale is another issue noted by assessors. Additionally, technical issues with the computer-based listening test and non-compliant features of the test contents with features of the target situation language use are among the main issues noted by both test-takers and assessors.
Keywords: ESP, Assessment, Aviation English, test validity, test design, aviation phraseology, language proficiency
Article history:
Submitted: 06 September 2023
Reviewed: 18 September 2023
Accepted: 22 September 2023
Published: 20 December 2023
Citation (APA):
Assassi, T, & Ghodbane, T. (2023). The Recurring Issue of Aviation English Test Validity: Echoes from Test-Takers and Assessors of the English for Aviation Language Testing System in Algeria. English Studies at NBU, 9(2), 239-269. https://doi.org/10.33919/esnbu.23.2.5
Copyright © 2023 Tarek Assassi and Tarek Ghodbane
This open access article is published and distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0), which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and source are credited. If you want to use the work commercially, you must first get the authors' permission.
References
Assassi, T. (2017). The Effect of Formulaic Language on ESP Learners' Conversation Competence: The Case of Aviation Professionals in Algeria. (Unpublished Doctoral Dissertation). Tlemcen University, Algeria.
Assassi, T. (2020). Algerian Aeronautics Professionals' Needs Taxonomy: A Target Situation Needs Analysis. International Journal of English Linguistics, 10, 90-103. https://doi.org/10.5539/ijel.v10n6p90
Assassi, T., & Benyelles, R. (2016). Formulaic Language for Improving Communicative Competence. Arab World English Journal, 7, 163-176. https://doi.org/10.24093/awej/vol7no2.11
Borowska, A. P. (2018). A Multilingual Speaker in Global Aviation Communication. Applied Linguistics Papers, 25(2), 11-19. https://doi.org/10.32612/uw.25449354.2018.2.pp.11-19
Brindley, G. (1998). Assessing listening abilities. Annual Review of Applied Linguistics, 18(1), 171–191. https://doi.org/10.1017/S0267190500003536
Brown, A. (1993). The role of test-taker feedback in the test development process: Test-takers' reactions to a tape-mediated test of proficiency in spoken Japanese. Language Testing, 10(3), 277-301. https://doi.org/10.1177/026553229301000305
Civil Aviation Authority (CAA International). (2012). English for Aviation Examinations. Handbook 2012. https://sciarium.com/file/283262
Civil Aviation Authority. (2016). CAP 413: Radiotelephony Manual. Intelligence, Strategy and Policy, Safety and Airspace Regulation Group. Civil Aviation Authority.
Cookson, S. (2009). Zagreb and Tenerife: Airline accidents involving linguistic factors. Australian Review of Applied Linguistics, 32(3), 22.1-22.14. https://doi.org/10.2104/aral0922
Cox. A., & Karimi. M. (2022). Creating a rubric for placement tests for Aviation English programs. In P. Tosqui-Lucks & J.C. Santana (Eds.), Aviation English a Global Perspective: Analysis, teaching, assessment (pp. 182-199). Bookerfield. https://doi.org/10.53268/BKF22080510
Cushing, S. (1994). Fatal words: Communication clashes and aircraft crashes. University of Chicago Press.
Davies, A., Brown, A., Elder, C., Hill, K., Lumley, T., & McNamara, T. (1999). Dictionary of language testing. Cambridge University Press.
Denscombe, M. (2014). The Good Research Guide: for small-scale social research projects. McGraw-Hill Education.
Dornyei, Z. (2003). Questionnaires in second language research: Construction, Administration, and Processing. Lawrence Erlbaum Associates.
Douglas, D. (2000). Assessing language for specific purposes. Cambridge University Press.
Douglas, D. (2005). Testing languages for specific purposes. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning (pp. 857-868). Lawrence Erlbaum.
Douglas, D., & Myers, R. (2000). Assessing the communication skills of veterinary students: Whose criteria? In A. J. Kunnan (Ed.), Fairness and validation in language assessment: Selected papers from the 19th Language Testing Research Colloquium, Orlando, Florida. Cambridge University Press.
Duff, P. (2008). Case Study Research in Applied Linguistics. Taylor & Francis.
Elder, C. (1993). How do subject specialists construe classroom language proficiency? Language Testing, 10(3), 235-254. https://doi.org/10.1177/026553229301000303
Federal Aviation Administration's Aeronautical Center (FAA). (2021, December 31). U.S. Civil Airmen statistics. https://www.faa.gov/data_research/aviation_data_statistics/civil_airmen_statistics
Field, J. (2019). Rethinking the Second Language Listening Test - From Theory to Practice. Equinox.
Field, J. (20022, July 6). Idle chatter: What really goes on in tests of interactive communication? CRELLA Symposium, 2020. Video. YouTube. https://www.youtube.com/watch?v=ONM7xcJTPD0
Fowler, R., Matthews, E., Lynch, J., & Roberts, J., (2021). Aviation English Assessment and Training. Collegiate Aviation Review International, 39(2), 26-42. https://doi.org/10.22488/okstate.22.100231
Hymes, D. (1972). On Communicative Competence. In J.B. Pride and J. Holmes (Eds.), Sociolinguistics. Selected Readings (pp. 269-293). Penguin.
International Civil Aviation Organization (ICAO). (2004). Manual on the Implementation of ICAO Language Proficiency Requirements (Doc. 9835). ICAO.
International Civil Aviation Organization (ICAO). (2006). ICAO Manual of Radiotelephony. Doc 9432-AN/925. Annex 10, Volume II, Chapter 5. ICAO.
International Civil Aviation Organization (ICAO). (2010a). GBAS CAT II/III development baseline SARPs. ICAO.
International Civil Aviation Organization (ICAO). (2010b). Manual on the Implementation of ICAO Language Proficiency Requirements (2nd ed.) (Doc. 9835). ICAO.
International Civil Aviation Organization (ICAO). (2013). Language Proficiency Requirements: Critical to aviation safety. ICAO Journal, 5, 64-64.
Jacoby, S., & McNamara, T. (1999). Locating competence. English for Specific Purposes, 18(3), 213-241. https://doi.org/10.1016/S0889-4906(97)00053-7
Katsarska, V. (2021). Formulaic Language in Aeronautical English-Friend or Foe. Journal of Teaching English for Specific and Academic Purposes, 9(2), 239-245. https://doi.org/10.22190/JTESAP2102239K
Lumley, T., & Brown, A. (1996). Specific-purpose language performance tests: Task and interaction. Australian Review of Applied Linguistics, Series S, 13, 105-136. https://doi.org/10.1075/aralss.13.06lum
Lynch, T., Mendelshon, D. (2010). Listening. In N. Schmitt (Ed.), An introduction to applied linguistics (2nd ed) (pp. 180-196). Hodder Arnold.
Mekkaoui, G. (2013). English for Aviation in the Algerian context: The case of Pilots and Air Traffic Controllers in Zenata-Messalli El Hadj Airport. (Unpublished master's thesis). Tlemcen University, Algeria.
Mekkaoui, G., & Mouhadjer, N. (2019). Addressing air traffic controllers' English language proficiency needs: Case of Zenata Airport. Global Journal of Foreign Language Teaching, 9(3), 167-183. https://doi.org/10.18844/gjflt.v9i3.4245
Moere, A.V., Suzuki, M., Downey, R., & Cheng, J. (2009). Implementing ICAO Language Proficiency Requirements in the Versant Aviation English Test. Australian Review of Applied Linguistics, 32, 27.1-27.17. https://doi.org/10.1075/aral.32.3.07van
Monteiro, A.L.T. (2022). The Assessment of English in Aeronautical Radiotelephony Communications: A mixed methods study. In P. Tosqui-Lucks & J.C. Santana (Eds.), Aviation English a Global Perspective: Analysis, teaching, assessment (pp. 224-245). https://doi.org/10.53268/BKF22080512
Netherlands Aviation Safety Board (1978). Final Report and Comments of the Netherlands Aviation Safety Board of the Investigation into the Accident with the Collision of KLM Flight 4805 and Pan American Flight 1736. https://www.faasafety.gov/files/gslac/courses/content/232/1081/finaldutchreport.pdf
Taylor, L., Geranpayeh, A. (2011). Assessing listening for academic purposes: Defining and operationalising the test construct. Journal of English for Academic Purposes, 10(2), 89-101. https://doi.org/10.1016/j.jeap.2011.03.002
Wagner, E. (2014). Assessing listening. In A. Kunnan (Ed.), Companion to language assessment Volume 1 (pp. 47-63). Wiley-Blackwell. https://doi.org/10.1002/9781118411360.wbcla094
Wu, Y. (1998). What do tests of listening comprehension test? - A retrospection study of EFL test-takers performing a multiple-choice task. Language Testing, 15, 21-44. https://doi.org/10.1177/026553229801500102
Zhou, Y., Yoshitomi, A. (2019). Test-taker perception of and test performance on computer-delivered speaking tests: the mediational role of test-taking motivation. Lang Test Asia 9(10), 1-19. https://doi.org/10.1186/s40468-019-0086-7
Review:
1. Reviewer's name: Undisclosed
Review Content: Undisclosed
Review Verified on Publons
2. Reviewer's name: Undisclosed
Review Content: Undisclosed
Review Verified on Publons
Handling Editor: Boris Naimushin, PhD, New Bularian University
Verified Editor Record on Publons