Exploring Saudi Students’ Perceptions of National Exams: A Washback Study
DOI:
https://doi.org/10.18662/rrem/13.1Sup1/393Keywords:
GAT & SAAT, Positive washback, Soft skills, Case study, High-stakes tests,Abstract
Research on washback and impact of tests (either positive or negative) on teaching and learning in the language assessment field has proliferated in approximately the last three decades. This study aims to explore the positive washback of the General Aptitude Test (GAT) and Scholastic Achievement Admission Test (SAAT) based on the perceptions of secondary school students in Saudi Arabia since this topic is underresearched in the literature. While the existing research has provided insight into the validity and reliability of these tests, we have a limited understanding of how these tests can positively impact the performance of students.
In this qualitative study, data were collected from 103 participants using a case study approach through the means of semistructured questionnaires and semistructured interviews. Data were analyzed inductively by employing a computer program called NVivo. The results of the study showed that the participants were positively influenced by the test. This impact was manifested in students’ low pressure test-taking experiences, their successful honing of soft skills needed for future university study instead of reliance merely on indoctrination or the memorization of information and the feasibility of enrollment in universities because the Secondary School Certificate (SSC) is no longer the sole indicator of academic performance. Students’ scores increased due to the method of testing, test preparation and test retaking, which played key roles in promoting score gains and thus accentuating a positive washback.
References
Al-Alwan, I. A. (2009). Association between scores in high school, aptitude and achievement exams and early performance in health science college. Saudi Journal of Kidney Diseases and Transplantation, 20(3), 448–453. Available from: https://www.sjkdt.org/text.asp? 2009/20/3/448/50777
Alanazi, N. K. (2014). The validity of the general aptitude test (GAT) to predict male Saudi students' academic performance in first semester. Unpublished PhD Dissertation, University of Northern Colorado. ProQuest Dissertations & Theses Global. https:// search.proquest.com/docview/1629828803?accountid=27804
Al-Hamly, M., & Coombe, C. (2005). To change or not to change: Investigating the value of MCQ answer changing for Gulf Arab students. Language Testing, 22(4), 509–531. https://doi.org/ 10.1191/0265532205lt317oa
Al-Owidha, A. A. (2013). The Saudi standardized achievement admission test (SAAT); A study of construct-related validity evidence (TR038). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Al-Qataee, A. (2012a, March 21). Test length and precision: How precise the GAT will be if the test is shortened? (TR008). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education. https://etec.gov.sa/ar/Researchers/Research-Studies/Studies/1 Full%20verses%20short%20form%20tests11%20final%20008.pdf
Al-Qataee, A. (2012b, August 3). Differential validity of the GAT admission test in Saudi Arabia (TR010). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Alazizi, E. (2016). The impact of a training program on the performance of a sample of students in the general abilities test in the province of Mahd Al-Thahab. The Educational International Specialized, Jordan, 5(7), 1–17. https://platform.almanhal.com/Files/2/100384
Aldahesh, A., Alnamer, M. & Alher, M. (2015). The impact of the students’ GPA in high school and the score in the GAT and SAAT on the GPA at university. Journal of Scientific Research in Education, Egypt, 16(5), 477–494. https://jsre.journals.ekb.eg/ issue_2656_2709_.html
Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14(2), 115–129. https://doi.org/10.1093/applin/ 14.2.115
Alghamdi, R. (2017). The reasons of the gap between results of female students tests in high school and results of general aptitude and achievement tests from the perspective of students, teachers, managers, supervisors. Journal of Culture and Development, Egypt, 16(101), 73–117. http://mcd.arkanorg.com/?s=%D8%A7% D9%84%D9%81%D8%AC%D9%88%D8%A9
Alghamdi, S. (2012). The GAT and its relationship with future anxiety on a sample of secondary school students in Saudi Arabia. Journal of College of Education, Egypt, 23(90), 120–152. https://content. mandumah.com/download?t=006748f28127725121603ef514e5211231080597&f=yIujVbp8RoaWsleq86KV1xryRgU1tI3hulA87odFJms=&s=1
Almehrej, H. (2009). The general aptitude test from the perspective of university students. Journal of Arabic Studies, 8(2), 205–262. http://search.mandumah.com/Record/82517
Alnahdi, G. H. (2015). Aptitude tests and successful college students: the predictive validity of the general aptitude test (GAT) in Saudi Arabia. International Education Studies, 8(4), 1–6. DOI:10.5539/ies.v8n4p1
Alnowibet, K., & Ahmad, S. (2015). Saudi school assessment system for predicting admissions to science colleges. The Online Journal of Quality in Higher Education, 2(1), 37–41. https://www.tojqih.net/ journals/tojqih/articles/v02i01/v02i01-05.pdf
Al Ruwaili, S. K. (2013). The Low Performance of High School Students in the General Aptitude Test in Al Jouf Region in Saudi Arabia and its Treatment: An Analytical Study. Unpublished MA Thesis, Yarmouk University. https://dirasat.ju.edu.jo/ES/Article/FullText/11746?volume=45&issue=4
Alshehri, M. (2007). The Predictive Validity of General Aptitude Test (GAT) and Secondary School Test (SST) on Grade Point Average (GPA) at Saudi Universities. Unpublished MA Thesis, Mutah University. https://dirasat.ju.edu.jo/ES/Article/FullText/11746?volume=45&issue=4
Alshehri, M. (2010). The relationship between the first year mathematics at university and the GAT, SAAT, and mathematics of secondary school. Education Journal for Educational, Psychological and Social Research, 144(3), 65–80. https://content-mandumah-com.sdl.idm.oclc.org/download?t=9aad857efb10f0cf2df92633c770075f101746bb&f=nPAYP9haJ/jK3vOZPhUuds7AtXO0nG8NXoOVg3eKllw=&s=1
Alshemeri, M. M. (2015). Equating Scores for Different Forms of Ability Test Among Secondary School Students in Saudi Arabia. Unpublished MA Thesis, Yarmouk University. https://search-mandumah-com.sdl.idm.oclc.org/Record/741294
Alshumrani, S. A. (2007). Predictive validity of the general aptitude test and high school percentage for Saudi undergraduate students. Unpublished PhD Dissertation, University of Kansas. ProQuest Dissertations & Theses Global. (304857763). Retrieved from https://search-proquest-com.sdl.idm.oclc.org/pqdtglobal/docview/3048577 63 / CE6F73F470B041CBPQ/8?accountid=142908
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Balkyoor, S. M. (2008). The Efficacy of CTT and IRT Equating Methods to the Forms of the General Ability Test (GAT). Unpublished PhD Dissertation, Umm Al-Qura University.
Bobsheet, A., & Al-Ghamdi, A. (2015). Perceptions of students at the university of Dammam towards general aptitude tests. The Saudi Journal for Higher Education, Saudi Arabia, 13, 137–168. http://chers.moe.gov.sa/ar/Publications/Publications1/HEJ-0013-Ar.pdf
Broadfoot, P. (1996). Education, assessment and society: A Sociological Analysis. Buckingham: Open University Press.
Cheng, L., Sun, Y., & Ma, J. (2015). Review of washback research literature within Kane’s argument-based validation framework. Language Teaching, 48(4), 436–470. DOI: https://doi.org/10. 1017/S0261444815000233
Cheng, L., Watanabe, Y., & Curtis, A. (2004). Washback in language testing: Research contexts and methods. Mahwah, NJ: Lawrence Erlbaum Associates.
Davies, A., Brown, A., Edler, C., Hill, K., Lumley, T., & McNamara, T. (1999). Dictionary of language testing. In Studies in language testing 7. Cambridge: Cambridge University Press.
Deygers, B., Van den Branden, K., & Van Gorp, K. (2018). University entrance language tests: A matter of justice. Language Testing, 35(4), 449–476. https://doi.org/10.1177/0265532217706196
Dimitrov, D. M. (2013a, January 22). Teacher tests-educational skills: Equating of test forms. Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Dimitrov, D. M. (2013b, March 12). General Aptitude Test-verbal: Testing for dimensionality and validation of factorial structure (TR011). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Dimitrov, D. M. (2013c, April 29). General Aptitude Test -quantitative: Testing for dimensionality and validation of factorial structure (TR012). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Dimitrov, D. M. (2013d, May 16). General Aptitude Test-verbal: IRT and true-score analysis (TR013). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Dimitrov, D. M. (2013e, June 30). General Aptitude Test-quantitative: IRT and true-score analysis (TR014). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Dimitrov, D. M. (2013f, August 19). Latent class analysis of GAT-quantitative data (TR015). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Dimitrov, D. M. (2013g, September 25). Latent class analysis of GAT-verbal data (TR016). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Dimitrov, D. M. (2013h, December 13). Testing for unidimensionality of GAT data (TR029). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Dimitrov, D. M. (2015). Psychometric features of the Arabic and English versions of the General Aptitude Test (GAT): An initial investigation with large-scale assessment data in Saudi Arabia (TR095). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Dimitrov, D. M., & Shamrani, A. R. (2015). Psychometric features of the General Aptitude Test–verbal part (GAT-V). Measurement and Evaluation in Counseling and Development, 48(2), 79–94. https://doi. org/10.1177/0748175614563317
Dimitrov, D. M., Raykov, T., & Al-Qataee, A. A. (2015). Developing a measure of general academic ability. Educational and Psychological Measurement, 75(3), 475–490. https://doi.org/10.1177/001316441 4543179
Gipps, C. (1994). Beyond testing: Towards a theory of educational assessment. London: Falmer Press.
Green, A. (2007). IELTS washback in context: Preparation for academic writing in higher education. Cambridge: Cambridge University Press.
Hamdan, A., & Al-Hattami, A. (2016). The accuracy of predicting university students’ academic success. Journal of Saudi Educational and Psychological Association, (54), 173–182. https://gesten.ksu.edu. sa/sites/gesten.ksu.edu.sa/files/8_0.pdf
Hamp-Lyons, L. (1997). Washback, impact and validity: Ethical concerns. Language Testing, 14(3), 295–303. https://doi.org /10.1177/026553229701400306
He, X., & Godfroid, A. (2019). Choosing words to teach: A novel method for vocabulary selection and its practical application. TESOL Quarterly, 53(2), 348–371. https://doi.org/10.1002/ tesq.483
Hughes, A. (2003). Testing for language teachers. Cambridge: Cambridge University Press.
Jehangir, K. (2015). Investigating the impact of proctor background characteristics on achievement scores in GAT (TR087). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Jehangir, K. (2016a, April 3). IRT model fit/residual analysis comparisons between 5 test forms of the computer based GAT quantitative test versus 5 corresponding test forms of the paper and pencil based GAT quantitative test (TR151). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Jehangir, K. (2016b, August 14). IRT model fit/residual analysis comparisons between 5 test forms of the computer based GAT verbal test versus 5 corresponding test forms of the paper and pencil based GAT verbal test (TR154). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Khaniya, T. R. (1990). Examinations as Instruments for Educational Change: Investigating the Washback Effects of the Nepalese English Examinations. Unpublished PhD Thesis. Edinburgh, University of Edinburgh. Edinburgh Research Archive. http://hdl.handle.net/1842/9862
Klasik, D., & Strayhorn, T. L. (2018). The complexity of college readiness: Differences by race and college selectivity. Educational Researcher, 47(6), 334–351. https://doi.org/10.3102/0013189X18 778598
Kunnan, A. J. (2004). Test fairness. In M. Milanovic, C. Weir, & S. Bolton (Eds.), Europe language testing in a global context: Selected papers from the ALTE conference in Barcelona (pp. 27–48). Cambridge, UK: Cambridge University Press.
Lamprianou, I. (2013a, October 11). Investigation for exposed items in the context of the General Aptitude Test (GAT) (TR023). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Lamprianou, I. (2013b, December 1). Investigation for potentially exposed items in the context of the verbal section of the General Aptitude Test (GAT) (TR030). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Lamprianou, I. (2014a, September 5). Confirmatory factor analysis as a method for assessing reliability and validity of the General Aptitude Test (GAT) for postgraduate students (TR050). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Lamprianou, I. (2014b, October 4). The dimensionality of the investigation and general prosecution scale: The General Aptitude Test (TR057). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Lamprianou, I. (2014c, November 9). Factorial invariance and latent mean differences of scores on GAT across school types (TR062). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Lamprianou, I. (2014d, December 21). Factorial invariance and latent mean differences of scores on GAT across gender (TR063). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Lamprianou, I. (2015a, November 21). Examining the effectiveness of the distractors in multiple-choice items using the differential distractor functioning (DDF) on SAAT scores across gender (TR091). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Lamprianou, I. (2015b, December 17). Factorial invariance and latent mean differences of scores on SAAT across gender (TR092). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Lee, M., & Aluhiedan, S. (2013). A comparison of students’ performance in Standard Achievement Admission Test (SAAT) across educational administrations, types of school, genders and nationalities (TR025). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241–256. https://doi.org/10.1177/0265532296013 00302
Ormond, B. M. (2019). The impact of standards-based assessment on knowledge for history education in New Zealand. Assessment in Education: Principles, Policy & Practice, 26(2), 143–165. https://doi.org/ 10.1080/0969594X.2018.1432564
Qiyas (2013). Measurement can never be an obstacle to admitting students at universities [Video file]. Resource document. National Center for Assessment. Retrieved from https://www.youtube.com/watch?v=qljmyqMZKH0
Saida, C., & Hattori, T. (2008). Post-hoc IRT equating of previously administered English tests for comparison of test scores. Language Testing, 25(2), 187–210. https://doi.org/ 10.1177/0265532207086781
Saif, S. (2006). Aiming for positive washback: A case study of international teaching assistants. Language Testing, 23(1), 1–34. https://doi.org/10.1191/0265532206lt322oa
Shih, C. M. (2010). The washback of the general English proficiency test on university policies: A Taiwan case study. Language Assessment Quarterly, 7(3), 234–254. https://doi.org/10.1080/ 154343010036 64196
Shohamy, E. (2001). The power of tests: A critical perspective of the uses of language tests. Harlow: Longman.
Siddiek, A. (2011). Standardization of the Saudi secondary school certificate examinations and their anticipated impact on foreign language education. International Journal of Humanities and Social Sciences, 1(3), 57–64. http://www.ijhssnet.com/view.php?u= https://www.ijhssnet.com/journals/Vol._1_No._3;_March_2011/9.pdf
Sideridis, G. D. (2014). Assessing differential achievement scores on the GAT due to examination center characteristics (TR048). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Sideridis, G. D. (2016). Effects of testlet exposure on students’ performance (TR158). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Sideridis, G. D., & Tsaousis, I. (2013a, December 15). DIF analysis for item and test on the NCA tests: The General Ability Test (GAT) art major (TR035). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Sideridis, G. D., & Tsaousis, I. (2013b, December 30). DIF analysis for item and test on the NCA tests: The General Ability Test (GAT) science major (TR036). Riyadh, Saudi Arabia: National Center for Assessment in Higher Education.
Sideridis, G. D., Tsaousis, I., & Al-harbi, K. A. (2015). Multi-Population Invariance With Dichotomous Measures: Combining Multi-Group and MIMIC Methodologies in Evaluating the General Aptitude Test in the Arabic Language . Journal of Psychoeducational Assessment, 33(6), 568–584. https://doi.org/10.11 77/07342829 14567871
Sinclair, J., Fox, G., Moon, R., & Stock, P. (1987). English language dictionary. London, UK: Collins.
Succi, C., & Canovi, M. (2019). Soft skills to enhance graduate employability: comparing students and employers’ perceptions. Studies in Higher Education. https://doi.org/10.1080 /03075079 .2019.1585420
Sulphey, M. M., Al-Kahtani, N. S., & Syed, A. M. (2018). Relationship between admission grades and academic achievement. Entrepreneurship and Sustainability Issues, 5(3), 648–658. DOI 10.9770/jesi.2018.5.3(17)
Taylor, L. (2005). Washback and impact. ELT Journal, 59(2), 154–155. https://doi.org/10.1093/eltj/cci030
Tsagari, D., & Cheng, L. (2017). Washback, impact, and consequences revisited. In E. Shohamy, I. G. Or, & S. May (Eds.), Language testing and assessment (pp. 359–372). Cham: Springer International Publishing.
Wall, D. (1997). Impact and washback in language testing. In C. Clapham & D. Corson (Eds.), Encyclopedia of language and education 7: Language testing and assessment (pp.291–302). Dordrecht: Kluwer Academic.
Wiggins, G. (1998). Educative assessment. San Francisco, CA: Jossey-Bass Publishers.
Wilkinson, A. (1968). The testing of oracy. In A. Davies (Ed.), Language testing symposium: A psycholinguistic perspective (pp. 117–132). Oxford, UK: Oxford University Press.
Wiseman, S. (1961). Examinations and English education. Manchester: Manchester University Press.
Wong, L. L. C., & Nunan, D. (2011). The learning styles and strategies of effective language learners. System, 39(2), 144–163. https://doi. org/10.1016/j.system.2011.05.004
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 The Authors & LUMEN Publishing House
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant this journalright of first publication, with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work, with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g. post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as an earlier and greater citation of published work (See The Effect of Open Access).
Revista Romaneasca pentru Educatie Multidimensionala Journal has an Attribution-NonCommercial-NoDerivs
CC BY-NC-ND
