Hostname: page-component-cb9f654ff-mnl9s Total loading time: 0 Render date: 2025-08-30T01:30:49.994Z Has data issue: false hasContentIssue false
  • English
  • Français

Usage-based analysis of L2 oral proficiency: Characteristics of argument structure construction use

Published online by Cambridge University Press:  04 August 2025

Hakyung Sung Open the ORCID record for Hakyung Sung [Opens in a new window]  and
Kristopher Kyle Open the ORCID record for Kristopher Kyle [Opens in a new window]
Hakyung Sung*
Affiliation:
Department of Linguistics, https://ror.org/0293rh119 University of Oregon , Eugene, OR, USA
Kristopher Kyle
Affiliation:
Department of Linguistics, https://ror.org/0293rh119 University of Oregon , Eugene, OR, USA
*
Corresponding author: Hakyung Sung; Email: hsung@uoregon.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Analyzing the relationship between argument structure construction (ASC) use and language learning has been an important area of investigation in second language (L2) studies from a usage-based constructionist approach. Previous studies have shown that advanced L2 learners’ language use demonstrates greater ASC diversity, less frequent ASC-verb combinations, and stronger ASC-verb associations. However, these investigations have been limited by methodological challenges in identifying ASCs and have predominantly focused on the written texts. To address these limitations, we employ a fine-tuned model to automatically extract ASCs from target and reference corpora, considering their semantic aspects. We then calculate ASC-based indices and, both alone and in combination with other lexicogrammatical indices, use them to predict L2 oral proficiency scores assigned by human judges. Our findings show that ASC-based indices alone explain 44% of the variance in scores. When combined with other indices, they provide complementary insights that enhance multivariate modeling of L2 oral proficiency.

Keywords

argument structure constructionsmultivariate analysesusage-based indicesL2 oral proficiency

Information

Type
Research Article
Information
Studies in Second Language Acquisition , First View , pp. 1 - 27
Check for updates
Open Practices
Open data Open materials Preregistered
Copyright
© The Author(s), 2025. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

ACTFL-ALC Press. (1996). Standard speaking test manual. ALC Press.Google Scholar
Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723.10.1109/TAC.1974.1100705CrossRefGoogle Scholar
Akbik, A., Chiticariu, L., Danilevsky, M., Li, Y., Vaithyanathan, S., & Zhu, H. (2015). Generating high quality proposition banks for multilingual semantic role labeling. In Zong, C. & Strube, M. (Eds.), Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 397–407). Association for Computational Linguistics.Google Scholar
Barðdal, J. (2006). Predicting the productivity of argument structure constructions. In Antić, Z., Chang, C. B., Cibelli, E., Hong, J., Houser, M. J., Sandy, C. S., Toosarvandani, M., & Yao, Y. (Eds.), Proceedings of the 32nd Annual Meeting of the Berkeley Linguistics Society: General Session and Parasession on Theoretical Approaches to Argument Structure (pp. 467–478). Berkeley Linguistics Society.Google Scholar
Barton, K. (2023). MuMIn: Multi-model inference (R package version 1.47.1). Comprehensive R Archive Network.Google Scholar
Berger, C., Crossley, S., & Skalicky, S. (2019). Using lexical features to investigate second language lexical decision performance. Studies in Second Language Acquisition, 41, 911–935.10.1017/S0272263119000019CrossRefGoogle Scholar
Berzak, Y., Kenney, J., Spadine, C., Wang, J. X., Lam, L., Mori, K. S., Garza, S., & Katz, B. (2016). Universal dependencies for learner English. In Erk, K. & Smith, N. A. (Eds.), Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 737–746). Association for Computational Linguistics.10.18653/v1/P16-1070CrossRefGoogle Scholar
Bestgen, Y., & Granger, S. (2014). Quantifying the development of phraseological competence in L2 English writing: An automated approach. Journal of Second Language Writing, 26, 28–41.10.1016/j.jslw.2014.09.004CrossRefGoogle Scholar
Boas, H. C. (2010). The syntax–lexicon continuum in Construction Grammar: A case study of English communication verbs. Belgian Journal of Linguistics, 24, 54–82.10.1075/bjl.24.03boaCrossRefGoogle Scholar
Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41, 977–990.10.3758/BRM.41.4.977CrossRefGoogle Scholar
Brysbaert, M., New, B., & Keuleers, E. (2012). Adding part-of-speech information to the SUBTLEX-US word frequencies. Behavior Research Methods, 44, 991–997.10.3758/s13428-012-0190-4CrossRefGoogle Scholar
Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. Behavior Research Methods, 46, 904–911.10.3758/s13428-013-0403-5CrossRefGoogle ScholarPubMed
Bulté, B., & Roothooft, H. (2020). Investigating the interrelationship between rated L2 proficiency and linguistic complexity in L2 speech. System, 91, 102246.10.1016/j.system.2020.102246CrossRefGoogle Scholar
Bybee, J. (2006). From usage to grammar: The mind’s response to repetition. Language, 82, 711–733.10.1353/lan.2006.0186CrossRefGoogle Scholar
Bybee, J. (2010). Language, usage and cognition. Cambridge University Press.10.1017/CBO9780511750526CrossRefGoogle Scholar
Chen, D., & Manning, C. D. (2014). A fast and accurate dependency parser using neural networks. In Moschitti, A., Pang, B., & Daelemans, W. (Eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 740–750). Association for Computational Linguistics.10.3115/v1/D14-1082CrossRefGoogle Scholar
Choi, J., & Sung, M. C. (2020). Utterance-based measurement of L2 fluency in speaking interactions: A constructionist approach. English Teaching, 75, 105–126.10.15858/engtea.75.s1.202006.105CrossRefGoogle Scholar
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Routledge.Google Scholar
Crossley, S. A., Skalicky, S., Kyle, K., & Monteiro, K. (2019). Absolute frequency effects in second language lexical acquisition. Studies in Second Language Acquisition, 41, 721–744.10.1017/S0272263118000268CrossRefGoogle Scholar
de Marneffe, M.-C., & Manning, C. D. (2008). Stanford typed dependencies manual. Stanford University.Google Scholar
Diessel, H. (2015). Usage-based construction grammar. In E. Dbrowska & D. Divjak (Eds.), Handbook of cognitive linguistics (pp. 295–321). Mouton de Gruyter.10.1515/9783110292022-015CrossRefGoogle Scholar
Eguchi, M., & Kyle, K. (2020). Continuing to explore the multidimensional nature of lexical sophistication: The case of oral proficiency interviews. Modern Language Journal, 104, 381–400.10.1111/modl.12637CrossRefGoogle Scholar
Ellis, N. C. (2002). Frequency effects in language processing: A review with implications for theories of implicit and explicit language acquisition. Studies in Second Language Acquisition, 24, 143–188.10.1017/S0272263102002024CrossRefGoogle Scholar
Ellis, N. C. (2006a). Language acquisition as rational contingency learning. Applied Linguistics, 27, 1–24.10.1093/applin/ami038CrossRefGoogle Scholar
Ellis, N. C. (2006b). Selective attention and transfer phenomena in L2 acquisition: Contingency, cue competition, salience, interference, overshadowing, blocking, and perceptual learning. Applied Linguistics, 27, 164–194.10.1093/applin/aml015CrossRefGoogle Scholar
Ellis, N. C. (2012). Formulaic language and second language acquisition: Zipf and the phrasal teddy bear. Annual Review of Applied Linguistics, 32, 17–44.10.1017/S0267190512000025CrossRefGoogle Scholar
Ellis, N. C., & Ferreira-Junior, F. (2009). Construction learning as a function of frequency, frequency distribution, and function. Modern Language Journal, 93, 370–385.10.1111/j.1540-4781.2009.00896.xCrossRefGoogle Scholar
Ellis, N. C., O’Donnell, M. B., & Römer, U. (2014). The processing of verb-argument constructions is sensitive to form, function, frequency, contingency and prototypicality. Cognitive Linguistics, 25, 55–98.10.1515/cog-2013-0031CrossRefGoogle Scholar
Ellis, N. C., Römer, U., & O’Donnell, M. B. (2016). Constructions and usage-based approaches to language acquisition. Language Learning, 66, 23–44.Google Scholar
Fillmore, C. J. (1976). Frame semantics and the nature of language. Annals of the New York Academy of Sciences, 280, 20–32.10.1111/j.1749-6632.1976.tb25467.xCrossRefGoogle Scholar
Fillmore, C. J., Johnson, C. R., & Petruck, M. R. (2003). Background to framenet. International Journal of Lexicography, 16, 235–250.10.1093/ijl/16.3.235CrossRefGoogle Scholar
Fillmore, C. J., Kay, P., & O’Connor, M. C. (1988). Regularity and idiomaticity in grammatical constructions: The case of let alone. Language, 64, 501–538.10.2307/414531CrossRefGoogle Scholar
Garner, J., Crossley, S., & Kyle, K. (2019). N-gram measures and L2 writing proficiency. System, 80, 176–187.10.1016/j.system.2018.12.001CrossRefGoogle Scholar
Gilquin, G. (2019). Light verb constructions in spoken L2 English: An exploratory cross-sectional study. International Journal of Learner Corpus Research, 5, 181–206.10.1075/ijlcr.18003.gilCrossRefGoogle Scholar
Goldberg, A. E. (1995). Constructions: A construction grammar approach to argument structure. University of Chicago Press.Google Scholar
Goldberg, A. E. (2013a). Argument structure constructions versus lexical rules or derivational verb templates. Mind & Language, 28, 435–465.10.1111/mila.12026CrossRefGoogle Scholar
Goldberg, A. E. (2013b). The emergence of the semantics of argument structure constructions. In MacWhinney, B. (Ed.), The emergence of language (pp. 197–212). Psychology Press.Google Scholar
Goldberg, A. E. (2019). Explain me this: Creativity, competition, and the partial productivity of constructions. Princeton University Press.Google Scholar
Goldberg, A. E., Casenhiser, D. M., & Sethuraman, N. (2004). Learning argument structure generalizations. Cognitive Linguistics, 15, 289–316.10.1515/cogl.2004.011CrossRefGoogle Scholar
Housen, A. (2008). A corpus-based study of the L2-acquisition of the English verb system. In S. Granger, J. Hung, & S. Petch-Tyson (Eds.), Computer learner corpora, second language acquisition and foreign language teaching (pp. 77–116). John Benjamins Publishing Company.Google Scholar
Hwang, H., & Kim, H. (2023). Automatic analysis of constructional diversity as a predictor of EFL students’ writing proficiency. Applied Linguistics, 44, 127–147.10.1093/applin/amac046CrossRefGoogle Scholar
Izumi, E., Uchimoto, K., & Isahara, H. (2004). The NICT JLE corpus: Exploiting the language learners’ speech database for research and education. International Journal of the Computer, the Internet and Management, 12, 119–125.Google Scholar
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2017). An introduction to statistical learning: With applications in R. Springer.Google Scholar
Kim, H., & Rah, Y. (2019). Constructional processing in a second language: The role of constructional knowledge in verb-construction integration. Language Learning, 69, 1022–1056.10.1111/lang.12366CrossRefGoogle Scholar
Kim, H., & Ro, E. (2023). Assessment of sentence sophistication in L2 spoken production: Expansion of verbs and argument structure constructions. System, 119, 103175.10.1016/j.system.2023.103175CrossRefGoogle Scholar
Kim, M., Crossley, S. A., & Kyle, K. (2018). Lexical sophistication as a multidimensional phenomenon: Relations to second language lexical proficiency, development, and writing quality. Modern Language Journal, 102, 120–141.10.1111/modl.12447CrossRefGoogle Scholar
Kiss, G. R., Armstrong, C., Milroy, R., & Piper, J. (1973). An associative thesaurus of English and its computer analysis. The Computer and Literary Studies, 13, 153–165.Google Scholar
Kobayashi, Y., & Abe, M. (2016). Automated scoring of L2 spoken English with random forests. Journal of Pan-Pacific Association of Applied Linguistics, 20, 55–73.Google Scholar
Koizumi, R., & Hirai, A. (2012). Comparing the story retelling speaking test with other speaking tests. JALT Journal, 34, 35–60.10.37546/JALTJJ34.1-2CrossRefGoogle Scholar
Kyle, K. (2019). Measuring lexical richness. In The Routledge handbook of vocabulary studies (pp. 454–476). Routledge.10.4324/9780429291586-29CrossRefGoogle Scholar
Kyle, K., & Crossley, S. A. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly, 49, 757–786.10.1002/tesq.194CrossRefGoogle Scholar
Kyle, K., & Crossley, S. (2017). Assessing syntactic sophistication in L2 writing: A usage-based approach. Language Testing, 34, 513–535.10.1177/0265532217712554CrossRefGoogle Scholar
Kyle, K., Crossley, S., & Berger, C. (2018). The tool for the automatic analysis of lexical sophistication (TAALES): Version 2.0. Behavior Research Methods, 50, 1030–1046.10.3758/s13428-017-0924-4CrossRefGoogle ScholarPubMed
Kyle, K., Crossley, S., & Verspoor, M. (2021). Measuring longitudinal writing development using indices of syntactic complexity and sophistication. Studies in Second Language Acquisition, 43, 781–812.10.1017/S0272263120000546CrossRefGoogle Scholar
Kyle, K., & Eguchi, M. (2023). Assessing spoken lexical and lexicogrammatical proficiency using features of word, bigram, and dependency bigram use. Modern Language Journal, 107, 531–564.10.1111/modl.12845CrossRefGoogle Scholar
Kyle, K., Eguchi, M., Miller, A., & Sither, T. (2022). A dependency treebank of spoken second language English. In Kochmar, E., Burstein, J., Horbach, A., Laarmann-Quante, R., Madnani, N., Tack, A., Yaneva, V., Yuan, Z., & Zesch, T. (Eds.), Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022) (pp. 39–45). Association for Computational Linguistics.10.18653/v1/2022.bea-1.7CrossRefGoogle Scholar
Kyle, K., & Sung, H. (2023). An argument structure construction treebank. In Bonial, C. & Madabushi, H. Tayyar (Eds.), Proceedings of the First International Workshop on Construction Grammars and NLP (CxGs+NLP, GURT/SyntaxFest 2023) (pp. 51–62). Association for Computational Linguistics.Google Scholar
Kyle, K., Sung, H., Eguchi, M., & Zenker, F. (2024). Evaluating evidence for the reliability and validity of lexical diversity indices in L2 oral task responses. Studies in Second Language Acquisition, 46, 278–299.10.1017/S0272263123000402CrossRefGoogle Scholar
Langacker, R. W. (1987). Nouns and verbs. Language, 63, 53–94.10.2307/415384CrossRefGoogle Scholar
Laufer, B., & Nation, P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16, 307–322.10.1093/applin/16.3.307CrossRefGoogle Scholar
Liu, Y., & Lu, X. (2024). Development of verb argument constructions in L2 English learners: A close replication of research question 3 in Römer and Berger (2019). Studies in Second Language Acquisition, 46, 1–19.10.1017/S027226312400024XCrossRefGoogle Scholar
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach [arXiv preprint arXiv:1907.11692]. arXiv. https://arxiv.org/abs/1907.11692Google Scholar
McDonald, S. A., & Shillcock, R. C. (2001). Rethinking the word frequency effect: The neglected role of distributional information in lexical processing. Language and Speech, 44, 295–322.10.1177/00238309010440030101CrossRefGoogle ScholarPubMed
Meara, P., & Bell, H. (2001). P-Lex: A simple and effective way of describing the lexical characteristics of short L2 tests. Prospect, 16, 5–19.Google Scholar
Menard, S. (2001). Applied logistic regression analysis. SAGE Publications.Google Scholar
Mostafa, T., & Crossley, S. A. (2020). Verb argument construction complexity indices and L2 writing quality: Effects of writing tasks and prompts. Journal of Second Language Writing, 49, 100730.10.1016/j.jslw.2020.100730CrossRefGoogle Scholar
Murakami, A., & Ellis, N. C. (2022). Effects of availability, contingency, and formulaicity on the accuracy of English grammatical morphemes in second language writing. Language Learning, 72, 899–940.10.1111/lang.12500CrossRefGoogle Scholar
Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (2004). The University of South Florida free association, rhyme, and word fragment norms. Behavior Research Methods, Instruments, & Computers, 36, 402–407.10.3758/BF03195588CrossRefGoogle ScholarPubMed
Ninio, A. (1999). Pathbreaking verbs in syntactic development and the question of prototypical transitivity. Journal of Child Language, 26, 619–653.10.1017/S0305000999003931CrossRefGoogle ScholarPubMed
O’Donnell, M., & Ellis, N. (2010). Towards an inventory of English verb argument constructions. In Sahlgren, M. & Knutsson, O. (Eds.), Proceedings of the NAACL HLT Workshop on Extracting and Using Constructions in Computational Linguistics (pp. 9–16). Association for Computational Linguistics.Google Scholar
Palmer, M., Gildea, D., & Kingsbury, P. (2005). The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31, 71–106.10.1162/0891201053630264CrossRefGoogle Scholar
Paquot, M. (2018). Phraseological competence: A missing component in university entrance language tests? Insights from a study of EFL learners’ use of statistical collocations. Language Assessment Quarterly, 15, 29–43.10.1080/15434303.2017.1405421CrossRefGoogle Scholar
Read, J. A. (2000). Assessing vocabulary. Cambridge University Press.10.1017/CBO9780511732942CrossRefGoogle Scholar
Römer, U., & Berger, C. M. (2019). Observing the emergence of constructional knowledge: Verb patterns in German and Spanish learners of English at different proficiency levels. Studies in Second Language Acquisition, 41, 1089–1110.10.1017/S0272263119000202CrossRefGoogle Scholar
Römer, U., O’Donnell, M. B., & Ellis, N. C. (2014). Second language learner knowledge of verb–argument constructions: Effects of language transfer and typology. Modern Language Journal, 98, 952–975.10.1111/modl.12149CrossRefGoogle Scholar
Rubin, R., Housen, A., & Paquot, M. (2021). Phraseological complexity as an index of L2 Dutch writing proficiency: A partial replication study. In S. Granger (Ed.), Perspectives on the second language phrasicon: The view from learner corpora (pp. 101–125). Multilingual Matters.Google Scholar
Saito, K. (2020). Multi-or single-word units? The role of collocation use in comprehensible and contextually appropriate second language speech. Language Learning, 70, 548–588.10.1111/lang.12387CrossRefGoogle Scholar
Schuler, K. K. (2005). VerbNet: A broad-coverage, comprehensive verb lexicon. University of Pennsylvania.Google Scholar
Silveira, N., Dozat, T., de Marneffe, M.-C., Bowman, S. R., Connor, M., Bauer, J., & Manning, C. D. (2014). A gold standard dependency corpus for English. In Calzolari, N., Choukri, K., Goggi, S., Declerck, T., Mariani, J., Maegaard, B., Moreno, A., Odijk, J., Mazo, H., Piperidis, S., & Loftsson, H. (Eds.), Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC 2014) (pp. 2897–2904). European Language Resources Association.Google Scholar
Sung, H., & Kyle, K. (2024a). Annotation scheme for English argument structure constructions treebank. In Henning, S. & Stede, M. (Eds.), Proceedings of the 18th Linguistic Annotation Workshop (LAW-XVIII) (pp. 12–18). Association for Computational Linguistics.Google Scholar
Sung, H., & Kyle, K. (2024b). Leveraging pre‑trained language models for linguistic analysis: A case of argument structure constructions. In Al‑Onaizan, Y., Bansal, M., & Chen, Y. N. (Eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 7302–7314). Association for Computational Linguistics.10.18653/v1/2024.emnlp-main.415CrossRefGoogle Scholar
Talmy, L. (1985). Lexicalization patterns: Semantic structure in lexical forms. Language Typology and Syntactic Description, 3, 36–149.Google Scholar
Tan, M. Y. J., & Biswas, R. (2012). The reliability of the Akaike information criterion method in cosmological model selection. Monthly Notices of the Royal Astronomical Society, 419, 3292–3303.10.1111/j.1365-2966.2011.19969.xCrossRefGoogle Scholar
Tomasello, M. (2005). Constructing a language: A usage-based theory of language acquisition. Harvard University Press.10.2307/j.ctv26070v8CrossRefGoogle Scholar
van der Laken, P., & Lambert, L. (2023). corrtable: Creates and saves out a correlation table with significance levels indicated (R package version 0.1.1). Comprehensive R Archive Network.Google Scholar
Yoon, S. Y., Lu, X., & Zechner, K. (2019). Features measuring vocabulary and grammar. In Evanini, K., Wang, X., & Loukina, A. (Eds.), Automated speaking assessment (pp. 123–137). Routledge.10.4324/9781315165103-8CrossRefGoogle Scholar