2.6 Automated Item Generation

Automatic item generation (AIG) is used to describe the process of generating items using computer technology, typically using some kind of models (e.g., cognitive models, Gierl, Lai, and Turner 2012). A template-based approach (Gierl and Lai 2013) formulates an item model (also called, for instance, schema or blueprint) containing the components of a task that can be varied to generate items. Item models can be described regarding the number of layers in which item clones (i.e., generated items) differ from a source. While AIG from an IRT perspective, for instance, generating items on the fly was suggested more than ten years ago (e.g., Embretson and Yang 2006), current research incorporating machine learning techniques such as deep learning (e.g., von Davier 2018) and models developed for natural language processing can be expected to provide promising new methods and applications (see for a review, e.g., Das et al. 2021; and for an example Attali et al. 2022).

References

Attali, Yigal, Andrew Runge, Geoffrey T. LaFlair, Kevin Yancey, Sarah Goodwin, Yena Park, and Alina A. von Davier. 2022. “The Interactive Reading Task: Transformer-based Automatic Item Generation.” Frontiers in Artificial Intelligence 5 (July): 903077. https://doi.org/10.3389/frai.2022.903077.

Das, Bidyut, Mukta Majumder, Santanu Phadikar, and Arif Ahmed Sekh. 2021. “Automatic Question Generation and Answer Assessment: A Survey.” Research and Practice in Technology Enhanced Learning 16 (1): 5. https://doi.org/10.1186/s41039-021-00151-1.

Embretson, Susan, and Xiangdong Yang. 2006. “Automatic Item Generation and Cognitive Psychology.” In Handbook of Statistics, 26:747–68. Elsevier. https://doi.org/10.1016/S0169-7161(06)26023-1.

Gierl, Mark J., and Hollis Lai. 2013. “Instructional Topics in Educational Measurement (ITEMS) Module: Using Automated Processes to Generate Test Items.” Educational Measurement: Issues and Practice 32 (3): 36–50. https://doi.org/10.1111/emip.12018.

Gierl, Mark J., Hollis Lai, and Simon R Turner. 2012. “Using Automatic Item Generation to Create Multiple-Choice Test Items: Automatic Generation of Test Items.” Medical Education 46 (8): 757–65. https://doi.org/10.1111/j.1365-2923.2012.04289.x.

von Davier, Matthias. 2018. “Automated Item Generation with Recurrent Neural Networks.” Psychometrika 83 (4): 847–57. https://doi.org/10.1007/s11336-018-9608-y.