Uses of Word Embeddings in Engineering Education

Natural Language Processing (NLP) techniques comprise all methods for human language comprehension by machines and Large Language Models (LLMs) are generative Artificial Intelligence (AI) models that are used for NLP tasks. An important aspect of LLMs are word embeddings, a way of representing text in numerical form such that words and phrases can be mathematically compared. The purpose of this review paper is to consider the existing research on word embeddings in an engineering education context and discuss future applications. After a brief explanation of the technology, a literature review details current uses in education. Of the 96 papers identified in a literature search, 13 are discussed further which highlight the use of word embeddings for applications like summarization and qualitative analysis. Finally, other potential applications are identified and discussed including knowledge-based assessments and syllabus analysis. The work suggests that applying embeddings as a method is similar in all contexts, showcasing this technology as a potentially multi-use method the engineering education community can implement for future work both in pedagogical settings and potential research projects.