Introduction

Large language models: Transforming artificial intelligence Since the release of BERT (Bidirectional Encoder Representations from Transformers) by Google in 2018 and GPT-3 by OpenAI in 2020, large language models (LLMs) have revolutionised the field of artificial intelligence (AI) and natural language processing. BERT pioneered the use of bidirectional transformers, which understand words by considering context from both sides, enabling a better grasp of natural language (Devlin et al., 2019). GPT-3 advanced this technology further with 175 billion parameters, providing unprecedented capabilities in text generation and comprehension (Frantar et al., 2023). These models are trained on vast amounts of textual data using deep learning techniques that capture the complex patterns and structures of human language. In analysing text sequences, LLMs use the transformer architecture for deep learning networks, allowing models to attend to different parts of the text simultaneously, thus improving the coherence and relevance of the generated responses. This capability enhances generative AI by helping it to create contextually appropriate content.

The transformer architecture was introduced in the seminal paper ‘Attention is All You Need’ (Vaswani et al., 2017). It demonstrated how self-attention mechanisms could outperform traditional recurrent neural networks in natural language processing tasks. This significantly improved the accuracy and efficiency of language models and laid the groundwork for subsequent developments.