Systematic reviews (SRs) synthesize evidence through a rigorous, labor-intensive, and costly process. To accelerate the title–abstract screening phase of SRs, several artificial intelligence (AI)-based semi-automated screening tools have been developed to reduce workload by prioritizing relevant records. However, their performance is primarily evaluated for SRs of intervention studies, which generally have well-structured abstracts. Here, we evaluate whether screening tool performance is equally effective for SRs of prognosis studies that have larger heterogeneity between abstracts. We conducted retrospective simulations on prognosis and intervention reviews using a screening tool (ASReview). We also evaluated the effects of review scope (i.e., breadth of the research question), number of (relevant) records, and modeling methods within the tool. Performance was assessed in terms of recall (i.e., sensitivity), precision at 95% recall (i.e., positive predictive value at 95% recall), and workload reduction (work saved over sampling at 95% recall [WSS@95%]). The WSS@95% was slightly worse for prognosis reviews (range: 0.324–0.597) than for intervention reviews (range: 0.613–0.895). The precision was higher for prognosis (range: 0.115–0.400) compared to intervention reviews (range: 0.024–0.057). These differences were primarily due to the larger number of relevant records in the prognosis reviews. The modeling methods and the scope of the prognosis review did not significantly impact tool performance. We conclude that the larger abstract heterogeneity of prognosis studies does not substantially affect the effectiveness of screening tools for SRs of prognosis. Further evaluation studies including a standardized evaluation framework are needed to enable prospective decisions on the reliable use of screening tools.

Bayesian optimal experiments that maximize the information gained from collected data are critical to efficiently identify behavioral models. We extend a seminal method for designing Bayesian optimal experiments by introducing two computational improvements that make the procedure tractable: (1) a search algorithm from artificial intelligence that efficiently explores the space of possible design parameters, and (2) a sampling procedure which evaluates each design parameter combination more efficiently. We apply our procedure to a game of imperfect information to evaluate and quantify the computational improvements. We then collect data across five different experimental designs to compare the ability of the optimal experimental design to discriminate among competing behavioral models against the experimental designs chosen by a “wisdom of experts” prediction experiment. We find that data from the experiment suggested by the optimal design approach requires significantly less data to distinguish behavioral models (i.e., test hypotheses) than data from the experiment suggested by experts. Substantively, we find that reinforcement learning best explains human decision-making in the imperfect information game and that behavior is not adequately described by the Bayesian Nash equilibrium. Our procedure is general and computationally efficient and can be applied to dynamically optimize online experiments.

Systematic reviews play important roles but manual efforts can be time-consuming given a growing literature. There is a need to use and evaluate automated strategies to accelerate systematic reviews. Here, we comprehensively tested machine learning (ML) models from classical and deep learning model families. We also assessed the performance of prompt engineering via few-shot learning of GPT-3.5 and GPT-4 large language models (LLMs). We further attempted to understand when ML models can help automate screening. These ML models were applied to actual datasets of systematic reviews in education. Results showed that the performance of classical and deep ML models varied widely across datasets, ranging from 1.2 to 75.6% of work saved at 95% recall. LLM prompt engineering produced similarly wide performance variation. We searched for various indicators of whether and how ML screening can help. We discovered that the separability of clusters of relevant versus irrelevant articles in high-dimensional embedding space can strongly predict whether ML screening can help (overall R = 0.81). This simple and generalizable heuristic applied well across datasets and different ML model families. In conclusion, ML screening performance varies tremendously, but researchers and software developers can consider using our cluster separability heuristic in various ways in an ML-assisted screening pipeline.

Regression is a fundamental prediction task common in data-centric engineering applications that involves learning mappings between continuous variables. In many engineering applications (e.g., structural health monitoring), feature-label pairs used to learn such mappings are of limited availability, which hinders the effectiveness of traditional supervised machine learning approaches. This paper proposes a methodology for overcoming the issue of data scarcity by combining active learning (AL) for regression with hierarchical Bayesian modeling. AL is an approach for preferentially acquiring feature-label pairs in a resource-efficient manner. In particular, the current work adopts a risk-informed approach that leverages contextual information associated with regression-based engineering decision-making tasks (e.g., inspection and maintenance). Hierarchical Bayesian modeling allow multiple related regression tasks to be learned over a population, capturing local and global effects. The information sharing facilitated by this modeling approach means that information acquired for one engineering system can improve predictive performance across the population. The proposed methodology is demonstrated using an experimental case study. Specifically, multiple regressions are performed over a population of machining tools, where the quantity of interest is the surface roughness of the workpieces. An inspection and maintenance decision process is defined using these regression tasks, which is in turn used to construct the active-learning algorithm. The novel methodology proposed is benchmarked against an uninformed approach to label acquisition and independent modeling of the regression tasks. It is shown that the proposed approach has superior performance in terms of expected cost—maintaining predictive performance while reducing the number of inspections required.

This paper reflects on a project-based curriculum employing constructed languages to teach linguistics, with a focus on phonology. In a special topics linguistics course, nine students were led through the construction of a language. While students in introductory linguistics courses sometimes struggle with phonology, active engagement with a semester-long language construction project endowed these students with the practical motivation to understand (1) what phonology is, (2) how phonological rules work, and (3) why rules surface in the first place. They readily captured generalizations based on natural classes of sounds, recognizing the systematicity of their constructed phonology. Student performance and engagement in this course support the use of constructed languages as a pedagogical tool in linguistics. Because an ongoing project builds in problem-solving opportunities and processual thinking, highlighting relationships among key concepts, students achieve a more comprehensive understanding of core areas in the broader linguistic picture.

This study creates a virtual space for language learning using a user-customizable metaverse platform and explores its potential for EFL learning. To this end, a virtual learning space, grounded in constructivist learning principles – contextualized learning, active learning, and collaborative learning – was created on a 2D metaverse platform. The metaverse was designed as a simulated deserted island for enjoyable and playful learning, allowing the students to actively explore, discover, and interact as they look for clues to escape the island. For educational application, 29 Korean middle school students participated in a two-hour activity. Data included screen recordings of student activities, student surveys, and interviews with the students and teachers. The findings showed that, as an EFL learning space of playful constructivism, the metaverse had great potential to embed contextualized learning and served as a medium for active learning that positively affected student interest and motivation. The results confirmed that the team-based approach combined with a game-like metaverse fostered student collaboration. Overall, the study showcased how language instructors can make use of a customizable metaverse for L2 learning and how a virtual space may serve as an arena for learner-centered instruction.

Motion planning for high-DOF multi-arm systems operating in complex environments remains a challenging problem, with many motion planning algorithms requiring evaluation of the minimum collision distance and its derivative. Because of the computational complexity of calculating the collision distance, recent methods have attempted to leverage data-driven machine learning methods to learn the collision distance. Because of the significant training dataset requirements for high-DOF robots, existing kernel-based methods, which require $O(N^2)$ memory and computation resources, where $N$ denotes the number of dataset points, often perform poorly. This paper proposes a new active learning method for learning the collision distance function that overcomes the limitations of existing methods: (i) the size of the training dataset remains fixed, with the dataset containing more points near the collision boundary as learning proceeds, and (ii) calculating collision distances in the higher-dimensional link $SE(3)^n$ configuration space – here $n$ denotes the number of links – leads to more accurate and robust collision distance function learning. Performance evaluations with high-DOF multi-arm robot systems demonstrate the advantages of the proposed active learning-based strategy vis-$\grave{\text{a}}$-vis existing learning-based methods.

In this article, we showcase the pilot scenario of The Trojan War, an educational self-directed game that combines text inspired by ancient Greek (as well as Roman) literature with graphics based on the ‘Geometric style’, an authentic Greek style of painting contemporary with the composition of the Homeric epics. Our game uses interactive scenarios to support active learning strategies of students interested in Classical Studies in both tertiary and secondary education. Players can take on the role of key characters, making choices that can prevent, start, or stop the Trojan War, as well as determine their own personal outcomes. The learners are thus presented with the opportunity to explore alternative pathways to rewrite the history of the War. In the process, they can apply their subject knowledge and develop their intellectual and critical skills. They also become familiar with a distinctive and expressive early Greek artistic style, the so-called Geometric. Rather than focusing on winning, the game aims to give students the opportunity to engage with important ideas and values of ancient Greek culture by exploring multiple perspectives on the topic. It also provides a valuable lesson on the potentially wide-ranging consequences of individual choices, which is a core element of responsible citizenship.

Environmental sensors are crucial for monitoring weather conditions and the impacts of climate change. However, it is challenging to place sensors in a way that maximises the informativeness of their measurements, particularly in remote regions like Antarctica. Probabilistic machine learning models can suggest informative sensor placements by finding sites that maximally reduce prediction uncertainty. Gaussian process (GP) models are widely used for this purpose, but they struggle with capturing complex non-stationary behaviour and scaling to large datasets. This paper proposes using a convolutional Gaussian neural process (ConvGNP) to address these issues. A ConvGNP uses neural networks to parameterise a joint Gaussian distribution at arbitrary target locations, enabling flexibility and scalability. Using simulated surface air temperature anomaly over Antarctica as training data, the ConvGNP learns spatial and seasonal non-stationarities, outperforming a non-stationary GP baseline. In a simulated sensor placement experiment, the ConvGNP better predicts the performance boost obtained from new observations than GP baselines, leading to more informative sensor placements. We contrast our approach with physics-based sensor placement methods and propose future steps towards an operational sensor placement recommendation system. Our work could help to realise environmental digital twins that actively direct measurement sampling to improve the digital representation of reality.

Named entity recognition (NER) aims to identify mentions of named entities in an unstructured text and classify them into predefined named entity classes. While deep learning-based pre-trained language models help to achieve good predictive performances in NER, many domain-specific NER applications still call for a substantial amount of labeled data. Active learning (AL), a general framework for the label acquisition problem, has been used for NER tasks to minimize the annotation cost without sacrificing model performance. However, the heavily imbalanced class distribution of tokens introduces challenges in designing effective AL querying methods for NER. We propose several AL sentence query evaluation functions that pay more attention to potential positive tokens and evaluate these proposed functions with both sentence-based and token-based cost evaluation strategies. We also propose a better data-driven normalization approach to penalize sentences that are too long or too short. Our experiments on three datasets from different domains reveal that the proposed approach reduces the number of annotated tokens while achieving better or comparable prediction performance with conventional methods.

Model order reduction (MOR) can provide low-dimensional numerical models for fast simulation. Unlike intrusive methods, nonintrusive methods are attractive because they can be applied even without access to full order models (FOMs). Since nonintrusive MOR methods strongly rely on snapshots of the FOMs, constructing good snapshot sets becomes crucial. In this work, we propose a novel active-learning-based approach for use in conjunction with nonintrusive MOR methods. It is based on two crucial novelties. First, our approach uses joint space sampling to prepare a data pool of the training data. The training data are selected from the data pool using a greedy strategy supported by an error estimator based on Gaussian process regression. Second, we introduce a case-independent validation strategy based on probably approximately correct learning. While the methods proposed here can be applied to different MOR methods, we test them here with artificial neural networks and operator inference.

Although an accurate reliability assessment is essential to build a resilient infrastructure, it usually requires time-consuming computation. To reduce the computational burden, machine learning-based surrogate models have been used extensively to predict the probability of failure for structural designs. Nevertheless, the surrogate model still needs to compute and assess a certain number of training samples to achieve sufficient prediction accuracy. This paper proposes a new surrogate method for reliability analysis called Adaptive Hyperball Kriging Reliability Analysis (AHKRA). The AHKRA method revolves around using a hyperball-based sampling region. The radius of the hyperball represents the precision of reliability analysis. It is iteratively adjusted based on the number of samples required to evaluate the probability of failure with a target coefficient of variation. AHKRA adopts samples in a hyperball instead of an n-sigma rule-based sampling region to avoid the curse of dimensionality. The application of AHKRA in ten mathematical and two practical cases verifies its accuracy, efficiency, and robustness as it outperforms previous Kriging-based methods.