Numerical solutions of partial differential equations require expensive simulations, limiting their application in design optimization, model-based control, and large-scale inverse problems. Surrogate modeling techniques aim to decrease computational expense while retaining dominant solution features and characteristics. Existing frameworks based on convolutional neural networks and snapshot-matrix decomposition often rely on lossy pixelization and data-preprocessing, limiting their effectiveness in realistic engineering scenarios. Recently, coordinate-based multilayer perceptron networks have been found to be effective at representing 3D objects and scenes by regressing volumetric implicit fields. These concepts are leveraged and adapted in the context of physical-field surrogate modeling. Two methods toward generalization are proposed and compared: design-variable multilayer perceptron (DV-MLP) and design-variable hypernetworks (DVH). Each method utilizes a main network which consumes pointwise spatial information to provide a continuous representation of the solution field, allowing discretization independence and a decoupling of solution and model size. DV-MLP achieves generalization through the use of a design-variable embedding vector, while DVH conditions the main network weights on the design variables using a hypernetwork. The methods are applied to predict steady-state solutions around complex, parametrically defined geometries on non-parametrically-defined meshes, with model predictions obtained in less than a second. The incorporation of random Fourier features greatly enhanced prediction and generalization accuracy for both approaches. DVH models have more trainable weights than a similar DV-MLP model, but an efficient batch-by-case training method allows DVH to be trained in a similar amount of time as DV-MLP. A vehicle aerodynamics test problem is chosen to assess the method’s feasibility. Both methods exhibit promising potential as viable options for surrogate modeling, being able to process snapshots of data that correspond to different mesh topologies.

Algorithmic automatic item generation can be used to obtain large quantities of cognitive items in the domains of knowledge and aptitude testing. However, conventional item models used by template-based automatic item generation techniques are not ideal for the creation of items for non-cognitive constructs. Progress in this area has been made recently by employing long short-term memory recurrent neural networks to produce word sequences that syntactically resemble items typically found in personality questionnaires. To date, such items have been produced unconditionally, without the possibility of selectively targeting personality domains. In this article, we offer a brief synopsis on past developments in natural language processing and explain why the automatic generation of construct-specific items has become attainable only due to recent technological progress. We propose that pre-trained causal transformer models can be fine-tuned to achieve this task using implicit parameterization in conjunction with conditional generation. We demonstrate this method in a tutorial-like fashion and finally compare aspects of validity in human- and machine-authored items using empirical data. Our study finds that approximately two-thirds of the automatically generated items show good psychometric properties (factor loadings above .40) and that one-third even have properties equivalent to established and highly curated human-authored items. Our work thus demonstrates the practical use of deep neural networks for non-cognitive automatic item generation.

Utilizing technology for automated item generation is not a new idea. However, test items used in commercial testing programs or in research are still predominantly written by humans, in most cases by content experts or professional item writers. Human experts are a limited resource and testing agencies incur high costs in the process of continuous renewal of item banks to sustain testing programs. Using algorithms instead holds the promise of providing unlimited resources for this crucial part of assessment development. The approach presented here deviates in several ways from previous attempts to solve this problem. In the past, automatic item generation relied either on generating clones of narrowly defined item types such as those found in language free intelligence tests (e.g., Raven’s progressive matrices) or on an extensive analysis of task components and derivation of schemata to produce items with pre-specified variability that are hoped to have predictable levels of difficulty. It is somewhat unlikely that researchers utilizing these previous approaches would look at the proposed approach with favor; however, recent applications of machine learning show success in solving tasks that seemed impossible for machines not too long ago. The proposed approach uses deep learning to implement probabilistic language models, not unlike what Google brain and Amazon Alexa use for language processing and generation.

Several neural networks have been proposed in the general literature for pattern recognition and clustering, but little empirical comparison with traditional methods has been done. The results reported here compare neural networks using Kohonen learning with a traditional clustering method (K-means) in an experimental design using simulated data with known cluster solutions. Two types of neural networks were examined, both of which used unsupervised learning to perform the clustering. One used Kohonen learning with a conscience and the other used Kohonen learning without a conscience mechanism. The performance of these nets was examined with respect to changes in the number of attributes, the number of clusters, and the amount of error in the data. Generally, the K-means procedure had fewer points misclassified while the classification accuracy of neural networks worsened as the number of clusters in the data increased from two to five.

Component-based approaches have been regarded as a tool for dimension reduction to predict outcomes from observed variables in regression applications. Extended redundancy analysis (ERA) is one such component-based approach which reduces predictors to components explaining maximum variance in the outcome variables. In many instances, ERA can be extended to capture nonlinearity and interactions between observed and components, but only by specifying a priori functional form. Meanwhile, machine learning methods like neural networks are typically used in a data-driven manner to capture nonlinearity without specifying the exact functional form. In this paper, we introduce a new method that integrates neural networks algorithms into the framework of ERA, called NN-ERA, to capture any non-specified nonlinear relationships among multiple sets of observed variables for constructing components. Simulations and empirical datasets are used to demonstrate the usefulness of NN-ERA. The conclusion is that in social science datasets with unstructured data, where we expect nonlinear relationships that cannot be specified a priori, NN-ERA with its neural network algorithmic structure can serve as a useful tool to specify and test models otherwise not captured by the conventional component-based models.

Neural Network models are commonly used for cluster analysis in engineering, computational neuroscience, and the biological sciences, although they are rarely used in the social sciences. In this study we compare the classification capabilities of the 1-dimensional Kohonen neural network with two partitioning (Hartigan and Späth k-means) and three hierarchical (Ward's, complete linkage, and average linkage) cluster methods in 2,580 data sets with known cluster structure. Overall, the performance of the Kohonen networks was similar to, or better than, the performance of the other methods.

The popularity of green, social and sustainability-linked bonds (GSS bonds) continues to rise, with circa US$939 billion of such bonds issued globally in 2023. Given the rising popularity of ESG-related investment solutions, their relatively recent emergence, and limited research in this field, continued investigation is essential. Extending non-traditional techniques such as neural networks to these fields creates a good blend of innovation and potential. This paper follows on from our initial publication, where we aim to replicate the S&P Green Bond Index (i.e. this is a time series problem) over a period using non-traditional techniques (neural networks) predicting 1 day ahead. We take a novel approach of applying an N-BEATS model architecture. N-BEATS is a complex feedforward neural network architecture, consisting of basic building blocks and stacks, introducing the novel doubly residual stacking of backcasts and forecasts. In this paper, we also revisit the neural network architectures from our initial publication, which include DNNs, CNNs, GRUs and LSTMs. We continue the univariate time series problem, increasing the data input window from 1 day to 2 and 5 days respectively, whilst still aiming to predict 1 day ahead.

With the emerging developments in millimeter-wave/5G technologies, the potential for wireless Internet of things devices to achieve widespread sensing, precise localization, and high data-rate communication systems becomes increasingly viable. The surge in interest surrounding virtual reality (VR) and augmented reality (AR) technologies is attributed to the vast array of applications they enable, ranging from surgical training to motion capture and daily interactions in VR spaces. To further elevate the user experience, and real-time and accurate orientation detection of the user, the authors proposes the utilization of a frequency-modulated continuous-wave (FMCW) radar system coupled with an ultra-low-power, sticker-like millimeter-wave identification (mmID). The mmID features four backscattering elements, multiplexed in amplitude, frequency, and spatial domains. This design utilizes the training of a supervised learning classification convolutional neural network, enabling accurate real-time three-axis orientation detection of the user. The proposed orientation detection system exhibits exceptional performance, achieving a noteworthy accuracy of 90.58% over three axes at a distance of 8 m. This high accuracy underscores the precision of the orientation detection system, particularly tailored for medium-range VR/AR applications. The integration of the FMCW-based mmID system with machine learning proves to be a promising advancement, contributing to the seamless and immersive interaction within virtual and augmented environments.

Machine vision–based herbicide applications relying on object detection or image classification deep convolutional neural networks (DCNNs) demand high memory and computational resources, resulting in lengthy inference times. To tackle these challenges, this study assessed the effectiveness of three teacher models, each trained on datasets of varying sizes, including D-20k (comprising 10,000 true-positive and true-negative images) and D-10k (comprising 5,000 true-positive and true-negative images). Additionally, knowledge distillation was performed on their corresponding student models across a range of temperature settings. After the process of student–teacher learning, the parameters of all student models were reduced. ResNet18 not only achieved higher accuracy (ACC ≥ 0.989) but also maintained higher frames per second (FPS ≥ 742.9) under its optimal temperature condition (T = 1). Overall, the results suggest that employing knowledge distillation in the machine vision models enabled accurate and reliable weed detection in turf while reducing the need for extensive computational resources, thereby facilitating real-time weed detection and contributing to the development of smart, machine vision–based sprayers.

Guaranteed minimum accumulation benefits (GMABs) are retirement savings vehicles that protect the policyholder against downside market risk. This article proposes a valuation method for these contracts based on physics-inspired neural networks (PINNs), in the presence of multiple financial and biometric risk factors. A PINN integrates principles from physics into its learning process to enhance its efficiency in solving complex problems. In this article, the driving principle is the Feynman–Kac (FK) equation, which is a partial differential equation (PDE) governing the GMAB price in an arbitrage-free market. In our context, the FK PDE depends on multiple variables and is difficult to solve using classical finite difference approximations. In comparison, PINNs constitute an efficient alternative that can evaluate GMABs with various specifications without the need for retraining. To illustrate this, we consider a market with four risk factors. We first derive a closed-form expression for the GMAB that serves as a benchmark for the PINN. Next, we propose a scaled version of the FK equation that we solve using a PINN. Pricing errors are analyzed in a numerical illustration.

In a Model Predictive Control (MPC) setting, the precise simulation of the behavior of the system over a finite time window is essential. This application-oriented benchmark study focuses on a robot arm that exhibits various nonlinear behaviors. For this arm, we have a physics-based model with approximate parameter values and an open benchmark dataset for system identification. However, the long-term simulation of this model quickly diverges from the actual arm’s measurements, indicating its inaccuracy. We compare the accuracy of black-box and purely physics-based approaches with several physics-informed approaches. These involve different combinations of a neural network’s output with information from the physics-based model or feeding the physics-based model’s information into the neural network. One of the physics-informed model structures can improve accuracy over a fully black-box model.

High-cardinality categorical features are pervasive in actuarial data (e.g., occupation in commercial property insurance). Standard categorical encoding methods like one-hot encoding are inadequate in these settings.

In this work, we present a novel Generalised Linear Mixed Model Neural Network (“GLMMNet”) approach to the modelling of high-cardinality categorical features. The GLMMNet integrates a generalised linear mixed model in a deep learning framework, offering the predictive power of neural networks and the transparency of random effects estimates, the latter of which cannot be obtained from the entity embedding models. Further, its flexibility to deal with any distribution in the exponential dispersion (ED) family makes it widely applicable to many actuarial contexts and beyond. In order to facilitate the application of GLMMNet to large datasets, we use variational inference to estimate its parameters—both traditional mean field and versions utilising textual information underlying the high-cardinality categorical features.

We illustrate and compare the GLMMNet against existing approaches in a range of simulation experiments as well as in a real-life insurance case study. A notable feature for both our simulation experiment and the real-life case study is a comparatively low signal-to-noise ratio, which is a feature common in actuarial applications. We find that the GLMMNet often outperforms or at least performs comparably with an entity-embedded neural network in these settings, while providing the additional benefit of transparency, which is particularly valuable in practical applications.

Importantly, while our model was motivated by actuarial applications, it can have wider applicability. The GLMMNet would suit any applications that involve high-cardinality categorical variables and where the response cannot be sufficiently modelled by a Gaussian distribution, especially where the inherent noisiness of the data is relatively high.

Aiming at alleviating the adverse influence of coupling unmodeled dynamics, actuator faults and external disturbances in the attitude tracking control system of tilt tri-rotor unmanned aerial vehicle (UAVs), a neural network (NN)-based robust adaptive super-twisting sliding mode fault-tolerant control scheme is designed in this paper. Firstly, in order to suppress the unmodeled dynamics coupled with the system states, a dynamic auxiliary signal, exponentially input-to-state practically stability and some special mathematical tools are used. Secondly, benefiting from adaptive control and super-twisting sliding mode control (STSMC), the influence of the unexpected chattering phenomenon of sliding mode control (SMC) and the unknown system parameters can be handled well. Moreover, NNs are employed to estimate and compensate some unknown nonlinear terms decomposed from the system model. Based on a decomposed quadratic Lyapunov function, both the bounded convergence of all signals of the closed-loop system and the stability of the system are proved. Numerical simulations are conducted to demonstrate the effectiveness of the proposed control method for the tilt tri-rotor UAVs.

Physics-informed neural networks (PINNs), which are a recent development and incorporate physics-based knowledge into neural networks (NNs) in the form of constraints (e.g., displacement and force boundary conditions, and governing equations) or loss function, offer promise for generating digital twins of physical systems and processes. Although recent advances in PINNs have begun to address the challenges of structural health monitoring, significant issues remain unresolved, particularly in modeling the governing physics through partial differential equations (PDEs) under temporally variable loading. This paper investigates potential solutions to these challenges. Specifically, the paper will examine the performance of PINNs enforcing boundary conditions and utilizing sensor data from a limited number of locations within it, demonstrated through three case studies. Case Study 1 assumes a constant uniformly distributed load (UDL) and analyzes several setups of PINNs for four distinct simulated measurement cases obtained from a finite element model. In Case Study 2, the UDL is included as an input variable for the NNs. Results from these two case studies show that the modeling of the structure’s boundary conditions enables the PINNs to approximate the behavior of the structure without requiring satisfaction of the PDEs across the whole domain of the plate. In Case Study (3), we explore the efficacy of PINNs in a setting resembling real-world conditions, wherein the simulated measurment data incorporate deviations from idealized boundary conditions and contain measurement noise. Results illustrate that PINNs can effectively capture the overall physics of the system while managing deviations from idealized assumptions and data noise.

The market for green bonds, and environmentally aligned investment solutions, is increasing. As of 2022, the market of green bonds exceeded USD 2 trillion in issuance, with India, for example, having issued its first-ever sovereign green bonds totally R80bn (c.USD1bn) in January 2023. This paper lays the foundation for future papers and summarises the initial stages of our analysis, where we try to replicate the S&P Green Bond Index (i.e. this is a time series problem) over a period using non-traditional techniques. The models we use include neural networks such as CNNs, LSTMs and GRUs. We extend our analysis and use an open-source decision tree model called XGBoost. For the purposes of this paper, we use 1 day’s prior index information to predict today’s value and repeat this over a period of time. We ignore for example stationarity considerations and extending the input window/output horizon in our analysis, as these will be discussed in future papers. The paper explains the methodology used in our analysis, gives details of general underlying background information to the architecture models (CNNs, LSTMs, GRUs and XGBoost), as well as background to regularisation techniques specifically L2 regularisation, loss curves and hyperparameter optimisation, in particular, the open-source library Optuna.

Antisocial behaviour arises from a complex interplay of innate and environmental factors, with the brain’s adaptability to shifting environmental demands playing a pivotal role. An important but scantly studied environmental factor – micro-geographic hot spots of crime – covers a broad array of problems that produce frequent triggers for antisocial behaviour. Despite the established influence of neural substrates and various environmental factors on antisocial behaviour, the impact of residing in high-risk, violent crime hot spots in Israel, as well as other global locales, remains understudied. This paper aims to elucidate the intricate interplay between neurobiological mechanisms and crime hot spots in the context of antisocial behaviour. Its objectives are twofold: first, to acquaint researchers with the existing literature on the subject; and second, to catalyse further research and robust discourse in this domain. The article commences by reviewing the behavioural manifestations of antisocial tendencies within the framework of crime hot spots. Subsequently, it delves into the influence of crime hot spots on neurocognitive substrates, particularly emphasizing their impact on developmental trajectories associated with antisocial tendencies and the expression of antisocial behaviours. In closing, the paper offers implications and conclusions pertinent to crime hot spots in Israel.

Given the peculiarly linguistic approach that contemporary philosophers use to apply St. Thomas Aquinas’s arguments on the immateriality of the human soul, this paper will present a Thomistic-inspired evaluation of whether artificial intelligence/machine learning (AI/ML) chatbots’ composition and linguistic performance justify the assertion that AI/ML chatbots have immaterial souls. The first section of the paper will present a strong, but ultimately crucially flawed argument that AI/ML chatbots do have souls based on contemporary Thomistic argumentation. The second section of the paper will provide an overview of the actual computer science models that make artificial neural networks and AI/ML chatbots function, which I hope will assist other theologians and philosophers writing about technology, The third section will present some of Emily Bender’s and Alexander Koller’s objections to AI/ML chatbots being able to access meaning from computational linguistics. The final section will highlight the similarities of Bender’s and Koller’s argument to a fuller presentation of St. Thomas Aquinas’s argument for the immateriality of the human soul, ultimately arguing that the current mechanisms and linguistic activity of AI/ML programming do not constitute activity sufficient to conclude that they have immaterial souls on the strength of St. Thomas’s arguments.

Flares on the Sun are often associated with ejected plasma: these events are known as coronal mass ejections (CMEs). These events, although are studied in detail on the Sun, have only a few dozen known examples on other stars, mainly detected using the Doppler-shifted absorption/emission features in Balmer lines and tedious manual analysis. We present a possibility to find stellar CMEs with the help of high-resolution solar spectra.