I. Introduction
While the metaverse hype may be over, some of the major technologies behind it, so-called immersive technologies, are still heavily invested in by big technology companies like Meta and Apple. A recent study by the Rathenau Instituut shows these technologies are accompanied by considerable human rights risks.Footnote 1 Partly based on the study, this article aims to raise awareness and understanding among business and human rights scholars about the characteristics of these technologies, technological trends and human rights risks involved. Early interventions are needed to avoid missteps made during the rise of new technologies such as social media, when benefits were encouraged, while risks were overlooked.
II. Immersive Technology
Immersive technologies allow people to immerse themselves in either a complete virtual environment or enhance the physical world with digital elements. The two predominant technologies are augmented reality (AR) and virtual reality (VR). AR provides a virtual layer over the physical world, like the smartphone game Pokémon Go; VR creates an entirely virtual three-dimensional (3D) environment. Immersive technologies are also known as extended reality (XR) because the physical environment is extended by merging or replacing it with a virtual one (see Figure 1).
Figure 1. Extended reality continuum.
Immersive technologies offer new kinds of experiences that feel real, even if fully or partially virtual. Through sensors, VR headsets, AR glasses and haptic devices, XR is literally more in touch with the skin than are laptops or smartphones. XR engages multiple senses, creating a feeling of ‘presence,’ resulting in lifelike and emotional experiences. Therefore, the distinction between fake and real does not seem apt for describing immersive experiences. More fitting would be to distinguish the physical from (partly) virtual experiences, which can all feel equally real from the user perspective.
A. Market Developments
In 2022, Meta, the company behind Facebook, WhatsApp and Instagram, was the global market leader in VR headsets, with a share of 81 per cent. Chinese DPVR and ByteDance, the company behind TikTok, both had a seven per cent share in 2022, making them the second and third biggest market players.Footnote 2 Apple made its XR debut in February 2024. The number of VR headsets sold in 2022 amounted to 9.1 million worldwide, according to statistical research firm Statista.Footnote 3 By comparison, 1.4 billion smartphones were sold that year.Footnote 4
There is a large gap between the optimistic predictions about the XR market in previous years and its actual size in 2022: $30 billion.Footnote 5 In 2017, Consulting firm McKinsey & Company predicted that the XR market would yield $60 billion in profits by 2022. Earlier, XR analytics firm Digi-Capital estimated it to be $150 billion by then.Footnote 6 Nevertheless, predictions for the future XR market remain substantial: the European Commission quotes predictions of a €800 billion market by 2030. Statista, however, now estimates it to be worth $62 billion in 2029, so predictions vary.Footnote 7
The impact of immersive technologies on human rights and society in general is strongly related to a large-scale consumer breakthrough, which has yet to happen. If it does, however, the human rights risks could be significant, both for consumers and non-consumers.
B. Use Cases
XR is already being applied and experimented with in sectors such as healthcare, training and education, entertainment, infrastructure, industry and art. Use cases include VR police training, trauma therapy, experiments with AR for anatomy lessons, 3D virtual environments for bridge design in international teams and AR assistance for order picking in distribution centres.Footnote 8
These applications build on two main benefits of XR experiences, which stem from the strong feeling of presence and emotion: ‘being’ together despite physical distance, and life-like, seamless experiences through (partly) virtual scenarios. A third advantage often mentioned is efficiency, especially in the logistics sector, for instance, by delineating an optimal route using virtual signage within distribution centres, for employees to efficiently fulfil order picking.Footnote 9 However, available studies on applications often used a small number of participants; therefore, conclusions should be taken with caution, and additional research is needed.
C. Data Collection and Analysis in XR
To understand the impact of XR on human rights, it is crucial to understand the interaction between users, software and devices. The Cybernetic Feedback Loop, a model developed by the Rathenau Instituut, describes this interaction in three steps: data collection, data analysis and application (see Figure 2 for an example).Footnote 10
Figure 2. Cybernetic Feedback Loop.
All XR devices need to collect and analyse data from users and their surroundings to run an immersive experience; that is, to render and constantly adjust virtual elements based on user behaviour. Within the scope of our research, it is impossible to say with certainty what kind of data are collected by current XR devices. However, it was feasible to develop an overview (Table 1) of possible types of analysis and application of physical and behavioural data collected through different XR devices, based on information about sensors on existing XR devices, prototypes in development and existing data processing practices on smartphones and smartwatches.
Table 1. Possible types of physical and behavioural data processed through XR devices
III. Three Technological Trends
Three main trends in the technological development of XR impact human rights: datafication, hyper-personalization and manipulation.
A. Datafication
Immersive technology is accelerating data collection and analysis. More and more information, of an increasingly intimate nature, is being processed. As Table 1 shows, hand movements, pupil reflexes and potentially brain activity can be used to match the XR experience to the physical environment, as well as to infer states of mind, characteristics and preferences.
Besides click behaviour or search history collected from desktop browsers and smartphones, companies behind XR systems can profile users based on information about physical and behavioural activities that is collected predominantly unconsciously, e.g. movements, pupil reflexes, heartbeat, et cetera, can be monitored by measuring systems in XR headsets. Technology companies are also developing (experimental) combinations with brain computer interfaces to make XR experiences more interactive.Footnote 11 Measuring neuroactivities can lead to the processing of even more sensitive types of data, also known as neurodata.
B. Hyper-personalization
The (prototype) XR glasses, Galea, analyse brain and muscle activity. According to the company, the purpose is to measure emotions, stress levels and arousal. As a possible application, Finnish XR company Varjo mentions changing the colour of a virtual environment to alert users of their stressed or fatigued mental state.Footnote 12 Such technological developments contribute to the immersiveness of the virtual world and make user experiences hyper-personal. The integration of generative AI into XR also furthers hyper-personalization, for example, with personalized advertisements in XR environments, or with applications such as Replika, which provides users with a digital ‘friend, partner or mentor’ that tailors itself to each user and with whom users can share AR experiences.Footnote 13 This far-reaching form of personalization can further exacerbate the personal ‘bubbles’ people are already experiencing as a consequence of social media algorithms, and complicate shared perceptions of reality.
C. Manipulation
Finally, technical advancements in XR make it easier to manipulate user experiences in both virtual and physical environments. This can be illustrated with Pokémon Go, which manipulated users’ perception and behaviour. The AR game managed to move players from public spaces to commercial locations in the physical world by planting game components there, nudging them to spend money. User movement in public spaces sometimes causes public nuisances due to the gathering of many Pokémon Go players, and causes harm when people are playing the game while driving.Footnote 14
IV. Risks for Human Rights
States have obligations to protect XR users and non-users against arbitrary human rights interferences by XR companies, which in turn must respect human rights. To effectively discuss the rights most affected by the technological characteristics and trends of XR, they are clustered below around four important values: privacy, autonomy, non-discrimination and a clean and healthy environment.
A. Privacy
Article 12 of the Universal Declaration of Human Rights (UDHR) covers the rights to private life and correspondence, freedom from surveillance and data protection. Although certain types of data processing are necessary for XR to function, the amount of data currently collected and the sensitive information that can (potentially) be derived from it are not justified from a privacy rights perspective.
First, some of the big technology companies developing XR base their revenue models on data analysis and advertisements. This incentivizes them to keep adding new sensors to the already large variety their devices contain, resulting in data processing on a much larger scale beyond that of current smartphones.
Second, as illustrated by Table 1, an increasing amount of intimate information is or could be derived from the data collection, compromising subjects’ privacy. Data thought to be anonymous proves to be far from it in practice. For instance, a 2023 study showed that based on five minutes of head and hand motion collected by VR devices, a user could be uniquely identified with 94.33% accuracy in less than two minutes, out of a pool of more than 55,000 users. With this, the scientists demonstrated VR biometrics’ comparability to facial recognition and fingerprint identification systems.Footnote 15
Other research shows physical and behavioural activities that are monitored, for example, measuring eye-tracking, can be linked to age, gender, ethnicity, sexual preference and medical diagnoses such as attention deficit hyperactivity disorder (ADHD) and autism.Footnote 16
Third, collected data may be used for other purposes than initially collected, creating ‘function creep.’ Examples from current widely used technologies illustrate the risk of function creep when data are collected. For example, the app Muslim Pro reminded users when to pray and showed them Mecca’s direction based on their location data, but then the US military accessed and used these data to inform special operations abroad.Footnote 17 Companies that process more and more information should realize that the more data that is collected, the higher the aforementioned risks are.
Lastly, risks that apply to users of immersive technologies may also apply to non-users, since sensors of immersive devices also analyse the user’s surroundings.
B. Autonomy
Human rights instruments protect both physical and mental autonomy. More generally, Article 3 of the UDHR refers to the right to life, liberty and security (or inviolability) of a person. Through intuitive and covert influence, XR can be used to manipulate users, putting pressure on their mental privacy and self-determination.
According to visual information researcher Bibri and sustainability researcher Allam, the combination of digital technology and biotechnology makes it increasingly possible to predict people’s thoughts, preferences and behaviour, and thus how they respond to certain cues.Footnote 18 This means the sensitive data and inferences discussed above could be used to hyper-personalize (partly) virtual environments and nudge users towards consumption, but also to influence their ideas and views, for instance via computer-generated avatars.Footnote 19
C. Non-discrimination
Article 2 of the UDHR states that everyone is entitled to the rights and freedoms set out in the Declaration, without distinction. Examples of protected discrimination grounds are race, sex, language, religion and political opinion.
First, widespread adoption of immersive technologies could amplify the risk of digitalization in general: the growing ‘digital divide,’ a widening gap between those who benefit from digital technology and those who do not, because they have unequal access to the technology, or are impacted more by its drawbacks.
Secondly, online harassment can be exacerbated. Harassment of groups already considered marginalized in gaming and VR, such as ethnic minorities, women and the LGBTQIA+ community, is an emerging (but understudied) problem. It ranges from racist and sexist comments to virtual violence and virtual touching without consent. In a study based on personal experiences of participants, it is noted that, unlike in other (two-dimensional [2D]) online social spaces, because of the focus on embodiment and immersion, these behaviours in social VR resemble offline harassment.Footnote 20
Thirdly, XR devices and environments can be non-inclusive. For example, cybersickness appears to be experienced considerably more by women than men, possibly because XR headsets are designed for a male target group, which has a wider eye distance.Footnote 21 Furthermore, XR experiences sometimes lack diversity in avatar options, making it difficult for underrepresented users to translate their identity through virtual visual cues (such as wheelchairs or accurate portrayals of race). This could even lead to minorities renouncing XR, furthering the lack of representation and the digital divide.Footnote 22 Currently, insufficient research exists on how discrimination manifests itself in XR. An ethnographic study that included interviews with industry experts showed more knowledge is needed to design safe, accessible and inclusive immersive environments.Footnote 23
D. Clean and Healthy Environment
In a general comment from 2019, the International Covenant on Civil and Political Rights (ICCPR) stated that ‘environmental degradation, climate change and unsustainable development constitute some of the most pressing and serious threats to the ability of present and future generations to enjoy the right to life,’ as protected by Article 6 of the ICCPR and Article 3 of the UDHR.Footnote 24 In 2022, the United Nations General Assembly adopted a resolution encompassing ‘the human right to a clean, healthy and sustainable environment.’Footnote 25
The data collection and processing for XR requires energy from data centres. Large-scale XR use will lead to the creation of many more data centres. Some innovators have claimed the opposite by speculating that XR applications could reduce travel CO2 emissions by improving remote collaboration, or replace part of the pollution of the fashion industry with a digital, more sustainable one.Footnote 26 But whether this outweighs the impact of additional data centres is unclear and needs to be studied.
V. Conclusion
VR and AR technologies are ‘immersive’ because of the life-like experience they offer, and the emotion and feeling of presence they invoke, resulting in an extension of reality. This makes XR suitable for all kinds of applications, ranging from entertainment to training to logistics. Big technology companies, including Meta and Apple, are investing heavily, with some predicting that these technologies will see a mass consumer breakthrough and will replace smartphones in the future. However, as this article has shown, the characteristics of XR together with the technological trends of datafication, hyper-personalization and manipulation also bring about significant risks for human rights, including risks to privacy, autonomy, non-discrimination and a clean and healthy environment.
This article has shown the urgency for governments to fulfil their duty to protect and companies to fulfil their responsibility to respect the rights of both XR users and non-users. The first harmful uses are already being witnessed. Technological developments are heading towards increased data processing, virtual environments are becoming hyper-personalized and more and more advanced manipulation techniques can be deployed. But a large-scale consumer uptake of XR has yet to come, which makes this the ideal time for policymakers and developers to take action and determine how to adjust and redirect the further development of immersive technologies from a human rights perspective.
Financial support
The research on which this paper is based was partly funded by the Dutch Ministry of the Interior and Kingdom Relations.
Competing interests
The Authors declare none.
Open access
