Over the past 15 years, Daniel Howe and Helen Nissenbaum, often working with other collaborators, have launched a series of projects that leverage obfuscation to protect people’s online privacy. Their first project, TrackMeNot, is a plug-in that runs in the background of browsers, automatically issuing false search queries and thereby polluting search logs, making it more difficult or impossible for search engines to separate people’s true queries from noise (TrackMeNot 2024). Howe and Nissenbaum later turned to online behavioural advertising, developing AdNauseam, another browser plug-in that automatically clicks on all ads. It is designed to obfuscate what people are actually interested in by suggesting – via indiscriminate, automatic clicks – that people are interested in everything (AdNauseam 2024).

Each of these projects has been accompanied by academic publications describing the teams’ experiences developing the tools but also reflecting on the value of and normative justification for obfuscation (Howe and Nissenbaum Reference Howe and Nissenbaum2017; Nissenbaum and Howe Reference Nissenbaum, Howe, Kerr, Lucock and Steeves2009). Among the many observations that they make in these papers, Howe and Nissenbaum conclude their article on AdNauseam by remarking that “trackers want us to remain machine-readable … so that they can exploit our most human endeavors (sharing, learning, searching, socializing) in pursuit of profit” (Howe and Nissenbaum Reference Howe and Nissenbaum2017). Online trackers don’t just record information but do so in a way that renders humans – their sharing, learning, searching, and socializing online – machine-readable and, as such, computationally accessible. For Howe and Nissenbaum, obfuscation is not only a way to protect people’s privacy and to protest the elaborate infrastructure of surveillance that has been put in place to support online behavioural advertising, it is specifically a way to resist being made machine-readable.

At the time of the paper’s publication, the concept of “machine readability” would have been most familiar to readers interested in technology policy from its important role in advocacy around open government (Yu and Robinson Reference Yu and Robinson2012), where there were growing demands that the government make data publicly available in a format that a computer could easily process. The hope was that the government would stop releasing PDFs of tables of data – from which data had to be manually and laboriously extracted – and instead release the Excel sheets containing the underlying data, which could be processed directly by a computer. “Machine readable” thus became a mantra of an open government movement, in service of the public interest. So why do Howe and Nissenbaum invoke machine-readability, in the context of online behavioural advertising, as a threat rather than a virtue, legitimating the disruptive defiance of obfuscation against an inescapable web of surveillance and classification?

In this paper, we address this question, in two parts, first, by theorizing what it means for humans to be machine-readable and, second, by exposing conditions under which machine-readability is morally problematic – the first, as it were, descriptive, the second, normative. Although the initial encounter with problematic machine-readability was in the context of online behavioural advertising and, as a justification for data obfuscation, our discussion aims for a coherent and useful account of machine-readability that can be decoupled from the practices of online, behavioural advertising. In giving the term greater definitional precision descriptively as well as normatively we seek to contribute to ongoing conversations about being human in the digital age.

6.1 Machine Readability: Top Down

The more established meaning of “machine readable” in both technical and policy discourse applied to material or digital objects that are rendered comprehensible to a machine through structured data, expressed in a standardized format, organized in a systematic manner. Typically, this would amount to characterizing an object in terms of data, in accordance with predefined fields of a database, in order to render them accessible to a computational system. Barcodes stamped on material objects, as mundane as milk cartons, render them machine readable, in this sense. When it comes to electronic objects, there are even different degrees of accessibility; for example, conversion tools can transform photographic images and electronic pdf documents into formats more flexibly accessible to computation according to the requirements of various machines’ data structures. Structure and standardization have been important for data processing because many computational operations cannot function over inconsistent types of inputs and cannot parse the different kinds of details in an unstructured record. In the near past, for example, a computer would not have been able to process a government caseworker’s narrative account of interviews with persons seeking public benefits. Instead, it would have been coded according to discrete fields and predefined sets of possible answers, enabling government agencies to automate the process of determining eligibility. Applied to people, machine readability, in this sense, would mean assigning data representations to them according to the predefined, structured data fields required by a given computational system. The innumerable forms we encounter in daily life, requiring us to list name, age, gender, address, and so forth, are instances of this practice.

If this were all there is to machine readability, it would not seem obviously wrong, nor would it legitimate, let alone valorize, disruptive protest, such as data obfuscation. It does, however, call to mind long-standing concerns over legibility in critical writing on computing. Here, the concept of legibility refers to the representation of people, their qualities, and their behaviours as structured data representations, the latter ultimately to be acted upon by computers. Representing people as data is not a passive act; rather data collection can be understood as an ontological project that defines what exists before seeking to measure it. Scholarship has tended to focus on how legibility is achieved by imposing categories on people which, in the very act of representing them in data, is morally objectionable in ways similar to practices of stereotyping and pigeon-holing. In the realm of computation, perhaps most famously, Bowker and Starr point out how information systems work to make people and populations legible to social and technical systems by subjecting them to rigid classificatory schemes, forcing them into often ill-fitting categories (Bowker and Starr Reference Bowker and Star2000). This wave of critical writing emphasizes the crude ways in which information systems turn a messy world into tidy categories and, in so doing, both elides differences that deserve recognition and asserts differences (e.g. in racial categories) where they are unwarranted by facts on the ground (Agre Reference Agre1994; Bowker and Starr Reference Bowker and Star2000). Scholars like Bowker and Starr were keenly aware of the importance of structured information systems representing humans and human activity for the purposes of subsequent computational analysis.

Critiques such as Bowker and Starr’s echo critical views of bureaucracies that go farther back in time, emphasizing the violence that their information practices inflict on human beings by cramming them into pigeonholes, blind to the complexities of social life (Scott Reference Scott2020). The legacy of much of this work is a deeper appreciation of the indignities of legibility that is achieved through information systems, which have been drawn with top-down, bright lines. Incomplete, biased, inaccurate, and weighted by the vested interests of the people and institutions who wield them (see Agre Reference Agre1994), these systems are also sources of inconvenience for humans who may have to contort themselves in order to be legible to them. Echoing these critiques, David Auerbach has observed that making oneself machine readable has historically demanded significant compromise, not just significant effort: “Because computers cannot come to us and meet us in our world, we must continue to adjust our world and bring ourselves to them. We will define and regiment our lives, including our social lives and our perceptions of ourselves, in ways that are conducive to what a computer can ‘understand.’ Their dumbness will become ours” (Auerbach Reference Auerbach2012). During this period, the terms in which we could make ourselves legible to computers were frequently so limited that achieving legibility meant sacrificing a more authentic self for the sake of mere recognition.

6.2 Machine Readability: A New Dynamic

While the operative concerns of this early work on legibility remain relevant, they do not fully account for the perils of machine readability as manifested in online behavioural advertising and similar practices of the present moment. If we agree that automation via digital systems requires humans to be represented as data constructs and that top-down, inflexible, possibly biased and prejudicial categories undermine human dignity and well-being, we may welcome new forms of data absorption and analytics that utilize increasingly powerful techniques of machine learning and AI. Why? To answer, we consider ways that they radically shift what it means to make people machine readable – the descriptive task – and how this new practice, which we call dynamic machine readability, affects the character of its ethical standing. To articulate this concept of machine readability, we provide a sketch – a caricature rather than a scientifically accurate picture (for which we beg our reader’s forbearance) – intended to capture and explain key elements of the new dynamic.

We begin with the bare bones: a human interacting with a machine. Less mysteriously, think of it as any of the myriad of computational systems, physically embodied or otherwise, that we regularly encounter, including websites, apps, digital services, or devices. The human in question may be interacting with the system in one of innumerable ways, for example, filing a college application, signing up for welfare, entering a contest, buying shoes, sending an email, browsing the Web, playing a game, posting images on social media, assembling a music playlist, creating a voice memo, and so on. In the approach we labeled “top down,” the machine reads the human via data that are generated by the interaction, typically, but not always, provided by the human as input that is already structured by the system. Of course, the structured data through which machines read humans may also be entered by other humans, for example, a clerk or a physician entering data into an online tax or health insurance form, respectively. To make the human legible, the data are recorded in an embedded classification scheme, which typically may also trigger an appropriate form of response – whether directly by the machine (e.g. an email sent) or a human-in-the-loop who responds accordingly (e.g. an Amazon warehouse clerk assembles and dispatches a package).

Keyboard and mouse, the dominant data entry medium of early days, have been joined by others, such as sound, visual images, or direct behavioural monitoring, limited by predefined fields for input data fields. The new dynamicFootnote ¹ accordingly also involves humans and machines engaging in interaction, either directly or indirectly. Thus, it allows a vastly expanded set of input modalities, beyond the keyboard and mouse of a standard computer setup involving data entry through alpha-numerics and mouse-clicks. Machines may capture and record the spoken voice, facial and other biometrics, a myriad of non-semantic sensory data generated by mobile devices, and streamed behavioural data, through active engagement or passively in the background (e.g. watching TV and movies on a streaming service).

The new dynamic also incorporates machine learning (ML) models, or algorithms, which are key to making sense of this input. Machines capture and record “raw” data inputs while the embedded models transform them into information that the system requires in order to perform its tasks, which may involve inferring facts about the individual, making behavioural predictions, deriving intentions, or surmising preferences, propensities, and even vulnerabilities. (Although we acknowledge readers who prefer the plain language of probabilities, we adopt – admittedly – anthropomorphizing terms, such as infer and surmise, which are more common.) Instead of structuring input in terms of pre-ordained categories, this version of machine reading accepts streams of data, which are structured and interpreted by embedded ML models. For example, from the visual data produced by a finger pad pressing on the glass of a login screen, identity is inferred, and a mobile device allows the owner of the finger to access its system. From the streams of data generated by sensors embedded in mobile devices, ML models infer whether we are running, climbing stairs, or limping (Nissenbaum Reference Nissenbaum2019), or, whether we are at home or at a medical clinic, and whether we are happy or depressed.

A transition to dynamic machine readability of humans means that it is unnecessary to make ourselves (and others) legible to computational systems in the formal languages that machines had been programmed to read. The caseworker we mentioned earlier may leapfrog the manual work of slotting as many details as possible into the discrete fields of a database and, instead, record the full narrative account (written or spoken) of meetings with their benefits-seeking clients. Language models, according to proponents, trained to extract relevant substantive details, would be able to parse these narratives, extract relevant information, and even, potentially, automate the decision-making itself.

A critical element of our high-level sketch has not yet been revealed, namely, key parties responsible for the existence of the systems we have been discussing – their builders (designers, software engineers, etc.) and their owners or operators (which may be their creators) or companies and other organizations for whom the systems have been developed. When attributing to ML models a capacity to make sense of a cacophony of structured and unstructured data, specifically, to read the humans with whom a system interacts, one must, simultaneously, bring to light the purposes behind the sense-making in turn dictated by the interests and needs of controlling parties, including its developers and owner-operators. To serve the purposes of online advertising (highly simplistically), for example, a model must be able to read humans who land on a given webpage as likely (or unlikely) to be interested in a particular ad. Moreover, making sense of humans through browsing histories and demographics, for example, according to the dynamic version of machine readability, does not require the classification of human actors in terms of human comprehensible properties, such as, “is pregnant,” or even in terms of marketing constructs, such as “white picket fence” (Nissenbaum Reference Nissenbaum2019). Instead, the concepts derived by ML models may be tuned entirely to their operational success as determined by how likely humans are to demonstrate interest in a particular range of products (however that is determined).

Advertising, obviously, is not the only function that may be served by reading humans in these ways. Although lacking insider access, one may suppose that they could serve other functions just as well, such as, helping private health insurance providers determine whether applicants are desirable customers and, if yes, what premiums they should be charged – not by knowing or inferring a diagnosis of, say, “early stage Parkinson’s disease” but by reading them as “desirable clients of a particular health plan.” Importantly, a model that has been tuned to serve profitable advertisement placement is different from one that has been tuned to the task of assessing the attractiveness of an insurance applicant.Footnote ² It is worth noting that machines reading humans in these functional terms may or may not be followed by a machine automatically executing a decision or an action on its grounds. Instances of the former include online, targeted advertising, and innumerable recommender systems, and, of the latter, human intervention in decisions to interview job applicants on the basis of hiring algorithms, or to award a mortgage on the basis of a credit score, etc. Earlier in this section, when we reported on natural language models that could extract relevant data from a narrative, we ought to have added that relevance itself (or efficacy, for that matter), a relational notion, is always tied to purposes. Generally, how machines read humans only makes sense in relation to the purposes embedded in them by operators and developers. The purposes themselves, of course, are obvious targets of ethical scrutiny.

Implicit in what we have described, thus far, is the dynamic nature of what we have labeled dynamic machine readability. ML models of target attributes, initially derived from large datasets, may continuously be updated on the basis of their performance. Making people legible to functional systems, oriented around specific purposes, is not a static, one-off business. Instead, systems are constantly refined on the basis of feedback from successive rounds of action and outcome. This means that, to be successful, dynamic approaches to reading humans must engage in continuous cycles of purpose-driven classification and, subsequently, modification based on outcomes. It explains why they are insatiably hungry for data – structured and unstructured – which may be collected unobtrusively as machines may monitor humans simply as they engage with the machines in question, to learn whether a particular advertisement yields a click from a particular individual, or a recommendation yields a match, and so on. Dynamic refinement, according to proponents, may be credited with their astonishing successes but also, we contend, a potential source of unethical practice.

To recap: dynamic machine readability is characterized: by an expansion of data input modalities and data types (structured and unstructured, semantic and non-semantic); by embedded ML models which are tuned to the purposes of machine operators and owners; and by the capacity of these models to be continuously refined in relation to these purposes. Dynamic machine readability releases us from the shackles of a limited lexicon – brittle and ill-fitting categories – and associated ethical issues. In a growing number of cases, the pressing concern is no longer whether we have to submit to a crass set of categories in order to be legible to computational systems; instead, many of the computational systems with which we interact take in massive pools of data of multifarious types and from innumerable sources, presumably, to read us as we are. Despite the scale and scope of the data and the power of ML, reading humans through models embedded in machines is constrained by the purposes laid out by machine operators and developers, which these models are designed to operationalize. From a certain perspective, the new dynamic is emancipatory. Yet, even if successful these model-driven machines raise a host of persistent ethical questions, which we reveal through a sequence of cases involving machines reading humans. Inspired by real and familiar systems out in the world whose functionality depends on making humans readable, we identified cases we considered paradigmatic in the types of ethical issues that machine readability raises. It turns out that, although the particular ways that a system embodies machine readability is relevant to its moral standing, moral standing depends on other elements of the larger system in which the human-reading subsystems are embedded.

6.3 Through the Lens of Paradigmatic Cases: An Ethical Perspective on Machine Readability

6.4 Pulling Threads Together

Machine-readable humanity is an evocative idea, whose initial impact may be to stir alarm, possibly even repulsion or indignation. Beyond these initial reactions, however, does it support consistent moral appraisal in one direction or another? “It depends” may be an unsurprising answer but it begs further explanation on at least two fronts: one, an elaboration of machine-readability to make it analytically useful, and another, an exploration of the conditions under which machine-readability is morally problematic (and when it is not). Addressing the first, we found it useful to draw a rough line between two relevant developmental phases of digital technologies, to which we attributed distinct but overlapping sets of moral problems, respectively. One, often associated with critical discussions of the late twentieth century, stems from the need to represent humans (and other material objects) in terms of top-down, predefined categories, in order to place them in databases, in turn making them amenable to the computational systems of the day. As discussed in Section 6.1, significant ethical critiques honed on the dehumanizing effects of forcing humans into rigid categories, which, as with any form of stereotyping and pigeon-holing, may mean that similar people are treated differently, and different people are lumped together without regard for significant differences. In some circumstances, it could be argued that well designed classification schemes serve positive values, such as efficient functioning, security, and fair treatment but it’s not difficult to see how the classification of humans into preordained categories could often lead to bias (or unfair discrimination), privacy violations, authoritarian oversight, and prejudice. In short, an array of harms may be tied, specifically, to making humans readable to machines by formatting them, as it were, in terms of information cognizable by computational systems.

Machine readability took on a different character, which we signaled with the term dynamic, in the wake of the successive advances of data and predictive analytics (“big data”), machine learning, deep learning, and AI. Although it addresses problems of “lumping people together” associated with top-down readability, ironically, its distinctive power to mass produce individualized readings of humanity introduces a new set of ethical considerations. The list we offer here, by no means exhaustive, came to us through the cases we analyzed seen through the lens of characteristic elements of a dynamic setup.

To begin, the broadening of data input modalities, about which promoters of deep learning are quick to boast, highlights two directions of questioning. One challenges whether all the data is relevant for the legitimate purposes that the model is claimed to serve (e.g. increasing the speed with which an IVR addresses a caller’s needs) or whether it is not (e.g., learning characteristics of callers that lead to unfair discrimination or violations of privacy) (Nissenbaum Reference Nissenbaum2009; Noble Reference Noble2018).

A second direction slices a different path through the issue of data modalities – in this instance, not about categories of data, such as race, gender, and so on, but about different streams feeding into the data pool. Of particular interest is the engagement of the human data subject (for lack of a better term), which is evident in personalized recommender systems. The Netflix case drew our attention because, over the years, it has altered course in how it engages subscribers in its recommender algorithm, a pendulum swinging from full engagement as choosers to no engagement to, at the present time, presumably somewhere in between. Similarly, in our IVR case, we noted that the powerful language processing algorithms that are able to grasp the meaning of spoken language and read and serve human-expressed intention are able to extract other features as well – unintended or against our will. Finally, in the context of behavioural advertising, paradigmatic of the dominant business model of the past three decades, the modality of recorded behaviour absent any input from expressed preference has prevailed in the reading of humanity by respective machines (Tae and Whang Reference Tae and Whang2021; Zanger-Tishler et al. Reference Zanger-Tishler, Nyarko and Goel2024).

In order to defend the legitimacy of including different modalities of input into the datasets from which models are extracted, an analysis would require a consideration of each of these streams – deliberate, expressed preference, behavioural, demographic, biometric, etc. – in relation to each of the cases, respectively. Although doing so, here, is outside the scope of this chapter, it is increasingly urgent to establish such practices as new ways to read humans are being invented, for example, in the growing field of so-called digital biomarkers (Adler et al. Reference Adler, Wang, Mohr, Estrin, Livesey and Choudhury2022; Coravos et al. Reference Coravos, Khozin and Mandl2019; Daniore et al. Reference Daniore, Nittas, Haag, Bernard, Gonzenbach and von Wyl2024), which are claimed to be able to make our mental and emotional states legible through highly complex profiles of sensory data from mobile devices (Harari and Gosling Reference Harari and Gosling2023). Another wave of access involves advanced technologies of brain–machine connection, which claims yet another novel modality for reading humans – our behaviours, thoughts, and intentions – through patterns of neurological activity (Duan et al Reference Duan, Zhou, Wang, Wang and Lin2023; Farahany Reference Farahany2023; Tang et al. Reference Tang, LeBel, Jain and Huth2023).

In enumerating ethical considerations, such as privacy, bias, and political freedom, we have skirted around, but not fully and directly confronted the assault on human autonomy, which ultimately may be the deepest, most distinctive issue for machine readable humanity. Acknowledging that the concept of autonomy is enormously rich and contested, we humbly advance its use, here, as roughly akin to self-determination inspired by the Kantian exhortation introduced in most undergraduate ethics courses,Footnote ⁸ to “act in such a way that you treat humanity whether in your own person or in the person of any other, never merely as a means to an end, but always at the same time as an end” (Kant Reference Kant1993, vii; MacKenzie and Stoljar Reference Mackenzie and Stoljar2000; Roessler Reference Roessler2021). When defending the automation of a given function, system, or institution, defenders cite efficiency, defined colloquially, as producing a desirable outcome with the least waste, expense, effort, or expenditure of resources. Our case of automated phone systems illustrated the point that efficiency for machine owners may produce less desirable outcomes for callers, more wasted time and expenditure of effort. In this relatively unsophisticated case, one may interpret the exploitation of callers as an assault on autonomy.

The expansive class of systems claiming to personalize or customize service (recommendations, information, etc.) illustrates a different assault on autonomy. Among the characteristic elements comprising dynamic machine readability is the dynamic revision of a model in relation to the goals or purposes for which a system was created. The general class of recommender systemsFootnote ⁹ largely reflect a two-sided marketplace because it serves two interested parties (possibly three-sided, if one includes the recommender system itself as an interested party.) The operators of personalized services imply that their systems are tailored to the individual’s interests, preferences, and choices but their performance, in fact, may be optimized for purposes of parties – commercial, political, etc. – seeking to be found or recommended. Purposes matter in other cases, too, specifically distinguishing between facial recognition systems, serving purposes of political repression of machine-readable humans, and mole identification systems, whose primary or sole criterion of success is an accurate medical diagnosis.

6.5 Conclusion: Human Beings as Standing Reserve

Martin Heidegger’s “The Question Concerning Technology” introduces the idea of standing reserve, “Everywhere everything is ordered to stand by, to be immediately at hand, indeed to stand there just so that it may be on call for a further ordering.” According to Heidegger, the essential character of modern technology is to treat nature (including humanity) as standing reserve, “If man is challenged, ordered, to do this, then does not man himself belong even more originally than nature within the standing-reserve?” (Heidegger Reference Heidegger1977, 17). Without defending Heidegger’s broad claim about the nature of technology, the conception of machine-readability that we have developed here triggers an association with standing-reserve, that is to say machine-readability as the transformation of humanity into standing reserve. Particularly evident in dynamic systems, humans are represented in machines as data in order to be readily accessible to the purposes of the controllers (owners, designers, engineers) that are embodied in the machine through the design of the model. The purposes in question may have been selected by machine owners with no consideration for ends or purposes of the humans being read. It is not impossible that goals and values of these humans (and of surrounding societies) are taken into consideration, for example, in the case of machines reading skin lesions; the extent this is so is a critical factor for a moral appraisal. Seen in the light of these arguments, AdNauseam is not merely a form of protest against behavioural profiling by the online advertising establishment. More pointedly, it constitutes resistance to the inexorable transformation of humanity into a standing reserve – humans on standby, to be immediately at hand for consumption by digital machines.

In this chapter I analyze different dilemmas regarding the use of robots to serve humans living in the digital age. I go beyond technical fields of knowledge to address how the design and deployment of carebots is embedded in multifaceted material and discursive configurations implicated in the construction of humanness in socio-technical spaces. Imagining those spaces necessarily entails navigating the “fog of technology,” which is always also a fog of inequality in terms of trying to decipher how the emerging architectures of our digitized lives will interface with pre-existing forms of domination and struggles of resistance premised upon our capacity to dissent. Ultimately, I contend that the absence of a “human nature” makes us human and that absence in turn makes us unpredictable. What it means to be human is thus never a fixed essence but rather must be strategically and empathically reinvented, renamed, and reclaimed, especially for the sake of those on the wrong side of the digital train tracks.

In Section 7.1, I open the discussion by critiquing Mori’s (1970) seminal theory on robot design, called the “uncanny valley,” by inscripting technologies in changing cultural practices and emergent forms of life. Section 7.2, through visual culture, gender, and race theories, sheds light on how the design of carebots can materialize complex dilemmas. In Section 7.3, I dissect Petersen’s (Reference Petersen2007, Reference Petersen, Lin, Abney and Bekey2011) perturbing theory and ethical defense of designing happy artificial people that “passionately” desire to serve. In the final Section 7.4, I offer some final thoughts on what I call the Carebot Industrial Complex, namely, the collective warehousing of aging people in automated facilities populated by carebots.

7.2 Visual Culture, Gender, and Race: Is It Possible to Design “Neutral” Robots?

Visual culture, gender, and race theories have had an extensive and transdisciplinary effect on debates concerning what is means to be human within the changing historical and cultural prisms of intersectionally related forms of inequality and struggles for equitable change. In this section I explore some angles of these theories to shed light on how the design of carebots can materialize complex discursive and symbolic configurations that impinge on the construction of humanness in the existing and emerging socio-technological architectures through which we signify our lives.

Visual culture, as a mode of critical visual analysis that questions disciplinary limitations, speaks of the visual construction of the social rather than the often-mentioned notion of the social construction of the visual. It focuses on the centrality of vision and the visual world in constructing meanings, maintaining esthetic values, as well as racialized, classed, and gendered stereotypes in societies steeped in digital technologies of surveillance and marketing. Visuality itself is understood as the intersection of power with visual representation (Mirzoeff Reference Mirzoeff and Mirzoeff2002; Rogoff Reference Rogoff and Mirzoeff2002).

Feminism and the analysis visual culture mutually inform each other. Feminism, by demanding an understanding of how gender and sexual difference figure in cultural dynamics coextensively with other modes of subjectivity and subjection such as sexual orientation, race, ethnicity, and class, among others, has figured prominently in the strengths of visual culture analysis. And, in turn, “feminism has long acknowledged that visuality (the conditions of how we see and make meaning of what we see) is one of the key modes by which gender is culturally inscribed in Western culture” (Jones Reference Jones2010, 2).Footnote ⁴

Relative to the design of robots, their gendering occurs at the level of the material body and the discursive and semiotic fields that inscript bodies (Balsamo Reference Balsamo1997). To the extent that subject positions “carry differential meanings,” according to de Lauretis, the representation of a robot as male or female is implicated in the meaning effects of how bodies are embedded in the semiotic and discursive formations (Robertson Reference Robertson2010, 4). Interestingly, however, Robertson contends that robots conflate bodies and genders:

Because the way “robot-makers gender their humanoids is a tangible manifestation of their tacit understanding of femininity in relation to masculinity, and vice versa” (Robertson Reference Robertson2010, 4), roboticists are entrenching their reified common-sense knowledge of gender and re-enact pre-existing sexist tropes and dominant stereotypes of gendered bodies without any critical engagement. Thus, despite the lack of physical genitalia, robots possess “cultural genitals” that invoke “gender, such as pink or grey lips” (Robertson Reference Robertson2010, 5).

This process of reification in the design of robots entrenches the mythological bubble of the “natural” in the context of sex/gender and male/female binaries that queer theorist Judith Butler bursts open very effectively by looking at the malleability of the body. The traditional feminist assumptions of gender as social and cultural and sex as biological and physical are recast by Butler, who contends that sex is culturally and discursively produced as pre-discursive. Sex is an illusion produced by gender rather than the stable bedrock of variable gender constructions.

Butler develops a theory of performativity wherein gender is an act, and the doer is a performer expressed in, not sitting causally “behind,” the deed. Performance reverses the traditional relation of identity as preceding expression in favor of performance as producing identity (Butler Reference Butler1990, Reference Butler1993). Gender for Butler thus becomes all drag and the “natural” becomes performative rather than an expression of something “pre-social” and “real.” Gender performances are not an expression of an underlying true identity, but the effects of regulatory fictions where individual “choice” is mediated by relations and discourses of power. In this way, queer theory’s contingent and fluid pluralization of gendered and sexual human practices stands in stark contrast to the fixed conflation of bodies and genders of humanoid robots posed by Robertson.

Considering these debates, is it possible to make gender-neutral robot designs? Even if designers purport to create gender-neutral robots, the robots will inevitably be re-gendered by the people who use them because of the pervasiveness of sexist/stereotyped tropes of femininity and masculinity in society. Re-gendering can occur, for instance, at the obvious level of languages such as romance languages that linguistically gender all things (e.g. from animate to inanimate, human to non-human) into either female or masculine nouns. Insofar as we conceive language as not merely an instrument or means of communication, but rather as constitutive of the very “reality” of which it speaks, we must contend with the gendering of even purportedly gender-neutral incarnations of robots. Moreover, in addition to language-related forms of gendering, it can also occur at the level of the activities performed by the service robots given that they can be culturally and discursively associated with female activities. In the case of caretaking functions that have historically been associated with female and poor labor, it would be unsurprising to see the replication of gendered stereotypes as applied to gender-neutral robots. As a result, any purported semiotic neutrality of robotic design is inevitably re-inscribed in discursive and cultural fields mined with tropes steeped in long histories of sexist and gendered stereotypes.

The raced and classed cultural tensions invoked by foreign caretakers provide further insight concerning the design of “neutral” service robots. Robertson discusses how elderly Japanese people prefer robots to the “sociocultural anxieties provoked by foreign labourers and caretakers” (Robertson Reference Robertson2010, 9). The Japanese perceived that robots did not have the historical baggage and cultural differences of migrant and foreign workers, which ultimately “reinforces the tenacious ideology of ethnic homogeneity” (Robertson Reference Robertson2010, 9). Here the purported neutrality of robots as opposed to racialized and discriminated human caretakers becomes a form of racist erasure and complicit celebration of sameness. Thus, an allegedly semiotically neutral robot design can be culturally embedded in a digitized society in highly contentious ways, simultaneously enacting processes of cultural re-gendering (associated with comforting stereotypes of female care) and racist erasure (associated with the discomforting use of stigmatized minorities in caretaking). Claims to neutrality in robot design are ultimately a discourse of power that must be dealt with cautiously in order to engage with how it reconfigures the positions of people within the techno-social imaginary of the emerging digital society.

The design of carebots must engage more self-reflexively with the problematic replication of gendered, racialized, and other stereotypes in robots, especially given the depth of the gendering and racializing process that can occur even when the objects are apparently gender-neutral and lack metallic genitalia and melanin in terms of their outward design. This suggests we have to think carefully about the ways in which digital technologies may entrench and deepen the lived experiences of both privilege and marginalization.

7.3 Happy Service Robots: Worse than Slavery?

After having discussed visual culture, gender, and race in relation to the design of robots, I now turn to explore the potential impact of the instrumentalization of robots who provide care for humans. Of particular interest in this section is the position that claims that it is possible to produce robots that are designed and built to happily work in service for humans and I will focus on the work of Steve Petersen as a foil to explore the implications of this claim.

There is a wide range of opinions concerning the ethics of service robots and whether or not they are considered ethical subjects. Levy believes that robots should be treated ethically, irrespective of whether they are ethical subjects, in order to avoid sending the wrong message to society, namely that treating robots unethically will make it “acceptable to treat humans in the same ethically suspect ways” (Levy Reference Levy2009, 215). In contrast, Torrance defends robot servitude, analogous to a kitchen appliance, because they are seen as incapable of being ethical subjects (Torrance Reference Torrance2008).

Petersen (Reference Petersen, Lin, Abney and Bekey2011), however, denaturalizes the status of the “natural” regarding robots and counters that a service robot can have full ethical standing as a person, irrespective of whether the artificial person is carbon or computationally-based. The important insight that personhood “does not seem to require being made of the particular material that happens to constitute humans,” but rather “complicated organizational patterns that the material happens to realize,” however, is combined with the much more controversial contention that it is nonetheless “ethical to commission them for performing tasks that we find tiresome or downright unpleasant” (Petersen Reference Petersen, Lin, Abney and Bekey2011, 248)Footnote ⁵ if they are hardwired to willingly desire to perform their tasks.

Petersen adds that the preferences hardwired into carebots could remain indecipherable to humans.

Petersen’s argument is built upon the premise of hardwiring robots to feel desires that impassion them toward fulfilling work that humans find unpleasant. And he believes this is analogous to the (“naturally”-produced) hardwiring of humans given that in the “carbon-based AP [artificial person] … the resulting beings would have to be no more ‘programmed’ than we are” (Petersen Reference Petersen, Lin, Abney and Bekey2011, 285).

The crux of Petersen’s position is that the robots freely choose to serve and thus do not violate the Kantian anti-instrumentalization principle of using a person as a mere means to an end.

By claiming that insofar as an artificial person is a “willing servant, … we can design people to serve us without thereby wronging them” (Petersen Reference Petersen, Lin, Abney and Bekey2011, 289), Petersen counters, first, the critique of creating a “caste” of people, particularly in the case of robots that do menial labors, and, second, the critique of designed servitude leading to “happy slaves” that have been paternalistically deprived of the possibility of doing something else with their lives (Walker Reference Walker2007).

Although Petersen defends the right of artificial persons to do otherwise, that is, not to serve, he believes that reasoning “themselves out of their predisposed inclinations [is as] unlikely as our reasoning ourselves out of eating and sex, given the great pleasure the APs derive from their tasks …” (Petersen Reference Petersen, Lin, Abney and Bekey2011, 292). Petersen’s caveat of accepting dissent, however, does not square with the premises of his theory. Ultimately, the caveat/exception does not legitimate the rule of hardwiring sentient submission but rather operates as a malfunction within a structural argument that is in favor of the hardwired design of “dissent-less” carebots, which is unethical from the start. This shows how Petersen’s writing is premised upon a strongly deterministic conception of behaviour as pre-determined by the hardwiring of living systems, be they organic or non-organic, and, as such, service robots are highly unlikely to finagle their way around their programmed “instinctive” impulses. In this way, despite Petersen’s valuable denaturalization of the status of the “natural” in terms of the distinction between human and robot, he re-entrenches it once again in his simplistic conception of a pre-deterministic causality from gene/hardware to behaviour. For Petersen, treating artificial people ethically entails “respecting their ends, encouraging their flourishing” by “permitting them to do laundry” because “[i]t is not ordinarily cruel or ‘ethically suspect’ to let people do what they want” (Petersen Reference Petersen, Lin, Abney and Bekey2011, 294). Precisely because they desire to serve, he contends that they must be distinguished from the institution of slavery. The inadequacy of the slave metaphor is such that, for Petersen, it would be irrational to preclude pushing buttons to custom design your artificial person who loves and desires to serve you because it could be an act of discrimination tantamount to the worst episodes of human history.

I find Petersen’s proposal of the happy sentient servant programmed to passionately desire servitude highly disturbing and problematic because, as I analyze and argue in this section, if materialized in future techno-social configurations of society, it would automate, reify, and legitimate the dissent-less submission of purportedly willing and happy swaths of sentient artificial humans and entrench hierarchical structures of oppression as natural divisions among artificial and non-artificial humans.

As part of setting out my analysis, I propose that robots can be historical in two senses, namely, as objects or subjects, although as objects they are historical in a much more limited sense than as subjects. Robots as historical objects are robots without sentience or self-awareness that are historical because of the values embedded in their design that are specifically situated in time and space. Furthermore, robots as objects are products of cultural translation within the technological frames and languages available at the time to materialize their functions. In contrast, robots as historical subjects are premised upon emergent forms of sentience and subjectivity that are analogous to those of “non-artificial” humans.

A digitized future in which we accept Petersen’s conception of sentient servitude as ethical through embedding hardwired desires to serve and be obedient could create a future in which we automate what Pierre Bourdieu calls symbolic violence. The work of Bourdieu offers a sophisticated theory to address how societies reproduce their structures of domination. Symbolic capital or cultural values are central to the processes of legitimizing structures of domination. Said cultural values are reified or presented as universal but are in fact historically and politically contingent or arbitrary. Of all the manners of “‘hidden persuasion,’ the most implacable is the one exerted, quite simply, by the order of things” (Bourdieu and Wacquant Reference Bourdieu, Wacquant, Scheper-Hughes and Bourgois2004, 272). Programming sentient servants to be happy with their servitude points to the automation of symbolic violence through the deployment of desire and pleasure to subjugate artificial persons into unquestioning servitude while being depicted as having “freely” chosen to wash laundry in saecula saeculorum. If we assume that their desire by design is successfully engineered to avoid wanting any other destiny than that of servitude, the symbolic violence of artificial persons à la Petersen lies in how the hardwired happiness becomes an embedded structure of the relation of domination and thus reifies their compliance with the status quo.

Petersen claims that creating sentient servers who enjoy their labors is an advance over histories of racism, sexism, colonialism, and imperialism. I contend, however, that Petersen’s Person-O-Matic culminates the imperial fantasy of biological determinism where you can have an intrinsically obedient population whose lesser sentience is engineered to feel grateful and happy to serve your needs. And I differ even further: It is not that Petersen’s artificial beings designed to serve are simply analogous to slaves, but actually they are worse than slaves because they are trapped in a programmed/hard-wired state of dissent-less and ecstatic submission.

The domination of humans by humans along the axes of colonial forms of hierarchizing the relationship of the civilized colonizer vis-à-vis the barbarian other, either seen as in need of being civilized or inevitably trapped in cultural and/or biological incommensurableness, always had to contend with the capacity of dissent or the native excesses that collapsed the discursive strictures of colonial otherness and destabilized the neat hierarchies imposed by the imperial discourses. Empires had to deal with the massive failure of their fantasies regarding the imaginary subservience and inferiority of others, but a digital future built on Petersen’s model is actually much more violent because desire by design hardwires pleasure in serving the masters. It basically precludes dissent, either because their lesser sentience has been designed effectively or because the dissent is a highly remote possibility. Hence the combination of limited sentience marked by a programmed incapacity to dissent poses the uncomfortable technological culmination of the fantasies of biological determinism that took definite shape as part of colonial endeavors.

An especially fascinating, powerful, and distinctive aspect of Petersen’s model is the commodification of custom designed servitude. Rather than the vision of the colonial administration of an empire, it is a consumerist vision of individuals who go to a vending machine to custom design a slave, an intimate subjectivity of empire for John Doe, who can now have his own little colony of sentient servers to “orgasmically” launder his clothes, presumably among many other duties.

The notions available for understanding and evaluating humanness in the digital future are impoverished by this kind of theorization. For Petersen, sentience is premised upon the capacity of the human to act upon one’s desires, which are hardwired into the robot. Desire here is a highly reductive and deterministic conception of desire by design. He articulates a linear theory of causality from design to desire, systemically hardwired to produce servants who happily do laundry. As already addressed, despite Petersen’s contention that the artificial persons that emerge from the Person-O-Matic act freely when choosing to serve, the fact that their design, if it is successful, precludes the possibility of not wanting to serve raises serious doubts as to whether the robots are actually exercising their free will to do their labors. This resonates with Murakami Woods’ critique, in Chapter 2, of digital imaginaries that nudge humanness into commercially profitable and instrumental boxes.

One of the central problems of Petersen’s conception of robot sentience – like Murakami Wood’s smart city denizen – is precisely that it is trapped within a narrow understanding of self-determination. This conception does not consider the social and historical constitution of subjectivities within cultural parameters that vary and produce contingent desires that are not reducible to the underlying hardwiring (genetic or otherwise). Desire is always multifaceted and leads to unintended consequences, always in excess, incomprehensible even for the subject that desires. Hence, once an artificial being acquires an emoting sentience of some sort, engineering identities is not a case of linear causalities that follow an imaginary teleology of desire in a genetic or computational fantasy of hardwiring. Hardwiring, genetic/computational engineering, and natural selection are the names of Petersen’s “game” and, as a result, Petersen is not engaging with the social production and cultural inscription of sentient robot subjectivities and, accordingly, he occludes the complex and contradictory process of interactive, mutually constitutive forms of sociability set out in Chapter 8, by Steeves.

An emoting sentience implies that robots become social and cultural subjects, not just objects inscribed with pre-existing values in their design. They become subjects who can engage with and resist the constraints of their architecture as well as those of the discursive and semiotic fields of signification within which they emote and represent their location within societies. Rather than define robots as human because they fulfill their hardwiring of desiring to serve, we can say that robots become “human” when they escape the constraints of their imaginary hardwiring, when there is an excess of desire that is not reducible to their programming. Desire, pleasure, and pain are ghosts in the hardwiring of machines. And it is the ghosts that make the robot “human” as a form of post-mortem ensoulment enabling a human-like sentience marked by contradiction and excess. Irreducibility makes human. Unpredictability makes human. Dissent makes human. The absence of a “human nature” makes human.

Although it is still technically impossible to create robots with a fully human-like sentience, the production of emoting robots with limited forms of sentience may not be such a remote possibility. For Petersen, artificial humans will be more trapped within their hardwiring than “non-artificial” humans and, thus, the exercise of dissent from the structural constraints of their design will be highly remote, if not precluded altogether. His defense of the ethical legitimacy of the artificial person that serves “passionately” raises very difficult questions that must be teased out. Is the production of a lesser sentient dissent-less robot worse than the production of a fully sentient robot capable of dissent? Should there be an ethical preclusion of a lesser sentience in the design of emoting robots? Should society err on the side of no sentience in order to avoid the perverse politics that underlies the design of terminally happy sentient beings incapable of dissent, that is, the ideal servant incapable of questioning the “ironic ramifications of [his/her/its] happiness”?Footnote ⁶

I contend that either no sentience or full sentience is more ethical than computationally or biologically wiring docility into an emoting and desiring being with a lesser sentience. When robots cease to be things to become sentient beings motivated by desires, pleasures, and happiness, the incapacity to dissent should not be a negotiable part of the design but rather should remain precluded. It seems much less unethical to create fully sentient robots with the capacity to dissent than to create a permanent underclass of unquestioningly obedient limited life forms motivated by passionate desires to serve.

Therefore, the techno-social imaginary of emerging digital societies must explicitly condemn the symbolic violence of automating the incapacity to dissent of artificial humans designed to happily launder the underwear of non-artificial humans. The ethical defense of the right to dissent in the design of emoting sentient beings crucially avoids creating the conditions for dystopic new forms of domination under a normalizing discursive smokescreen of the “order of things.”

7.4 The Carebot Industrial Complex: Some Final Thoughts

Although I do not underestimate what carebots could mean for elderly people, in this final section I raise some final thoughts about what I call the Carebot Industrial Complex, namely, the collective warehousing of aging populations in automated facilities populated by carebots.

For Latour, the relationship of people to machines is not reducible to the sum of its parts but rather adds up to a complex emergent agency. Beyond Winner’s concern over the embedding of social values in technologies, Latour deploys the notion of the delegation of humanness into technologies. Technological fixes can thus deskill people in a moral sense (Latour Reference Latour, Bijker and Law1992). This process of deskilling acquires special relevance in the context of using carebots and how it can undermine human learning and development acquired in caregiving settings. Of special relevance here is Vallor’s work on the ethical implications of carebots in terms of a dimension that has been ignored within the literature, that is, “the potential moral value of caregiving practices for caregivers” (Vallor Reference Vallor2011, 251). By examining the goods internal to caring practices, she attempts to “shed new light on the contexts in which carebots might deprive potential caregivers of important moral goods central to caring practices, as well as those contexts in which carebots might help caregivers sustain or even enrich those practices, and their attendant goods” (Vallor Reference Vallor2011, 251).

The Industrial Carebot Complex can deprive people living in the digital age of the moral value of caregiving practices for human caregivers and, in the process simultaneously, stigmatize the decaying bodies of the elderly, subject to the disciplinary effects of a shifting built environment and intimate technology of carebots, whose name exudes the oxymoronic concern of whether there can be care without caring. In addition, it can undermine the process of human intergenerational learning of caregiving skills for the elderly and stymy opportunities for emotional and social growth that occur when we act selflessly and out of concern for others.

Turkle’s arguments on the evocative instantiations of robotic pets, some of which have been deployed as part of elder care, can be extended to the broader concern over the emotional identification of elderly people with future carebots. For instance, in her fieldwork on elders’ interaction with pet robots, specifically Paro, a pet seal, Turkle asks:

One of Turkle’s central concerns is how our affection “can be bought so cheap” relative to robots that are incapable of feeling:

The potential attribution of affection to robots by a lonely and relegated population is particularly problematic and raises the specter of how the emotionless management of elderly bodies is ultimately not care. The emergent agency of the Industrial Carebot Complex can dehumanize aging populations even more dramatically than current forms of warehousing the elderly where the warmth of human hands, even those of a stranger, can make a radical difference in terms of the ethics of care experienced. Thus, the integration of carebots to elder care must never be in exclusion of human care, but rather complementary and subordinate to the moral value of human caregiving skills and practices.

In this chapter I have analyzed different dilemmas regarding the use of robots to serve humans living in the digital age. The design and deployment of carebots is inscribed in complex material and discursive landscapes that affect how we think of humanness in the socio-technological architectures through which we signify our lives. As stated at the outset of this chapter, imagining those spaces necessarily entails navigating the “fog of technology,” which is also always a fog of inequality in terms of trying to decipher how the emerging architectures of our digitized lives will interface with pre-existing forms of domination and struggles of resistance premised upon our capacity to dissent. My main contention is anti-essentialist, namely, that the absence of a “human nature” makes us human and unpredictable. There is no underlying fixed essence to being human. Instead, we should be attentive to how what it means to be human, as I said in the opening and want to repeat here, must be strategically and empathically reinvented, renamed, and reclaimed, especially for the sake of those on the wrong side of the digital train tracks.

In this chapter, I use qualitative research findings to explore how algorithmically driven platforms impact the experience of being human. I take as my starting point Meadian scholar Benhabib’s reminder that, “the subject of reason is a human infant whose body can only be kept alive, whose needs can only be satisfied, and whose self can only develop within the human community into which it is born. The human infant becomes a ‘self,’ a being capable of speech and action, only by learning to interact in a human community” (Benhabib Reference Benhabib1992, 5). From this perspective, living in community and participating in communication with others is a central part of the human experience; it gives shape to Nafisi’s dance between public and private worlds and enables an agential path by which we come to know ourselves and forge deep bonds in community with others.

Certainly, early commentators celebrated the emancipatory potential of new online communities as they first emerged in the 1990s as spaces for both self-expression and community building (Ellis et al. Reference Ellis, Oldridge and Vasconcelos2004). Often designated communities of shared interest rather than communities of shared geography, they were expected to strengthen social cohesion by enabling people to explore their own interests and deepen their connection with others in new and exciting ways (see, e.g. Putnam Reference Putnam2000). Critics, on the other hand, worried that networked technology would further isolate people from each other and weaken community ties (Ellis et al. Reference Ellis, Oldridge and Vasconcelos2004). The advent of social media, those highly commercialized community spaces with all their hype of self-expression, sharing and connection, simply amplified the debate (Haythornthwaite Reference Haythornthwaite, Joinson, McKenna, Postmes and Reips2007).

For my part, I am interested in what happens to the human experience when community increasingly organizes itself algorithmically. What do we know about the ways in which people manage the interaction between self and others in these communities? What kind of language can we use to come to a normative understanding of what it means to be human in these conditions? How do algorithms influence both our interactions and this normative understanding?

To date, platform owners have encouraged the use of the language of control to describe life in the online community, calling upon individuals to make their own choices about withholding or disclosing personal information to others so they can enjoy what in 1984 the German Supreme Court called informational self-determination (Eichenhofer and Gusy Reference Eichenhofer, Gusy, Brkan and Psychogiopoulou2017). From this perspective, the human being interacting with others in community online is conceptualized apart from any relationship with others, and their agency is exercised by a binary control: zero, they withhold and stay separate and apart from others; one, they disclose and enjoy the fruits of publicity. As I have argued elsewhere (Steeves Reference Steeves, Roessler and Mokrosinska2015, Reference Steeves and DiGiacomo2016), this perspective has consistently failed to capture the complicated and constrained interactions described by people living in these environments.

More socially grounded critiques of this understanding of being human online have underscored the anaemic protection that individual control provides, largely by displacing the autonomous individual with a more social understanding of subjectivity (see, e.g. Cohen Reference Cohen2012; Koskela Reference Koskela and Lyon2006; Liu Reference Liu2022; Mackenzie Reference Mackenzie2015). This approach is interesting precisely because it can account for moments of human agency exercised in the context of a variety of resistive behaviours. For example, 11- and 12-year olds often report that they enjoy asking Siri and Alexa nonsensical questions that the machine cannot answer, as a way of asserting their mastery over the technology. Like the Rickroll memeFootnote ¹ and the Grown Women Ask Hello Barbie Questions About Feminism videoFootnote ², this is a playful way for people to deconstruct the ways in which they are inserted into technical systems and collectively resist the social roles they are offered by the platforms they use.

However, as Los (Reference Los and Lyon2006) notes, resistance is a poor substitute for agential action, precisely because current platforms are “intrinsically bound to social, political, and economic interests” (Thatcher et al. Reference Thatcher, O’Sullivan and Mahmoudi2016, 993) that may overpower the resister by co-opting their resistance and repackaging it to fit within the features that serve those interests. In this context, observers too often interpret the networked human as overly determined through the internalized norms of the platform or as restricted to a form of apolitical transgression/resistance similar to the Rickroll meme and other examples. Either way, we are left with critique but no path forward.

The project of being human in the digital world accordingly requires a better set of metaphors (Graham Reference Graham2013), a richer conceptualization that can capture the human experience within performances, identities and interactions shaped by algorithmic nudges. I suggest that Benhabib’s insight that we come to know ourselves and others by living in community is a productive starting point for developing such a lexicon, not least because the Meadian assumptions upon which it is based set the stage to reunite the search for human agency and the embrace of the social (Koopman Reference Koopman and Fairfield2010). It is also a useful way to extend the insights of relational autonomy scholars (Mackenzie Reference Mackenzie, Armstrong, Green and Sangiacomo2019; Roessler Reference Roessler2021) to do what Pridmore and Wang (Reference Pridmore and Wang2018) call for in the context of digital life – to theorize human agency without severing it from our social bonds to others.

To help give this shape, I start my discussion by revisiting some data I collected from young Canadians in 2017Footnote ³ about their experiences on algorithmically driven platforms. These data were first collected to see how young people navigate their online privacy by making decisions about what photos of themselves to post; we reported that they described a complicated negotiation driven by the need to be seen but not to be seen too clearly, given the negative consequences of a failed performance (Johnson et al. Reference Johnson, Steeves, Shade and Foran2017). However, the data also provide an interesting window into how young people make sense of their self-presentation and interactions with others in networked community spaces that are shaped by algorithms.

Accordingly, I conducted a secondary analysis of the data to explore these elements. I start this chapter by reviewing the findings of that analysis, focusing on the ways in which my participants responded to the algorithms that shaped their online experiences by projecting a self made up of a collage of images designed to attract algorithmic approval as evidenced by their ability to trigger positive responses from a highly abstract non-personalized online community. I then use Meadian notions of sociality to offer a theoretical framing that can explain the meaning of self, other and community found in the data. I argue that my participants interacted with the algorithm as if it were another social actor and reflexively examined their own performances from the perspective of the algorithm as a specific form of generalized other. In doing so, they paid less attention to the other people they encountered in online spaces and instead oriented themselves to action by emulating the values and goals of this algorithmic other. Their performances can accordingly be read as a concretization of these values and goals, making the agenda of those who mobilize the algorithm for their own purposes visible and therefore open to critique. I then use Mead’s notion of the social me and the indeterminate I to theorize the limited and constrained moments of agency in the data when my participants attempted – sometimes successfully, sometimes not – to resist the algorithmic logics that shape networked spaces.

8.1 What Self? What Other? What Community?

As noted, in 2017 we conducted qualitative research to get a better sense of young people’s experiences on social media. Our earlier work (Bailey and Steeves Reference Bailey and Steeves2015) suggested that young people rely on a set of social norms to collaboratively manage both their identities and their social relationships in networked spaces and that they are especially concerned about the treatment of the photos they post of themselves and their friends. We wanted to know more about this, so we asked 18 teenagers between 13 and 16 years of age from diverse backgrounds, 4 of whom identified as boys and 14 of whom identified as girls, to keep a diary for 1 week of the photos they took. They then divided the photos into three categories:

• Those photos they were comfortable sharing with lots of people;

• Those photos they were comfortable sharing with a few people; and

• Those photos they were not comfortable sharing with anyone.Footnote ^4, Footnote ⁵

The photos we collected through this process were largely what we expected to see – school events, group shots, food, lots of landscapes. But when we sat down and talked to our participants, the discussion was not at all what we expected. It quickly became clear that the decisions they were making about what photos to share with many people really had very little to do with their personal interests or their friendships. Although they described the networked world as a place where they could connect with their community of family and friends, the decision-making process itself did not focus on what their friends and family would like to see or what they would like to show them of themselves. Instead, it focused on “followers”, an abstract and anonymous audience they assumed was paying attention to a particular platform. Because of this, they positioned themselves less as people exploring their own sense of identity in community and more as apprentice content curators responsible for feeding the right kind of content to that abstract audience.

The right kind of content was determined by a careful read of the algorithmic prompts they received from the platform. Part of this involved them doing the work of the platform (Andrejevic Reference Andrejevic2009); for example, they universally reported that they maintained Snapchat streaks by posting a photo a day, even when it was inconvenient, because it was what the site required of them. Interestingly, they did this most easily by posting a photo of nothing. For example, one participant was frequently awakened by an alert just before midnight to warn her that her streaks were about to end, so she would cover her camera lens with her hand, take a photo of literally nothing and post the photo as required. It was very clear that the posts were not to communicate anything about themselves or to connect with other people, but to satisfy the demands of the algorithmic prompt.

However, the bulk of their choices rested on a careful analysis of what they thought the audience for a particular platform would be interested in seeing. To be clear, this audience was explicitly not made up of friends or other teens online; it was an abstraction that was imbued with their sense of what the algorithm that organized content on the site was looking for. For example, they all agreed that a careful read of the platform and the ways content was algorithmically fed back to them on Instagram indicated that it was an “artsy” space that required “artsy” content. Because of that, if you had an Instagram page, you needed to appear artsy, even if you were not. Moreover, given the availability of Insta themes, it was important to coordinate the posts to be consistently artsy in a “unique” way, even though that uniqueness did not align with your own personal tastes or predilections.

From this perspective, the self-presentations that they offered to the site revealed very little of themselves aside from their fluency in reading the appropriate algorithmic cues. The digital self was accordingly a fabricated collage of images designed to attract algorithmic approval; in their words, “post worthy” content was made up of photos that said something “interesting” not from their own point of view but from the point of view of the abstract audience that would judge how well they had read the algorithmic prompts. One of the girls explained it this way:

She then explained that the point was not to share her interest in Harry Potter with people she knew on or offline (in fact, she was not a fan); it was to identify a theme that would appeal to the algorithm so her content would be pushed up the search results page and attract likes from unknown others. Moreover, she felt that her choice was vindicated when two followers added the photo to their collections. In this context, a pre-existing audience had already made its preferences known and the online marketplace then amplified those preferences by algorithmically prioritizing content that conformed to them; accordingly, the most “interesting” self to portray was one that mirrored the preferences of that marketplace independent of whether or not those preferences aligned with the preferences of the poster.

This boundary or gap between their own preferences and the preferences they associated with their online self was intentionally and aggressively enforced. This was best illustrated when one of the participants was explaining why a photo she took of an empty white bookcase against a white wall could not be shared with anyone. She told me that she had originally planned on posting it to her Instagram page (her theme was “white” because monochromatic themes were “artsy”) but then she noticed a small object on the top shelf. She expanded the photo to see that it was an anime figurine. It was there because she was an ardent anime fan. However, she was distraught that she had almost inadvertently posted something that could, if the viewer expanded the photo, reveal her interest online. She eschewed that type of self-exposure because it could be misconstrued by the algorithm and then made public in ways that could open her up to judgement and humiliation. As another participant explained, photos of your actual interests and close relationships aren’t

To avoid these harms, all our participants reported that they collaborated with friends to collectively curate each other’s online presences, paying special care to images. Specifically, no one posted photos of faces, unless they were part of a large group and taken from a distance. Even then, each photo would be perused to see if everyone “looked good” and, before posting, it would be vetted by everyone in the photo to make sure they were comfortable with it going online.

Interestingly, there were two prominent exceptions to these rules. The first occurred when they were publicly showing affection to friends in very specific contexts that were unambiguous to those who could see them. This included overtly breaching the rules on birthdays: publicly posting a “bad” photo of a friend’s face without permission, so long as it was tagged with birthday good wishes, was a way to demonstrate affection and friendship, akin to embarrassing them by decorating their lockers at school with balloons. The second exception involved interacting with branded commercial content. For example, one of the girls had taken a series of shots of herself at Starbucks showing her face with a Starbucks macchiato beside it. She was quite confident that this photo would be well received because Starbucks was a popular brand. Similarly, our participants were confident that photos and positive comments posted on fan sites would be well-received because they were part of the online marketplace.

All other purposes – actually communicating with friends or organizing their schedules, for example – occurred in online spaces, such as texting or instant messaging apps, that were perceived to be more private. But even there, they restricted the bulk of their communications to sharing jokes or memes and reserved their most personal or intimate conversations for face-to-face interactions where they couldn’t be captured and processed in ways that were outside their control.

8.2 Understanding Algorithmic Community

The snapshot in Section 8.1 paints a vivid picture of a digital self that seeks to master the algorithmic cues embedded in networked spaces to self-consciously fabricate a collage of images that will attract approval from an abstracted, highly de-personalized community. From my research participants’ perspective, networked interaction is therefore not simply about the self expressing itself to others, as throughout this process personal preferences are carefully and meticulously hidden. Rather, it is about the construction of an online self that is “unique” in the sense that it is able to replicate the preferences of the online marketplace in particularly successful ways. Success is determined through feedback from an abstracted and anonymous group of others who view and judge the construction but, to attract the gaze of those others, content must first be algorithmically selected to populate the top of search results. To do this well, the preferences, experiences and thoughts that are unique to the poster must be hidden and kept from the algorithmic gaze, and the poster must post content that will both be prioritized by the algorithm and conform to the content cues embedded by the algorithm in the platform. This is a collaborative task; individuals carefully parse what online self they choose to present but also rely on friends and family to co-curate the image of the self, by helping hide the offline self from the algorithmic gaze and by posting counter content to repair any reputational harm if the online self fails to resonate with the preferences of the online marketplace (see also Bailey and Steeves Reference Bailey and Steeves2015).

To better understand these emerging experiences of the online self, other and community, I suggest we revisit Mead’s understanding of self and community as co-constructed through inter-subjective dialogue. For Mead, an essential part of being human is the ability to anticipate how the other will respond to the self’s linguistic gestures, to see ourselves through the other’s eyes. This enables us to put ourselves in the position of the other and reflexively examine ourselves from the perspective of the community as a whole. He calls this community perspective the “generalized other” (Mead Reference Mead1934; see also Aboulafia Reference Aboulafia2016; Prus Reference Prus, Herman and Reynolds1994).

Martin (Reference Martin2005) argues that this ability to take the perspective of the other is a useful way to understand community because it calls upon us to pay attention to “our common existence as interpretive beings within intersubjective contexts” (232). Certainly, my participants can be understood as interpreters of the social cues they found embedded in networked spaces, exemplifying Martin’s understanding of perspective taking as “an orientation to an environment that is associated with acting within that environment” (234). What is new here is that my participants described a process in which they gave less attention to their interactions with other people in that environment and instead oriented themselves to action by carefully identifying and emulating the perspective of the algorithm that shaped the environment itself.

This was often an explicit process. When they explained how they were trying to figure out the algorithm’s preferences and needs, they were not merely seeking to reach through the algorithm to the social actors behind it to interpret the expectations of the human platform owners or even the human audience that would see their content. Rather, by carefully reading the technical cues to determine what kind of content was preferred by the platform and offering up a fabricated collage of images designed to attract its approval, they both talked about and interacted with the algorithm as if it were another subject.

This is a kind of reverse Turing Test. They were not fooled into thinking the algorithm was another human. Instead, they injected the algorithm with human characteristics, seeking to understand what was required of them by identifying the algorithm’s preferences and interacting with it as if it were another subject. They did this both directly (by feeding the platform information and watching to see what response was communicated back to them) and indirectly through the “followers” who acted as the algorithm’s proxies. Moreover, the importance they accorded to these algorithmic preferences was demonstrated by the kinds of identities my participants chose to perform in response to this interaction – such as Harry Potter Fan and Starbucks Consumer – even when these identities did not align with the selves and community they co-constructed offline and on private apps with friends and family.

Daniel (Reference Daniel2016) provides an entry point into exploring this gap between online and offline selves when he rejects the notion of the unitary generalized other and posits a multiplicity of generalized others that can better take into account experiences of social actors who are located in a multiplicity of communities. Certainly, his insight that “the self is constituted by its participation in multiple communities but responds to them creatively by enduring the moral perplexity of competing communal claims” (92) describes the difficulties my participants talked about as they sought to be responsive to the multiple perspectives of the various social actors in their lives, including family, friends, schoolmates and algorithms. But reconceiving these various audiences as a “plurality of generalized others” (Martin Reference Martin2005, 236), each of which reflects a self based on a specific set of expectations and aspirations shared by those inhabiting a particular community space (Daniel Reference Daniel2016, 99), makes it possible to conceptualize – and analyze – the algorithm as the algorithmic other, with its own commercially-driven values and goals, that shapes selves and interactions in networked spaces.

To date, the most comprehensive critique of the commercial values and goals that shape the online environment has been made by Zuboff (Reference Zuboff2019). She argues that algorithms act as a form of Big Brother or, in her words, “a Big Other that encodes the ‘otherized’ viewpoint of radical behaviorism as a pervasive presence” (20). From this perspective, the problem rests in the fact that the algorithmic other does not operate to reflect the self back to the human observer so the human can see its performances as an object, but instead quantifies the fruits of social interaction in order to (re)define the self as an object that can be nudged, manipulated and controlled (Lanzing Reference Lanzing2019; McQuillan Reference McQuillan2016; Steeves Reference Steeves2020). In this way, the algorithmic other serves to:

However, Zuboff’s critique is dissatisfying as it gives us no way to talk about agency: if we are fully automated, then we have been fully instrumentalized. It also fails to capture the rich social-interactive context in which my research participants sought to understand and respond to the algorithms that shape their public networked identities. Once again, I suggest that Mead can help us because he lets us unpack the instrumentalizing logic of the nudge without giving up on agency altogether.

Certainly, my participants’ experiences suggest that the kinds of identities that we can inhabit in networked spaces are constrained to those that conform to the commercial imperatives of the online ecology. However, the notion that a particular community constrains the kinds of identities we are able to experiment with is not new. As Daniel (Reference Daniel2016) notes:

From this perspective, the types of identities that we see performed online in response to the algorithmic other concretize the values and goals embedded in online spaces by platform owners who mobilize algorithms for their own profit; and, by making those values visible, they open them up to debate. This makes the algorithm a key point of critique because it is a social agent that operates to shape and instrumentalize human interactions for the purposes of the people who mobilize it. From this perspective, to solve the kinds of polarization, harassment and misinformation we see in the networked community we must start by analyzing how algorithms create a fruitful environment for those kinds of outcomes. Unpacking how this works is the first step in holding those who use algorithms for their own profit to public account.

The sociality inherent in my participants’ interactions with the algorithmic other also lets us account for those small moments of agency reflected in the data. Mead posits that the self interacts with the generalized other in two capacities. The first is the social me that is performed for the generalized other and reflected back to the self so the self can gauge the success of its own performance. As noted in Section 8.1, the intrusiveness of the algorithmic other constrains the social me that can be performed in networked spaces. This is exemplified by my participants’ concern that their networked selves – artsy consumers of branded products – conform to the expectations of the algorithmic other even when they can’t draw a stick figure or don’t like coffee. However, the second capacity of the self is the I, the self that observes the social me as an object to itself and then decides what to project next.

Mead accordingly helps us break out of algorithmic determination by anchoring agency in the indeterminacy of the I as a future potentiality. This indeterminacy is constrained because it is concretized as the social me as soon as the I acts. But its emergent character reasserts the possibility of change and growth precisely because it can only act in the future. By situating action in a future tense of possibility, we retain the ability to resist, to choose something different, to be unpredictable, to know things about ourselves that have not yet come into being. In this sense, the algorithm can constrain us, but it cannot fully determine us because we continue to emerge.

Certainly, my research participants sought to exercise agency over their online interactions by revealing and hiding, making choices as part of an explicitly conscious process of seeing the objective self reflected back to them. They also wrested online space away from the algorithm on occasion. Birthday photos, for example, were consciously posted in order to break the algorithmic rules and to connect not with the abstract audience but with the humans in their lived social world, a social world which both interpolates with and extends beyond networked spaces. This demonstrates both a familiarity with and an ability to pull away from the algorithmic other in favour of the generalized other they experience in real world community.

8.3 Conclusion

I argue that my participants’ experiences demonstrate the paucity of identities available to networked humans who interact on sites that are shaped by the instrumental goals of profit and control. But those same experiences also underscore the rich sociality with which humans approach algorithmically driven ecologies, shaping their own interactions with the environment by injecting social meaning into the algorithm through their reading of the algorithmic other.

Certainly, the algorithmic positioning of human as object for its own instrumental purposes rather than for the social purposes of the self leaves us uneasy. Although we may feel reduced to an online self that is “compactified” into “a consumable package” and wonder if we can “know what it means to exist as something unsellable” (Fisher-Quann Reference Fisher-Quann2022), the point is we still wonder. Once again, agency exists as a potentiality in the moment of our own perusal of the self as object, in spite of – or perhaps because of – our interactions with the aggressive and intrusive nature of all generalized others (Daniel Reference Daniel2016).

Moreover, by conceiving of the algorithm as a social actor, we can extend the moment of human agency and bring the values and goals embedded in the algorithm out of the background and into the foreground of social interaction. From this perspective we can open up the algorithmic black box and read its instrumental intentions through the performances it reflects back to us because we recognize and interact with the algorithmic other as other. From this perspective, the algorithm only “masquerades as uncontested, consensus reasons, grounds, and warrants when they are anything but” (Martin Reference Martin2005, 251). Acknowledging the algorithm as an inherent part of online sociality helps us begin the hard task of collectively confronting the politics inherent in the algorithmic machine (McQuillan Reference McQuillan2016, 2).

Mead’s Carus Lecture in 1930 is prophetic in this regard. He said:

Hildebrandt and Backhouse (Reference Hildebrandt and Backhouse2005) make the same point when they argue that the data that algorithms use to sort us are a representation, constructed from a particular point of view, of a messy, complicated, nuanced and undetermined person. They warn us that, if our discourse confuses the representation of an individual with the lived sense of self, we will fail to account for the importance of agency in the human experience. We will also be unable to unmask the values and goals of those humans who mobilize algorithms in the networked world for their own purposes.

This book contends with various ways of being human in the digital era, and this chapter intends to describe what it means to have a human body in our time. Much has been written about the colonial, racializing and gendered continuities of perceiving, sorting and discriminating bodies in a digital world. However, nothing like the digital has transformed the materiality of the body in its very flesh and bone. It seems redundant to say that the body is the prerequisite to being human, yet this superfluous fact questions how bodies function in in-between worlds: they flow in this world’s digital veins and yet rigidly represent decisive characteristics. They seem unreal, an amalgamation of data sometimes, while at other times fingerprints, iris scans and bone tests portray a cage, a trap, a body that betrays. This contrast is especially visible in uncertain spaces, where identity becomes crucial and only certain categories of humans can pass, such as borders and refugee camps. These spaces are not only obscuring the body while exposing it; they also exist in a complex mixture of national jurisdiction, international regulations and increasingly private “stakeholders” in immigration management. In addition to the severity of experiencing datafication of bodies in these spaces, the deliberate unruliness paves the way for these spaces to become technological testing grounds (Molnar Reference Molnar2020); for example, technologies developed for fleeing populations were used for contact tracing during the COVID-19 pandemic.

The relationship between body, datafication and surveillance has been scrutinized from the early days of digital transformation. Today’s most debated issues, such as algorithmic bias, were already warned about, and the ramifications of their discriminatory assumption for marginalized people were highlighted at the end of the 1980s (Gandy Reference Gandy1989). Similarly, the predictive character of aggregated data and the consequences of profiling were analysed (Marx Reference Marx1989). From these early engagements, many instances of showing how routinely technologies are used to govern, datafy and surveil the body developed (see, e.g. Bennett et al. Reference Bennett, Haggerty, Lyon and Steeves2014). Additionally, surveillance scholars discussed how the “boundary between the body itself and information about that body” is increasingly transforming (Van der Ploeg Reference Van der Ploeg, Ball, Haggerty and Lyon2012, 179). Building on this rich body of literature and personal experiences of immigration, exile and entrapment, this chapter revisits the body, being uncomfortable in/with/within it and yet being aware of its power to define if one is considered human enough to bear rights, feelings and existence. Similar to the chapter’s movement between boundaries of the material and virtual, the text also oscillates between academic thinking, autobiographical accounts, pictures and poesy; denoting the discomfort of being in a Code/Body.Footnote ³ In this chapter, poetic language remedies the absence of the performative to help with the linguistic distress for finding the right words to describe embodied feelings.

9.1 From Data Doubles to Embodiment

The scholarship on datafication, surveillance and digital transformation in the 2000s is infatuated with what can be called the demise of the material body. The speed of datafication and digital change lead to the idea that the surveillance society gives rise to disappearing bodies (Lyon Reference Lyon2001); the body is datafied and represented through data in a way that its materiality is obscured. Although such conceptualizations had been formerly discussed, especially by feminist and queer scholars, the liberatory nature of these feminist interpretations of cyborg bodies (Haraway Reference Haraway1985) and body assemblages were not transferred into these new understandings of datafied and surveilled body. In their influential essay on surveillant assemblages, Haggerty and Ericson compare the digital era with Rousseau’s proclamation, “man was born free, and he is everywhere in chains” by claiming that nowadays “humans are born free, and are immediately electronically monitored” (Haggerty and Ericson Reference Haggerty and Ericson2000, 611). The subjectivating effect of surveillance, then, is instantly interlinked with basic rights and the meaning of being human. The body, they argue, is positioned within this surveillance assemblage: it is “broken down into a series of discrete signifying flows” (Haggerty and Ericson Reference Haggerty and Ericson2000, 612). Contrary to the Foucauldian way of monitoring, the body needs to be fragmented to be observed. Fragments can be combined or re-combined into “data-doubles”: ones that “transcend human corporeality and reduce flesh to pure information” (Haggerty and Ericson Reference Haggerty and Ericson2000, 613).

The scholarly debates on bodies in the following two decades were centred around the transformation of the body “via practices of socio-technical intermediation” (French and Smith Reference French and Smith2016, 9). The body and its datafication, visualization, mediation and multiplication have become increasingly important. Research about sorting, profiling and reification of marginal identities (or race/gender/class/etc.), inclusion and exclusion proliferates and successfully demonstrates how bias, racism, oppression and discrimination are injected into digital lives. The data double revealed the concurrent processes of the body’s objectification – to transform its characteristics to data – and its subjectivation due to the socio-technical processes of datafication. As Zuboff assertively writes in The Age of Surveillance Capitalism, “the body is simply a set of coordinates in time and space where sensation and action are translated as data” (Zuboff Reference Zuboff2019, 203). In this reading of the body, behavioural surplus is the engine of surveillance capitalism and the body is only another source of data. However, recent technological advancements, especially in using bodily features for identification, have started to expand and reconfigure such accounts. More recent studies underline the body’s centrality, for example, in big data surveillance and manipulation of the “surveilled subject’s embodied practices” (Ball et al. Reference Ball, Di Domenico and Nunan2016) or critically examine how biometric technologies transform the relationships between the body and privacy (Epstein Reference Epstein2016). It is argued that data body is not only a change in how bodies are represented but there exists an ontological change: the materiality of the body “and our subjective forms of embodiment that are caught in this historical process of change” are transforming (Van der Ploeg Reference Van der Ploeg, Ball, Haggerty and Lyon2012, 179). This chapter contributes to these later discussions, where the body is not only central as the source of data but has its own agency as an actant in data assemblages.

9.2 The Birth of Code/Body

Following the global digital transformation, discussions on issues of privacy, data protection, algorithmic harm and similar have entered the academic discourse and public debate. The recent years have seen an increase in reporting about the Big Tech companies as emerging new actors in the international governance realm. However, only those events that entail geopolitical or socio-economic relations to the Western countries are deemed relevant. For example, the news of Chinese payment methods through facial recognition technology rapidly reached the Western media (Agence France-Presse 2019) but much less attention was paid to the internal politics of digitalization in the Global South or the new e-governance measures of international governance institutions. This reluctance is intensified when digital technologies target communities that are marginalized, stateless or economically disadvantaged. UNHCR’s use of iris scanning for refugee cash assistance illustrates a case of extreme datafication of the body against people in dire need of assistance with hardly any voice to consent to or refuse the imposed technologies. Ninety per cent of refugees in Jordan are registered through EyeCloud, “a secure and encrypted network connection that can be used to authenticate refugees against biometric data stored in the UNHCR database” (UNHCR 2019). Iris scanning is then used for payment in the camp’s supermarket to calculate and pay the wages for working inside the camp and it replaces any monetary transaction. The EyeCloud demonstrates how current datafication practices do not only stop at using the datafied body for identification and representation but actively integrate the body as a part of data machinery. This instrumentalized body simultaneously carries the gaze of surveillance and guards itself against itself. The consequences are painful: more than a decade ago, The Guardian newspaper reported that asylum seekers burn their fingertips on electric stoves or with acid to avoid the Dublin regulations and to avoid being returned to their point of arrival, usually in Greece or Italy (Grant and Domokos Reference Grant and Domokos2011). The betraying body, however, regenerates fingertips after 2 weeks. Similarly, in cases where the age assessment of a claimed minor proves inconclusive, the person could be referred for a bone density test of the wrist by x-ray in Malta (Asylum Information Database Reference Akbari2023) or a “dental x-ray of the third molar in the lower jaw and MRI of the lower growth plate of the femur bone” in Sweden (Rättsmedicinalverket 2022). In these cases, the immigration authorities believe the body’s truthfulness and the accuracy of medical sciences against mendacious and deceitful asylum seekers. Table 9.1 shows the extent of data categories gathered on visa, immigration or asylum applicants travelling to Europe. The body increasingly becomes a vehicle for knowing the real person behind the application.

Table 9.1Data categories stored in European immigration data banks.⁴

Table 9.1Long description

The table outlines the data categories stored in various European Union immigration data banks. The table is divided into types of personal data and lists whether each data type is stored in the Schengen Information System S I S, the Visa Information System V I S, E U R O D A C, the entry or exit system E E S, E T I A S, and E C R I S-T C N. An x indicates that the data type is stored in the respective system, and x indicates conditional storage.

1. 1. Alphanumeric Data

   • General information, namely name, age, gender, and nationality are stored in S I S, V I S, conditionally in E U R O D A C, E E S, E T I A S, and E C R I S-T C N.

   • Occupation is stored in V I S and E T I A S.

   • Education is stored in E T I A S.

   • Reason for travel is stored in V I S.

   • Information about funds for living expenses is stored in V I S.

   • Address, phone number, email address, and I P address are stored in E T I A S.

   • Information about past or present felonies is stored in S I S, E T I A S, and E C R I S-T C N.

   • Information about a recent stay in a war or conflict region is stored in E T I A S.

• Biometric Data

  • Fingerprints are stored in S I S, V I S, E U R O D A C, E E S, and E C R I S-T C N.

  • Facial image is stored in S I S, V I S, E U R O D A C conditionally, E E S, and E C R I S-T C N.

  • Genetic data is stored in S I S.

The body acts as a trap. It transcends the current argumentations about profiling, sorting or bias based on personal data. What we witness is not just the datafication of the body but its function as ID card, debit card or labour hours registration sheet. If the cash machines, IDs and punched cards were technologies of yesterday, today these features are transferred to the body. The body becomes the payment system, the surveillance machine, the border. It is integrated into the datafied society’s infrastructure. It is platformized humanity. It is an integral material part of the bordering. On the Eastern European borders, heartbeat detectors, thermal-vision cameras and drones are used to unlawfully return the asylum seekers who manage to pass the border (Popoviciu Reference Popoviciu2021). The border is not a line on the map; it is everywhere (Balibar Reference Balibar2012, 78). The border is simultaneously a body on the move and a vehicle to keep out a body that does not belong. Consequently, the body/border can efficiently prevent flight since it entraps. When the Taliban got hold of biometric data banks that Western governments, the UN and the World Bank left behind in 2021, many activists and experts who collaborated with the coalition went into hiding because any border passage would put them in immediate danger of identification (Human Rights Watch 2022). They went into indefinite house arrest within the skeleton of their own bodies. This notion of corporeal entrapment or embodied surveillance resonates with the new conceptualization of how we understand space in the era of datafication. Coded space is defined as “spaces where software makes a difference to the transduction of spatiality, but the relationship between code and space is not mutually constituted” (Kitchin and Dodge Reference Kitchin and Dodge2011, 18). The digitalization of border security at airports or the use of digital technologies in the classroom are examples of coded space. In all these instances, when technology fails, there are still ways to finish the intended task: if the machine at a fully-automated high-tech airport does not recognize you, there is always an officer who can legitimize the authenticity of your ID. However, in the code/space the existence of space is dependent on the code and vice versa. If you are attending an online presentation and the technology fails, that would end your interaction. The code/space highlights the dyadic relationship between the two and their co-constructive nature (Kitchin and Dodge Reference Kitchin and Dodge2011). The dyadic relationship also explains the sense of corporeal entrapment. The datafied or coded body still exists, moves and functions. It has a mutual relationship with the data it produces but is not entirely constituted through it. We have our virtual profiles in social media platforms or wear smart watches but, as soon as we leave such spaces, we resign from being part of their universe. The Code/Body, however, is born in co-construction with the code and ceases existence if the code fails.

The Code/Body is an extension of code/space argumentation to the Foucauldian corporeal space, constantly subject to governmentality. Consequently, the surveillant assemblage introduced by Haggerty and Ericson (Reference Haggerty and Ericson2000) is not only about the production of data double. Their use of the Deleuzian conceptualization of the “body without organs” as an abstraction of the material body does not reflect the co-construction of the virtual and the material. Code/Body, on the other hand, offers a way to understand how the materiality of the body remains integral to our understanding of the body’s datafication while transcending the virtual–material dichotomy. The Code/Body carries manifold wounds: the bodily pain of being wounded – burnt fingertips, lungs full of water, starved behind border walls – and the hidden wounds of datafied exclusion. Although algorithms try to hide their bias, Code/Body reveals that race has a colour, ethnicity has an accent and gender could be “scientifically” examined. Underlining such material features of the body and their role in defining the Code/Body emphasizes “the co-constitution of humans and non-humans” (Müller Reference Müller2015, 27) and brings our attention to how things are held together and how datafied societies function. Extending the assemblage point of view, Actor Network Theory (ANT) provides a better empirical ground to understand the politics of the networks. It moves the focus more on outward associations and less on the intrinsic characteristics of a thing or its abstraction. The Code/Body highlights the co-constitution of these outward–inward associations and the body’s agency in changing the flows and associations within the assemblage. From this perspective, things have an open and contested character (Mol Reference Mol, Law and Hassard1999, 75), and the body is performative, meaning that its position within an assemblage can redefine its reality. Consequently, if one thing could be shaped by a variety of practices and networked connections, it can be configured in multiple and ambivalent ways. Lawful immigrants from internationally undesirable countries experience this multiple configuration throughout their border experiences. Visas to countries that have been visited before are rejected; border officers ask irrelevant questions to make the entry unpleasant or surprisingly act extra friendly. Automated passport check stations flicker a red light for double control but, on the next visit, go green. The assemblage changes and the integrated body in it changes accordingly. The Code/Body is, then, the ultimate device to realize and fulfil this fluidity. As a result, it is highly political how assemblages take shape, what actants dominate the flows, and which of the multiple realities of a thing are given preference. The ontological politics (Mol Reference Mol, Law and Hassard1999, 74) of Code/Body define the conditions of the possibility of being a human. Depending on their position in an assemblage, a person’s body could be reconfigured very differently. Heartbeats mean one thing on a smartwatch at a spinning class and another when sitting behind a lie detector machine at a border detention centre. Such politics of being are not only about positionality and “where we are” but also include temporality and “when we are.” I was held twice at the UK border detention centre despite having a valid visa. On both occasions, a sympathetic border officer took upon himself the time-consuming task of removing me from the “bad list.” It feels like a wonder that, within 40 minutes, a detained suspicious person, banished to a corner of the airport under the watchful eyes of a guard, turns into a legal traveller. Like a thing, the body can be understood as “a temporary moment in an endless process of assembling materials, a partial stabilisation and a fragile accomplishment that is always inexorably becoming something else, somewhere else” (Gregson et al. Reference Gregson, Crang, Ahamed, Akhter and Ferdous2010, 853). Code/Body, again, facilitates this temporary and mutable process of re-configuration. The more the body is datafied, the more physical it becomes.

9.3 A Moebius Body

The Code/Body blurs not only the virtual–material binaries but also nuances the politics of sorting by questioning the discourse of inclusion–exclusion through digital technologies, platforms and algorithms. Code/Body extends concepts such as the mediated body or the quantified self to propose an existential situation where the self stops to exist outside the code. This newly contended notion of self is the prerequisite of citizenship in the smart cities – the utopian dream of an urban life, which all societies are ambitiously moving towards. In future smart cities, we will not only witness behaviour nudging or the gamification of obedience (Botsman Reference Botsman2017); cities will be transformed into experimental labs, where the urban citizen is produced through measuring (Mattern Reference Mattern2016). On top of gathering data through sensors, following the movements and urban flows, and closely watching the bodies, the body becomes an instrument of belonging. To be in, it needs to be outed. The body needs to be thoroughly datafied to become integrated into the smart infrastructure of the city. Living in the Code/Body is a constant ride on a Moebius ring: the inside and outside depend on how one defines their situation or how their situation is defined for them. The Code/Body could belong to an urban assemblage at a specific time and lose all its association by a slight change in the code in the next second. In Figure 9.1, I have drawn a Moebius ring on the verdict of my complaint against UK immigration’s refusal of my tourist visa.

Figure 9.1 Mobius strip on Immigration Courts’ verdict

© Azadeh Akbari, Reference Akbari2023

Figure 9.1Long description

The legal document from the United Kingdom first-tier tribunal immigration and asylum chamber, dated April 21, 2015, details the appeal. Azadeh Akbari Kharazi against a visa decision by the entry clearance officer in Istanbul. The document includes official markings such as the crown emblem and the appeal number, which is partially redacted, and is identified as a decision and reasons notice. It outlines that the appellant is an Iranian citizen who applied for a United Kingdom general long-term visitor visa intending to visit the United Kingdom for a year with frequent visits planned over a ten-year period. The document mentions her employment as a Professor Assistant and her past travel history to the United Kingdom. The hearing took place at Columbus House in Newport and was handled as a paper case without legal representation. Some personal information is obscured with redaction and marker.

I had lived in London for 4 years and, after giving up my residency and returning to Iran, my tourist visa application was rejected. I was confused: I used to belong, work, live and actively participate in British society. Why was I suddenly out? Curiously, the judge had suggested since I can use technologies such as Skype to contact my friends in the UK, my human rights are not deemed to be violated. The code kept my body outside through its affordances to bring us closer. The movement between inside and outside makes bodily functions fuzzy; as if one can die while breathing and live forever, even after the heart stops. The following quote from a Somali refugee (now residing in Europe) initially shocks the reader: Did they drown?

Immediately after this thought, it seems his body has been revived from a mass of drowned refugees.

Another female Kurdish Iranian protestor during the Woman, Life, Freedom movement – a movement of Iranian women against compulsory Islamic dress code and discriminatory laws – reflects on how her body experiences the images she had previously seen on (social) media. She writes about how the physical and digital blend into each other and, despite the fear of pain instigated by watching social media videos, the real batons or pellets do not cause the expected physical pain.

The body’s agency leaks into the consciousness only after it has performed a task. In moments of upheaval, where the oppressed body stands up to its oppressors, it tries to distort its entrapment. Despite being surveilled, controlled and censored the body lives the unpermitted imaginary: it kicks the security forces, it runs and hides, it shows skin. It revolts against the sensory limitations imposed on it. In Figure 9.2, Woman, Life, Freedom protestors have covered a subway CCTV camera with female menstrual pads. Their female bodies withstand the gaze that controls, hides, oppresses and objectifies them. Next to the camera is a hashtag with an activist’s name: this time, virtual campaigns fuse into the material reality of the city. The Code/Body which is meant to be a part of the surveillant machinery through CCTV camera and facial recognition technologies, blinds the omnipresent eye with its most female bodily function: menstruation.

Figure 9.2 Blocking CCTV cameras in Public transportation with menstruation pads.

(Akbari Reference Akbari2023, 24)

Similar to silencing some bodily features, some bodies are marked as intangible, unrepresentable and unfathomable. Despite being embedded within different streams of data and code, our collective imagination still does not register the precarity of some bodies. At the time that artificial intelligence claims to further the limits of our creative powers by creating historical scenes or impossible fantasies, I inserted the poem by Warsan Shire at the beginning of this chapter in three popular AI-based text-to-image generation platforms. The results in Figure 9.3 show irrelevant pictures of mostly men depicting some keywords of the poem. The messiness of the poetry – and the poet’s feelings – does not translate into clear cut images. The machine fails to grasp even the theme of the poem. The wounded Code/Body remains hidden. The skin bears the pain of these wounds without bleeding and without any algorithm capturing its suffering. The person is caught in a body that can be datafied, but its emotions cannot be perceived.

Figure 9.3 AI-generated pictures created by Azadeh Akbari based on the poem “Conversations about Home (at the Deportation Centre)” by Warsan Shire, using three popular AI-based platforms

Figure 9.3Long description

The twelve grayscale panels include a variety of scenes arranged in a 3 by 4 grid. The left column features different trucks parked or driving through sparse desert-like landscapes, often accompanied by solitary male figures standing nearby. The middle column features close-up views of individuals, including a man gesturing during a conversation, an older man with striking facial features in a suit, and another man wearing glasses with a caption beneath him. The right column includes a person lying on a hillside, a densely packed urban scene with a comic-style speech bubble that reads My dream went this way, and a scene of three figures in light clothing, viewed from behind, standing together.

This chapter does not aim to investigate the political, economic or social reasons or structures that construct the Code/Body. The biopolitical and necropolitical, the Foucauldian corporeal space and its governmentality have been the subject of many scholarly debates. How surveillance and datafication affect these spaces is also not a new matter of discussion. However, it seems persistently new how uncomfortable the body feels for some people. The more some lives are exposed to precarity of intense datafication, some bodies are forced to give away their unscrupulous owner. Surveilling and constant measuring of the Code/Body assures that these lives remain precarious. Some bodies, it seems, could be easily deleted, like a line of dead code.