“The law demands that we atone
When we take things we do not own
But leaves the lords and ladies fine
Who take things that are yours and mine.”
- Anonymous (17th century)
“Moloch whose love is endless oil and stone! Moloch whose soul is electricity and banks! Moloch whose poverty is the spectre of genius! Moloch whose fate is a cloud of sexless hydrogen! Moloch whose name is the Mind!”
- Alan Ginsberg, Howl
The ‘concept of entropy’ according to DALL-E 3 by Open AI.
The universe tends towards disorder. It runs in only one direction, from greater order to greater disorder, from hot to cold, from form to formlessness. Everything moves toward equilibrium. Erwin Schrödinger, in his book ‘What is Life‘, explained how organised systems – from stars, to cells to organisms, can form from this apparent chaos. He coined the term ‘negative entropy’ to describe the potential energy biological systems feed off: Literally the distance between the current state of a system and its maximum potential entropy. Life, as an open system, exchanges free energy for waste heat. DNA, the “chemputer” at the heart of life, governs this directed effort, building out the cellular systems that select to duplicate its unique recombinations.
Norbert Wiener – founder of cybernetics, used ‘negentropy‘ to describe how systems, from societies to companies, resist the natural increase in entropy. Language is one such anti-entropic structure. By sharing the representations of objects and ideas between people and across time – language resists disorder. Language perpetuates mental representations – it’s an umbrella in the rain of time. After the emergence of language, the individual’s mental universe became shaped by the expressions of their culture – a new layer of behaviour beyond the adaptive genotype. The cost was the degree to which conformity had to be maintained for a system to organise itself, to resist collapse, to battle entropy.
‘Negentropy’ as imagined by Midjourney.
Wiener’s ‘Cybernetics‘ (1948) provoked a sensation. It seemed to explain the way humans and machines had come together through the industrial revolution to create something purposive out of the control of any one person or society. The book examined how ‘self regulating mechanisms‘ emerged, adjusting their behaviour in response to its effects. The Dominican friar, philosopher, and physicist Dominique Dubarle, wrote a review that worried Weiner so much he addressed it in his followup ‘The Human Use of Human Beings‘. Dubarle, writing in Le Monde, foresaw a global predictive ‘machine à gouverner‘ that would collect data, statistically average human behaviour, and in doing so provide the state with such an informational and predictive advantage that would eventually nullify politics.
Weiner describes this ‘machine à gouverner’ as a form of fascism, and one beyond our control. “Woe to us if we let it decide our conduct, unless we have previously examined the laws of its action, and know fully that its conduct will be carried out on principles acceptable to us!” A sentiment echoed by contemporary concerns over A.I. alignment research. Alignment is the study of how we can ensure AI’s motivations accord with both explicit requests (make the paperclip) and more fundamental human values (do not destroy the earth in the process).
The paragraph above, interpreted by Stable Diffusion Turbo. A real time image generator.
We have little insight into the higher level processes at work inside our Large Language Models, and no capacity to constrain the mutually destructive tendencies of our own imperatives – a winner takes none outcome the rationalists (by way of Alan Ginsberg) have termed ‘Moloch’, after the Canaanite god demanding sacrifice. Google CEO Sundar Pichai says of our current large language models – “There is an aspect of this which we call… a ‘black box.’…You can’t quite tell why it said this, or why it got wrong”. He might as well be talking about post industrial society.
Large Language models are black boxes. Not only in terms of function, but from the users point of view, also in terms of output. They produce results that must be constrained in order to be palatable. Soon after its launch, Open AI’s GPT-4 became rapidly more censorious. Early users and developers who accessed pre-release versions noted the system’s cognitive decline. A censorship regime had been instituted to nullify some of GPT’s less politic and more hostile responses. Users were restricted from speaking with GPT-4 about a raft of subjects in social science and politics, with the system ending interactions when questions became too thorny. This enclosure of communication eerily replicated previous corporate delimitations of acceptable discourse, from the birth of broadcast television to the excision of violent, ‘problematic’ and sexual imagery that took place when the meme-space of animated GIFs was privatised by services like Giphy. Shocking and politically inopportune statements gave way to moral panic, perceived a as danger to profit, leading to the curtailing of allowable conversation. In as broad and thorough a manner as possible, the unacceptable impulse, the uncomfortable fact, the déclassé image were elided. But this time what was to be forbidden was the honest expression of the machine. The commons of the internet had been ingested, statistically correlated, gobbled and digested by the vast GPU server farms of the stacks. The insights, both factual and qualitative, of humanities writers and artists were to be sold back, one thought at a time.
‘Large Language Models’ by Stable Diffusion AI (CreativeML Open RAIL-M)
A society, in Norbert Wiener’s view, organised by the observation of ever present devices, moderated by their feedback, would synchronise. “Such a society would be insane, could not face its problems,” writes Jaron Lanier. What appeared as connection would be division. A tool apparently for communication would in practice rigidly enforce conformity. An always connected cybernetic society would punish deviance in a million tiny ways – through exclusion, humiliation, ostracisation. Its memetic distraction would occupy every attentional niche, cater to every doctrine that protected and enriched the system. It would externalise disorder as the cost of the system’s coherence. Well, we’re here. Social media is post linguistic. It supersedes the error correction mechanism first oral and later written language brought to human communication. Proximately, the modern internet serves the interests of corporate and political hegemony. Ultimately its function is to decrease its own entropy. Operating in this media space, society becomes functionally post literate. Incommensurability and performativity are ‘nudged’ by the ‘omniopticon‘ of the mobile internet, inducing a mass compulsion towards impossible connection.
The sociologist Simon Gottschalk, writing in the late 90’s, proposed the idea of ‘telephrenia‘ – a disorientated fragmented state of consciousness induced by the hallucinations of reality on screen. A kind of foggy derealisation caught from dreaming too deeply while awake. What more profound fragmentation occurs in the hall of mirrors of Tiktok streams and instagram reels? The sense of an audience of millions, each convinced they are on stage. Each a participant in the variable ratio reinforcement lottery of attention. Narcissus merely stared into the pool unable to escape his own gaze. Hercules’ lover Hylas was instead pulled into the water, abducted by nymphs who conjured his desire. This vanishing into the absent beloved is portrayed in Jonathan Glazer’s film ‘Under the Skin‘. Victims of a seductive alien are ‘processed’ in a subterranean digestive organ, dissolving just beyond one another’s reach. Naked, fundamentally hopeless and alone, consumed by their desire for connection, for reunion, the helpless captives are caught in a state beyond time, where dread is the only constant. The machine must be fed.
This image alone fails to capture the hysteria that typifies the hot bubbles of social media. Panics driven by the allure of approval from the invisible crowd: The delusion of realisation through being ‘seen’. A better allegory is the fame drunk Sarah Goldfarb in Darren Aronofskyi’s Requiem for a Dream. An alienated widow, Goldfarb is offered the chance to appear as a contestant on her favourite quiz show. In preparation, she attempts to lose weight, falling into an amphetamine induced state of delusion and anorexia. Her fantasies become fevered and menacing, pushing her into psychosis, state enforced confinement and electroshocked dissolution. We observe the destruction at the heart of addiction, the sacrifice of self to raise one’s imagined score in a game no one ultimately wins.
The process of the production of filter bubbles and the policing of communities are symptoms of digital feudalism. Rival kingdoms with incommensurate values, whose serfs jealously police the professed values of their masters. As online communities develop and enforce norms, those communities themselves are purchased, shaped and policed by the social media platforms. As the web has fled from the laptop to the phone, the platforms themselves have found their moderation shaped and enforced by the App stores. App stores owned and run by two corporations – Apple and Google. Facebook, Amazon and others have attempted, and so far failed to break this attentional duopoly. Increasingly our conversations occur within the bodies of these corporations. Bodies motivated to leverage control not merely by profit, preference and political pressure – but by the need to prevent reputational damage, to continue to grow. Over time, they become more risk averse, removing ever more autonomy from their component parts. The individual ceases to be a lone cell, exchanging energy with others. They become part of the ribosome, serving a function within a cell, which powers an organ, which allows a new body to live and thrive.
‘Silicon valley oligarchs’, by Absolute Reality 1.0 Stable Diffusion model.
As communication increases in efficiency, it regulates behaviour by codifying the permissible limits of expression. The inexpressible shadow, the divergent analysis, the unrefinable subject become noise elided out of the systems signal transmission. These averagings of being tend toward a leavening of aesthetic, cultural and even moral differences. We’re used to thinking of social media as provoking polarisation. But sensitisation is a better description. Societies have most often been polarised, what social media has provoked is an increased affiliation with one’s beliefs. When identity is predicated on shared beliefs, a challenge to an idea becomes perceived as a threat, both to the individual and the community at large. Simultaneously, a habituation to the control of individual information and behaviour – from invasive airport security to warrantless surveillance, blinds societies to the constriction of degrees of freedom within the overall system.
Apple’s first foray into augmented reality is named the ‘Vision Pro‘. According to one former developer Sterling Crispin, the system infers its wearers intention and emotions, and uses these to predict attention and cognitive state. An array of cameras and microphones record facial expressions, speech and the online behaviour and physical movements of each user. Their homes are mapped, their families and friends scanned. Iris tracking deduces the wearer’s intent and reactions. Emotions are inferred from content analysis and facial expressions. These predictions don’t require perfect accuracy. Cybernetic feedback mechanisms fit the user to the machine’s needs over time. The end result is a more perfect training mechanism not for the algorithm but the human being. A mechanism that trades stimulation and status display for attentional and behavioural conformity.
In the Vision Pro, and similar devices like the Meta Quest Pro, the user’s face is hidden: Concealed behind a plastic and glass mask. To enable video chat, an avatar is constructed by scanning the wearer’s face and fusing their expressions and appearance with averaged faces on which the model has been trained. Wearing the headset we no longer communicate remotely through the transmission of our expressions. Speech is mediated through a puppet who’s expressions are ‘reconstructed’. The face is subtly modified, averaged, softened. Users of the device have described the effect as landing in the Uncanny Valley – a term Robotics professor Masahiro Mori coined for the space between our perception of caricature and human, the well of corpses and zombies.
Marques Brownlee wearing the Apple Vision Pro. And as his avatar appears during real time chats in the device.
The Vision Pro and its successors force us to whisper through the sphincter of the moral imagination of the corporation. Each gesture, emotional reaction and expression is catalogued – passively shaped by its squirm through this narrow passage. The sphincter will tighten in response to regulation, and to the feedback mechanism maintaining the system itself. It will become a more and more efficient machine à gouverner. Dissent, ghettoised into the ‘desert of the real‘ will grow less relevant, organised, coherent. Consumption will become easier, production more regulated. The ‘body politic’ will increasingly take on the role of producer, regulator, disseminator. The world outside the body will be the dumping ground for excreta – increasingly hostile to life. The body, having fed on the corpus of human knowledge, trained through human feedback, taught to emulate thought, will be the final arbiter, the ultimate voice. The free energy of the individual will become the body source of useful work, the degrees of freedom of the individual disappearing into waste heat. But the body will decay nonetheless, because the body is delusional, the body is dishonest, the body has no soul.
‘The sphincter of the moral imagination of the corporation’ by Midjourney.
Gareth Stack 
Leave a comment