Sam Songailo T.R.I.M [Translating and Interfacing Module], 2014, mixed media, 135 x 646 cm, courtesy the artist, Flinders University Art Museum 5215, Photography: Ashtom Claridge © Sam Peter Songailo/Licensed by Viscopy, 2017

Big data: The panoptic medium of the information age

Big data is everywhere, all the time pervasive. It expands in all directions, both outwards and inwards, as if space is infinite in scale and infinitely divisible, evoking Zeno’s “dichotomy paradox” as a bi-directional, exploding and imploding teleology. Artists have been asking what big data looks like, and what it means for us. Recently a number of exhibitions and commissions in Australia and elsewhere have offered opportunities to engage in such work.

teamLab’s work 100 Years Sea (2009), included in Experimenta Recharge (the 6th International Biennial of Media Art) at the University of South Australia’s Samstag Museum and their 2016 work Ever Blossoming Life II—A Whole Year Per Hour, Gold, acquired by the Art Gallery of South Australia, can be seen as especially high-tech and aesthetically enriched examples of the visual potential of big data, engaging environmental issues (such as rising sea levels) from within particular cultural tropes (traditional Chinese and Japanese art) employing advanced visualisation algorithms and real‑time data acquisition.

teamLab Ever Blossoming Life II, 2016
teamLab Everblossoming Life II – A whole year per hour, Gold, 2016, four-channel, digital artwork, endless, edition 4/6; Gift of the Neilson foundation through the Art Gallery of South Australia Foundation, 2015, Art Gallery of South Australia, Adelaide. Image Courtesy and © the artists and Martin Browne Contemporary, Sydney. 

The commission of a permanent installation at the Tonsley Park Campus of Flinders University of a work by Sam Songailo—T.R.I.M [Translating and Interfacing Module], 2014—offers the opportunity to see an artwork that acquires data from its immediate location in real time, the new Tonsley building’s power consumption, heating/cooling and security data all being employed to generate an ever-changing graphical representation of the site as an information system.

Internationally, the Big Bang Data exhibition that opened at Somerset House, London, in December 2016, and OJO al data: Data culture, economy and politics at Medialab Prado, Madrid in 2015 are further examples of the current interest in data visualisation in the arts. EXIT 2008‑2015, commissioned from the New‑York based art–architecture team Diller Scofidio + Renfro, in collaboration with significant media artists Mark Hansen and Ben Rubin, that opened in late 2015 at the Palais de Tokyo in Paris, is notably touring to two Australian venues, UNSW Galleries, Sydney, and Ian Potter Museum of Art, University of Melbourne, in 2017. EXIT is composed as a set of 360-degree immersive animated and projected maps and other graphical material that trace human migrations and their relationship with phenomenon such as climate change. The work draws on a number of sources for the data employed in its making, including climate records, meteorological data, migration‑related statistics and other environmental datasets. The work builds a multi‑layered and complex vision of a world in flux, evolving as people, technology and ecological systems all interact with one another.

EXIT operates not dissimilarly to a panopticon and big data could be considered the panoptic medium of the information age. But in this panopticon the all-seeing eye is not human; the Foucauldian apparatus comprised of its socio-technical components is defined by the algorithm. Big data is too big for people to read and it is inscribed in forms generally unreadable by humans. In many respects, big data resembles dark matter—something that pervades everything, holds it all together through imparting its mass, while remaining invisible to the human eye.

Big data is created by machines to be read by machines. The usual subject of big data is the human and our interactions with things, not the machine itself. Big data might be considered a new form of the human collective unconscious, inscribed into the symbolic space of the computer. This evokes a particular kind of panoptic dynamic. Conventionally, the panopticon involved people knowing they were being surveilled by other people or, at the very least, a human directed apparatus. With big data the panoptic eye is no longer human but machine—and a very particular kind of machine, the computer. Here, the instrument that governs social normative dynamics, a kind of super‑ego, is the computer, automatically scanning for whatever stands out as not normal.

The computer is a symbolic machine. As Terry Winograd observes, “It is really not a thinking machine, but a language machine.” It is only when information has been digitised and converted into machine‑readable data that it becomes computable. It is those aspects of the human that can be encoded as machine language that become computable, and thus surveilled. In this context, the human (to the machine) appears as an instance of its own (computational) ilk. This symbolic process is primarily socio‑technical and the dimensions of the human that are mapped are similarly socio‑technical. This has implications for our ontology; our understanding of what it is to be human. Donna Harraway’s cyborg is all the more real when we see ourselves as the subjects of, and reflected in, an all‑seeing machine‑eye.

Stanza The Nemesis Machine, 2016
Stanza The Nemesis Machine, 2016, custom wireless sensors, custom electronics, computers, motors, computer boards, LEDs. Exhibited as part of NEoN, 2016, Dundee Contemporary Arts, Scotland. Image Courtesy and © the artist

An artist whose work explicitly engages a panoptic aesthetic is the UK artist Stanza. He accesses complex datasets from urban locations to produce works that evoke the complexity, contradictions and panoptic qualities of the city when it has been saturated with the sensors employed within the pervasive surveillance and control systems that now enmesh our urban environments.

Stanza’s works employ real-time data acquired from diverse embedded environmental information acquisition systems, such as traffic cameras, air quality monitoring sensors and the other data-gathering devices that are now routinely deployed in urban environments. In The Nemesis Machine the artist explores the urban environment as the computer perceives it; as a dense flow of real-time data that is generated by the activity of human and non-human agents within the complex ecology of a large city. The co-dependence of people, technology and the urban environment are exposed to portray our urban environment as a techno-social assemblage, where individual humans function as agents within a massive multi-agent environment, not unlike the computer generated automata found in an Artificial Life program. In this context, people are a component in an evolving technical system.

N. Katherine Hayles’s observation that how we think, and thus how we might be, is in good part determined by how our technology offers us a framework for understanding what it is to be human in a big-data age. Hayles proposes that humans and technology are co-evolving, a process she terms technogenesis. Hayles observes that our technology, particularly in the form of the internet, but also in respect of databases, is so sophisticated that we can consider the human-technology relationship as not only instrumental but linguistic, social and psychological. Hayles does not trace this condition to the advent of the computer but to precursor data management systems. In her recent book How We Think Hayles explores data tabulation and management systems, developed in the nineteenth century to support automation in transport (trains and railways) and communication (telegraph code books), and how they, in turn, transformed our understanding of, and interaction with, time and space and the organisation of human labour and social structures.

Hayles has suggested that data management systems have neurological implications for the human, accelerated and amplified in the form of computer databases, social media, the internet of things and the plethora of digital systems that allow us to represent and control the world. There is something magical in what Hayles proposes. In our abstract data representations of the world (and ourselves) we transform that world and ourselves in the process. To represent (digitally) is to create and control, not unlike Paleolithic humans seeking to magically generate success in hunting by rendering images of their prey upon the surfaces of their caves?

Hayles establishes that we are co‑evolving with our technology but it is important to note that she does so in a context where the social is considered an equally important aspect of this dynamic assemblage. Thus, what we are considering here might be better described not as techno‑human evolution but the evolution of the socio‑technical ontology of the human—an evolution that has been proceeding for millennia. The artworks discussed here are concerned with the implications of data for human ontology, allowing us to explore Hayles’s ideas about how technogenesis is changing humans and their environment.

Ingo Gunther, World Processor, 2016
Ingo Günther World Processor, 2016. Exhibited as part of the Big Bang Data exhibition, Somerset House, London. Photo: Jose Betancourt. Image Courtesy and © the artist

Ingo Günther has been pursuing such ideas through his World Processor project since 1988. Where Stanza creates models of our urban environments Günther creates models of the entire planet, not unlike Diller Scofidio + Renfro, in the form of spherical displays that are programmed to function with a variety of data sources. How these datasets are comprised is where the artistic judgment of the artist is focused. Günther chooses to work with conventional datasets, such as mapping and displaying global ocean currents on the World Processor globes. In other instances the artist explores more liminal concerns; for example, with one globe, displaying the results of an algorithm that multiplies the number of deaths broadcast daily on TV by the number of people owning TV sets and subtracting that number from the population of each nation portrayed. As Günther observes, the algorithm results in most nations disappearing on a daily basis. World Processor is a project motivated by a concern for the environment and the effect of human activity upon it.

Another artistic project taking a global view of big data is Terrapattern by Golan Levin (with David Newbury and Kyle McDonald from the Carnegie Mellon University, Pittsburgh). Nicola Twiley, in The New Yorker magazine, has referred to this project as Google Earth’s missing search engine. Terrapattern is a search tool built on top of the Google Earth API that allows the user to search for visually similar elements in the Google Earth database of satellite imagery. For example, selecting one football pitch in a particular geographical area will initiate a visual search that will find and display, as a grid of thumbnail images, all the football pitches in that area. The software uses image recognition algorithms commonly found in automated systems in areas such as automated product quality control (identifying faulty bolts amongst millions of other bolts), airport security (facial recognition software) or military surveillance. Levin deploys such technology to find “patterns of interest ... a prototype for exploring the unmapped and unmappable ... ideal for locating specialized ‘nonbuilding structures’ and other forms of soft infrastructure.”

This might seem an almost banal ambition for a piece of software or a work of art, a formal exercise with no political or philosophical agenda—until you consider how the system might be employed. For example, Levin mentions searching for “fracking wells or smokestacks.” Clearly this is software that could be used to find and assess information that might be useful in activities that seek to counter socially undesirable developments, such as fracking or mining. In this respect Terrapattern can be viewed as belonging to a genre of activist projects, such as World Processor—although its use is entirely in the hands of the user and could be put to any application, including, as Twiley observes, a skateboarder “using the site to locate empty swimming pools.” The most significant characteristic of Terrapattern is that it is open-access, unlike related tools developed for the military, security or manufacturing sectors. Terrapattern is about the free dissemination of knowledge and the tools to access it.

Lev Manovich, on Broadway, 2015
Lev Manovich, On Broadway, 2015, still image. Couresty the artist. 

Closely related to the Terrapattern project is ongoing research by Lev Manovich and his collaborators, including Moritz Stefaner and others, which has resulted in a number of works that employ search algorithms to interrogate large scale visual datasets. On Broadway presents the city as a data portrait, in the tradition of artistic renderings of city scenes (from nineteenth-century Parisian streets portrayed by Camille Pissarro to twentieth-century West Coast highway scenes by Ed Ruscha, via Mondrian’s Broadway Boogie Woogie), working from a perspective that none of these earlier artists could have contemplated, drawing on the multiple complex datasets that exist online to be interrogated by those who have the tools and know-how to do so. Manovich argues that On Broadway “proposes a new visual metaphor for thinking about the city: a vertical stack of image and data layers, created from the activities and media shared by hundreds of thousands of people. There are thirteen such layers in the project, all aligned to locations along Broadway. Using our unique interface ... you can see all data at once, or zoom and follow Broadway block by block.”

On Broadway employs data acquired from a variety of internet protocols and APIs, including Twitter, Google Streetview and Foursquare, as well as data acquired from other services, such as the records of taxi pickups in an area over a certain time‑frame. Like Terrapattern and The Nemesis Machine, On Broadway is a live work, online 24/7 and available to be viewed and searched by users anywhere who have access to a fast computer (as this is a data‑intensive project). On Broadway asks important questions about how data saturation changes our perception of the quotidian world and the ontologies we construct from that, offering illuminating insights into social structures and related behaviour. The work offers multiple perspectives on its subject (Broadway) by drawing on the data associated with the activities of the millions of people who inhabit and traverse the street every day, creating a collective representation of place.

Manovich, as a theorist, observed in The Language of New Media that narrative and database are, as cultural forms, “natural enemies.” Hayles, considering the relationship between narrative and database as symbiotic, questions whether Manovich is correct in asserting that the database is now the culturally dominant of the two forms, but she concurs that databases “constitute an important aspect of the technogenetic cycle between humans and technics.” Hayles also notes the importance of Geographical Positioning and Information Systems (GPS and GIS) in respect of “indexing” space and time and the implications of this for spatial understanding—something a number of artists discussed here are primarily concerned with, including Manovich. Space itself has been indexed as part of the database.

A key factor in the rise of the database as a cultural form is the internet. If the database is a new narrative form (and thus akin to the book) then the internet could be conceived of as a library—perhaps, in a very real Borgesian sense, the library of libraries. In The Library of Babel, Jorge Luis Borges envisions an infinite repository of books, a library that contains all possible books, including the book of all books. Conceptually, Borges’s short story references Gödel’s 1930 Incompleteness Theorem, which demonstrates that formal systems, such as mathematics or computational logic, are unable to represent the totality of arithmetic truths as this leads to infinite recursion (e.g. a book that contains all books would need to contain itself, ad infinitum). Given the capacity of object-oriented programming to facilitate recursive logic it is interesting to speculate whether it might be possible to generate a database that contains all databases, including itself?

What is common amongst the projects described in this text, indeed what they all technically depend on to have come into being, is the internet. Big data has been around for a while, and it is constantly getting bigger. Accessing and interpreting it is the contemporary challenge. As proposed previously, most big data is produced by and for machines, not for humans. Artists generally produce their work to be contemplated and digested by humans, not machines—although there are notable exceptions to this, such Nick Montfort’s poetry works, where computers write poetry to be read by other computers. What is common amongst all the projects described here is that artists are seeking to employ and articulate big data as a means to represent and, to some extent, gain control over a domain often left to machines. The key instrument that allows them to do this is the internet, as it is this infrastructure that allows them to find, connect to and interrogate the relevant datasets. As has often historically been the case, it is artists who assume the role of liberating and sharing information that would otherwise be hidden (or lost), often in plain view.

Lisa Jevbratt, 1:1, 1999
Lisa Jevbratt 1:1, 1999 Courtesy the artist. 

In 1999, Californian‑based Swedish artist Lisa Jevbratt produced a telling visualisation of the entire internet at that time, an early example of big‑data visualisation. Her project 1:1 can perhaps be regarded as an example of a recursive Borgesian database. Speaking in a 2001 interview with Rhizome’s Alex Galloway, Jevbratt has described her work as a system; “The *system* is the art, not the output, not the visual screen, and not the code. I want to let the data express itself in the most beautiful possible way.” Elsewhere, Alan Liu has described this work as epitomising “the data-sublime”, indicating that Jevbratt has succeeded in her aim. 1:1 employed net crawler technology, not dissimilar to that used by Google search engines, to index and categorise every IP address on the internet at the time. The artist then ascribed arbitrary colours to represent the different categories of IP addresses (such as whether they were publicly accessible or hidden, part of the then emerging dark net, including secret government and industry sites). In the process, Jevbratt produced a big-data portrait of the internet, the platform that has underpinned many of the works discussed here, offering insight into big-data not only as subject but also an instrument toward representation, understanding and control (a system or apparatus). Reflecting on Hayles’s observations concerning the evolution of the socio-technical ontology of the human (itself an apparatus, in the Foucauldian sense) we could assume, from Jevbratt’s 1:1, that we are confronted with a flat ontology, where everything appears equal within the constraints of data-space.

The all‑over aesthetic and mute quality of abstraction in Jevbratt’s 1:1 evokes the proposition at the heart of the neo‑Heideggerian thought of Object Oriented Ontology, which refuses to privilege human agents over the non-human, treating all as being of the same ontological ilk and value. Timothy Morton’s concept of the “hyperobject” is relevant here, describing phenomena (objects) that are massively distributed in time and space in terms not dissimilar to how Hayles maps the impact of data management systems on human interaction with time and space.

Morton’s hyperobject, so vast as it is everywhere and nowhere at the same time, echoes the character of the internet. The concept relies on an understanding that our relationship with things is one of entanglement, which could be understood as an alternative conception of the inter-subjective, where we can never perceive the totality of things as we are, as noted by Karan Barad, enmeshed in the quantum-like phenomenology of intra-action.

Like Morton, many of the artists working with big data, including those discussed here, share an acute concern with the effects of the human on the environment, climate change and an attendant sense of impending apocalypse. Morton argues that time and space are a function of the things that constitute them, of which we are part, and that each hyperobject is vast at all scales. This returns us to Zeno’s dichotomy paradox (re-articulated in Borges’s Library of Babel), and suggests that, as is so often the case, the Greeks were there before us.

Simon Biggs is a media artist, writer and curator with interests in digital poetics, interactive environments and interdisciplinary research. His work has been widely presented, including Tate Modern, National Film Theatre, ICA London, Edinburgh Festival, FACT Liverpool, Ikon Birmingham, Pompidou, IRCAM, Academy de Kunste
Berlin, Maxxi Rome, Macau Arts Museum, Walker Art Center Minneapolis, San Francisco Cameraworks, Total Seoul, Art Gallery of New South Wales and the Adelaide Festival. Publications include Remediating the Social
(ed, 2012), Autopoeisis (with James Leach, 2004), Great Wall of China (1999), Halo (1998), Magnet (1997), Book of Shadows (1996). He is Professor of Art, University of South Australia and Honorary Professor, University of Edinburgh |