SITE : STAGE : STRUCTURE is an archival documentary project. It is a Transmedia Installation that integrates books, objects, photographs, short films, audio narratives, and heritage walks as a way of revitalizing memories and telling a history that is absent from the formal narratives.
Mazgaon is a port and dock area of Mumbai with a rich heritage and complex history. An integral and once central part of the city seems to be almost abandoned, its history neglected, its heritage slowly corroded by the passages of Time and the amnesia of people.
The series of photographs is a cross section of a layered and multi-tiered space, which comprises historic architecture and iconic landmarks amidst more eclectic mix of temporary setups, chawls, shanties, tenements and its inhabitants. The highlight of the exhibition are the paintings, each presenting an important aspect of the city’s history and heritage; as well as photobooks of Mazgaon that document the changing demographic and visual identity of the space. The project maps Mathar Pacady – a catholic village, the remnants of Chinatown, a Bohri (a sect within the Shia Islam) family’s apartment and Darukhana where ships are brought to be torn apart on the dry dock. The project works in multiple directions by documenting the life and living conditions of these diverse communities within the area and the relationships and dynamics of people and places spanning 300 years into the past. Here I fight nostalgia by documenting it, recreating it through videos, conversations and staging ethnographic reconstructions of people’s homes.
Mazgaon has been important to me – my grandparents have lived in Mazgaon for 42 years, for more than my entire life. During school summer vacations, I visited their home and was taken to the famous Mazgaon Gardens. The Garden located on a hilltop offers stunning views of the docks, the ships and the urban landscape of Mazgaon and beyond. This vivid memory has remained with me and I was keen to revisit the space now as an artist.
Historically, Mazgaon has been important to Bombay. Slowly over a period of time, this importance has shifted and declined. Mazgaon has moved from the center of the city to its sidelines. Sites are explored through the lens of history and heritage by looking at the importance of each of these spaces to Mazgaon and to Bombay, and how collectively they formed Mazgaon.
Mazgaon is a vast and changing space. Between the heritage and historical spaces, there are newer spaces. Some of these are devoid of significant history, while others are waiting to be assimilated into the historical and cultural narratives of the future.
Sites revisited on a daily basis over a period of two years have resulted in these spaces becoming ‘stages set for the performances of the everyday’. This transformation is something that I want to make transparent as an experience.
Slowly documenting the Site gives way to the viewing of specific elements – doors, windows, elevations and props. As individual elements they provide the setting and the drama of life – Trivia and personal histories form the script; the seemingly ordinary architecture as well as the normal unassuming people of Mazgaon transform into the set, location and protagonists of the narrative.
Structure is involved with the architectural aspects of the space – the individual houses, individual structures, and personal spaces – the interiors. This leads to another level of enquiry that is concerned with the intimacy and privacy of space as memory.
The Project is a focused attempt to document the history and the present conditions of the people of Mazgaon. They contribute to the city’s self-knowledge as a place with a conflicted and tangled cosmopolitan past. Such projects enrich Mumbai’s art scene by offering something other than aesthetic wall hangings and floor pieces, or theory-laden group shows. In an era in which rightwing groups continue to insist on Mumbai narrowly as a Hindu Marathi city, counter-historical practices like SITE : STAGE : STRUCTURE serve much more than ethnographic curiosity.
Each aspect of the current exhibition can be extrapolated into a substantial body of work. At the same time this existing body of work needs to be disseminated beyond Bombay because it’s not the story of Bombay in itself, or the way Mazgaon has changed. Mazgaon is exemplary of the way several spaces/places in the country and the world are changing – silently, and without our knowing about it. It is like a falling teacup that we don’t see till it shatters. Fortunately Mazgaon and other places like that haven’t shattered yet. But they’re falling. It is up to us how soon we can look at them and how soon we can arrest that fall.
Memories matter in any day and age - I believe in the saying, “Those who forget history, are bound to repeat it”. In the context of the city and its people, the easiest thing would be to forget people who live beyond the perimeter of our immediate vision, but in doing so we condemn ourselves to be forgotten by others – A fairy tale, the origin of a deity, stories of old men recounting childhood and parents, origins of their home, recounting of a 75 year old friendship that lasted relocation to Calcutta and back, the memories of ‘Paradise Lost’ – accounts that do not enter a formal narrative – information that in the grander scheme of things seem not as important, but which we cannot afford to forget.
In this meandering process, I had the good fortune of being introduced to Rafique Baghdadi, a journalist and amateur historian living in Mazgaon for the last 60 years. His unparalleled knowledge, insight and his long lasting relationships with the various integral communities have allowed this project to take the shape it now has.
This project is a documentation of my intimate connection with various people, strangers whose lives are an open book to me. I realized how something as intangible as a community could be reflected very concretely in architecture, colours and spaces, and most importantly in the people, those who hold on and treasure its past as well as those who look towards its future.
This is Mazgaon from 2012 to 2014.
Ali Akbar Mehta
Ali Akbar Mehta talks to Kevin Lobo about his project ‘256 Million Colours of Violence’, Manifestations of Violence, Big Data Ethics and the urgency of asking questions. They discuss tools of inquiry used and attempt to dissect the various narratives presented through the survey. Inquiries into colour, violence and their perceptions can lead to multiple directions – the conversation looks at how the narrative experience of colour may be embodied, embedded and extended in the contexts of these meanings.
The conversation was conducted in Piramal Museum of Art, as part of Limitations of Liability.
Limitations of Liability was an eight day program between March 17 to 28, 2017, structured around ‘256 Million Colours of Violence’ – an interactive survey-based-research project by contemporary transmedia artist, Ali Akbar Mehta. The project began in 2016, and now includes more than 200 entries of people from all walks of life. 256 Million Colours of Violence is available to the public for free. The exposition of the past survey results were installed along with a participation booth at The Mumbai Assembly and Harkat studios inviting everyone to participate in the ongoing survey.
256 Million Colours of Violence Complete Artist Note: http://www.256millioncoloursofviolence.com/about
Limitations of Liability was held across three venues in Mumbai:
- The Mumbai Assembly (Bandra),
- The Piramal Art Foundation (Lower Parel) and
- Harkat Studios (Versova)
The program also included spoken word performances, talks and discussions, and a series of films screenings concerned with the subject of violence and trauma.
In the relation between art and image, photography has played a symptomatic and often paradoxical role. Baudelaire made of it the sinister instrument of the triumph of technical reproduction over artistic imagination. And yet we also know of the long struggle of photographers (pictorialistes) to affirm that photography was not merely mechanical reproduction, but rather an interpretation of the world. But scarcely had they won their battle to endow the technical medium of photography with the status of artistic medium, when Benjamin turned the game on its head. He made mechanical reproduction the principle of a new paradigm of art: the productions of the mechanical arts were for him the means towards a new sensible education, the instruments of the formation of a new class of experts in art, namely in the art of interpreting signs and documents. Cinema was a series of tests of our world. Atget’s photos were indices to interpret; Sander’s collections were notebooks for teaching combatants in the social struggle to readily identify allies and adversaries. The photographic medium participated in the construction of a sensible world where men of the age of the masses could affirm their existence as both possible subjects of art and experts in its use.
It seems, nevertheless, that the destiny of the art of photography has no more confirmed Benjamin’s diagnostic than that of Baudelaire. To support this claim, we can point to two phenomena more or less contemporary to one another that concern both photography and its interpretation. On the one hand, the 1980s saw photography invade art museums and exhibitions, taking on the dimensions of monumental paintings. These large-format photographs, amidst the proliferation of installations and video installations, assure, in a certain sense, the continuity of the pictorial surface. But, at the same time, what they present to us on this surface seems to turn its back on the forms of the pictorial revolutions of the twentieth century. Without even speaking of extreme examples like Jeff Wall’s revival of the historical tableau, we can think of the multiplication of portraits and the new status of the portrait, illustrated by, for example, photographer Rineke Dijkstra’s monumental portraits of otherwise indifferent individuals, represented without any particular aura: slightly awkward-looking adolescents on working-class beaches, young mothers still burdened by their babies, or apprentice toreadors, whose red- faced figures clash with the bull fighter’s traditional suit of lights. On the one hand, these full-length portraits present themselves as documents on social types or age groups undergoing transformation. On the other, the absence of expression, combined with the formal- ism of the pose and the size of the image, gives these indifferent gures something mysterious: something that for us also inhabits the portraits of Florentine and Venetian nobility which populate the museums. The teenager in the green swimsuit photographed on a Polish beach, with her slender body, her swaying hips, and her unfurled hair (below) is like an awkward replica of Botticelli’s Venus. Photography is thus not content to occupy the place of painting. It presents itself as the rediscovered union between two statuses of the image that the modernist tradition had separated: the image as representation of an individual and as operation of art.
How should we think this new coincidence and tension between the grand pictorial form and simply the images of indifferent individuals? The interpretation seems, at first sight, split between two extremes: at the one end, an exacerbation of the sensible presence of the photographed subject, in its provocative power with respect to modernist logic; at the other, an integration of this photographic realism – or hyperrealism – into the modernist scheme. In the first instance, we think of course of Barthes and Camera Lucida, the absolute reference for thought on photography in the 1980s. Barthes’s manoeuvre was to break the representation of the indifferent in two. The indifferent is, on the one hand, that which is identifiable by the intersection of a certain number of general traits. On the other, it is the absolute singularity of that which imposes its brute presence, and affects by this brute presence. We recognize here the principle of the opposition between the studium, conceived as the informative content of the photograph, and the punctum, conceived as its affective force, irreducible to transmission of knowledge. This affective force is the transfer of an absolute singularity, that of the represented subject, to another absolute singularity, that of the viewing subject. It is easy to underline the double paradox of this theorisation in light of the ulterior evolution of photography. It privileges a vision of photographic reproduction where it is the having-been of the body that comes to imprint itself on the sensitive plate, and from there touches us without mediation. This raising of the stakes concerning the indexical conception of photography was immediately countered by the digital invasion. At the moment when large-format photography is about to overrun the museum walls and affirm itself as a visual art, it transforms the photographic gaze into the gaze of an individual who pages through albums. But this historical contretemps refers us back to a more fundamental torsion concerning the relation between photography, art and modernism. In a certain manner, Barthes contorts the formalist modernist, who opposed the form (artistic/pictorial) to the anecdote (empiricist/photographic). Barthes diverts the opposition by transferring the anecdote to the studium, in order to pit it against not the artistic form, but an experience of the unique that refutes the pretension to art as well as the platitude of information. However, this opposition between art and photography is perhaps more profoundly the leave given to another modernity, to which Benjamin’s essay bore witness, and that inscribed photography among the instruments of a new social sensibility and a new social consciousness (three elements and not two). It is from this point of view that it seems useful to me to examine more closely the examples through which Barthes operates the opposition between studium and punctum. Let us take, for example, Lewis Hine’s photograph of the two mentally disabled children (below).
Barthes tells us not to look at the monstrous heads or the pitiful profiles that signify the disability. Instead, he opposes to these the force of fascination that is exerted on him by the details without signification: the boy’s Danton collar, the bandage on the little girl’s finger.1 But the punctum thus marked, in fact, obeys the same formal logic as the repudiated studium. It concerns, in both cases, features of disproportion. The privilege of the punctum here is simply to privatize this formal effect. We can read this analysis as the exact reversal of the critical logic previously put to work by the Barthes of Mythologies. What was at stake for him there, in a Brechtian logic, was to make visible the social hidden in the intimate, the history dissimulated as the appearance of nature. From this point of view, the very choice of the photograph is significant. The photo of the two disabled children appears as a hapax (ἅπαξ λεγόμενον ‘[something] said only once’) in the career of a photographer who devoted numerous series to the representation of work and the campaign against child labor. The ‘stupidity’ of the detail drawn from the irreducible hardship and misfortune of the two disabled children can be read like a screen placed before other photos of children: that of the Polish child, ‘Willie’, working in a mill in Rhode Island, or Francis Lance, the 5-year- old newspaper ‘salesman’. Yet, these ‘documentary’ photographs are the bearers of a tension between visuality and signification that is perhaps more interesting than the image of the two disabled children. They are in effect made for the purpose of denouncing the scandal of child labor. Yet, Willie’s attitude, as he sits nonchalantly (taking his midday rest) in a doffer-box, or Francis Lance’s, proudly standing his ground on a train platform with his newspapers tucked under his arm, do not testify to any suffering. What strikes us is precisely the opposite: it is the selfsame ease with which they show themselves capable of both adapting to their work and posing for the camera, thus obliging Lewis Hine to insist, in his commentary, on the dangers of their work, which they themselves seem so unconcerned about.
The activity of the commentator seems to respond, in advance, to the ‘Benjaminian’ demand. It is, in particular, the relation between the child workers, the camera, the photo and the text that follows this logic, linking the appreciation of the photographic performance to new forms of ‘expertise’ and to the experimentation of a new sensible world. The Danton collar suffices to silently settle the accounts with this logic. The only sensible world that the photo witnesses is the relation of the absolute singularity of the spectacle to the absolute singularity of the gaze. Much the same can be said about Avedon’s photograph of the old slave.2 Here the procedure is reversed: no detail distracts from a sociopolitical reading. On the contrary, the mask of the photographed subject speaks of nothing else than the condition of slavery. But the effect is the same: it is slavery in person, as a historical singularity, that offers itself entirely in the singularity of a single face. To declare slavery to be present in person, in front of our eyes, between our hands, is, in fact, to diminish the singularity of the other photographs that speak to us about what took place between the abolition of slavery and our present. For example, John Vachon’s photo, which shows us only the sign reading Colored, nailed high up on the trunk of a pine tree, next to which is the likely object of its discrimination: a simple drinking fountain. The multiplicity of racial discrimination’s forms of sensible existence, and the multiple singularity of these photographs that vary, and thereby tell us of, the visual forms of the metaphor and of the metonymy, come to be crushed against this black mask that presents slavery in person. But this being of slavery identifies itself with its having-been. Avedon’s photo represents the slavery that is no longer on the face of a man who, himself, is no longer, at the time when Barthes wrote his commentary. When all is said and done, the singularity of slavery written on a singular face is nothing other than the universality of the having-been; in other words, death.
It is to this singularity that the image of the two disabled children, which conceals those of the playing children of the factories, ultimately comes down. But this singularity of the image is itself determined by the power of words alone. Taking up again the two traits of the punctum of this photo, it is rst of all the bandage on the finger of the little girl. The French word with which Barthes refers to the bandage is poupée. Yet the French reader who does not know this usage of the word immediately has another image. The ordinary sense of the word in French is ‘doll’. And the identification of this poignant detail with the poupée inevitability evokes a whole series of images: from Hoffmann’s automaton, commented on by Freud, to the dismembered dolls that are a part of the surrealist imaginary, and that contributed more than a little to the transformation of Winnicott’s transitional object into Lacan’s object petit a. In short, the effect attributed to the phraseless singularity of the detail is the power of a word. And this power of the word is further accentuated by the proper name that quali es another poignant detail: the Danton collar. The French reader has no idea what a Danton collar might be. However, the name is immediately associated with that of a revolutionary who had his head sliced off by the guillotine. The punctum is nothing other than death foretold.
The analysis of the photo of the two mentally disabled children is therefore linked with that which Barthes devotes to the photo of the handcuffed young man. The photo is beautiful, Barthes tells us, and so is the young man, but that is the studium. The punctum is that ‘he is going to die’.3 Yet this death foretold is not visible in any of the features of the photograph. Its presumed effect rests on the combination of the brown colouring of the old photographs and the acquaintance with the individual represented, (in this case) Lewis Payne, condemned to death in 1885 for an attempted assassination of the then American secretary of state. But this affirmation of present death once again employs words to deny what constitutes the visual singularity of the photograph – that is, precisely that its present refuses any readings of the young man’s history, of the past that led him there, and of the future that awaits him. The half-nonchalant, half-curious attitude of the young man says nothing about this history, much the same as Willie’s relaxed pose said nothing about the hardships of factory work, and the gaze of the Polish teenager on the beach nothing about what reasons she might have had for exposing herself, nor her thoughts as she stands in front of the camera. What they speak to us of is only this capacity to expose one’s body at the request of the camera, without, for all that, surrendering to it the thought and the feeling that inhabit it. This tension between exposition and retreat vanishes in the pure relation of the viewer with the death that comes to view him.
This disappearance is not only due to the fact that Camera Lucida is first of all a eulogy addressed to Barthes’s dead mother. Behind the expression of personal grief, there is the expression of another grief, that of the gaze that endeavoured to tie the appreciation of the beauty of an image to that of the social reality that it expressed. Yet, his second grief also manifests itself in a type of reading which, contrary to Barthes, sees in the new modes photographic exposition the reaffirmation of a certain idea of the objectivity of the photograph. It is this thesis that was defended in 1988 by a period defining exhibition entitled ‘Another Objectivity’ (Une autre objectivité). The accompanying text, by Jean-François Chevrier and James Lingwood, redefined, in its own way, the relation between two fundamental aspects of the modernist norm: on the one hand, the fidelity to the law of the medium; on the other, the fidelity to a certain type of exhibition surface, the forme-tableau in its formal separation from the multiple social uses of the image. The fact is that the law of the photographic medium does not offer itself up to a simple interpretation. We can liken it to the instrumental conception that makes the camera a means to furnish some objective information about what is in front of it. But, from this, we still have not defined the specificity of the art of photography. We can liken it to the reproducible character of the photographic image. But it is hardly possible to discern the specific quality of an image from the fact that it is reproducible. This is why the theoreticians of photographic objectivity displaced the idea of multiplication in favour of the idea of a multiple unity. Reproducibility thus becomes seriality.
Benjamin based his argumentation on the typologies of August Sander, while Chevrier and Lingwood favoured the works of Bernd and Hilla Becher. But the analogy is problematic. Benjamin expected that Sander’s series would help the combatants in the
social struggle to recognize allies and enemies. There is manifestly nothing of the sort to be expected from the Bechers’ series of water towers or disused industrial sites. They would even fall easily within the scope of Brecht’s critique, which was taken up by Benjamin: photos of factories say nothing about the social relations that manifest themselves there. The interest of the series can therefore no longer be looked for in what it enables us to say about social relations. It boils down to an ethical virtue accorded to the multiple as such, in that it rules out the prestige of the one and of the aura, of the unique moment and of the ecstatic contemplation. But this principle is purely negative. Its artistic ‘positivity’ must thus come from a second manner of thinking the ‘objectivity’ of the medium. This is summed up, for Chevrier and Lingwood, in the notion of the forme-tableau, exemplified by Jeff Wall’s backlit photographs. But what relation should we think between these large scenes in the form of historical tableaux and the identical rectangles that make the Bechers’ views of water towers and smoke- stacks resemble pedagogical charts? None, perhaps, if not the Greenbergian idea of the surface that encloses the artist’s performance and prohibits him from leaving himself, from showing empathy for his subject or from considering himself as a form of social experimentation. In this sense, the Bechers’ industrial sites are a manner of concluding the dream of the artist engineers and factory builders of Peter Behrens’s era, in much the same way as Barthes’s fascination with the Danton collar served to repress photographer Lewis Hine’s engagement on the side of the oppressed and forgot- ten of the factories and hospices. The reference to the essence of the medium is again here a manner of settling accounts with the epoch where the medium was thought of as the organ of a new collective world. Simply put, this settling of accounts is more complex in the case of the Bechers and the theoreticians of ‘objective photography’, for whom the repression of the constructivist dream also wants to be the affirmation of a fidelity to the values linked to the industrial universe and the workers’ struggle: the sobriety of the documentary gaze that refuses the humanist pathos, the formal principles of the frontal perspective, the uniform framing, and the presentation in series that links scientific objectivity and the disappearance of the subjectivity of the artist.
It remains the case: that which is given to see by the objectivist mindset is fundamentally an absence – disused edifices in the place of social classes and types. Yet, photographing absence can be interpreted in two ways: it can be a manner of showing the programmed departure of the industrial world and worker; but it is just as much a manner of playing on the aesthetic affect of the disused (desaffecté) that sends us back to the side of Barthes’s ‘having-been’. This tension in the objectivist idea of the medium is more perceptible still in the series of containers taken by a follower of the Bechers, Frank Breuer, presented during the 2005 Rencontres photographiques in Arles, in the transept of an ancient church, along with two other series, devoted to warehouses and to logos. From afar the spectator perceived them as abstract scenes or as reproductions of minimalist sculptures. Upon approaching, however, one discovered that the colored rectangles on a white background were containers stacked in a large deserted space. The impact of the series was down to the tension between this minimalism and the signification that it concealed. These containers were to be, or were to have been, filled with merchandise unloaded at Antwerp or Rotterdam, and probably were produced in a distant country, perhaps by faceless workers in Southeast Asia. They were, in short, filled with their own absence, which was also that of every worker engaged to unload them, and, even more remotely, that of the European workers replaced by these distant laborers.
The ‘objectivity’ of the medium thus masks a deter- mined aesthetic relation between opacity and transparency, between the containers as brute presence of pure colored forms and the containers as representatives of the ‘mystery’ of the merchandise – that is to say, of the manner in which it absorbs human work and hides its mutations. It consists in the relation between presence and absence, in the double relation of a visible form to a signi cation and an absence of sense. Jean- François Chevrier bases his argument on the idea of an ‘impoverished ontology’ of photography. On one level, this is to say that photography does not have the strong ontological consistency that would enable its artistic forms to be deduced from its materiality. But we can give this poverty a more positive signification. If photography is not under the law of a proper ontological consistency, linked to the specificity of its technical mechanism, it lends itself to accomplishing the ideas about art formed by the other arts. This capacity of the mechanical art to realize what other arts had tried to accomplish by their own means was developed at length by Eisenstein, in relation to cinematic editing, which, via the temporal sequencing of shots, realized what painting had tried to accomplish in fragments. Serov, for example, tried to bring out on canvas the energy of the actress Yermolova through cutting, with the help of the lines of the mirrors and of the mouldings of a room, several different framings for the different parts of the body. The editing of the different shots of the stone lions in The Battleship Potemkin realized this dream of the painter. Photography allows an accomplishment of the same order by capturing a motionlessness that literature tried to attain through the movement of the phrase or the power of the mystery sought in the contortion of the uses of language. The poverty of photography permits it to realize this inclusion of non-art that literature or painting can only imitate by artistic means.
This is what can be demonstrated by a photograph situated in the interval between Barthes’s ‘having- been’ and the objectivity of the Becher School. Walker Evans’s photograph (left) represents to us a detail of the kitchen in a farm in Alabama. It responds, first of all, to a documentary function at the heart of a major investigation commissioned by the Farm Security Administration. Nevertheless, something happens in the photo that exceeds the task of providing information concerning a miserable situation: a kitchen with neither sideboard nor cupboard, tinplate silverware held in a makeshift rack, a lopsided wooden board nailed to a wall of disjointed and worm-ridden planks. What strikes us is a certain aesthetic disposition marked by disorder: the parallels are not parallel, the silverware is ordered in disorder, the objects on the high beam (functioning as a shelf) are placed in a dissymmetrical manner. This lopsided assemblage composes, in total, a harmonious dissymmetry, the cause of which remains uncertain: is it the effect of chance, the fact that the objects found themselves in front of the objective? Is it the gaze of the photographer, who chose a closeup of a detail, thus transforming a completely random or simply functional layout into an artistic quality? Or is it the aesthetic taste of an inhabitant of the premises, making art with the means available by hammering in a nail or putting a can here rather than there? It is possible that the photographer wanted to show the destitution of the farmers. It is also possible that he simply photographed what was in front of him without any particular intention, and that the photo thus benefits from the beauty of the random. And, it is possible that he took pleasure in seeing a quasi-abstract minimalist scene or, conversely, that he wanted to underline a certain beauty of the functional: the sobriety of the plank and of the rack could, in effect, satisfy a certain aesthetic of design, attracted by the simple and brute material, and the art of living and doing transmitted by generations of simple people. All in all, the aesthetic quality of the photograph stems from a perfect equilibrium, a perfect indecision between the two forms of beauty that Kant distinguished: beauty adherent to the form adapted to its function, and the free beauty of the finality without end.
We don’t know what was going through Walker Evans’s mind in framing his photo as he did. But we do know that he had an idea about art that he inherited, not from a photographer or painter, but from a writer, Flaubert. The idea is that the artist must remain invisible in his work, like God in his creation. But it would be going a bit too far to say that the camera realizes on the cheap – that is, by its mechanism alone – that which, for the writer, involves a never-ending work of subtraction. For impersonality is not the same thing as the objectivity of the camera, and the issue is perhaps not so much to subtract but rather to make the ‘impersonalisation’ of the style coincide with the grasping of the opposite movement: that by which indifferent lives appropriate the aesthetic capacities that subtract them from a simple social identification. The photographer’s gaze upon the singular arrangement of the silverware in a poor Alabama kitchen might remind us of the gaze that Flaubert lent to Charles Bovary as he looked at the head of Minerva, drawn by young Emma for her father on the peeling walls of Father Rouault’s farm. This is not merely to say that the camera directly expresses a poetry of the banal that the writer could only make felt through laborious work on each sentence. It is also the power to transform the banal into the impersonal, forged by a literature that hollows out from the inside the apparent evidence, the apparent immediacy of the photo, just as pictorial silence overran the ‘Flaubertian’ phrase. But this effect of painting on literature and of literature on photography is not the same as a simple shared capacity to trans gure the banality of life into the artistic splendor of indifference. This ‘indifference’ is also the meeting point, the point of tension, between the subtraction of the artistic effect that characterizes the work of the artist and the supplement of aesthetic sensibility that is adjoined to the lives of indifferent beings.
The consideration of both the punctum and the objectivism of the forme-tableau also lacks this relation between social banality and aesthetic power that inhabits the photographic portrait of the indifferent being. To understand what the ‘indifference’ of the photograph of the kitchen in Alabama or of the Polish teenager has in common with that of ‘Flaubertian’ literature, and to what type of ‘modernity’ this indifference bears witness, one must no doubt integrate these images into a completely different evolution of representation ( figuration). To sketch out this history, I would like to dwell for a moment on a singular analysis that Hegel devotes, in his Lessons on Aesthetics, to Murillo’s paintings of the child beggars of Seville, which he saw in the Royal Gallery in Munich. He evokes these paintings in a development whereby he attempts to reverse the classic evaluation of the value of pictorial genres according to the dignity of their subjects. But Hegel does not content himself with telling us that all subjects are equally proper to painting. He establishes a close relation between the virtue of this painting and the activity specific to these young beggars, an activity that consists precisely in doing nothing and not worrying about anything. There is in them, he tells us, a total disregard towards the exterior, an inner freedom in the exterior that is exactly what the concept of the artistic ideal calls for. They are like the young man in one of the portraits at the time attributed to Raphael, whose idle head gazes freely into the distance. Better still, they testify to a beatitude that is almost similar to that of the Olympian gods.
There is one notion in particular in this passage that grabs our attention, that of being carefree. It seems to reply in advance to an analysis of the aesthetic revolution that holds sway today, that by which Michael Fried characterizes the theorizing and the practice of painting implemented by the contemporaries of Diderot. Presenting the characters in the scene as completely absorbed by their task is, for him, the means by which the painters of that period, following the example of Greuze, posed and resolved the big question of artistic modernity: how can a work be made coherent by excluding the spectator from its space? This ‘anti-theatricality’ is for him the essence of pictorial modernity, defined not in a ‘Greenbergian’ manner as simple concentration of the artist on his medium, but rather as definition of the place that it gives to the person who looks upon it. The forme-tableau of Jeff Wall’s lightboxes or of the large-format cibachromes and chromogenic prints by Rineke Dijkstra, Thomas Struth, Andreas Gursky or Thomas Demand seems to Fried to renew, in exemplary fashion, the tradition of this modernity. But it comes at a price, and the active ‘absorption’ of the pictorial character, originally illustrated with such impassioned attention by Greuze’s characters, increasingly becomes an inability to see and to feel seen. Thus, for example, the tourists in Thomas Struth’s photographs of museums are represented in the absence of what they look upon in the Accademia (Michelangelo’s David) or blurred in the darkness in Tokyo in front of a Liberté guidant le peuple, itself separated by a glass pane. Likewise, Rineke Dijkstra’s teenagers are valued first of all for the awkwardness proper to their age, for their lack of control over their bodies which makes them unconscious of what they offer to be seen.7 The window cleaner who, in Jeff Wall’s famous photo, washes the windows of Mies van der Rohe’s pavilion, is not only separated from us by the back that he turns to us and by his relegation outside of the area directly illuminated by the sun; he is also ‘deliberately forgetful’ of the great event signifying the new day, ‘the in ux of the warm morning light’.8 As for the traders at the Hong Kong stock exchange or the workers at the basket factory in Nha Trang, their ‘absorption’ excludes the spectator all the more effectively as it renders them almost invisible by depriving them of all interiority and making of their attention an entirely mechanical process. It would be off-key, Fried emphasizes, to see here any form of representation of capitalist dehumanization. This ‘flattening of absorption’ bears witness, on the contrary, to ‘the consistency with which this artist resists or indeed repudiates all identification by the viewer with the human subjects of his images – the project of severing calls for nothing less’.
‘Objective’ photography therefore demonstrates here the exacerbation of a modernist project of separation. The visual attention that is paid by the modest people, in Greuze’s paintings, to each other and their surroundings is replaced by their ant-sized representation in Gursky’s photographs. But this transformation, in turn, reveals the presuppositions of the analysis: the active absorption of characters by their task is, ultimately, only their passive absorption into the space of the painting. What they are or do matters little, but what is important is that they are put in their place. It is with regard to this positing named absorption that Hegel’s insistence on the carefree inactivity of the young beggars becomes meaningful. Inactivity is not laziness. It is the suspension of the opposition between activity and passivity that aligned an idea of art with a hierarchical vision of the world. Murillo’s child beggars belonged to the type of picturesque paintings that eighteenth-century aristocrats collected as documents on the exotic life of the working classes. Hegel’s analysis removes them from there by giving them a quality which they share with the Olympian gods. This ‘carefree’ attitude is more striking than the new indifference of subjects and their common capacity to be ‘absorbed’. It posits as the exemplary subject of art this ‘doing nothing’, this common aesthetic neutralization of the social hierarchy and of the artistic hierarchy.
The aesthetic capacity shared by the Olympian god, the young noble dreamer and the carefree street child neutralizes the opposition between the subjects of art and the anonymous forms of experience. ‘We have the feeling that for a young person of this type any future is possible’, says Hegel.10 It is a peculiar comment, which makes the gures represented in a seventeenth- century painting contemporary beings whose future we consider. The young beggars testify, in fact, for another modernism far removed from that of Michael Fried’s absorbed characters, without, for all that, becoming identified with the young velocipede racing experts extolled by Benjamin. The future that they bear is the blurring of the opposition between the world of work and the world of leisure, between the naked forms of life and the experiences of the aestheticised world. It is to this modernity that the assertion of Walker Evans’s master, Flaubert, on the indifference of the subject, belongs. This does not mean the possibility for the artist to apply the ‘project of severing’, symbolic of Greenberg’s or Fried’s modernism, to any subject. It is realized only in that space where the artist rids himself of all the habitual attributes of the artist style and comes to encounter the attempts of obscure beings to introduce art into their sensible life, or any other of those forms of experience which their social condition is supposed to forbid. Flaubert may ridicule Emma’s artistic pretension, but her art is forever linked to this artistic aspiration of a farmer’s girl.
It is, similarly, a form of this encounter that James Agee and Walker Evans try to capture, one by bran- dishing Whitmanian enumerations and Proustian reminiscences to describe the houses of poor peasants, the other by rendering minimalist art and social document indiscernible when framing a dozen or so pieces of cutlery in front of four planks of brute wood. Before our gaze, there is thus neither simple objective information about a situation nor a wound inflicted by the ‘it has been’. The photo does not say whether it is art or not, whether it represents poverty or a game of uprights and diagonals, weights and counterweights, order and disorder. It tells us neither what the person who laid the planks and cutlery in this manner had in mind nor what the photographer wanted to do. This game of multiple gaps perfectly illustrates what Kant designated under the name of aesthetic idea: ‘a presentation of the imagination which prompts much thought, but to which no determinate thought whatsoever, i.e., no [determinate] concept can be adequate.’11 The aesthetic idea is the indeterminate idea that connects the two processes that the destruction of the mimetic order left separated: the intentional production of art which seeks an end, and the sensible experience of beauty as finality without end. Photography is exemplarily an art of aesthetic ideas because it is exemplarily an art capable of enabling non-art to accomplish art by dispossessing it. But it is also such through its participation in the construction of a sensible environment which extends beyond its own specificity. What we are shown by the young beggars seen by Hegel, the head of Minerva on the walls of the Normandy farm, the lopsided cans on the beams of the Alabama kitchen, the nonchalant demeanour of the child-worker in his doffer-box, or the swaying hips of the Polish teenager, is that this dispossession which makes art cannot be thought independently of the de-specification which removes all of these characters from their social identity. But this de-specification itself is not the making of an artistic coup de force. It is the correlate of the ability acquired by the characters themselves to play with the image of their being and of their condition, to post it to walls or to set it up before the lens. Judgements about photography are also appreciations of this ability and of what it means for art. This link between artistic purity and aesthetic impurity both fascinated and worried the authors of Spleen de Paris and Madame Bovary. Walter Benjamin wanted to integrate it in a global vision of the new man in the new technical world. Barthes brought it down to the intimacy of the private gaze. Michael Fried now proposes to bring it down to the interminable task of separation attributed to artistic modernity. But this theoretical coup de force would not be possible if the art of photography today was not already the bearer of this tendency to break the historical complicity between the art of the photographer and the aesthetic capacity of his subjects.
Translated by Darian Meacham
- Ibid., p. 34.
- Ibid., p. 96.
- Jean-François Chevrier and James Lingwood, Une autre
1. Roland Barthes, Camera Lucida, Hill & Wang, New York, 1981, p. 51.
objectivité, Prato, Paris, 1989.
5. S.M. Eisenstein, ‘Yermolova’, in Selected Works, vol. 2:
Towards a Theory of Montage, ed. Misha Glenny and Richard Taylor, British Film Institute, London, 1994, pp. 82–105.
6. G.W.F.vonHegel,VorlesungenüberÄsthetikI,Suhrkamp Verlag, Frankfurt, 1986, p. 224.
7. Michael Fried, Why Photography Matters as Art as Never Before, Yale University Press, New Haven CT, 2008, pp. 211–12.
8. Ibid., p. 75
- Ibid., p. 173.
- Vorlesungen über Ästhetik I, p. 224.
- Immanuel Kant, Critique of Judgement, trans. W.S. Plu-har, Hackett, Indianapolis, 1987, p. 182.
From biofeedback to biophysical performance
by Miguel Ortiz
This article describes the evolution of the field of biosignal-driven art. It gives an account of the various historical periods of activity related to this practice and outlines current practices. It poses the question of whether this is an artistic field or if it is only a collection of disassociated practices with mere technical aspects in common. The aim is to draw commonalities and artistic differences between the works of different practitioners and encourage academic and artistic discussion about the key issues of the field.
Biosignal monitoring in interactive arts, although present for over fifty years, remains a relatively little known field of research within the artistic community as compared to other sensing technologies. Since the early 1960s, an ever-increasing number of artists have collaborated with neuroscientists, physicians and electrical engineers, in order to devise means that allow for the acquisition of the minuscule electrical potentials generated by the human body. This has enabled direct manifestations of human physiology to be incorporated into interactive artworks. However, the evolution of this field has not been a continuous process. It would seem as if there has been little communication amongst practitioners, and historically there have been various sudden periods of activity that seem to have very little relation to past works and also a very limited influence in later works beyond some technical methodologies and general operating metaphors.
This paper presents an introduction to this field of artistic practice that uses human physiology as its main element.
The Human Nervous System
It is possible to think of the human nervous system as a complex network of specialised cells that communicate information about the organism and its surroundings (Maton et al., 1994). In gross anatomy, the nervous system is divided into two sub-systems: the Central Nervous System (CNS) and the Peripheral Nervous System (PNS). The CNS is the largest part of the nervous system. For humans, it includes the brain and the spinal cord. It is responsible for coordinating the activity of all parts of the body. It processes information, is responsible for controlling the activity of the peripheral nervous system, and plays a fundamental role in the control of behaviour.
Figure 1. Flowchart of the Nervous System, Taxonomy and Organisation. Nervous System Divisions (http://en.wikipedia.org/wiki/File:NSdiagram.png) by unknown author, CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0). [Click image to enlarge]
The PNS extends the CNS by providing a connection to the body’s limbs and organs. The PNS provides a means for sensing the outside world and for manifesting volitional actions upon it. The PNS is further divided into: Autonomic Nervous System (ANS) and Somatic Nervous System (SNS). The SNS is a component of the peripheral system that is concerned with sensing external stimuli from the environment and is responsible for the volitional control of the skeletal muscles that allow us to interact with the outside world (Knapp, Kim and André 2010). The ANS controls the internal sensing of the various elements that form the nervous system. It regulates involuntary responses to internal and external events and is further sub-divided into Sympathetic Nervous Systems (SNS), which are responsible for physiological changes during times of stress, and Parasympathetic Nervous Systems (PNS), which control salivation, lacrimation, urination, digestion and defecation during the resting state. Figure 1 illustrates the taxonomy and organisation of the Central Nervous System.
There are various techniques and methodologies available to monitor the operation of the nervous system. Changes in human physiology manifest themselves in various ways, ranging from changes in physical properties (i.e. dilatation of the pupils) to changes in electrical properties of organs or specialised tissues (i.e. changes in electrical conductivity of the skin).
Biosignal is a generic term that encompasses a wide range of continuous phenomena related to biological organisms. In common practice, the term is used to refer to signals that are bio-electrical in nature, and that manifest as the change in electrical potential across a specialised tissue or organ in a living organism. They are an indicator of the subject’s physiological state. Biosignals are not exclusive to humans, and can be measured in animals and plants. Excitable tissues can be roughly divided into tissues that generate electrical activity, such as nerves, skeletal muscles, cardiac muscle and soft muscles. Passive tissues that also manifest a small difference of potential include the skin and the eyes. Valentinuzzi defines the latter as “non-traditional sources of bioelectricity” (Valentinuzzi 2004, 219).
Biosignal monitoring has had a large tradition in healthcare ever since Italian physician Luigi Galvani discovered “animal electricity” in 1791 (Galvani 1791; Galvani 1841; Piccolino 1998) which was confirmed three years later by Humboldt and Aldini (Aldini 1794; Swartz and Goldensohn 1998). 1[1. For a more detailed definition of biosignals and their use in the fields of medicine, psychology and bioengineering instrumentation, please see Cacioppo, Tassinary, Berntson (2007) and Webster (1997).]
Galvanic Skin Response (GSR)
Galvanic Skin Response (GSR) is the change of the skin’s electrical conductance properties caused by stress and/or changes in emotional states (McCleary 1950). It reflects the activity of sweat glands and the changes in the sympathetic nervous system (Fuller 1977), and is an indicator of overall arousal state. The signal is measured at the palm of the hands or the soles of the feet using two electrodes between which a small, fixed voltage is applied and measured. Changes in the skin’s resistance are caused by activity of the sweat glands; for example, when a subject is presented with a stress-inducing stimulus; his/her skin conductivity will increase as the perspiratory glands secrete more sweat
The GSR signal is easy to measure and reliable. It is one of the main components of the original polygraph or “lie detector” (Marston 1938) and is one of the most common signals used in both psycho-physiological research and the field of affective computing (Picard 1997).
Figure 2. An ideal ECG signal. [Click image to enlarge]
The ECG is a measurement of the electrical activity of the heart as it progresses through the stages of contraction. Figure 2 shows the components of an ideal ECG signal.
In Human Computer Interaction (HCI) systems for non-clinical applications, the Heart Rate (HR) and Heart Rate Variability (HRV) are the most common features measured. For example, low and high HRs can be indicative of physical effort. In affective computing research, if physical activity is constant, a low HRV is commonly correlated to a state of relaxation, whereas an increased HRV is common to states of stress or anxiety (Haag et al., 2004).
EOG is the measurement of the Corneal-Retinal Potentials (CRP) across the eye using electrodes. In most cases, electrodes are placed in pairs to the sides or above/below the eyes. The EOG is traditionally used in HCI to assess eye-gaze and is normally used for interaction and communication by people that suffer from physical impairments that hinder their motor skills (Patmore and Knapp 1998).
Electromyography is a method for measuring the electrical signal that activates the contraction of muscle tissue. It measures the isometric muscle activity generated by the firing of motor neurons (De Luca and Van Dyk 1975). Motor Unit Action Potentials (MUAPs) are the individual components of the EMG signal that regulate our ability to control the skeletal muscles. Figure 3 illustrates a typical EMG signal and its amplitude envelope.
Figure 3. Example of an EMG signal. [Click image to enlarge]
EMG-based interfaces can recognise motionless gestures (Greenman 2003) across users with different muscle volumes without calibration, measuring only overall muscular tension, regardless of movement or specific coordinated gestures. They are commonly used in the fields of prosthesis control and functional neuromuscular stimulation. For musical applications, EMG-driven interfaces have traditionally been used as continuous controllers, mapping amplitude envelopes to control various musical parameters (Tanaka 1993).
MMG is an alternative technology to measure muscle activity. Unlike EMG, MMG is not an electrical signal, but a mechanical one. The contraction of skeletal muscles creates a mechanical change in the shape of the muscle and subsequent oscillations of the muscle fibres at the resonant frequency of the muscles. These vibrations can be audible and are effectively measured by contact microphones and/or accelerometers (Miranda and Wanderley 2006). Due to the characteristics of the signal, the MMG is also known as the acoustic myogram (AMG), phonomyogram (PMG) and viromyogram (VMG).
EMG and MMG signals share many of their characteristics and could be regarded as different techniques to measure the same phenomena. However, there is an important difference that can be explored in artistic contexts. EMG measures muscle fibre action potentials, that is it measures the activation and firing of motor neurons to trigger contraction of the muscle. MMG measures the actual mechanical contraction of the muscle fibres. This means that there is an important decoupling between both signals as repeated actions and muscle fatigue develop during any given activity (Barry, Geiringer and Ball 1985).
The Electroencephalogram (EEG) monitors the electrical activity caused by the firing of cortical neurons across the brain’s surface. In 1924, German neurologist Hans Berger measured these electrical signals in the human brain for the first time and provided the first systematic description of what he called the Electroencephalogram (EEG). In his research, Berger noticed spontaneous oscillations in the EEG signals (Rosenboom 1999) and identified rhythmic changes that varied as the subject shifted his or her state of consciousness. These variations, which would later be given the name of alpha waves, were originally known as Berger rhythms (Berger 1929, 355; Gloor 1969; Adrian and Matthews 1934).
Brainwaves are an extremely complex signal. In surface EEG monitoring, any given electrode picks up waves pertaining to a large number of firing neurons, each with different characteristics indicating different processes in the brain. The resulting large amount of data that represents brain activity creates a difficult job for physicians and researchers attempting to extract meaningful information.
Brainwaves have been categorised into four basic groups or bands of activity related to frequency content in the signals: Alpha, Beta, Theta and Delta (Lusted and Knapp 1996). Figure 4 shows each of the frequency bands as displayed by an EEG monitoring system.
Figure 4. EEG frequency bands. [Click image to enlarge]
This categorisation however, is the source of certain controversy as some researchers recognise up to six different frequency bands (Miranda et al., 2003). Furthermore, the exact frequency at which each band is divided from the rest is not cast in stone and one might find discrepancies of up to 1 Hz in various texts dealing with the subject. The following categorisation is based on the guidelines provided by the International Federation of Electrophysiology and Clinical Neurophysiology (Steriade et al., 1990):
Delta waves are slow periodic oscillations in the brain that lie within the range of 0.5–4 Hz and appear when the subject is in deep sleep or under the influence of anæsthesia.
Theta waves lie within the range of 4–7 Hz and appear as consciousness slips toward drowsiness. It has been associated with access to unconscious material, creative inspiration and deep meditation.
Alpha rhythm has a frequency range that lies between 8 and 13 Hz. Alpha waves have been thought to indicate both a relaxed awareness and the lack of a specific focus of attention. In holistic terms, it has been often described as a “Zen-like state of relaxation and awareness”.
Beta refers to all brainwave activity above 14 Hz and is further subdivided into 3 categories:
- Slow beta waves (15–20 Hz) are the usual waking rhythms of the brain associated with active thinking, active attention, focus on the outside world or solving concrete problems.
- Medium beta waves (20–30 Hz): this state occurs when the subject is undertaking complex cognitive tasks, such as making logical conclusions, calculations, observations or insights (Rosenboom 1999).
- Fast beta waves (Over 30 Hz): this frequency band is often called Gamma and is defined as a state of hyper-alertness, stress and anxiety (Miranda et al., 2003). It is found when performing a reaction-time motor task (Sheer 1989).
Biosignal-driven Interactive Arts
In 1919, German poet Rainer Maria Rilke wrote an essay entitled Primal Sound, in which he stresses the visual similarity between the surface of the human skull and that of early phonograph wax cylinders. He then speculated about the possibility of transducing the skull’s grooves into this primal sound using a similar technology as it was used for the playback of wax cylinders. In the author’s own words:
The coronal suture of the skull has — let us assume — a certain similarity to the closely wavy line which the needle of a phonograph engraves on the receiving, rotating cylinder of the apparatus. What if one changed the needle and directed it on its return journey along translation of a sound, but existed of itself naturally — well, to put it plainly, along the coronal suture, for example. What would happen? A sound would necessarily result, a series of sounds, music.… Feelings — which? Incredulity, timidity, fear, awe — which of all the feelings here possible prevents me from suggesting a name for the primal sound which would then make its appearance in the world… (Rilke 1978)
Although Rilke never implemented the necessary interface to generate the primal sound, his idea is extremely seductive in its conception and the artistic-æsthetic implications it proposes. Rilke’s text captures the fascination that many artists hold for the possibility of using physiological phenomena to create art. The implication being that something inherently “human-body-like” might result from the direct sonification of bodily features and physiological functions.
Early Pieces and the Biofeedback Paradigm
In the 1960s, a whole generation of artists indeed reappropriated medical tools and developed systems to harness the subtle physiological changes of the human body. These pioneers slowly created a decentralised movement that sought inspiration in medical science to create works that relate to the human being at a physiological level.
In 1964, American composer Alvin Lucier had begun working with physicist Edmond Dewan and became the first composer to make use of biosignals in an artistic context. His piece Music for Solo Performer, scored for “enormously amplified brainwaves”, was premiered at Brandeis University in 1965 (Holmes 2002).
Lucier’s piece explores the rhythmic modulations of the alpha band of brainwaves by means of direct audification and with the addition of percussion instruments — namely cymbals, drums and gongs — which were coupled to large speakers (Teitelbaum 1976). High bursts of alpha activity would cause the speakers to excite the acoustic instruments, which in turn activated a disembodied percussion ensemble.
Music for Solo Performer challenged the very notions of performance and composition. The work is not prescriptive and the composer does not have full control over the end sound result. Similarly, there is no active performance activity, at least not in the way it was understood at the time. The process itself is the music, a feature that is characteristic of Lucier’s musical æsthetic. Regarding the piece Lucier stated:
To release alpha, one has to attain a quasi-meditative state while at the same time monitoring its flow. One has to give up control to get it. In making Music for Solo Performer (1965), I had to learn to give up performing to make the performance happen. By allowing alpha to flow naturally from mind to space without intermediate processing, it was possible to create a music without compositional manipulation or purposeful performance. (Siegmeister et al., 1979)
Lucier’s pioneering use of EEG signals for music composition was quickly adopted by other composers, most notably Richard Teitelbaum and David Rosenboom. Teitelbaum had been working in Rome since the early 1960s as part of the group Musica Elettronica Viva (MEV). In 1967, he presented his work Spacecraft, in which EEG and ECG signals of five performers were used to control various sound and timbre parameters of a Moog synthesiser (Arslan et al., 2005). A year later, Finnish multimedia artist Erkki Kurenniemi attended a music conference organised by the Teatro Comunale in Florence, Italy. During the conference, he was exposed to Mandord Eaton’s ideas of biofeedback as a source for musical practice (Ojanen et al., 2007). Kurenniemi then designed the Dimi-S and Dimi-T. Two electronic musical instruments that measure changes in the electroresistance of the human skin and the production of brainwaves respectively.
During the following years, Teitelbaum explored biosignals further. His 1968 compositions Organ Music and In Tune incorporated the use of the voice and breathing sounds in order to create a close relationship between the resulting music and the human body that generated it (Teitelbaum 1976).
David Rosenboom carried on Teitelbaum’s explorations and, in 1970, presented Ecology of the Skin, a work that measures EEG and ECG signals of performers and audience members (Rosenboom 1999). He was the first artist to undertake systematic research into the potential of brainwaves for artistic applications, creating a large body of works and developing a series of systems that increasingly improved the means of detecting cognitive aspects of musical thinking for real-time music making.
The following year, musique concrète pioneer Pierre Henry began collaborating with scientist Roger Lafosse who was undertaking research into brainwave systems. This collaboration spawned a highly complex and sophisticated live performance system entitled Corticalart (Henry 1971). During the same year, Manford Eaton, who was working at Orcus Research in Kansas City, published Bio-Music (Eaton 1971), a manifesto in which he describes in great detail the apparatus and methods to implement a full biofeedback system for artistic endeavours and calls for a completely new biofeedback-based art in which the intentions of the composer are “fed directly” to the listener by means of careful monitoring and manipulation of the listener’s physiological signals.
Eaton’s system consisted of both audio and visual stimuli for the listener, designed to elicit pre-defined psycho-physiological states which are controlled by the composer. Therefore, his Bio-Music ethos abandons the division between performer and audience. Bio-Music compositions are neither to be “listened to” nor “witnessed” by a large audience, but to be experienced by individual listeners. The composer / performer, adapts his or her algorithms and the presented stimuli to the subject’s individual physiological responses and delivering a consistent “message” or experience for each individual that experiences the work. In Eaton’s Bio-Music, the specific sounds or images presented to the listener are irrelevant as long as they succeed in modulating the subject’s physiological state to that desired by the composer.
Post Biofeedback Practice
Towards the end of the 1980s, the advent of digital signal processing systems and the wide availability of powerful personal computer systems made it possible for researchers to further develop the existing techniques for biosignal analysis in real-time applications. In 1988, California-based scientists Benjamin Knapp and Hugh Lusted introduced the BioMuse system (Knapp and Lusted 1988), which consisted of a signal-capturing unit that sampled eight channels of biosignals, which were then amplified, conditioned and translated to midi messages. The sensors were implemented as simple limb-worn velcro bands that were able to capture EMG, EEG, EOG, ECG and GSR signals. The BioMuse system, facilitated not only the analysis of the signals, but also the ability to use the results of the analysis to control other electronics in a precise and reproducible manner that had not been previously possible (Knapp and Lusted 1990). This allowed them to introduce the concept of biocontrol, an important conceptual shift from the original biofeedback paradigm that had reigned unchallenged during the 1970s. Whilst biofeedback allowed for physiological states to be monitored and, relatively passively, translated to other media by means of sonification, biocontrol proposed the idea and means to create reproducible volitional interaction using physiological data as input (Tanaka 2009).
In order to fully demonstrate the possibilities afforded by their system, Knapp and Lusted commissioned composer Atau Tanaka to write the first piece for their new interface. The BioMuse’s maiden concert took place in Stanford California in 1989. In that concert, Tanaka premiered Kagami, a piece that used EMG signals measuring muscular tension on his forearms (Keislar et al., 1993). This introduced a novel biosignal performance practice that consisted of a highly personal visual and sonic style of biosignal-driven music and stage presence, moving from the archetypal image of the motionless centre-stage-seated bio-performer pioneered by Lucier, to a dynamic musician that explored arm gestures in a highly engaging way.
In 1998, Teresa Marrin-Nakra and Rosalind Picard, who were carrying out research in the field of affective Computing at the Massachusetts Institute of Technology (MIT), created The Conductor’s Jacket, a wearable computing device that facilitated the measuring and recording of physiological and kinematic signals from orchestra conductors (Marrin and Picard 1998). Even though The Conductor’s Jacket was originally conceived as a recording and monitoring device for scientific enquiry, its ability to stream data in real time allowed Nakra to use it in performance contexts, where it functioned not as a passive monitoring device but as a disembodied musical instrument.
Biosignals in the 21st Century
The Early 2000s
The turn of the 21st Century brought with it a renewed worldwide interest in biosignals for artistic applications, as many favourable factors converged. On the one hand, personal computers became powerful enough to deal with these types of signals. Likewise, the evolution of the BioMuse and other biosignal measuring devices created by the affective computing team at MIT meant that it was now possible to ecologically measure physiological signals from performers in stage situations in a transparent and effective way. 2[2. The term “ecological” is often used in the field in similar contexts to refer to measurement techniques that do not impede the natural execution of actions by performers.] Moreover, commercially available medical equipment such as the g.MOBIlab, Emotiv’s EPOC, MindMedia’s Nexus units and Thought Technology’s Infiniti systems have become more affordable and easy to use. This contrasts with the complicated and extremely costly systems that were available for artists in the 1970s. The popularisation of the internet, as well as the establishing of international conferences and symposia that dealt with musical interactive systems, meant that information could be shared by artists and researchers at faster rates than ever before, allowing for a steady incremental development of biosignal interfaces and art works created for these novel devices.
This makes the issue of meaning and content even more relevant than ever. The various technologies that facilitate the measurement of biosignals as well as their correlates to human emotion have undergone a great development, yet the associated approaches and metaphors that artists use to create works using these technologies remain relatively unchanged.
In 2002, Australian artist Tina Gonsalves began to explore the ways in which her artworks could monitor the audience’s emotional states, through “the use of bio-metric sensors as triggers for emotional video narratives, leading to both more immersive installations, as well as intimate ubiquitous works.” (Gonsalves 2009).
Her work explores issues of intimacy, empathy and emotional behaviour in humans through immersive audio-visual installations. According to Gonsalves:
Nothing seems as private as the bodily experience of raw emotion. Emotions are a common thread that every human being can read, understand, and share. Emotions influence all aspect of behaviour and subjective experience; grabbing attention, enhancing or blocking memories and swaying logical thought. Emotions spread in social collectives almost by contagion. In cohesive social interactions, we are highly attuned to subtle and covert emotional signals, Our behaviours often mirror each other in minute detail. At times, we may voluntarily suppress our emotional reactions, temporarily disguising our intentions or vulnerability, though “true” emotions are nevertheless evident in a pattern of internal bodily responses that set an underlying tone for behaviour. (Gonsalves 2009).
In 2004, during the International Conference on Auditory Displays (ICAD), Stephen Barrass organised a practice-led research project entitled Listening to the Mind Listening. The project consisted of an open call for composers to write a piece based on the brain activity of a person listening to a piece of music. The data-set was recorded from the brain activity generated by the chief executive officer of the Brain Resource company, Evian Gordon, as he listened to David Page’s composition Dry Mud (1997). The essential criteria for the pieces submitted were as follows (Barrass 2004):
- Data-driven. Sonification is a mapping of data into sounds for some purpose. The sonification should be the result of an explicit mapping from the data into sounds. The listener should be able to understand relations and structures in the data from the sonification.
- Time is the binding. The timeline of the data must map directly to the timeline of the sonification. All other mapping decisions are completely open but we need to be able to compare pieces across time, and also compare them with the original data set and source piece of music. This means that the final sonification pieces will all be exactly the same duration as the data set, and original piece of music.
- Reproducibility. The mapping of the data into sound must be described in a manner than can be reproduced by others. Mappings should be described explicitly. Different mappings will enable different perceptions of information in the data. The experiment should lay a foundation for scientific and æsthetic observations and ongoing development by the research community.
The concert was followed by a set of reviews carried out by the composers themselves, sonification scientists, brain scientists and members of the attending audience. According to Barrass, the experiment resulted not only in a successful concert which presented pieces that were æsthetically well received by the audience, but in providing a clear auditory display that could be used for scientific purposes, thanks to the systematic sonification approach to composition (Barrass et al., 2006).
During the same year, London-based artist Christian Nold began the Bio Mapping project, a community-led initiative to emotionally map urban environments. According to Nold, throughout the duration of the project, over 1500 people took part in it. The project consisted of equipping participants with a GSR measuring device and a Global Positioning System (GPS) unit. As the participants walk and interact with the urban surroundings, their arousal level is measured by the GSR sensor and linked to the specific location where important variations on the signal occur, thus creating detailed communal emotion maps which display areas within the city that people feel strongly about.
A year later, Ben Knapp and Perry Cook extended the original concept of biocontrol and introduced a conceptual framework which they called the Integral Music Controller (IMC) [Knapp and Cook 2005]. They define four categories of possible musical interfaces based on the type of interaction between performer and the resulting sound as follows:
- Traditional Physical Interface. All acoustic instruments fall into this category. There is a direct physical-acoustic coupling between the performer’s actions and the production of sound.
- Augmented Interface. When one embeds additional sensing modalities to an existing traditional instrument. HyperInstruments fall under this category, the idea being that the interface is augmented to sense events which would not normally produce changes in the instrument’s sound producing or modulating capabilities. For example, a trumpet with pressure sensors on the pistons that can change the sound as more pressure is applied to them, a gesture that does not make any discernable sound difference in a non-augmented trumpet.
- Remote Interface. When there is no direct physical connection between the performance action and the sound being produced. The iconic example for this type of interface is the computer itself, specifically laptop computers which have become pervasive in concert environments.
- Emotion Interface. An interface that allows for motionless emotion-driven interaction between the performer and the resulting sounds. Any of the biofeedback interfaces reviewed so far in this chapter could fit in this category, but only once proper analysis has been carried out to identify and assess emotion.
Based on this categorisation, the IMC can be seen as a framework within which specific musical interfaces (or instruments) that afford all possible interactions can be developed in a transparent way. According to Knapp and Cook (2005), the IMC is defined as a controller that:
- Creates a direct interface between emotion and sound production unencumbered by the physical interface.
- Enables the musician to move between this direct emotional control of sound synthesis and the physical interaction with a traditional acoustic instrument and through all of the possible levels of interaction in between.
Figure 5. The Integral Music Controller Pyramid (Knapp and Cook 2005). [Click image to enlarge]
Figure 5 shows the IMC pyramid, which illustrates the growing complexity of available interfacing options from simple physico-mechanical interaction to the emotion-driven interaction model. According to Knapp and Cook, the image also illustrates the number of available interfaces for each category which narrows down as we climb up the pyramid. I would add to this, that our understanding as creators of each interactive model follows a similar shape, from the well understood traditional instruments to the complex interaction with biosignal-driven interfaces.
In 2006, the artist duo Terminalbeach collaborated with the Trondheim Sinfonietta to create the Heart Chamber Orchestra (HCO). The HCO project consisted of an audio-visual composition where real-time ECG signals were measured from a group of twelve classically trained musicians. The data generated by the performer’s heart beats was used to control a computer composition and visualisation environment which generated a score in real time for the musicians to play, as well as the generation of electronic sound and visual content. During the performance, the score was constantly changed by the state of the musicians’ heart beats and vice versa, creating a closed feedback loop system. 3[3. See Peter Votava and Erich Berger’s article “The Heart Chamber Orchestra: An audio-visual real-time performance for chamber orchestra based on heartbeats” in this issue of eContact! for a discussion and videos of the project.]
Brazilian composer Eduardo Reck Miranda, a researcher at the Interdisciplinary Centre for Computer Music Research at the University of Plymouth, has carried out important research in the field of EEG monitoring by proposing Brain-Computer Musical Interfaces (BCMI) [Miranda and Brouse 2005a, 2005b; Miranda 2006]. His research focuses on the identification and classification of music-related cognitive processes for the direct control of generative musical algorithms. Miranda has published a series of articles describing the technical and musical implications of BCMIs. Canadian composer Andrew Brouse has collaborated with Miranda, using BCMIs to create meditative musical compositions. Miranda’s research has also fed back into the medical sciences field.
Current State of the Field
The second half of the 2000s has seen an explosion in the use of biosensing technologies in interactive art practice. This has started a process of diversification of practitioners and their original disciplines. Whilst the early works were pioneered by composers linked to the western contemporary or electroacoustic traditions, the turn of the century has seen a large number of artists from very diverse mediums that span from fine arts, media arts, cinema and philosophy.
In 2005, Belgian researcher Benoît Macq directed a project at the eNTERFACE Summer Workshop on Multimodal Interfaces dedicated to the creation of a musical instrument driven by biosignals (Arslan et al., 2005). The project had great success, and for the next four years the core team assembled by Macq continued participating in the eNTERFACE workshops improving their design implementation for biosignal-driven musical instrument. This allowed not only for the team members to work directly in this field but various artists who participated for a single year had the opportunity of approaching the use of these technologies in collaboration with highly skilled technicians as they were developing the instruments. Amongst some of the artists who had these opportunity we have Hanna Drayson and Cumhur Erkut.
A year later, BioMuse creator Benjamin Knapp joined the Sonic Arts Research Centre (SARC) at Queen’s University Belfast. There, he founded the Music, Sensors and Emotion (MuSE) research group. MuSE is a multidisciplinary team focused on both qualitatively and quantitatively measuring the relationship between music and emotion and using this information to inform musical composition and performance. The main areas of interest for the team are:
- Integral music control: Using both gestural and emotional interactions to control digital musical instruments.
- The use of physiological and kinematic ambulatory monitoring to measure gesture and emotion during performance.
- Physiological monitoring of audiences during performance.
- Contagion of emotion and physiological signals between performer and audience.
Within these areas, the MuSE team has produced various biosignal-driven interactive installations (Coghlan et al., 2009; Jaimovich 2010) and high quality scientific research (Jaimovich et al., 2011).
During 2006, my own work with biosignal interfaces started at Queen’s University as one of the founding members of the MuSE team. Following on the tradition of contemporary classical music practice, my work has largely focused in the integration of physiological sensing technologies with traditional acoustic instruments. In a way that compliments the ongoing research within MuSE, my artistic practice does not focus on the assessment of emotional states but on the intrinsic audible characteristics of biosignals themselves as a source for composing materials and as medium of performance. In a way that could be compared to hyperinstruments (Machover and Chung 1989), I have focused on extending not the instruments, but the performer. I.e., taking the physical actions required to perform on a traditional instrument and the physiological states that these actions produce on the performer as the driving element of sound output. A good example of this is my 2010 work S&V for saxophone, violin and heart rate monitor.
Video 1 (12:38). Gascia Ouzounian and Franziska Schroeder performing Miguel Ortiz’ S&V (2010) at the Sonic Arts Research Centre, Belfast (UK) 2010.
During his time as director of MuSE, Knapp went from instrument designer to performer as he founded the Biomuse Trio in 2008 to perform computer chamber music integrating traditional classical performance, laptop processing of sound and the transduction of bio-signals for the control of musical gesture. The work of the ensemble encompasses hardware design, audio signal processing, bio-signal processing, composition, improvisation and gesture choreography. The Biomuse Trio consists of Gascia Ouzounian (violin), Ben Knapp (BioMuse) and Eric Lyon (computer). 4[4. MuSE research and projects and compositions using the BioMuse system are discussed in “The Biomuse Trio in Conversation: An Interview with R. Benjamin Knapp and Eric Lyon” by Gascia Ouzounian, in this issue of eContact!]
In December 2010, new media artist and sonic artist Marco Donnarumma presented Music for Flesh I at the University of Edinburgh, a first public performance using his Xth Sense system. Xth Sense is an MMG-driven interactive system for the biophysical generation and control of music (Donnarumma 2011). Drawing from the growing fields of open source software and hardware, Xth Sense is not only a bespoke system for the sole use of its creator, but a set of tools that can be easily implemented by the community at large and at a very low cost. This marks an important milestone as it opens the doors for a grassroots physiological-driven arts practice outside university walls.
In April 2012, the Sonorities Festival at Queen’s University had a special theme titled The Body’s Music. The festival focused on innovative relationships between music and the body. In particular, explorations of the combination of body and technology, while delving into the sonic world of the body itself. During the parallel Two Thousand + TWELVE symposium, a plenary session was held with the participation of experts Ben Knapp and Atau Tanaka and some of the more active new practitioners Marco Donnarumma, Miguel Ortiz and Gascia Ouzounian. During this open discussion some of the most relevant questions that had previously only been made in conference hallways and late night pub sessions were tackled, namely:
- Is this a field or just a collection of decentralised individual artistic practices?
- Are there any connections or direct progress from the initial intentions of the early pioneers of the 1960s and current practices?
- Are there any intrinsic musical properties in these signals or are they just a bridge between higher cognitive and emotional states and musical outputs?
- Are there any opportunities for collaboration and focused development of biosignal-driven musical practice or are the individual interests of practitioners too focused for any commonalities?
Despite long discussions during the plenary session and throughout the festival, this (and other) questions remain open…
The use of biosignal monitoring technologies in interactive art contexts has been present for over sixty years. From Alvin Lucier’s pioneering work Music for Solo Performer to the current practice of biosignal-driven performance and sound installation, the field has advanced both in its technical implementations and the artistic affordances that the medium provides. Developments in medicine and psychophysiology allow us to understand better the meaning and implication of human-generated electrical signals and their correlation to emotion. The work carried out by the Affective Computing Group at MIT and the Music, Sensors and Emotion team at SARC has facilitated the technical aspects of biosignal monitoring for interactive artistic practice. Furthermore, the technical and social advancements in the wider electronic music field that appeared in the early 2000s has evolved and matured into the establishment of specific interest groups that allow for thorough research and artistic practice in the field.
It is now easier than ever to incorporate physiological measurements onto the stage; thus, biosignal-driven art can now be carried out in a practical way, without the need for the large and expensive equipment used in the early 60s and 70s. This opens the door for deeper artistic and æsthetic explorations, which in our opinion should become the central focus of creative work.
The opportunities seem open ended and it is up to the artistic community interested in harvesting the physiologic mechanisms of the human body to fully realise their potential beyond the mere novelty of the interface.
Adrian, Edgar D. and Bryan H.C. Matthews. “The Berger Rhythm: Potential changes from the occipital lobes in man.” Brain 57/4 (1934), pp. 355–383.
Aldini Giovanni. De animali electricitate dissertationes duae. Typographia Instituti Scientiarum: Bonomiae, 1794.
Arslan, Burak, Andrew Brouse, Julien Castet, Jean-Julien Filatriau, Rémy Lehembre, Quentin Noirhomme and Cédric Simon. “Biologically-driven Musical Instrument.” eNTERFACE 05: Summer Workshop on Multimodal Interfaces (Mons, Belgium, 17 July – 11 August 2005). ISCA, 2005.
Arslan, Burak, Andrew Brouse, Julien Castet, Jean-Julien Filatriau, Rémy Lehembre, Quentin Noirhomme and Cédric Simon. “From Biological Signals to Music.” Enactive05. Proceedings of the 2nd International Conference on Enactive Interfaces (Genoa, Italy, 17–18 November 2005). Available online at http://tcts.fpms.ac.be/publications/papers/2005/enactive05_abbacjfjjlrnqsc.pdf [Last accessed 26 April 2010]
Barrass, Stephen. “Listening to the Mind Listening.” Call for submissions for a Concert of Sonifications at the Sydney Opera House during ICAD 2004. http://www.icad.org/websiteV2.0/Conferences/ICAD2004/concert_call.htm
Barrass, Stephen, Michael Whitelaw and Freya Bailes. “Listening to the Mind Listening: An Analysis of sonification reviews, designs and correspondences.” Leonardo Music Journal 16 (December 2006) “Noises Off: Sound Beyond Music,” pp. 13–19.
Barry, Daniel T., Steven R. Geiringer and Richard D. Ball. “Acoustic Myography: A Noninvasive Monitor of Motor Unit Fatigue.” Muscle Nerve 8/3 (March-April 1985), pp. 189–94.
Berger, Hans. “Über das Elektroenkephalogramm des Menschen. I. Mitteilung.” Archiv für Psychiatrie und Nervenkrankheiten 87 (1929), pp. 527–570.
Cacioppo, John T., Louis G. Tassinary and Gary G. Berntson (eds.). Handbook of Psychophysiology. Cambridge, UK: Cambridge University Press, 2007.
De Luca, Carlo J. and Egbert J. Van Dyk. “Derivation of Some Parameters of Myoelectric Signals Recorded During Sustained Constant Force Isometric Contractions.” Biophysical Journal 15/12 (1975), pp. 1167–1180.
Coghlan, Nial, Javier Jaimovich, R. Benjamin Knapp, Donald O’Brien and Miguel Ortiz. “AffecTech: an Affect-Aware Interactive AV Artwork.” ISEA 2009. Proceedings of the 15th International Symposium of Electronic Arts (Belfast: University of Ulster, 23 August – 1 September 2009).
Donnarumma Marco. “Xth Sense: A Study of muscle sounds for an experimental paradigm of musical performance.” ICMC 2011: “innovation : interaction : imagination”. Proceedings of the International Computer Music Conference (Huddersfield, UK: Centre for Research in New Music (CeReNeM) at the University of Huddersfield, 31 July – 5 August 2011).
Eaton, Manford. Bio-Music: Biological feedback, experiential music system.Kansas City, USA: Orcus, 1971.
Fuller, George D. Biofeedback: Methods and procedures in clinical practice. San Francisco: Biofeedback Press, 1977.
Galvani, Luigi. “De viribus electricitatis in motu musculari commentarius.” Comment Bonon Scient et art Inst Bologna 7 (1791), pp. 363–418.
_____. Opere edite ed inedite del Professore Luigi Galvani raccolte e pubblicate dall’Accademia delle Science dell’Istituto di Bologna. Edited by Silvestro Gherardi. Accademia delle Science dell’Istituto di Bologna: Dall’Olmo, Bologna. 1841.
Gloor, Pierre. Hans Berger on the Electroencephalogram of Man. The Fourteen Original Reports on the Human Electroencephalogram. New York: Elsevier, 1969.
Gonsalves, Tina. “Tina Gonsalves’ Digital Folio.” 2009. http://www.tinagonsalves.com/INTERFrame.html [Last accessed 26 April 2010].
Greenman, Phillip E. Principles of Manual Medicine. Philadelphia, USA: Lippincott Williams & Wilkins, 2003.
Haag, Andreas, Silke Goronzy, Peter Schaich and Jason Williams. “Emotion Recognition Using Bio-Sensors: First Steps Towards an Automatic System.” In Affective Dialogue Systems. Berlin/Heidelberg: Springer Verlag, 2004, pp. 36–48.
Henry, Pierre. Mise en musique du corticalart de Roger Lafosse (1971). Prospective 21e Siècle [6521 022]. 1971.
Holmes, Thomas B. Electronic and Experimental Music: Pioneers in Technology and Composition. New York: Routledge, 2002.
Jaimovich, Javier. “Ground Me!: An Interactive Sound Art Installation.” NIME 2010. Proceedings of the 10th International Conference on New Instruments for Musical Expression (Sydney, Australia: University of Technology Sydney, 15–18 June 2010). pp. 391–394.
Jaimovich, Javier, Nicholas Gillian, Paolo Coletta, Esteban Maestre and Marco Marchini. “Workshop on Multi-Modal Data Acquisition for Musical Research.” NIME 2010. Proceedings of the 10th International Conference on New Instruments for Musical Expression (Sydney, Australia: University of Technology Sydney, 15–18 June 2010). p. 551.
Juslin, Patrik N. and John A. Sloboda. Music and Emotion: Theory and Research. Series in Affective Science Oxford, UK: Oxford University Press, 2001.
Knapp, R. Benjamin, Jonghwa Kim and Elisabeth André. “Physiological Signals and their Use in Augmenting Emotion Recognition for Human-Machine Interaction.” In Emotion-Oriented Systems, Cognitive Technologies. Edited by Roddy Cowie, Catherine Pelachaud and Paolo Petta. Berlin/Heidelberg: Springer Verlag, 2011, pp. 133–159.
Knapp, R. Benjamin and Perry R. Cook. “The Integral Music Controller: Introducing a direct emotional interface to gestural control of sound synthesis.” ICMC 2005: Free Sound. Proceedings of the International Computer Music Conference (Barcelona, Spain: L’Escola Superior de Música de Catalunya, 5–9 September 2005), pp. 4–9.
Knapp, R. Benjamin and Hugh S. Lusted. “A real-time digital signal processing system for bioelectric control of music.” Acoustics, Speech, and Signal Processing, 1988. ICASSP-88. 1988 5, pp. 2556–2557.
Knapp, R. Benjamin and Hugh S. Lusted. “A Bioelectric Controller for Computer Music Applications.” Computer Music Journal 14/1 (Spring 1990) “New Performance Interfaces (1),” pp. 42–47.
Keislar, Doug, Robert Pritchard, Todd Winkler, Heinrich Taube, Mara Helmuth, Jonathan Berger, Jonathan Hallstrom and Brad Garton. “1992 International Computer Music Conference, San Jose, California USA, 14–18 October 1992.” Computer Music Journal 17/2 (Summer 1993) “Synthesis and Transformations,” pp. 85–98.
Kreibig, Sylvia D., Frank H. Wilhelm, Walton T. Roth and James J. Gross. “Cardiovascular, Electrodermal and Respiratory Response Patterns to Fear- and Sadness-Inducing Films.” Psychophysiology 44/5 (September 2007), pp. 787–806.
Lusted, Hugh S., and R. Benjamin Knapp. “Controlling Computers with Neural Signals.” Scientific American275/4 (October 1996), pp. 82–87.
Machover, Tod and Joe Chung. “Hyperinstruments: Musically Intelligent and Interactive Performance and Creativity Systems.” ICMC 1989. Proceedings of the International Computer Music Conference (USA: Ohio State University, 1989). Key: citeulike:794359
Marrin, Teresa and Rosalind W. Picard. “The Conductor’s Jacket: A Device for recording expressive musical gestures.” ICMC 1998. Proceedings of the International Computer Music Conference (Ann Arbor MI, USA: University of Michigan 1998), pp. 215–219.
Marston, William M. Lie Detector Test. New York: R.R. Smith, 1938.
Maton, Anthea, Jean Hopkins, Susan Johnson, David LaHart, Maryanna Q. Warner and Jill D. Wright. Human Biology And Health. Englewood Cliffs NJ: Prentice Hall, 1994.
McCleary, Robert A. “The Nature of the Galvanic Skin Response.” Psychological Bulletin 47 (1950), pp. 97–117.
Miranda, Eduardo Reck. “Brain-Computer Interface for Generative Music.” ICDVRAT 2006. Proceedings of the 6th International Conference on Disability, Virtual Reality and Associated Technologies (Esbjerg, Denmark, 18–20 September 2006), pp. 295–302.
Miranda, Eduardo Reck and Andrew Brouse. “Toward Direct Brain-Computer Musical Interfaces.” NIME 2005. Proceedings of the 5th International Conference on New Instruments for Musical Expression (Vancouver: University of British Columbia, 26–28 May 2005), pp. 216–219.
_____. “Interfacing the Brain Directly with Musical Systems: On Developing systems for making music with brain signals.” Leonardo 38/4 (August 2005), pp. 331– 336, 2005b. doi: 10.1162/0024094054762133. URL http://dx.doi.org/10.1162/ 0024094054762133.
Miranda, Eduardo Reck, Ken Sharman, Kerry Kilborn and Alexander Dunca. “On Harnessing the Electroencephalogram for the Musical Braincap.” Computer Music Journal 27/2 (Summer 2003) “Live Electronics in Recent Music of Luciano Berio,” pp. 80–102.
Miranda, Eduardo Reck and Marcelo Wanderley. New Digital Musical Instruments: Control And Interaction Beyond the Keyboard. Computer Music and Digital Audio Series. Madison WI, USA: A-R Editions, Inc. 2006. http://portal.acm.org/citation.cfm?id=1201683
Nold, Christian (ed.). Emotional Cartography — Technologies of the Self. Softbook published under a Creative Commons Attribution, 2009. http://emotionalcartography.net [Last accessed 26 April 2010].
Ojanen, Mikko, Jari Suominen, Titti Kallio and Kai Lassfolk. “Design Principles and User Interfaces of Erkki Kurenniemi’s Electronic Musical Instruments of the 1960s and 1970s.” NIME 2007. Proceedings of the 7th International Conference on New Instruments for Musical Expression (New York: New York University, 6–10 June 2007), pp. 88–93. DOI=10.1145/1279740.1279756 http://doi.acm.org/10.1145/1279740.1279756
Ortiz, Miguel. “Towards an Idiomatic Compositional Language for Biosignal Interfaces.” Unpublished PhD Thesis. Belfast, Northern Ireland: Queen’s University, 2010.
Patmore, David W. and R. Benjamin Knapp. “Towards an EOG-based Eye Tracker for Computer Control.” Proceedings of the 3rd International ACM Conference on Assistive Technologies (New York, USA, 1998), pp. 197–203.
Picard, Rosalind W. Affective Computing. Cambridge MA, USA: MIT Press. 1997.
Piccolino, Marco. “Animal Electricity and the Birth of Electrophysiology: The Legacy of Luigi Galvani.” Brain Research Bulletin 46/5 (1998), pp. 381–407.
Rilke, Rainer Maria. Where Silence Reigns: Selected Prose. New York: New Directions Publishing Corporation, 1978.
Rosenboom, David. “Extended Musical Interface with the Human Nervous System: Assessment and prospectus.” Leonardo 32/4 (August 1999), p. 257.
Siegmeister, Elie, Alvin Lucier and Mindy Lee. “Three Points of View.” The Musical Quarterly 65/2 (April 1979), pp. 281–295. ISSN 00274631. URL http://www.jstor.org.ezproxy. qub.ac.uk/stable/741709. ArticleType: primary article / Full publication date: Apr., 1979 / Copyright © 1979 Oxford University Press.
Sheer, Daniel E. “Sensory and Cognitive 40–Hz Event-Related Potentials: Behavioral correlates, brain functions and clinical application.” Brain Dynamics. Edited by E. Basar and T.H. Bullock. Springer series in Brain Dynamics vol. 2. Berlin/Heidelberg: Springer Verlag, 1989, pp. 339–374
Steriade, Mircea, Pierre Gloor, Rodolfo R. Llinas, Fernando H. Lopes da Silva and M. Marsel Mesulam. “Basic Mechanisms of Cerebral Rhythmic Activities.” Electroencephalography and Clinical Neurophysiology 76/6 (December 1990), pp. 481–508.
Swartz, Barbara E. and Eli S. Goldensohn. “Timeline of the History of EEG and associated fields.” Electroencephalography and Clinical Neurophysiology 106/2 (February 1998), pp. 173–176.
Tanaka, Atau. “Musical Technical Issues in Using Interactive Instrument Technology with Application to the BioMuse.” ICMC 1993. Proceedings of the International Computer Music Conference (Japan: Waseda University, 1993), pp. 124–126.
_____. “Sensor based Musical Instruments and Interactive Music.” In The Oxford Handbook of Computer Music. Edited by Roger T. Dean. Oxford, UK: Oxford University Press, 2009, pp. 233–257.
Tanaka, Atau and R. Benjamin Knapp. “Multimodal Interaction in Music using the Electromyogram and Relative Position Sensing.” NIME 2002. Proceedings of the 2nd International Conference on New Instruments for Musical Expression (Dublin: Media Lab Europe, 24–26 May 2002), pp. 1–6.
Teitelbaum, Richard. “In tune: Some early experiments in biofeedback music (1966–1974).” In Biofeedback and the Arts: Results of early experiments. Edited by David Rosenboom. Vancouver: Aesthetic Research Centre of Canada, 1974.
Valentinuzzi, Max E. Understanding the Human Machine: A Primer for bioengineering. Singapore: World Scientific Publishing Company, 2004.
Webster, John G. Medical Instrumentation-Application and Design. 3rd Editions. New York: John Wiley and Sons, 1997.
Wishart, Trevor. “Transforming Sounds: Confirming and Confounding Expectations.” Paper presented at the International Conference on Music and Emotion (Durham, UK: Durham University, 31 August – 3 September 2009.
Deborah Lupton, Faculty of Arts & Design, University of Canberra This is a pre-print of a chapter to be published in Routledge Handbook of Physical Cultural Studies, edited by D. Andrews, M. Silk and H. Thorpe. London: Routledge.
Human bodies have always interacted with technologies. However the nature of the technology has changed over the millennia. In the contemporary digital era, bodies are digitised as never before, both by individuals on their own behalf and by other actors and agencies seeking to portray and monitor their bodies. From Facebook status updates and images, Instagram selfies, YouTube videos and tweets to exergames, sophisticated digital medical imaging technologies and the ceaseless generation of data from sensor-based devices and environments, human bodies now emit vast quantities of digital data. A major change in digitised embodiment is the ways in which detailed data are now generated on the geolocation, movements, appearance, behaviours and functions of bodies and the uses to which these data are put as part of the digital data knowledge economy. The cyborg body has transformed into the digital body, whose data outputs possess commercial, managerial and research as well as personal value and status to a range of actors and agencies beyond the individual.
In this chapter I examine the ways in which human bodies interact with and are configured by digital technologies and how these technologies generate new knowledges and practices in relation to bodies. I use infants and young children as a case study to explain these aspects. From before they are even born, children’s bodies are now frequently represented and monitored by digital technologies, including medical imaging and monitoring devices as well as social media sites, surveillance and self-tracking technologies. In my discussion I draw on literature from sociocultural theorising of the body, childhood, digital technologies and big data, particularly that by scholars adopting the sociomaterial perspective. The chapter is divided into two main parts. The first presents a general overview of theoretical approaches to conceptualising the interactions between bodies and technologies, while the second part is devoted to outlining the ways in which infants’ and young children’s bodies are digitised.
Theorising digital bodies
Scholars in the sociology of the body and technocultures developed an interest in the entanglements of human bodies with computerised technologies following the advent of personal computing in the mid-1980s. The terms ‘cyborg’ and ‘cyberspace’ (among many other ‘cyber’ neologisms) were adopted to discuss the ways in which computer users interacted with their PCs and with each other online. Donna Haraway’s work on the political implications of the cyborg as a heterogeneous, ambiguous and hybrid entity has been particularly important in drawing attention to the fluidities of embodiment and selfhood (Haraway 1991, 1997). Many other social researchers into the 1990s and early 2000s seized on the concept of the cyborg to investigate the forms of embodiment that are generated or mediated by digital technologies across a range of contexts: including, for example, computer users, IVF embryos, menopausal women, athletes and older people (Buse 2010, Franklin 2006, Leng 1996, Lupton 1995, Rayvon 2012)
Cyber terminology is not as often employed in discussions of the social, cultural and political dimensions of computer technology use now that academic terminology has moved more to a focus on the ‘digital’ (Lupton 2015b). However the important work of Haraway and others writing on cyborg bodies developed an argument that acknowledges the complexity of relationships between human and nonhuman actors and calls into question ideas about the fixed nature of identity and embodiment (Lupton 2015c). Such a perspective is now often referred to as ‘sociomaterialism’. It recognises that subject and object co-configure each other as part of a relationship. Objects are viewed as participating in specific sets of relations, including those with other artefacts as well as with people (Fenwick and Landri 2012, Latour 2005, Law 2008, Law and Hassard 1999). The term ‘assemblage’ is often used to capture these entanglements. Assemblages of human flesh and nonhuman actors are constantly configured and reconfigured. They facilitate modes of knowing and living the body.
People domesticate technologies by bringing them into their everyday worlds, melding them to their bodies/selves and bestowing these objects with their own biographically- specific meanings. They become ‘territories of the self’, marked by individual use, and therefore redolent of personal histories (Nippert-Eng 1996). This concept of territories of the self acknowledges that bodies and selves are not contained to the fleshly envelope of the individual body, but extend beyond this into space and connect and interconnect with other bodies and objects. These processes are inevitably relational because they involve embodied interactions and affective responses (Labanyi 2010, Lupton 2015b, forthcoming). As Merleau-Ponty (1968) argues, our embodiment is always inevitably interrelational or intercorporeal. We experience the world as fleshly bodies, via the sensations and emotions configured through and by our bodies as they relate to other bodies and to material objects and spaces. We touch these others and they touch us. Our bodies are distributed throughout the spaces we inhabit, just as these spaces and the others within them inhabit. Embodiment, then, is primarily a relational assemblage. The concept of ‘the person’ (including the person’s body) becomes distributed between the interactions of heterogeneous elements (Lee 2008).
In the digital age, practices of embodiment are increasingly becoming enacted via digital technologies. We now no longer refer to the separate environment of ‘cyber space’ as our everyday worlds have become so thoroughly digitised. Where once the figure of the cyborg was a science-fiction creation of superhuman powers (Lupton 1995), our bodies now engage routinely with digital technologies to the extent that it is taken-for-granted. It is now frequently argued that online and offline selves cannot be distinguished from each other any longer, given the pervasiveness and ubiquity of online participation. Instead categories of flesh, identity and technology are porous and intermeshed (Elwell 2014, Hayles 2012). Our bodies are digital data assemblages (Lupton 2015c).
Digital social theorists have drawn attention to the increasingly sensor-saturated physical environments in which people move, which add to the pre-existing technologies for visually observing and documenting human movements in public spaces, such as CCTV cameras (Kitchin 2014, Kitchin and Dodge 2011, Lyon and Bauman 2013). Kitchin and Dodge (2011) use the term ‘code/space’ to describe the intersections of software coding with the spatial configurations of humans and nonhumans. They underline the power of code to shape, manage, monitor and discipline the movements of bodies in space and place, including both public and private domains. Digital representations of bodies and digital data on many aspects of embodiment are generated from the various sites, devices and spaces which with individuals interact daily: the transactional data produced via routine encounters with surveillance cameras in public spaces, sensors or online websites, platforms and search engines or from the content that people upload voluntarily to social media sites or collect on themselves using self-tracking devices. These technologies create and recreate certain types of digital data assemblages which can then be scrutinised, monitored and used for various purposes, including intervention (Elmer 2003, Haggerty and Ericson 2000, Lupton 2012b).
The collection and analysis of digitised information about people’s behaviours are now becoming increasingly advocated and implemented in many social contexts and institutions, including the workplace, education, medicine and public health, insurance, government, marketing, advertising and commerce, the military, citizen science and urban planning and management. The growing commodification and commercial value of digital data sets and their use in these domains are blurring the boundaries between small and big data, the private and the public. People are now encouraged, obliged or coerced into using digital devices for monitoring aspects of their lives to produce personal data that are employed not only for private and voluntary purposes but also for the purposes of others. These data have begun to be appropriated by a range of actors and agencies, including commercial, managerial, research and governmental (Lupton forthcoming).
Critical data scholars have drawn attention to the valorisation of quantifiable information in the digital data economy and the algorithmic processing of this information as part of new forms of soft power relations and the production of inequalities (Cheney-Lippold 2011, Kitchin 2014, Lupton 2015b). Digital data can have tangible material effects on people’s actions, including the ways in which their bodies are conceptualised, managed and disciplined by themselves and others. The calculations and predictions that are generated by software algorithms are beginning to shape people’s life chances and opportunities such as their access to insurance, healthcare, credit and employment and their exclusion from spaces and places, as in the identification of potential criminals and terrorists (Crawford and Schultz 2014).
It is difficult, if not impossible, to separate digital technologies from their users, as both are viewed as mutually constituted. Technologies discipline the body to better assimilate it to their requirements, their ways of seeing, monitoring and treating human flesh. Bodies about to be scanned by MRI technology, for example, must be adapted and customised to a specific physical norm: they cannot be too tall or overweight, suffer from claustrophobia, wear jewellery or spectacles or contain metallic implants, hearing aids or pacemakers. The patient must stay still and calm according to the directions of the technologies and physicians taking the scan (Burri 2007).
However bodies also shape technologies. The new mobile and wearable devices are carried or worn on the body, becoming a body prosthetic, an extension of the body. When people handle or touch technologies, they may leave the marks of their bodies on the devices: body oils, sweat, skin flakes. Software is also transformed by use. Now that digital technologies are increasingly used as part of the practices of selfhood, digital archives have become important storage places for personalised bodily data. Images, descriptions and markers of users’ bodies are entered into the memories of their digital devices: photographs and videos of themselves, records of their geolocation, the detailed biometric information that is generated by self-tracking apps. Digital devices and software have become repositories of selfhood and embodiment (Lupton 2015b, forthcoming).
Young children’s embodiment and digital technologies
All human bodies are understood to be in the process of constant transformation, requiring engaging in work on the self and reflexive self-monitoring as part of performing selfhood and embodiment. Foucault refers to these ethical practices of citizenship as ‘technologies of the self’ (Foucault 1986, 1988), while Beck uses the term ‘reflexive biography’ (Beck 1992, Beck and Beck-Gernsheim 1995) to denote the ways in which people are encourage to seek knowledge and use it to improve their life chances, health and wellbeing. The idea of the unfinished body is particularly true of children’s bodies, which are viewed as requiring constant monitoring, assessment and improvement from themselves and other actors and agencies to achieve the ideal of the civilized body (Jenks 2005, Lupton 2013a, Uprichard 2008).
While developing in utero and following birth, children’s bodies are measured and observed for signs of ‘normal’ growth and development, and they are continually subjected to practices that seek to socialise and normalise their bodies. Children’s bodies – and especially those of the unborn, infants and the very young – are regarded as particularly precious and vulnerable, requiring the intense surveillance of their caregivers as part of efforts to protect them from risk and ensure their optimum health and development (Lupton 2013a, 2014). These efforts are now often rendered into digital forms with the use of an array of devices and software.
The sociomaterialist perspective has been taken up by several scholars writing about children’s bodies, particularly within cultural geography, but also by some sociologists and anthropologists (Horton and Kraftl 2006a, 2006b, Lee 2008, Prout 1996, Woodyer 2008). Researchers using a sociomaterialist approach have conducted studies on, for example, children’s use of asthma medication (Prout 1996), the surveillant technologies that have developed around controlling children’s body weight in schools (Rich et al. 2011), children’s sleep and the objects with which they interact (Lee 2008), the interrelationship of objects with pedagogy and classroom management of students’ bodies (Mulcahy 2012) and sociomaterial practices in classrooms that lead to the inclusion or exclusion of children with disabilities (Söderström 2014). Outside sociomaterialist studies, young children’s interactions with digital technologies have attracted extensive attention from social researchers, particularly in relation to topics such as the potential for cyber-bullying, online paedophilia and for children to become unfit and overweight due to spending too much time in front of screens (Holloway et al. 2013). However few researchers thus far have directed their attention to the types of digital technologies that visually represent children’s bodies or render their body functions, activities and behaviours into digital data; or, in other words, how children’s bodies become digital data assemblages.
From the embryonic stage of development onwards, children’s bodies are now routinely monitored and portrayed using digital technologies. A plethora of websites provide images of every stage of embryonic and foetal development, from fertilisation to birth, using a combination of digital images taken from embryo and foetus specimens and digital imaging software (Lupton 2013c). 3/4D ultrasounds have become commodified, used for ‘social’ or ‘bonding’ purposes instead of the traditional medical diagnostic and screening scan. Many companies offering 3/D ultrasounds now come to people’s homes, allowing expectant parents to invite family and friends and turn a viewing of the foetus into a party event. This sometimes involves a ‘gender reveal’ moment, in which the sonographer demonstrates to all participants, including the parents, the sex of thefoetus . Some companies offer the service of using 3D ultrasound scan files to create life- sized printed foetus replica models for parents.
The posting to social media sites such as Facebook, Twitter, Instagram and YouTube of the foetus ultrasound image has become a rite of passage for many new parents and often a way of announcing the pregnancy. Using widgets such as ‘Baby Gaga’, expectant parents can upload regular status updates to their social media feeds automatically that provide news on the foetus’s development. While a woman is pregnant, she can use a range of digital devices to monitor her foetus. Hundreds of pregnancy apps are currently on the market, including not only those that provide information but those that invite users to upload personal information about their bodies and the development of their foetus (Tripp et al. 2014). Some apps offer a personalised foetal development overview or provide the opportunity for the woman to record the size of her pregnant abdomen week by week, eventually creating a time-lapse video. Other apps involve women tracking foetal movements or heart beat. Bella Beat is a smartphone attachment and app that allows the pregnant women to hear and record the foetal heart beat whenever she likes and to upload the audio file to her social media accounts.
YouTube has become a predominant medium for the representation of the unborn entity in the form of ultrasound images and of the moment of birth. Almost 100,000 videos showing live childbirth, including both vaginal and Caesarean births, are available for viewing on that site, allowing the entry into the world of these infants to be viewed by thousands and, in the case of some popular videos, even millions of viewers. Some women even choose to live-stream the birth so that audiences can watch the delivery in real time. Following the birth, there are similar opportunities for proud parents to share images of their infant online on social media platforms. In addition to these are the growing number of devices on the market for parents to monitor the health, development and wellbeing of their infants and young children. Apps are available to monitor such aspects as infants’ feeding and sleeping patterns, their weight and height and their development and achievements towards milestones. Sensor- embedded baby clothing, wrist or ankle bands and toys can be purchased that monitor infants’ heart rate, body temperature and breathing, producing data that are transmitted to the parents’ devices. Smartphones can be turned into baby monitors with the use of apps that record the sound levels of the infant.
As children grow, their geolocation, educational progress and physical fitness can be tracked by their parents using apps, other software and wearable devices. As children themselves begin to use digital technologies for their own purposes, they start to configure their own digital assemblages that represent and track their bodies. With the advent of touchscreen mobile devices such as smartphones and tablet computers, even very young children are now able to use social media sites and the thousands of apps that have been designed especially for their use (Holloway et al. 2013). Some such technologies encourage young children to learn about the anatomy of human bodies or about nutrition, exercise and physical fitness, calculate their body mass index, collect information about their bodies or represent their bodies in certain ways (such as manipulating photographic images of themselves). These technologies typically employ gamification strategies to provide interest and motivation for use. Some involve combining competition or games with self-tracking using wearable devices. One example is the Leapfrog Leapband, a digital wristband connected to an app which encourages children to be physically active in return for providing them with the opportunity to care for virtual pets. Another is the Sqord interactive online platform with associated digital wristband and app. Children who sign up can make an avatar of themselves and use the wristband to track their physical activity. Users compete with other users by gaining points for moving their bodies as often and as fast as possible.
In the formal educational system there are still more opportunities for children’s bodies to be monitored measured and evaluated and rendered into digitised assemblages. Programmable ‘smart schools’ are becoming viewed as part of the ‘smart city’, an urban environment in which sensors that can watch and collect digital data on citizens are ubiquitous (Williamson 2014). The monitoring of children’s educational progress and outcomes using software is now routinely undertaken in many schools, as are their movements around the school. In countries such as the USA and the UK, the majority of schools have CCTV cameras that track students, and many use biometric tracking technologies such as RFID chips in badges or school uniforms and fingerprints to identify children and monitor their movements and their purchases at school canteens (Selwyn 2014, Taylor 2013). A growing number of schools are beginning to use wearable devices, apps and other software for health and physical education lessons, such as coaching apps that record children’s sporting performances and digital heart rate monitors that track their physical exertions (Lupton 2015a).
We can see in the use of digital technologies to monitor and represent the bodies of children a range of forms of embodiment. Digitised data assemblages of children’s bodies are generated from before birth via a combination of devices that seek to achieve medical- or health-related or social and affective objectives. These assemblages may move between different domains: when, for example, a digitised ultrasound image that was generated for medical purposes becomes repurposed by expectant parents as a social media artefact, a way of announcing the pregnancy, establishing their foetus as new person and establishing its social relationships. Parents’ digital devices, and later those of educational institutions and those of children themselves when they begin to use digital devices, potentially become personalised repositories for a vast amount of unique digital assemblages on the individual child, from images of them to descriptions of their growth, development, mental and physical health and wellbeing, movements in space, achievements and learning outcomes. These data assemblages, containing as they do granular details about children, offer unprecedented potential to configure knowledges about individual children and also large groups of children (as represented in aggregated big data sets).
As I have shown in this chapter, new forms of bodies are being configured via contemporary digital technologies. Devices that are able to monitor, portray, measure and compare bodies generate unceasing flows of data about individuals which then move into the digital data economy and are repurposed by a range of actors and agencies. I have employed the example of young children’s bodies to demonstrate the manifold ways in which such digitised bodily assemblages are created and the uses to which they are put. Digital data are forms of ‘lively capital’ in three major ways. First they are generated from life itself, in terms of documenting humans’ bodies and selves. Second, as digital data they are labile and fluid as they are generated and circulate in the digital data economy. And third, because with the advent of interconnected smart objects, aggregated data sets and predictive analytics , personal digital data have potential effects on the conduct of life and life opportunities (Lupton forthcoming).
In this age of unceasing collection of often very intimate and personal information about people via digital technologies, questions of data security and data privacy have become paramount. Once personal digital data enter the computing cloud, people lose control over how they are protected and controlled. Recent scandals and controversies, such as the former CIA and the US National Security Agency contractor Edward Snowden’s release of documents that demonstrate how national security agencies in western countries are conducting surveillance on citizens’ online interactions and various events of hacking into personal data databases have revealed the precariousness of personal data security and privacy.
Thus far we know very little about how people are engaging with the digital data assemblages that are generated on them, how they contribute to, manage, manipulate and make sense of these assemblages and what impacts they have on people’s sense of selfhood and embodiment. This is a particularly pressing issue for individuals such as the current generation of children whose lives and bodies have been so thoroughly digitally documented. As humans are entering into technological entanglements that are able to document their lives from pre-birth to death in ever-finer detail, many issues and implications remain to be explored. These include who has the right to collect data on people, who controls and has access to the repositories of personal data that are now configured on individuals, how these data are used by those who do have access and what happens to people’s data assemblages after death.
Digital data assemblages are always mutable, dynamic and responsive to new inputs. A recursive feedback loop is established in which information is generated from digital technologies which then are used by the individual to assess her or his activities and behaviour and modify them accordingly, which then configure a renewed data assemblage – and on the cycle goes (Lupton 2012a, 2013b, forthcoming). Indeed one major novel aspect of people’s encounters with digital technologies is the ways in which these technologies are now often designed to ‘nudge’ users into taking up certain practices. Instead of merely providing information, as in older forms of internet engagement, software is coded to algorithmically manipulate users’ personal data and send them ‘push’ notifications to encourage them to purchase more goods and services or change their behaviour to optimise their health, wellbeing or productivity. More and more, our digital machines are taking on the role of managers, task-masters or disciplinarians of our bodies. Commentators are now beginning to envisage a world in which interconnected smart devices, as part of the Internet of Things, interact with the personalised data that each generate to provide advice to users. Thus, for example, the wearable body tracker can interact with smart objects in the user’s home (such as the smart fridge, smart thermostat, smart television and smart bed) to determine what kind of food users should consume, what types of television programs they should watch, what temperature level their home should be set at and for how long and what time they should go to sleep and wake up, based on such features as their mood, body weight, calories burnt and physical activity data.
Such entanglements of human bodies with technological devices potentially represent further major changes to concepts and practices of embodiment. For the field of physical cultural studies, they constitute a new and important element of understanding how knowledges, practices, objects, emotion, discourse, data and humans intertwine.
Beck, U., 1992. Risk society: Towards a new modernity. London: Sage.
Beck, U. and Beck-Gernsheim, E., 1995. The normal chaos of love. Cambridge: Polity. Burri, R., 2007. Sociotechnical anatomy: Technology, space, and body in the MRI unit. In:
R. Burri & J. Dumit (eds.) Biomedicine as culture: Instrumental practices,
technoscientific knowledge, and new modes of life. New York: Routledge, 109-121. Buse, C.E., 2010. E-scaping the ageing body? Computer technologies and embodiment in
later life. Ageing & Society, 30 (6), 987-1009.
Cheney-Lippold, J., 2011. A new algorithmic identity: Soft biopolitics and the modulation
of control. Theory, Culture & Society, 28 (6), 164-181.
Crawford, K. and Schultz, J., 2014. Big data and due process: Toward a framework to
redress predictive privacy harms. Boston College Law Review, 55 (1), 93-128. Elmer, G., 2003. A diagram of panoptic surveillance. New Media & Society, 5 (2), 231-
Elwell, J.S., 2014. The transmediated self: Life between the digital and the analog.
Convergence: The International Journal of Research into New Media Technologies,
20 (2), 233-249.
Fenwick, T. and Landri, P., 2012. Materialities, textures and pedagogies: Socio-material
assemblages in education. Pedagogy, Culture & Society, 20 (1), 1-7. Foucault, M., 1986. The care of the self: The history of sexuality volume 3. New York:
Foucault, M., 1988. Technologies of the self. In: L. Martin, H. Gutman & P. Hutton (eds.)
Technologies of the self: A seminar with Michel Foucault. London: Tavistock, 16-
Franklin, S., 2006. The cyborg embryo. Theory, Culture & Society, 23 (7/8), 167. Haggerty, K. and Ericson, R., 2000. The surveillant assemblage. British Journal of
Sociology, 51 (4), 605–22.
Haraway, D., 1991. Simians, cyborgs and women: The reinvention of nature. London: Free
Haraway, D., 1997. Modest_witness@second_millennium.
Femaleman©meets_oncomousetm: Feminism and technoscience. New York:
Hayles, N.K., 2012. How we think: Digital media and contemporary technogenesis.
Chicago, IL: University of Chicago Press.
Holloway, D., Green, L. and Livingstone, S., 2013. Zero to eight: Young children and their
internet use. London, 2045-256X.
Horton, J. and Kraftl, P., 2006a. Not just growing up, but going on: Materials, spacings,
bodies, situations. Children’s Geographies, 4 (3), 259-276.
Horton, J. and Kraftl, P., 2006b. What else? Some more ways of thinking and doing
‘children’s geographies’. Children’s Geographies, 4 (1), 69-95.
Jenks, C., 2005. Childhood: Critical concepts in sociology. New York: Routledge.
Kitchin, R., 2014. The data revolution: Big data, open data, data infrastructures and their consequences. London: Sage.
Kitchin, R. and Dodge, M., 2011. Code/space: Software and everyday life. Cambridge, Mass: MIT Press.
Labanyi, J., 2010. Doing things: Emotion, affect, and materiality. Journal of Spanish Cultural Studies, 11 (3/4), 223-233.
Latour, B., 2005. Reassembling the social: An introduction to actor-network-theory. Oxford: Clarendon.
Law, J., 2008. On sociology and sts. The Sociological Review, 56 (4), 623-649. Law, J. and Hassard, J., 1999. Actor network theory and after. Oxford: Blackwell
Lee, N., 2008. Awake, asleep, adult, child: An a-humanist account of persons. Body &
Society, 14 (4), 57-74.
Leng, K.W., 1996. On menopause and cyborgs: Or, towards a feminist cyborg politics of
menopause. Body & Society, 2 (3), 33-52.
Lupton, D., 1995. The embodied computer/user. Body & Society, 1 (3-4), 97-112. Lupton, D., 2012a. M-health and health promotion: The digital cyborg and surveillance
society. Social Theory & Health, 10 (3), 229-244.
Lupton, D. (2012b) The quantified self movement: Some sociological perspectives.
movement-some-sociological-perspectives/ , accessed 22 December 2012. Lupton, D., 2013a. Infant embodiment and interembodiment: A review of sociocultural
perspectives. Childhood, 20 (1), 37-50.
Lupton, D., 2013b. Quantifying the body: Monitoring and measuring health in the age of
mhealth technologies. Critical Public Health, 23 (4), 393-403.
Lupton, D., 2013c. The social worlds of the unborn. Houndmills: Palgrave Macmillan. Lupton, D., 2014. Precious, pure, uncivilised, vulnerable: Infant embodiment in
australian popular media. Children & Society, 28 (5), 341-351.
Lupton, D., 2015a. Data assemblages, sentient schools and digitised health and physical
education (response to Gard). Sport, Education and Society, 20 (1), 122-132. Lupton, D., 2015b. Digital sociology. London: Routledge.
Lupton, D., 2015c. Donna Haraway: The digital cyborg assemblage and the new digital
health technologies. In: F. Collyer (ed.) The Palgrave Handbook of social theory in
health, illness and medicine. Houndmills: Palgrave Macmillan, 567-581. Lupton, D., forthcoming. The quantified self: A sociology of self-tracking. Cambridge:
Lyon, D. and Bauman, Z., 2013. Liquid surveillance: A conversation. Oxford: Wiley. Merleau-Ponty,M.,1968.Thevisibleandtheinvisible. Evanston,IL:Nortwestern
Mulcahy, D., 2012. Affective assemblages: Body matters in the pedagogic practices of
contemporary school classrooms. Pedagogy, culture and society, 20 (1), 9-27. Nippert-Eng, C., 1996. Home and work: Negotating boundaries through everyday life.
Chicago: University of Chicago Press.
Prout, A., 1996. Actor-network theory, technology and medical sociology: An illustrative analysis of the metered dose inhaler. Sociology of Health and Illness, 18 (2), 198- 219.
Rayvon, F., 2012. Aren’t athletes cyborgs?: Technology, bodies, and sporting competitions. Women’s Studies Quarterly, 40 (1), 281-293.
Rich, E., Evans, J. and De Pian, L., 2011. Children’s bodies, surveillance and the obesity crisis. In: E. Rich, L.F. Monaghan & L. Aphramor (eds.) Debating obesity: Critical perspectives. Houndsmills: Palgrave Macmillan, 139–63.
Selwyn, N., 2014. Data entry: Towards the critical study of digital data and education. Learning, Media and Technology), 1-19.
Söderström, S., 2014. Socio-material practices in classrooms that lead to the social participation or social isolation of disabled pupils. Scandinavian Journal of Disability Research, online first.
Taylor, E., 2013. Surveillance schools: Security, discipline and control in contemporary education. Houndmills: Palgrave Macmillan.
Tripp, N., et al., 2014. An emerging model of maternity care: Smartphone, midwife, doctor? Women and birth : journal of the Australian College of Midwives, 27 (1), 64.
Uprichard, E., 2008. Children as ‘being and becomings’: Children, childhood and temporality. Children & Society, 22 (4), 303-313.
Williamson, B. (2014) Smart schools in sentient cities. dmlcentral. http://dmlcentral.net/blog/ben-williamson/smart-schools-sentient-cities, accessed 14 May 2015.
Woodyer, T., 2008. The body as research tool: Embodied practice and children’s geographies. Children’s Geographies, 6 (4), 349-362.