Introduction

This entry is part 3 in the series An Opus to AR.

Augmented Reality (AR) is a theme of computer research which deals with a combination of real world and computer generated data. AR is just one version of a Mixed Reality (MR) technology, where digital and real elements are mixed to create meaning. In essence AR is any live image that has an overlay of information that augments the meaning of these images.

Digital graphics are commonly put to work in the entertainment industry, and ‘mixing realities’ is a common motif for many of today’s media forms. There are varying degrees to which The Real and The Virtual can be combined. This is illustrated in my Mixed Reality Scale:

mixed-reality-scale
My Mixed Reality Scale, a simplified version of Milgram & Kishino’s (1994) Virtuality Continuum

This is a simplified version of Milgram and Kishino’s (1994) Virtuality Continuum; simplified, because their research is purely scientific, without an explicit interest in media theory or effects, therefore not wholly applicable to my analysis. At the far left of my Mixed Reality Scale lies The Real, or physical, every-day experiential reality. For the longest time we lived solely in this realm. Then, technological innovation gave rise to the cinema, and then television. These media are located one step removed from The Real, a step closer to The Virtual, and can be considered a window on another world. This world is visually similar to our own, a fact exploited by its author to narrate believable, somewhat immersive stories. If willing, the viewer is somewhat ‘removed’ from their grounding here in physical reality, allowing them to participate in the construction of a sculpted, yet static existence. The viewer can only observe this contained reality, and cannot interact with it, a function of the viewing apparatus.

Later advancements in screen media technologies allowed the superimposition of graphical information over moving images. These were the beginnings of AR, whereby most of what is seen is real with some digital elements supplementing the image. Indeed, this simple form of AR is still in wide use today, notably in cases where extra information is required to make sense of a subject. In the case of certain televised sports, for example, a clock and a scoreboard overlay a live football match, which provides additional information that is useful to the viewer. Television viewers are already accustomed to using information that is displayed in this way:

Simple Augmented Reality, televised football matches augment meaning with digital graphics
Simple Augmented Reality, televised football matches augment meaning with digital graphics

More recently, computing and graphical power gave designers the tools to build wholly virtual environments. The Virtual is a graphical representation of raw data, and the furthest removed from physical reality on my Mixed Reality Scale. Here lies the domain of Virtual Reality (VR), a technology that uses no real elements except for the user’s human senses. The user is submersed in a seemingly separate reality, where visual, acoustic and sometimes haptic feedback serve to transpose them into this artificial, yet highly immersive space. Notice the shift from viewer to user: this is a function of the interactivity offered by digital space. VR was the forerunner to current AR research, and remains an active realm of academic study.

Computer graphics also enhanced the possibilities offered by television and cinema, forging a new point on the Mixed Reality Scale. I refer to the Augmented Virtuality (AV) approach, which uses mainly digital graphics with some real elements superimposed. For example, a newsreader reporting from a virtual studio environment is one common application. I position AV one step closer towards The Virtual to reflect the ratio of real to virtual elements:

An Augmented Virtuality, the ITV newscasters sit at a real table in a virtual studio
An Augmented Virtuality, the ITV newscasters sit at a real table in a virtual studio

There is an expansive realm between AV and VR technologies, media which offer the user wholly virtual constructions that hold potential for immersion and interactivity. I refer to the media of video games and desktop computers. Here the user manipulates visually depicted information for a purpose. These media are diametrically opposed to their counterpart on my scale, the cinema and television, because they are windows this time into a virtual world, actively encouraging (rather than denying) user interactivity to perform their function. Though operating in virtuality, the user remains grounded in The Real due to apparatus constraints.

Now, further technological advancements allow the fusion of real and virtual elements in ways not previously possible. Having traversed our way from The Real to The Virtual, we have now begun to make our way back. We are making a return to Augmented Reality, taking with us the knowledge to manipulate wholly virtual 3D objects and the computing power to integrate digital information into live, real world imagery. AR is deservedly close to The Real on my scale, because it requires physicality to function. This exciting new medium has the potential to change the way we perceive our world, forging a closer integration between our two binary worlds. It is this potential as an exciting and entirely new medium that has driven me to carry out the following work.

To begin, I address the science behind AR and its current applications. Next, I exploit an industry connection to inform a discussion of AR’s development as an entertainment medium. Then, I construct a methodology for analysis from previous academic thought on emergent technologies, whilst addressing the problems of doing so. I use this methodology to locate AR in its wider technologic, academic, social and economic context. This discussion opens ground for a deeper analysis of AR’s potential socio-cultural impact, which makes use of theories of media and communication and spatial enquiry. I conclude with a final critique that holds implications for the further analysis of Mixed Reality technology.

Published by

Tom Saunter

I like to think about the media, technology, pop-culture & the future. When not blogging, I tweet @freedimensional & work @MediaComUK. Feel free to visit my Personal Bio to learn more about me.

  • Thought I’d point you in the direction of a similar assessment/adaptation of the Virtuality Continuum that I wrote last year as part of the _Augmentology 1[L]0[L]1_ project:

    http://arsvirtuafoundation.org/research/2008/04/25/reality-mixing-the-geospecificity-complex/

    “_Reality Mixing + the Geospecificity Complex_

    …Paul Milgram suggested the Reality-Virtuality Continuum as a type of linear reality scale where at one end lies Geophysical Reality [“The Real Environment”] and the opposite end houses the Virtual. In-between lies an area defined as Mixed Reality: a mixture of augmented virtuality and the corporeal.

    This Reality-Virtuality Continuum as such offers a vectored compartmentalisation of reality within scientific confines. An elastic, contemporized version of this Continuum might read:

    [Geophysical][Synthetic]

    …with each mode spawning distinct “swarmic variables” or “notional massing”. Conditional examples of such masses/variables are:

    [Synthetic] = Avatar Fluctuations / Non-Player Character Annexing / Auxiliary Proprieception / Networked Socialisation

    [Mixed] = Layered Attention / Identity Extensions / Augmented States of Consciousness / Multiple Theories-of-Mind[s]

    [Cartesian] = Euclidian / Non-Euclidian / Human Area Networks / Dimensional Consciousness

    [Geophysical] = Primary Consciousness / Ego-Mediation / Geospecificity / Geolocation

    These masses could parallel volume [in the audio sense] in terms of measurement and production of reality clusters that map and mix composite modes simultaneously. For instance, the phenomena of Geocaching or mobile gaming such as _Parallel Kingdom_ which: “brings new meaning to Role Playing Games by using GPS to place the virtual world on top of the real world“.

    In relation to the creation of such a _Reality Spectrum_, one intriguing by-product could be a redefinition of established principles regarding conceptions of consciousness, perception, identity construction and associated mental health conditions related to “normalcy”. If reality isn’t what we once thought it was, perhaps the inclusion of an Internet Addiction Disorder in the DSMV should not be a desired aim. Instead, should there be a proposal to include a range of disorders that diagnose the inability of a subject to adapt to reality contouring? For instance, a type of _Geospecificity Complex_ that highlights a dysfunction perceiving modalities that exist apart from concrete geophysical markers?”

    I’d also recommend you read the series of _Social Tesseracting_ posts, as they also foreground the majority of the issues you’re writing about:

    http://arsvirtuafoundation.org/research/2009/07/24/_social-tesseracting_-part-3/

    Regards,
    Mez Breeze

  • Thought I’d point you in the direction of a similar assessment/adaptation of the Virtuality Continuum that I wrote last year as part of the _Augmentology 1[L]0[L]1_ project:

    http://arsvirtuafoundation.org/research/2008/04/25/reality-mixing-the-geospecificity-complex/

    “_Reality Mixing + the Geospecificity Complex_

    …Paul Milgram suggested the Reality-Virtuality Continuum as a type of linear reality scale where at one end lies Geophysical Reality [“The Real Environment”] and the opposite end houses the Virtual. In-between lies an area defined as Mixed Reality: a mixture of augmented virtuality and the corporeal.

    This Reality-Virtuality Continuum as such offers a vectored compartmentalisation of reality within scientific confines. An elastic, contemporized version of this Continuum might read:

    [Geophysical][Synthetic]

    …with each mode spawning distinct “swarmic variables” or “notional massing”. Conditional examples of such masses/variables are:

    [Synthetic] = Avatar Fluctuations / Non-Player Character Annexing / Auxiliary Proprieception / Networked Socialisation

    [Mixed] = Layered Attention / Identity Extensions / Augmented States of Consciousness / Multiple Theories-of-Mind[s]

    [Cartesian] = Euclidian / Non-Euclidian / Human Area Networks / Dimensional Consciousness

    [Geophysical] = Primary Consciousness / Ego-Mediation / Geospecificity / Geolocation

    These masses could parallel volume [in the audio sense] in terms of measurement and production of reality clusters that map and mix composite modes simultaneously. For instance, the phenomena of Geocaching or mobile gaming such as _Parallel Kingdom_ which: “brings new meaning to Role Playing Games by using GPS to place the virtual world on top of the real world“.

    In relation to the creation of such a _Reality Spectrum_, one intriguing by-product could be a redefinition of established principles regarding conceptions of consciousness, perception, identity construction and associated mental health conditions related to “normalcy”. If reality isn’t what we once thought it was, perhaps the inclusion of an Internet Addiction Disorder in the DSMV should not be a desired aim. Instead, should there be a proposal to include a range of disorders that diagnose the inability of a subject to adapt to reality contouring? For instance, a type of _Geospecificity Complex_ that highlights a dysfunction perceiving modalities that exist apart from concrete geophysical markers?”

    I’d also recommend you read the series of _Social Tesseracting_ posts, as they also foreground the majority of the issues you’re writing about:

    http://arsvirtuafoundation.org/research/2009/07/24/_social-tesseracting_-part-3/

    Regards,
    Mez Breeze