New Interactive Augmented Reality Experience by T-Immersion

Total Immersion have launched a new kind of interactive attraction using mobility and augmented reality. This new attraction developed in cooperation with Hanwa Co. (Japan) is a state of the art walk through where guests are swept into an experience merging videogame and real live adventure:

Before entering in the attraction, guests are equipped with a backpack and a video gun (with an infrared camera, one LCD display and a trigger). Guests have to use the video gun to progress inside the horror house and find ghosts. As soon as the ghost is found, guests have to shoot it to collect points and succeed their mission.

T-Immersion are using AR really effectively in this instance, in what I think is a quite impressive videogame-like scenario. This is a departure for T-Immersion, who have formerly been focused on installation-based & and webcam-based AR.

Does this give anyone any good ideas for other, more meaningful AR interactions? Let me know in the comments.

Bibliography

So that’s it, my series is over. All that’s left to do now is credit the academic sources that influenced and aided in the construction of my argument. Thanks to everyone below, and thanks to you, dear reader, for coming along for the ride.

References:

Baudrillard, Jean (1983). Simulations. New York: Semiotext(e).

Baudrillard, Jean (1988). Selected Writings, ed. Mark Poster. Cambridge: Polity Press.

Baumann, Jim (date unknown). ‘Military applications of virtual reality’ on the World Wide Web. Accessed 20th March 2007. Available at http://www.hitl.washington.edu/scivw/EVE/II.G.Military.html

Benjamin, Walter (1968). ‘The Work of Art in the Age of Mechanical Reproduction’, in Walter Benjamin Illuminations (trans. Harry Zohn), pp. 217–51. New York: Schocken Books.

Bolter, J. D., B. Mcintyre, M. Gandy, Schweitzer, P. (2006). ‘New Media and the Permanent Crisis of Aura’ in Convergence: The International Journal of Research into New Media Technologies, Vol. 12 (1): 21-39.

Botella, Cristina.M, & M.C. Juan, R.M. Banos, M. Alcaniz, V. Guillen, B. Rey (2005). ‘Mixing Realities? An Application of Augmented Reality for the Treatment of Cockroach Phobia’ in CyberPsychology & Behaviour, Vol. 8 (2): 162-171.

Clark, N. ‘The Recursive Generation of the Cyberbody’ in Featherstone, M. & Burrows, R. (1995) Cyberspace/Cyberbodies/Cyberpunk, London: Sage.

Featherstone, Mike. & Burrows, Roger eds. (1995). Cyberspace/ Cyberbodies/ Cyberpunk: Cultures of Technological Embodiment. London: Sage.

Future Image (author unknown) (2006). ‘The 6Sight® Mobile Imaging Report’ on the World Wide Web. Accessed 22nd March 2007. Available at http://www.wirelessimaging.info/

Genosko, Gary (1999). McLuhan and Baudrillard: The Masters of Implosion. London: Routledge.

Kline, Stephen, DePeuter, Grieg, & Dyer-Witheforde, Nick (2003). Digital Play: The Interaction of Technology, Culture, and Marketing. Kingston & Montreal: McGill-Queen’s University Press.

Levinson, Paul (1999). Digital McLuhan: a guide to the information millennium. London: Routledge.

Liarokapis, Fotis (2006). ‘An Exploration from Virtual to Augmented Reality Gaming’ in Simulation Gaming, Vol. 37 (4): 507-533.

Manovich, Lev (2006). ‘The Poetics of Augmented Space’ in Visual Communication, Vol. 5 (2): 219-240.

McLuhan, Marshall (1962). The Gutenberg galaxy: The Making of Typographic Man. Toronto, Canada: University of Toronto Press.

McLuhan, Marshall (1964). Understanding Media: The Extensions of Man. New York: McGraw-Hill.

McLuhan, Marshall and Powers, Bruce R. (1989). The Global Village: Transformations in World Life in the 21st Century. Oxford University Press: New York.

Milgram, Paul & Kishino, Fumio (1994). ‘A Taxonomy of Mixed Reality Visual Displays’ in IEICE Transactions on Information Systems, Vol. E77-D, No.12 December 1994.

Reitmayr, Gerhard & Schmalstieg, Dieter (2001). Mobile Collaborative Augmented Reality. Proceedings of the IEEE 2001 International Symposium on Augmented Reality, 114–123.

Roberts, G., A. Evans, A. Dodson, B. Denby, S. Cooper, R. Hollands (2002) ‘Application Challenge: Look Beneath the Surface with Augmented Reality’ in GPS World, (UK, Feb. 2002): 14-20.

Stokes, Jon (2003). ‘Understanding Moore’s Law’ on the World Wide Web. Accessed 21st March 2007. Available at http://arstechnica.com/articles/paedia/cpu/moore.ars

Straubhaar, Joseph D. & LaRose, Robert (2005). Media Now: Understanding Media, Culture, and Technology. Belmont, CA: Wadsworth.

Thomas, B., Close. B., Donoghue, J., Squires, J., De Bondi, I’,. Morris, M., and Piekarski, W. ‘ARQuake: An outdoor/indoor augmented reality first-person application’ in Proceedings of the Fourth International Symposium on Wearable Computers, (Atlanta, GA, Oct. 2000), 139-141.

Wagner, D., Pintaric, T., Ledermann, F., & Schmalstieg, D. (2005). ‘Towards massively multi-user augmented reality on handheld devices’. In Proc. 3rd Int’l Conference on Pervasive Computing, Munich, Germany.

Weiser, M. (1991) ‘The Computer for the Twenty-First Century’ in Scientific American 265(3), September: 94–104.

Williams, Raymond (1992). Television: Technology and Cultural Form. Hanover and London: University Press of New England and Wesleyan University Press

Further Reading:

Bolter, Jay D. & Grusin, Richard (1999). Remediation: Understanding New Media. Cambridge, MA: MIT Press.

Cavell, Richard (2002). McLuhan in Space: a Cultural Geography. Toronto: University of Toronto Press.

Galloway, Alexander R. (2006). Gaming: Essays on Algorithmic Culture. Minneapolis: University of Minnesota Press.

Horrocks, Christopher (2000). Marshall McLuhan & Virtuality. Cambridge: Icon Books.

Jennings, Pamela (2001). ‘The Poetics of Engagement’ in Convergence: The International Journal of Research into New Media Technologies, Vol. 7 (2): 103-111.

Lauria, Rita (2001). ‘In Love with our Technology: Virtual Reality A Brief Intellectual History of the Idea of Virtuality and the Emergence of a Media Environment’ in Convergence: The International Journal of Research into New Media Technologies, Vol. 7 (4): 30-51.

Lonsway, Brian (2002). ‘Testing the Space of the Virtual’ in Convergence: The International Journal of Research into New Media Technologies, Vol. 8 (3): 61-77.

Moos, Michel A. (1997). Marshall McLuhan Essays: Media Research, technology, art, communication. London: Overseas Publishers Association.

Pacey, Arnold (1983). The Culture of Technology. Oxford: Basil Blackwell.

Salen, Katie & Zimmerman, Eric. (2004) Rules of Play: Game Design Fundamentals. Cambridge, MA: MIT.

Sassower, Raphael (1995). Cultural Collisions: Postmodern Technoscience. London: Routledge.

Wood, John ed. (1998). The Virtual Embodied: Presence/Practice/Technology. London: Routledge.

Conclusion

I set out to assess the implications of a wholly new medium, one which had received little academic attention written from a media theoretical perspective. I made clear use of an industry connection to gain inside knowledge of the developments occurring to bring this medium to the mainstream. Building a methodology that could sustain the level of analysis that I hoped to achieve, I observed the interactions between technology and industry, market forces and cultural influences. Having positioned my subject at the crest of a curling wave, I employed critical media theory to explore the potential implications of my subject in its wider context of social reality. This ambitious task has granted me insight into how the complex interactions of various fields give rise to social change. Along the way I have revealed seams rich in potential for further analysis.

McLuhan is proven to apply to yet another medium, the perspective he offers served my analysis quite well. A further exploration might make use of his Acoustic and Visual Space probe, Cavell’s basis for McLuhanistic spatial enquiry in his book McLuhan in Space (2002) would be a good starting point for such work, since it applies McLuhanism to the media of time and space, thus a good start for work on the presence of virtual objects. Media analysts occupied with screen design might wish to extend Bolter and Grusin’s (1999) work on remediation to the emergent Mobile AR technology, perhaps from an explicit digital gaming perspective. Those with interest in advertising or business as applied to Augmented Reality would do well to continue Benjaminian thought to its logical end: manipulating a virtual object to hold added-value for commercial enterprise. Those with a more creative bent might enjoy a study of the public perception of AR artworks using Benjamin also. There is scope for research into AR-based social interactions; gaming styles; immersion and identity formation, but this sort of work necessitates that first Mobile AR spends at least some time in public consciousness.

Finally, I believe that I have convincingly laid out an argument showing that AR is currently being developed and packaged as an entertainment technology, but its potential for community-driven, self-proliferating excitement of user-created content makes AR a significant and culturally-transformative technology. Convergence between media types will enable and drive the creation of innovative content which if successful will itself rely on new ways of accessing and viewing content and ultimately new forms of content and user experience entirely. We are at the crest of a wave. Will it wither and let a larger wave pass above it, or will it grow to reach tidal proportions? Despite my predictions, only time will tell.

Five New Interfaces from SIGGRAPH 2009

Just saw these over at MIT’s Technology Review, and thought I’d share…

Touchable Holography:

Researchers at the University of Tokyo have developed a display that lets users “touch” holograms. Virtual objects appear to float in mid-air thanks to an LCD and a concave mirror. The sensation of touching the objects is created using an ultrasound device positioned below the LCD and mirror, creating an area of condensed air.

Augmented Reality Toys:

One student of France’s L’École de Design has developed a way to ‘hack’ toys using AR. His Scope display automatically recognizes ordinary toys that have been mounted onto platforms covered with hexagonal patterns, as seen below. With AR, these hexagons become interactive buttons that are used to make virtual modifications to the toy.


Augmented Reality Toys.v2 (Work in progress) from Frantz Lasorne on Vimeo.

Hyper-Real Virtual Reality:

Another French project, this time from INRIA and Grenoble Universities, could revive the dying science of Virtual Reality. Their new VR system, Virtualization Gate, tracks users’ movements very accurately using multiple cameras, allowing them to interact with virtual objects with never-before-reached realism. This interface demonstrates true physics, as well as crispy graphics, so a cluster of PCs is needed to perform the necessary image capture and 3D modeling.

3D Teleconferencing:

Researchers at the University of Southern California will demo Headspin, a 3D teleconferencing system that maintains eye contact between a three-dimensional head and several participants on the other end of a connection.

To capture an image, a polarized beam-splitter “places” the camera virtually near the eyes of the speaker. The 3D display works by projecting high-speed video onto a rapidly spinning aluminum disk to generate an accurate image for each viewer.


HeadSPIN: A One-to-Many 3D Video Teleconferencing System from MxR on Vimeo.

Scratchable Input:

One researcher from Carnegie Mellon University will demonstrate his new scratch input technology. The system turns any surface into an instant input device by sensing the unique sound produced when a fingernail is dragged across it. The interface is small enough to fit into a mobile device (though I have concerns about calibration) and could thereby turn any surface the device is placed upon into an interface.

Applying Benjamin

AR technology grants virtual objects presence in physicality. This is a concept ripe with potential for academic study. Baudrillardian thought states that we would seek to assign these objects similar values to other, real world objects. The School of Economy offers the truth that scarcity creates value. Our physical, tangible world is finite, but The Virtual is infinite. Now, any object in space denies the opportunity for another object to exist there, however we know this not to be true in an Augmented Reality. I proffer that virtual objects taking up space in a finite world hold economic value, but Benjamin might argue against this view.

In his piece The Work of Art in the Age of Mechanical Reproduction (1968) which was written in the thirties, Walter Benjamin argues that film and other ‘reproductive’ media diminished or destroyed the aura that had belonged to earlier art. As Bolter et al. (2006: 24) put it, “Aura belongs to works of art that are unique, as most art was before technologies of mechanical reproduction. Aura is the sense of the ‘here and now’ that each such work possesses because of its history of production and transmission. This uniqueness lends to each painting or sculpture a special quality, which can in turn evoke an attitude of reverence on the part of the viewer”. Benjamin (1968: 222-223) said that:

We define the aura as the unique phenomenon of a distance, however close it may be. If, while resting on a summer afternoon, you follow with your eyes a mountain range on the horizon or a branch which casts its shadow over you, you experience the aura of those mountains, of that branch.

Benjamin (1968: 222-223)

In this we see that Benjamin equates Aura with immediacy and the uniqueness of the moment of an artefact’s viewing. Since each moment is unique for each of us, it is economically valuable, auratic even. The information age of perfect digital reproduction does not detract from the personal uniqueness of experiential reality, only diminishes the aura offered by certain mass-produced elements within it. In Benjamin’s time, art was sanctified and extremely auratic. Now, most view classical art as a digital copy or encyclopaedia reference, which embodies little of the aura of the original piece. Travelling to the Louvre offers only the added auratic experience not gained from a home computer, where one’s experience is more like anybody else’s experience of the subject, but visitors must get very close to a works in order to experience what Aura remains there.

Benjamin might say that in a world of perfect digital reproduction, virtual objects can have no Aura. The Magic Lens is not spiritually void, however, and my Reality 2.0 is not a stagnant place. It buzzes with an aura of its own, borne from the uniqueness of each interaction, live and in context. A virtual object can be duplicated infinitely, but its location in space and the immediacy of its presence in that moment that helps shape its unique meaning for every AR denizen. Physicality offers the backdrop to the unique experiences offered by virtuality, indeed, experiential reality is improved through the wider opportunity for unique moments.

Returning to the economics of virtual space, virtual objects cannot hold currency in and of themselves, but the context surrounding them has value. Advertisers could exploit this if they act fast in commercialising AR space. More interesting to me is the potential for unique virtual art objects and sculptures that could hold purely aesthetic value. In this case, interested parties would seek out the object and view its intricacies and movements in the space it was designed to fill. There is much room for spatial enquiry in this field, some of which I will come onto in my next section.

Applying Benjamin

Applying Baudrillard

For Jean Baudrillard (1983), “at any moment in the course of our modernity, a particular arrangement of signifying objects and images conditions the way we see the world” (Clark, 1995). “Each major transformation is accompanied by a feeling of disorientation and discomfort over the loss of the previous ‘reality’. This effects a recourse into the imagined certainties of the receding order to ground or stabilise that which is new. In this way, “reality loops around itself”, as “each phase of value integrates into its own apparatus the anterior apparatus as a phantom reference, a puppet or simulation reference”” (Baudrillard, 1988: 145, 121; cited in Clark, 1995). In these words, we see Baudrillard’s perspective can apply neatly to my analysis of Mobile AR. Taking up where McLuhan left us- a view of the Magic Lens constrained by its deterministic overtones- Baudrillard injects the much-needed element of an actively social construction of Mixed Reality, whilst grounding my work in his Postmodern thought on Virtuality.

I am interested in the view that iterations of reality, whilst overlapping and viewable through the Magic Lens, support and influence each other’s existence within a wider structure. I could live wholly in The Virtual, and bring to it conceptions of the reality from whence I came. We see a similar behaviour in Alternate Reality games such as Second Life (Linden Lab: 2003) or The Sims (Maxis: 2000) whereby developers program known physical world causalities, behaviours and actions despite the near-limitless formal opportunities offered by the medium. Users, when given freedom, will likely bring their own conceits and personal experiences to these alternate realities, thereby foregoing what else might be possible in favour of their own culturally-inherited drives and ambitions. The Magic Lens presents a wholly new canvas for the social construction of reality. The collaborative and democratic Mobile 2.0 ethos that Nokia hope to breathe into Mobile AR could falter if users bring too much of our present iteration of reality to it. The Magic Lens offers an opportunity to reshape The Real, not solely through tagging buildings or leaving messages floating in mid-aid, but through the lessons we might learn through engaging with each other in a new way.

Baudrillard focused his work on how we interface with information, and how we build it into our view of reality. He posited that The Media had hijacked reality, becoming a powerful force in the construction of hyper-reality, a social reality that has become more powerful than we exert control over. Through the Magic Lens, we might give form to some aspects of hyper-reality. The medium allows for virtual elements to co-exist with real objects occupying space in the user’s own hyper-reality. In this way, each user can choose which hyper-reality they want to exist in, whether it is one in which 3D AR avatars walk the streets and go about their virtual lives; or one where arrows and directions graphically point out where to go to fulfil a shopping list’s requirements. The Magic Lens makes a shift from mass-media control to personalised, user-focused context-based reality: Reality 2.0 if you will.

Assuming AR does present a new layer to reality, there are certain Baudrillardian imperatives that we will bring to this landscape. One such imperative links the physical properties of real-world space- gravity, mass, optics- to our new environment. To make sense of virtual elements in their context we will employ what we already know about the environment we are in. This means that the most prized virtual objects will exhibit expected behaviour, intuitive interactivity and will be visually suited to its surroundings. Similarly, an object’s location in space alters its perceived importance. I would argue that should a common Mixed Reality exist, governing bodies would write entire protocol for the positioning and size of virtual objects so that one contributor could not take up more than his worth. Important to consider is that even writing hypothetically I am bringing Baudrillardian imperatives to task, applying democracy to a non-existent world! Baudrillard’s “reality loops around itself” has a troublesome effect on my analysis. Let me instead take a fresh perspective, in my next section written from the perspective of Walter Benjamin…

Deconstructing AR

The next four entries in my series on AR will assess it’s potential socio-cultural impact, which has now started to begin.

We have now reached a depth of analysis that has taken in the historical, economic and technologic paths leading to Mobile AR, but there are greater depths to plunge. From this vantage point, the question arises: how will the user experience this new medium? This section assesses the implications of Mobile AR from a critical and media theoretical perspective, and offers the other half of my close analysis.

There are very few published articles on the subject of AR, let alone Mobile AR, and fewer still that offer a media theory perspective on user-interactions in augmented environments. There are a clutch of articles that refer to such an area, but like my own analysis, these academics have resorted to applying old frameworks to a new medium, with McLuhan, Baudrillard and Benjamin seeming the most applicable theorists. My own thoughts on Mobile AR resonate with these thinkers yet no such analysis has involved all three epistemologies, so I must tread carefully…