Bibliography

So that’s it, my series is over. All that’s left to do now is credit the academic sources that influenced and aided in the construction of my argument. Thanks to everyone below, and thanks to you, dear reader, for coming along for the ride.

References:

Baudrillard, Jean (1983). Simulations. New York: Semiotext(e).

Baudrillard, Jean (1988). Selected Writings, ed. Mark Poster. Cambridge: Polity Press.

Baumann, Jim (date unknown). ‘Military applications of virtual reality’ on the World Wide Web. Accessed 20th March 2007. Available at http://www.hitl.washington.edu/scivw/EVE/II.G.Military.html

Benjamin, Walter (1968). ‘The Work of Art in the Age of Mechanical Reproduction’, in Walter Benjamin Illuminations (trans. Harry Zohn), pp. 217–51. New York: Schocken Books.

Bolter, J. D., B. Mcintyre, M. Gandy, Schweitzer, P. (2006). ‘New Media and the Permanent Crisis of Aura’ in Convergence: The International Journal of Research into New Media Technologies, Vol. 12 (1): 21-39.

Botella, Cristina.M, & M.C. Juan, R.M. Banos, M. Alcaniz, V. Guillen, B. Rey (2005). ‘Mixing Realities? An Application of Augmented Reality for the Treatment of Cockroach Phobia’ in CyberPsychology & Behaviour, Vol. 8 (2): 162-171.

Clark, N. ‘The Recursive Generation of the Cyberbody’ in Featherstone, M. & Burrows, R. (1995) Cyberspace/Cyberbodies/Cyberpunk, London: Sage.

Featherstone, Mike. & Burrows, Roger eds. (1995). Cyberspace/ Cyberbodies/ Cyberpunk: Cultures of Technological Embodiment. London: Sage.

Future Image (author unknown) (2006). ‘The 6Sight® Mobile Imaging Report’ on the World Wide Web. Accessed 22nd March 2007. Available at http://www.wirelessimaging.info/

Genosko, Gary (1999). McLuhan and Baudrillard: The Masters of Implosion. London: Routledge.

Kline, Stephen, DePeuter, Grieg, & Dyer-Witheforde, Nick (2003). Digital Play: The Interaction of Technology, Culture, and Marketing. Kingston & Montreal: McGill-Queen’s University Press.

Levinson, Paul (1999). Digital McLuhan: a guide to the information millennium. London: Routledge.

Liarokapis, Fotis (2006). ‘An Exploration from Virtual to Augmented Reality Gaming’ in Simulation Gaming, Vol. 37 (4): 507-533.

Manovich, Lev (2006). ‘The Poetics of Augmented Space’ in Visual Communication, Vol. 5 (2): 219-240.

McLuhan, Marshall (1962). The Gutenberg galaxy: The Making of Typographic Man. Toronto, Canada: University of Toronto Press.

McLuhan, Marshall (1964). Understanding Media: The Extensions of Man. New York: McGraw-Hill.

McLuhan, Marshall and Powers, Bruce R. (1989). The Global Village: Transformations in World Life in the 21st Century. Oxford University Press: New York.

Milgram, Paul & Kishino, Fumio (1994). ‘A Taxonomy of Mixed Reality Visual Displays’ in IEICE Transactions on Information Systems, Vol. E77-D, No.12 December 1994.

Reitmayr, Gerhard & Schmalstieg, Dieter (2001). Mobile Collaborative Augmented Reality. Proceedings of the IEEE 2001 International Symposium on Augmented Reality, 114–123.

Roberts, G., A. Evans, A. Dodson, B. Denby, S. Cooper, R. Hollands (2002) ‘Application Challenge: Look Beneath the Surface with Augmented Reality’ in GPS World, (UK, Feb. 2002): 14-20.

Stokes, Jon (2003). ‘Understanding Moore’s Law’ on the World Wide Web. Accessed 21st March 2007. Available at http://arstechnica.com/articles/paedia/cpu/moore.ars

Straubhaar, Joseph D. & LaRose, Robert (2005). Media Now: Understanding Media, Culture, and Technology. Belmont, CA: Wadsworth.

Thomas, B., Close. B., Donoghue, J., Squires, J., De Bondi, I’,. Morris, M., and Piekarski, W. ‘ARQuake: An outdoor/indoor augmented reality first-person application’ in Proceedings of the Fourth International Symposium on Wearable Computers, (Atlanta, GA, Oct. 2000), 139-141.

Wagner, D., Pintaric, T., Ledermann, F., & Schmalstieg, D. (2005). ‘Towards massively multi-user augmented reality on handheld devices’. In Proc. 3rd Int’l Conference on Pervasive Computing, Munich, Germany.

Weiser, M. (1991) ‘The Computer for the Twenty-First Century’ in Scientific American 265(3), September: 94–104.

Williams, Raymond (1992). Television: Technology and Cultural Form. Hanover and London: University Press of New England and Wesleyan University Press

Further Reading:

Bolter, Jay D. & Grusin, Richard (1999). Remediation: Understanding New Media. Cambridge, MA: MIT Press.

Cavell, Richard (2002). McLuhan in Space: a Cultural Geography. Toronto: University of Toronto Press.

Galloway, Alexander R. (2006). Gaming: Essays on Algorithmic Culture. Minneapolis: University of Minnesota Press.

Horrocks, Christopher (2000). Marshall McLuhan & Virtuality. Cambridge: Icon Books.

Jennings, Pamela (2001). ‘The Poetics of Engagement’ in Convergence: The International Journal of Research into New Media Technologies, Vol. 7 (2): 103-111.

Lauria, Rita (2001). ‘In Love with our Technology: Virtual Reality A Brief Intellectual History of the Idea of Virtuality and the Emergence of a Media Environment’ in Convergence: The International Journal of Research into New Media Technologies, Vol. 7 (4): 30-51.

Lonsway, Brian (2002). ‘Testing the Space of the Virtual’ in Convergence: The International Journal of Research into New Media Technologies, Vol. 8 (3): 61-77.

Moos, Michel A. (1997). Marshall McLuhan Essays: Media Research, technology, art, communication. London: Overseas Publishers Association.

Pacey, Arnold (1983). The Culture of Technology. Oxford: Basil Blackwell.

Salen, Katie & Zimmerman, Eric. (2004) Rules of Play: Game Design Fundamentals. Cambridge, MA: MIT.

Sassower, Raphael (1995). Cultural Collisions: Postmodern Technoscience. London: Routledge.

Wood, John ed. (1998). The Virtual Embodied: Presence/Practice/Technology. London: Routledge.

Web Discoveries for June 24th

These are my del.icio.us links for June 24th

Where is freedimensional?

You’ve probably read about Google Latitude, and maybe even used it yourself. I’ve been using it mostly without meaning to, because I activated the service on my N95’s Google Maps and the bloody thing never turns off. Here’s where I am right now:



Locative technologies are a growing area of interest for me. I believe that GPS, cell-tower triangulation and even good old Bluetooth will play a large part in making cloud-computing extra-relevant to consumers.

I know that people get a bit funny with the blend of real locations and virtual space (see Google Street View debacle) but once we’re all using our next-gen pieces of UI, your networked device could begin to act as a portal to new layers of information useful to you about the city, street, or shop you are in.

I am talking about location-based advertising. An implementational nightmare, but it is foreseeable that Semantic technologies could serve geographically relevant messages, charging advertisers on a cost per impact basis. Google kind of do this with their local search results. It’s a bit shit at the moment though.

The nearest we have to the kind of next-gen solution I’m thinking of is lastminute.com’s free service NRU, available on the Android OS. It lets you scan around your environment with your phone acting as a viewfinder, where cinemas, restaurants and theatres are overlaid in a sonar-like interface. These services pay a small amount to lastminute.com on an affiliate basis, or are paid inclusions:

NRU for Android, from lastminute on the G1

There’s one locative service I’m disappointed never took off in the UK, despite being around for a while. BrightKite is a kind of location-based Twitter, and it had real promise until Google came stomping all over them with the release of Latitude.

If I were to ‘check in’ at The Queens Larder on Russell Square, BrightKite users would see my marker and message on a map of the area, as well as other people checked in nearby. The potential for social interaction is high, because through using the service one feels proximity with other users.

With all this in mind, I’d like my readers to ‘feel closer’ to me, so as well as in this post I’ll be placing my Latitude Location Badge on my Contact Page. If you’re in the vicinity, go ahead and either serve me an advert or say hello. I won’t mind which.

Virtual Reality

AR is considered by some to be a logical progression of VR technologies (Liarokapis, 2006; Botella, 2005; Reitmayr & Schmalstieg, 2001), a more appropriate way to interact with information in real-time that has been granted only by recent innovations. Thus, one could consider that a full historical appraisal would pertain to VR’s own history, plus the last few years of AR developments. Though this method would certainly work for much of Wearable AR- which uses a similar device array- the same could not be said for Mobile AR, since by its nature it offers a set of properties from a wholly different paradigm: portability, connectivity and many years of mobile development exclusive of AR research come together in enhancing Mobile AR’s formal capabilities. Despite the obvious mass-market potential of this technology, most AR research continues to explore the Wearable AR paradigm. Where Mobile AR is cousin to VR, Wearable AR is sister. Most published works favour the Wearable AR approach, so if my assessment of Mobile AR is to be fair I cannot ignore its grounding in VR research.

As aforementioned, VR is the realm at the far right of my Mixed Reality Scale. To explore a Virtual Reality, users must wear a screen array on their heads that cloak the user’s vision with a wholly virtual world. These head-mounted-displays (HMD’s) serve to transpose the user into this virtual space whilst cutting them off from their physical environment:

A Virtual Reality HMD, two LCD screens occupy the wearer's field of vision
A Virtual Reality HMD, two LCD screens occupy the wearer's field of vision

The HMD’s must be connected to a wearable computer, a Ghostbusters-style device attached to the wearer’s back or waist that holds a CPU and graphics renderer. To interact with virtual objects, users must hold a joypad. Aside from being a lot to carry, this equipment is restrictive on the senses and is often expensive:

A Wearable Computer array, this particular array uses a CPU, GPS, HMD, graphics renderer, and human-interface-device
A Wearable Computer array, this particular array uses a CPU, GPS, HMD, graphics renderer, and human-interface-device

It is useful at this point to reference some thinkers in VR research, with the view to better understanding The Virtual realm and its implications for Mobile AR’s Mixed Reality approach. Writing on the different selves offered by various media, Lonsway (2002) states that:

“With the special case of the immersive VR experience, the user is (in actual fact) located in physical space within the apparatus of the technology. The computer-mediated environment suggests (in effect) a trans-location outside of this domain, but only through the construction of a subject centred on the self (I), controlling an abstract position in a graphic database of spatial coordinates. The individual, of which this newly positioned subject is but one component, is participant in a virtuality: a spatio-temporal moment of immersion, virtualised travel, physical fixity, and perhaps, depending on the technologies employed, electro-magnetic frequency exposure, lag-induced nausea, etc.”

Lonsway (2002: 65)

Despite its flaws, media representations of VR technologies throughout the eighties and early nineties such as Tron (Lisberger, 1982), Lawnmower Man (Leonard, 1992) and Johnny Mnemonic (Longo, 1995) generated plenty of audience interest and consequent industrial investment. VR hardware was produced in bulk for much of the early nineties, but it failed to become a mainstream technology largely due to a lack of capital investment in VR content, a function of the stagnant demand for expensive VR hardware (Mike Dicks of Bomb Productions: personal communication). The market for VR content collapsed, but the field remains an active contributor in certain key areas, with notable success as a commonplace training aid for military pilots (Baumann, date unknown) and as an academic tool for the study of player immersion and virtual identity (Lonsway, 2002).

Most AR development uses VR’s same array of devices: a wearable computer, input device and an HMD. The HMD is slightly different in these cases; it is transparent and contains an internal half-silvered mirror, which combines images from an LCD display with the user’s vision of the world:

An AR HMD, this model has a half-mirrored screen at 45 degrees. Above are two LCDs that reflect into the wearer's eyes whilst they can see what lies in front of them
An AR HMD, this model has a half-mirrored screen at 45 degrees. Above are two LCDs that reflect into the wearer's eyes whilst they can see what lies in front of them

 

What Wearable AR looks like, notice the very bright figure ahead. If he was darker he would not be visible
What Wearable AR looks like, notice the very bright figure ahead. If he was darker he would not be visible

There are still many limitations placed on the experience, however: first, the digital graphics must be very bright in order to stand out against natural light; second, they require the use of a cumbersome wearable computer array; third, this array is at a price-point too high for it to reach mainstream use. Much of the hardware used in Wearable AR research is bought wholesale from liquidized VR companies (Dave Mee of Gameware: personal communication), a fact representative of the backward thinking of much AR research.

In their work New Media and the Permanent Crisis of Aura Bolter et al. (2006) apply Benjamin’s work on the Aura to Mixed Reality technologies, and attempt to forge a link between VR and the Internet. This passage offers a perspective on the virtuality of the desktop computer and the World Wide Web:

“What we might call the paradigm of mixed reality is now competing successfully with what we might call ‘pure virtuality’ – the earlier paradigm that dominated interface design for decades.
In purely virtual applications, the computer defines the entire informational or perceptual environment for the user … The goal of VR is to immerse the user in a world of computer generated images and (often) computer-controlled sound. Although practical applications for VR are relatively limited, this technology still represents the next (and final?) logical step in the quest for pure virtuality. If VR were perfected and could replace the desktop GUI as the interface to an expanded World Wide Web, the result would be cyberspace.”

Bolter et al. (2006: 22)

This account offers a new platform for discussion useful for the analysis of the Internet as a component in Mobile AR: the idea that the Internet could exploit the spatial capabilities of a Virtual Reality to enhance its message. Bolter posits that this could be the logical end of a supposed “quest for pure virtuality”. I would argue that the reason VR did not succeed is the same reason that there is no “quest” to join: VR technologies lack the real-world applicability that we can easily find in reality-grounded media such as the Internet or mobile telephone.

What is AR and What is it Capable Of?

Presently, most AR research is concerned with live video imagery and it’s processing, which allows the addition of live-rendered 3D digital images. This new augmented reality is viewable through a suitably equipped device, which incorporates a camera, a screen and a CPU capable of running specially developed software. This software is written by specialist software programmers, with knowledge of optics, 3D-image rendering, screen design and human interfaces. The work is time consuming and difficult, but since there is little competition in this field, the rare breakthroughs that do occur are as a result of capital investment: something not willingly given to developers of such a nascent technology.

What is exciting about AR research is that once the work is done, its potential is immediately seen, since in essence it is a very simple concept. All that is required from the user is their AR device and a real world target. The target is an object in the real world environment that the software is trained to identify. Typically, these are specially designed black and white cards known as markers:

An AR marker, this one relates to a 3D model of Doctor Who's Tardis in Gameware's HARVEE kit
An AR marker, this one relates to a 3D model of Doctor Who's Tardis in Gameware's HARVEE kit

These assist the recognition software in judging viewing altitude, distance and angle. Upon identification of a marker, the software will project or superimpose a virtual object or graphical overlay above the target, which becomes viewable on the screen of the AR device. As the device moves, the digital object orients in relation to the target in real-time:

armarker2
Augmented Reality in action, multiple markers in use on the HARVEE system on a Nokia N73

The goal of some AR research is to free devices from markers, to teach AR devices to make judgements about spatial movements without fixed reference points. This is the cutting edge of AR research: markerless tracking. Most contemporary research, however, uses either marker-based or GPS information to process an environment.

Marker-based tracking is suited to local AR on a small scale, such as the Invisible Train Project (Wagner et al., 2005) in which players collaboratively keep virtual trains from colliding on a real world toy train track, making changes using their touch-screen handheld computers:

crw_80271
The Invisible Train Project (Wagner et al., 2005)

GPS tracking is best applied to large scale AR projects, such as ARQuake (Thomas et al, 2000), which exploits a scale virtual model of the University of Adelaide and a modified Quake engine to place on-campus players into a ‘first-person-shooter’. This application employs use of a headset, wearable computer, and a digital compass, which offer the effect that enemies appear to walk the corridors and ‘hide’ around corners. Players shoot with a motion-sensing arcade gun, but the overall effect is quite crude:

100-0007_img_21
ARQuake (Thomas et al, 2000)

More data input would make the game run smoother and would provide a more immersive player experience. The best applications of AR will exploit multiple data inputs, so that large-scale applications might have the precision of marker-based applications whilst remaining location-aware.

Readers of this blog will be aware that AR’s flexibility as a platform lends applicability to a huge range of fields:

  • Current academic work uses AR to treat neurological conditions: AR-enabled projections have successfully cured cockroach phobia in some patients (Botella et al., 2005);
  • There are a wide range of civic and architectural uses: Roberts et al. (2002) have developed AR software that enables engineers to observe the locations of underground pipes and wires in situ, without the need schematics
  • AR offers a potentially rich resource to the tourism industry: the Virtuoso project (Wagner et al., 2005) is a handheld computer program that guides visitors around an AR enabled gallery, providing additional aural and visual information suited to each artefact;

The first commercial work in the AR space was far more playful, however: AR development in media presentations for television has led to such primetime projects as Time Commanders (Lion TV for BBC2, 2003-2005) in which contestants oversee an AR-enabled battlefield, and strategise to defeat the opposing army, and FightBox (Bomb Productions for BBC2, 2003) in which players build avatars to compete in an AR ‘beat-em-up’ that is filmed in front of a live audience; T-Immersion (2003- ) produce interactive visual installations for theme parks and trade expositions; other work is much more simple, in one case the BBC commissioned an AR remote-control virtual Dalek meant for mobile phones, due for free download from BBC Online:

A Dalek, screenshot taken from HARVEE's development platform (work in progress)
A Dalek, screenshot taken from HARVEE's development platform (work in progress)

The next entry in this series is a case study in AR development. If you haven’t already done so, please follow me on Twitter or grab an RSS feed to be alerted when my series continues.

How I Feel About AR Right Now

I found this video clip which does well to sum up my current opinion on marker-based Augmented Reality, which seems to be breaking into the mainstream via desktop applications and fixed webcams:


Me too (doing some AR stuff)! from Anatoly Zenkov on Vimeo.

You’ve seen it around the office – someone prints out a marker and everyone huddles around the only computer with a webcam. Anatoly Zenkov has noticed it too, and he wants you to realise that all you’re really looking at is a static 3D image overlaid on a piece of paper. Nothing culture-shifting about that now is there?

Let me be among the first of the dissenters:
There is far more to AR than barcodes and webcams.

There are thousands of helpful & exciting uses for the technology. Think past ‘tethered’ AR experiences and consider what good Mobile AR can do with GPS and with markerless tracking. Endless potential lies that way. I assure you, the future is far brighter than we think.

Where I initially believed that the Advertising and Entertainment industries would drive innovation and push AR into popular consiousness (they are currently doing so), I now believe that good AR (as distinct from any old AR) will be driven by paid applications on next-gen handsets.

I think I’ll serialise posts about the best applications for Augmented Reality over the coming weeks. Why not subscribe now to hear about it first? You know it makes sense.