Summary So Far

In summary, Mobile AR has many paths leading to it. It is this convergence of various paths that makes a true historical appraisal of this technology difficult to achieve. However, I have highlighted facets of its contributing technologies that assist in the developing picture of the implications that Mobile AR has in store. A hybridisation of a number of different technologies, Mobile AR embodies the most gainful properties of its three core technologies: This analyst sees Mobile AR as a logical progression from VR, but recognises its ideological rather than technological founding. The hardware basis of Mobile AR stems from current mobile telephony trends that exploit the growing capabilities of Smartphone devices. The VR philosophy and the mobile technology are fused through the Internet, the means for enabling context-based, live-updating content, and housing databases of developer-built and user-generated digital objects and elements, whilst connecting users across the world.

I have shown that where the interest in VR technologies dwindled due to its limited real-world applicability, Mobile Internet also lacks in comparison to Mobile AR and its massive scope for intuitive, immersive and realistic interpretations of digital information. Wearable AR computing shares VR’s weaknesses, despite keeping the user firmly grounded in physical reality. Mobile AR offers a solution that places the power of these complex systems into a mobile telephone: the ubiquitous technology of our generation. This new platform solves several problems at once, most importantly for AR developers and interested Blue-chip parties, market readiness. Developing for Mobile AR is simply the commercially sensible thing to do, since the related industries are already making the changes required for its mass-distribution.

Like most nascent technologies, AR’s success depends on its commercial viability and financial investment, thus most sensible commercial developers of AR technologies are working on projects for the entertainment and advertising industries, where their efforts can be rewarded quickly. These small-scale projects are often simple in concept, easily grasped and thus not easily forgotten. I claim here that the first Mobile AR releases will generate early interest in the technology and entertainment markets, with the effect that press reportage and word-of-mouth behaviour assist Mobile AR’s uptake. I must be careful with my claims here however, since there is no empirical evidence to suggest that this will occur for Mobile AR. Looking at the emergence of previous technologies, however, the Internet and mobile telephony grew rapidly and to massive commercial success thanks to some strong business models and advancements in their own supporting technologies. It is strongly hoped by developers like Gameware and T-Immersion that Mobile AR can enjoy this same rapid lift-off. Both technologies gained prominence once visible in the markets thanks to a market segment called early adopters. This important group gathers their information from specialist magazine sources and word of mouth. Mobile AR developers would do well to recognise the power of this group, perhaps by offering shareware versions of their AR software that encourage a form of viral transmission that exploit text messaging.

Gameware have an interesting technique for the dissemination of their HARVEE software. They share a business interest with a Bluetooth technology firm, which has donated a prototype product the Bluetooth Push Box, which scans for local mobile devices and automatically sends files to users in acceptance. Gameware’s Push Box sends their latest demo to all visitors to their Cambridge office. This same technology could be placed in public places or commercial spaces to offer localised AR advertising, interactive tourist information, or 3D restaurant menus, perhaps.

Gameware, through its Nokia projects and HARVEE development program is well placed to gain exposure on the back of a market which is set to explode as mobile offerings become commercially viable, ‘social’, powerful, multipurpose and newsworthy. Projects like HARVEE are especially interesting in terms of their wide applicability and mass-market appeal. It is its potential as a revolutionary new medium that inspires this very series.

Colour Picker by Jinsun Park

Colour Picker is an innovative design of a concept pen that can scan colours from anything around and instantly use the colour for drawing:

After placing the pen against an object, the user just presses the scan button. The colour is being detected by the colour sensor and the RGB cartridge of the pen mixes the required inks to create the target colour:

This superb device will help people to observe the changing colours of nature. With colour picker, all range of artists will be able to create a more sensorial and visual insight of their surrounding nature’s colours:

via Colour Picker by Jinsun Park | Future Technology.

Mobile Telephone

The Internet and the mobile phone are two mighty forces that have bent contemporary culture and remade it in their form. They offer immediacy, connectivity, and social interaction of a wholly different kind. These are technologies that have brought profound changes to the ways academia consider technoscience and digital communication. Their relationship was of interest to academics in the early 1990’s, who declared that their inevitable fusion would be the beginning of the age of Ubiquitous Computing: “the shift away from computing which centered on desktop machines towards smaller multiple devices distributed throughout the space” (Weiser, 1991 in Manovich, 2006). In truth, it was the microprocessor and Moore’s Law- “the number of transistors that can be fit onto a square inch of silicon doubles every 12 months” (Stokes, 2003) that led to many of the technologies that fall under this term: laptops, PDA’s, Digital Cameras, flash memory sticks and MP3 players. Only recently have we seen mobile telephony take on the true properties of the Internet.

The HARVEE project is partially backed by Nokia Corp. which recognises its potential as a Mobile 2.0 technology: user-generated content for mobile telephony that exploits web-connectivity. Mobile 2.0 is an emerging technology thematically aligned with the better established Web 2.0. Nokia already refer to their higher-end devices as multimedia computers, rather than as mobile phones. Their next generation Smartphones will make heavy use of camera-handling systems, which is predicated on the importance of user-generated content as a means to promote social interaction. This strategic move is likely to realign Nokia Corp.’s position in the mobile telephony and entertainment markets.

Last year, more camera phones were sold than digital cameras (Future Image, 2006). Nokia have a 12 megapixel camera phone ready for release in 2009, and it will be packaged with a processing unit equal to the power of a Sony PSP (Nokia Finland: non-public product specification document). MP3 and movie players are now a standard on many handsets, stored on plug-in memory cards and viewed through increasingly higher resolution colour screens. There is a growing mobile gaming market, the fastest growing sector of the Games Industry (Entertainment & Leisure Software Publishers Association (ELSPA) sales chart). The modern mobile phone receives its information from wide-band GPRS networks allowing greater network coverage and faster data transfer. Phone calls are the primary function, but users are exploiting the multi-media capabilities of their devices in ways not previously considered. It is these factors, technologic, economic and infrastructural that provide the perfect arena for Mobile AR’s entry into play.

Mobile Internet is the natural convergence of mobile telephony and the World Wide Web, and is already a common feature of new mobile devices. Mobile Internet, I would argue, is another path leading to Mobile AR, driven by mobile users demanding more from their handsets. Mobile 2.0 is the logical development of this technology- placing the power of location-based, user-generated content into a new real-world context. Google Maps Mobile is one such application that uses network triangulation and its own Google Maps technologies to offer information, directions, restaurant reviews or even satellite images of your current location- anywhere in the world. Mobile AR could achieve this same omniscience (omnipresence?) given the recent precedent for massively multi-user collaborative projects such as Wikipedia, Flickr and Google Maps itself. These are essentially commercially built infrastructures designed to be filled with everybody’s tags, comments or other content. Mobile AR could attract this same amount of devotion if it offered such an infrastructure and real-world appeal.

There is a growing emphasis on Ubiquitous Computing devices in our time-precious world, signified by the increased sales in Smartphones and WiFi enabled laptops. Perhaps not surprisingly, Mobile Internet use has increased as users’ devices become capable of greater connectivity. Indeed, the mobile connected device is becoming the ubiquitous medium of modernity, as yet more media converge in it. It is the mobile platform’s suitability to perform certain tasks that Mobile AR can take advantage of, locating itself in the niche currently occupied by Mobile Internet. Returning to my Mixed Reality Scale, Mobile AR serves the user better than Mobile Internet currently can: providing just enough reality to exploit virtuality, Mobile AR keeps the user necessarily grounded in their physical environment as they manipulate digital elements useful to their daily lives.

Crowdsourced Protein Shakes

I read about Foldit in Wired US yesterday, a game that takes the foundations laid by SETI@home, which uses thousands of computers’ idle time to decode frequencies from Space, and crowdsources solutions to the protein folding problems that are currently baffling the smartest machines in the world.

The difference with Foldit is that it’s not PC idle time that is tapped into here, but players’ idle time. There is no algorithm that can yet match humans’ depth perception; natural ability to recognise patterns; and see causal links in their actions. These traits make us humans the ideal CPU to solve these ‘protein-puzzles’:

Foldit provides a series of tutorials in which the player manipulates simple protein-like structures, and a periodically updated set of puzzles based on real proteins. The application displays a graphical representation of the protein’s structure which the user is able to manipulate with the aid of a set of tools.

As the structure is modified, a “score” is calculated based on how well-folded the protein is, based on a set of rules. A list of high scores for each puzzle is maintained. Foldit users may create and join groups, and share puzzle solutions with each other; a separate list of group high scores is maintained.

Indeed, the creators report that groups working together have led to breakthroughs not matched by either individuals or heavy-duty computing power. It is the power of the engaged-masses that the Baker Lab, research team behind the game are hoping will bring forth potential cures for HIV/AIDS, Cancer and Alzheimer’s.

More info on the game and it’s background on their Science Portal.

Does this remind anyone of War Games?

The Internet

The Internet, or specifically the World Wide Web, requires a limited virtuality in order to do its job. The shallow immersion offered to us by our computer screens actually serves our needs very well, since the Internet’s role in our lives is to connect, store and present information in accessible, searchable, scannable, and consistent form for millions of users to access simultaneously, to be dived in and out of quickly or to surround ourselves in the information we want. The naturally-immersive VR takes us partway towards Mobile AR, but its influence stops at the (admittedly profound) concept of real-time interaction with 3D digital images. What the Internet does is bring information to us, but VR forces us to go to it.

This is a function of the Mixed Reality Scale, and the distance of each from The Real. The closer we can bring artefacts from The Virtual to The Real, the more applicable these can be in our everyday lives. The self-sufficient realm of The Virtual does not require grounding in physical reality in order to exist, whereas the Internet and other MR media depend on The Real to operate. AR is the furthest that a virtual object can be ‘stitched into’ our reality, and in doing so we exploit our power in this realm to manipulate and interact with these digital elements to suit our own ends, as we currently do with the World Wide Web.

The wide-ranging entertainment resources offered by the Internet are having a profound effect on real-world businesses, a state of flux that Mobile AR could potentially exploit. There is a shift in the needs of consumers of late that is forcing a change in the ways that many blue-chip organisations are handling their businesses: Mobile data carriers (operators), portals, publishers, content owners and broadcasters are all seeking new content types to face up to the threat of VOIP (Voice Over Internet Protocol) – which is reducing voice traffic; and Web TV/ Internet – reducing (reduced?) TV audiences, particularly in the youth market.

T-Mobile, for example, seeks to improve on revenues through offering unique licensed mobile games, themes, ringtones and video-clips on their T-Zones Mobile Internet Portal; NBC’s hit-series ‘Heroes’ is the most downloaded show on the Internet, forcing NBC to offer exclusive online comics on their webpage, seeking to recoup advertising revenue losses through lacing the pages of these comics with advertising. Mobile AR represents a fresh landscape for these businesses to mine. It is no surprise, then, that some forward-thinking AR developers are already writing software specifically for the display of virtual advertisement billboards in built-up city areas (T-Immersion).

The Internet has changed the way we receive information about the world around us. This hyper-medium has swallowed the world’s information and media content, whilst continuing to enable the development of new and exciting offerings exclusive to the desktop user. The computing capacity required to use the Internet has in the past constrained the medium to the desktop computer, but in the ‘Information Age’ the World Wide Web is just that: World Wide.

Where is freedimensional?

You’ve probably read about Google Latitude, and maybe even used it yourself. I’ve been using it mostly without meaning to, because I activated the service on my N95’s Google Maps and the bloody thing never turns off. Here’s where I am right now:



Locative technologies are a growing area of interest for me. I believe that GPS, cell-tower triangulation and even good old Bluetooth will play a large part in making cloud-computing extra-relevant to consumers.

I know that people get a bit funny with the blend of real locations and virtual space (see Google Street View debacle) but once we’re all using our next-gen pieces of UI, your networked device could begin to act as a portal to new layers of information useful to you about the city, street, or shop you are in.

I am talking about location-based advertising. An implementational nightmare, but it is foreseeable that Semantic technologies could serve geographically relevant messages, charging advertisers on a cost per impact basis. Google kind of do this with their local search results. It’s a bit shit at the moment though.

The nearest we have to the kind of next-gen solution I’m thinking of is lastminute.com’s free service NRU, available on the Android OS. It lets you scan around your environment with your phone acting as a viewfinder, where cinemas, restaurants and theatres are overlaid in a sonar-like interface. These services pay a small amount to lastminute.com on an affiliate basis, or are paid inclusions:

NRU for Android, from lastminute on the G1

There’s one locative service I’m disappointed never took off in the UK, despite being around for a while. BrightKite is a kind of location-based Twitter, and it had real promise until Google came stomping all over them with the release of Latitude.

If I were to ‘check in’ at The Queens Larder on Russell Square, BrightKite users would see my marker and message on a map of the area, as well as other people checked in nearby. The potential for social interaction is high, because through using the service one feels proximity with other users.

With all this in mind, I’d like my readers to ‘feel closer’ to me, so as well as in this post I’ll be placing my Latitude Location Badge on my Contact Page. If you’re in the vicinity, go ahead and either serve me an advert or say hello. I won’t mind which.

Virtual Reality

AR is considered by some to be a logical progression of VR technologies (Liarokapis, 2006; Botella, 2005; Reitmayr & Schmalstieg, 2001), a more appropriate way to interact with information in real-time that has been granted only by recent innovations. Thus, one could consider that a full historical appraisal would pertain to VR’s own history, plus the last few years of AR developments. Though this method would certainly work for much of Wearable AR- which uses a similar device array- the same could not be said for Mobile AR, since by its nature it offers a set of properties from a wholly different paradigm: portability, connectivity and many years of mobile development exclusive of AR research come together in enhancing Mobile AR’s formal capabilities. Despite the obvious mass-market potential of this technology, most AR research continues to explore the Wearable AR paradigm. Where Mobile AR is cousin to VR, Wearable AR is sister. Most published works favour the Wearable AR approach, so if my assessment of Mobile AR is to be fair I cannot ignore its grounding in VR research.

As aforementioned, VR is the realm at the far right of my Mixed Reality Scale. To explore a Virtual Reality, users must wear a screen array on their heads that cloak the user’s vision with a wholly virtual world. These head-mounted-displays (HMD’s) serve to transpose the user into this virtual space whilst cutting them off from their physical environment:

A Virtual Reality HMD, two LCD screens occupy the wearer's field of vision
A Virtual Reality HMD, two LCD screens occupy the wearer's field of vision

The HMD’s must be connected to a wearable computer, a Ghostbusters-style device attached to the wearer’s back or waist that holds a CPU and graphics renderer. To interact with virtual objects, users must hold a joypad. Aside from being a lot to carry, this equipment is restrictive on the senses and is often expensive:

A Wearable Computer array, this particular array uses a CPU, GPS, HMD, graphics renderer, and human-interface-device
A Wearable Computer array, this particular array uses a CPU, GPS, HMD, graphics renderer, and human-interface-device

It is useful at this point to reference some thinkers in VR research, with the view to better understanding The Virtual realm and its implications for Mobile AR’s Mixed Reality approach. Writing on the different selves offered by various media, Lonsway (2002) states that:

“With the special case of the immersive VR experience, the user is (in actual fact) located in physical space within the apparatus of the technology. The computer-mediated environment suggests (in effect) a trans-location outside of this domain, but only through the construction of a subject centred on the self (I), controlling an abstract position in a graphic database of spatial coordinates. The individual, of which this newly positioned subject is but one component, is participant in a virtuality: a spatio-temporal moment of immersion, virtualised travel, physical fixity, and perhaps, depending on the technologies employed, electro-magnetic frequency exposure, lag-induced nausea, etc.”

Lonsway (2002: 65)

Despite its flaws, media representations of VR technologies throughout the eighties and early nineties such as Tron (Lisberger, 1982), Lawnmower Man (Leonard, 1992) and Johnny Mnemonic (Longo, 1995) generated plenty of audience interest and consequent industrial investment. VR hardware was produced in bulk for much of the early nineties, but it failed to become a mainstream technology largely due to a lack of capital investment in VR content, a function of the stagnant demand for expensive VR hardware (Mike Dicks of Bomb Productions: personal communication). The market for VR content collapsed, but the field remains an active contributor in certain key areas, with notable success as a commonplace training aid for military pilots (Baumann, date unknown) and as an academic tool for the study of player immersion and virtual identity (Lonsway, 2002).

Most AR development uses VR’s same array of devices: a wearable computer, input device and an HMD. The HMD is slightly different in these cases; it is transparent and contains an internal half-silvered mirror, which combines images from an LCD display with the user’s vision of the world:

An AR HMD, this model has a half-mirrored screen at 45 degrees. Above are two LCDs that reflect into the wearer's eyes whilst they can see what lies in front of them
An AR HMD, this model has a half-mirrored screen at 45 degrees. Above are two LCDs that reflect into the wearer's eyes whilst they can see what lies in front of them

 

What Wearable AR looks like, notice the very bright figure ahead. If he was darker he would not be visible
What Wearable AR looks like, notice the very bright figure ahead. If he was darker he would not be visible

There are still many limitations placed on the experience, however: first, the digital graphics must be very bright in order to stand out against natural light; second, they require the use of a cumbersome wearable computer array; third, this array is at a price-point too high for it to reach mainstream use. Much of the hardware used in Wearable AR research is bought wholesale from liquidized VR companies (Dave Mee of Gameware: personal communication), a fact representative of the backward thinking of much AR research.

In their work New Media and the Permanent Crisis of Aura Bolter et al. (2006) apply Benjamin’s work on the Aura to Mixed Reality technologies, and attempt to forge a link between VR and the Internet. This passage offers a perspective on the virtuality of the desktop computer and the World Wide Web:

“What we might call the paradigm of mixed reality is now competing successfully with what we might call ‘pure virtuality’ – the earlier paradigm that dominated interface design for decades.
In purely virtual applications, the computer defines the entire informational or perceptual environment for the user … The goal of VR is to immerse the user in a world of computer generated images and (often) computer-controlled sound. Although practical applications for VR are relatively limited, this technology still represents the next (and final?) logical step in the quest for pure virtuality. If VR were perfected and could replace the desktop GUI as the interface to an expanded World Wide Web, the result would be cyberspace.”

Bolter et al. (2006: 22)

This account offers a new platform for discussion useful for the analysis of the Internet as a component in Mobile AR: the idea that the Internet could exploit the spatial capabilities of a Virtual Reality to enhance its message. Bolter posits that this could be the logical end of a supposed “quest for pure virtuality”. I would argue that the reason VR did not succeed is the same reason that there is no “quest” to join: VR technologies lack the real-world applicability that we can easily find in reality-grounded media such as the Internet or mobile telephone.