Web Discoveries for June 24th

These are my del.icio.us links for June 24th

What is AR and What is it Capable Of?

Presently, most AR research is concerned with live video imagery and it’s processing, which allows the addition of live-rendered 3D digital images. This new augmented reality is viewable through a suitably equipped device, which incorporates a camera, a screen and a CPU capable of running specially developed software. This software is written by specialist software programmers, with knowledge of optics, 3D-image rendering, screen design and human interfaces. The work is time consuming and difficult, but since there is little competition in this field, the rare breakthroughs that do occur are as a result of capital investment: something not willingly given to developers of such a nascent technology.

What is exciting about AR research is that once the work is done, its potential is immediately seen, since in essence it is a very simple concept. All that is required from the user is their AR device and a real world target. The target is an object in the real world environment that the software is trained to identify. Typically, these are specially designed black and white cards known as markers:

An AR marker, this one relates to a 3D model of Doctor Who's Tardis in Gameware's HARVEE kit
An AR marker, this one relates to a 3D model of Doctor Who's Tardis in Gameware's HARVEE kit

These assist the recognition software in judging viewing altitude, distance and angle. Upon identification of a marker, the software will project or superimpose a virtual object or graphical overlay above the target, which becomes viewable on the screen of the AR device. As the device moves, the digital object orients in relation to the target in real-time:

armarker2
Augmented Reality in action, multiple markers in use on the HARVEE system on a Nokia N73

The goal of some AR research is to free devices from markers, to teach AR devices to make judgements about spatial movements without fixed reference points. This is the cutting edge of AR research: markerless tracking. Most contemporary research, however, uses either marker-based or GPS information to process an environment.

Marker-based tracking is suited to local AR on a small scale, such as the Invisible Train Project (Wagner et al., 2005) in which players collaboratively keep virtual trains from colliding on a real world toy train track, making changes using their touch-screen handheld computers:

crw_80271
The Invisible Train Project (Wagner et al., 2005)

GPS tracking is best applied to large scale AR projects, such as ARQuake (Thomas et al, 2000), which exploits a scale virtual model of the University of Adelaide and a modified Quake engine to place on-campus players into a ‘first-person-shooter’. This application employs use of a headset, wearable computer, and a digital compass, which offer the effect that enemies appear to walk the corridors and ‘hide’ around corners. Players shoot with a motion-sensing arcade gun, but the overall effect is quite crude:

100-0007_img_21
ARQuake (Thomas et al, 2000)

More data input would make the game run smoother and would provide a more immersive player experience. The best applications of AR will exploit multiple data inputs, so that large-scale applications might have the precision of marker-based applications whilst remaining location-aware.

Readers of this blog will be aware that AR’s flexibility as a platform lends applicability to a huge range of fields:

  • Current academic work uses AR to treat neurological conditions: AR-enabled projections have successfully cured cockroach phobia in some patients (Botella et al., 2005);
  • There are a wide range of civic and architectural uses: Roberts et al. (2002) have developed AR software that enables engineers to observe the locations of underground pipes and wires in situ, without the need schematics
  • AR offers a potentially rich resource to the tourism industry: the Virtuoso project (Wagner et al., 2005) is a handheld computer program that guides visitors around an AR enabled gallery, providing additional aural and visual information suited to each artefact;

The first commercial work in the AR space was far more playful, however: AR development in media presentations for television has led to such primetime projects as Time Commanders (Lion TV for BBC2, 2003-2005) in which contestants oversee an AR-enabled battlefield, and strategise to defeat the opposing army, and FightBox (Bomb Productions for BBC2, 2003) in which players build avatars to compete in an AR ‘beat-em-up’ that is filmed in front of a live audience; T-Immersion (2003- ) produce interactive visual installations for theme parks and trade expositions; other work is much more simple, in one case the BBC commissioned an AR remote-control virtual Dalek meant for mobile phones, due for free download from BBC Online:

A Dalek, screenshot taken from HARVEE's development platform (work in progress)
A Dalek, screenshot taken from HARVEE's development platform (work in progress)

The next entry in this series is a case study in AR development. If you haven’t already done so, please follow me on Twitter or grab an RSS feed to be alerted when my series continues.

Abstract

This series addresses the development and emergence of a new media technology known as Augmented Reality.
I hold the view that this is a culturally significant innovation that holds implications for the user in society.

My discussion reaches a depth of analysis not yet met by contemporary media theorists, but I employ previous academic thought on emergent technologies to provide a framework for analysis, whose work also guides and highlights certain key points that I make.

I address this work with a methodical, structural approach that leaves room for exploration of themes such as virtuality, experiential reality, economics, art, the aura, space, technological determinism, and hyper-reality.

Gutter Talk

Gutter Talk: Hot & Cool in the World of Comic Books

The medium we call comics is based on a simple idea: the idea of placing one picture after another to show the passage of time. Legendary comic artist Will Eisner defines comics as ‘sequential art’. The idea that art can be ‘joined up’ to create a narrative can be seen in practice throughout the ages and is therefore not new. Examples from history include Egyptian hieroglyphics, European stained glass windows, Greek wall frescoes, Japanese scrolls and prehistoric wall paintings. In the year 1066 the Bayeux Tapestry, a 230 foot long piece of sequential art, was completed. Reading from left to right, it illustrates the events of the Norman Conquest of England unfolding in deliberate, chronological order before its viewers. Today the formula for making sequential art remains the same, though Scott McCloud’s definition is more relevant to the type of comics we have come to know. In his book ‘Understanding Comics’ McCloud states that comics are:

“Juxtaposed pictorial and other images in deliberate sequence, intended to convey information and/or to produce an aesthetic response in the viewer”

(McCloud, 1993, page 9)

This definition ignores single-panel comics such as Larson’s ‘The Far Side’, and other visual mediums like animation, but includes the use of the ‘speech bubble’ as a critical element in a comic’s narrative. McCloud’s definition lends itself to the most common forms of comics: newspaper strips, magazine-format comic books and graphic novels.

So how do modern comics work? Each panel in a comic is contained within a border. Each panel is a self-contained piece of art, and a fragment of a narrative. When read in a sequence, the gap between each panel is ‘filled in’ by the reader’s imagination. It is the reader that ‘animates’ these still pictures, not the medium itself. This happens because the mind is designed to close gaps in our awareness. Gestalt psychologists call this process ‘closure’. The real power of comics, then, lies not in the characters involved, nor what they are saying, or even the stories themselves, nor any of the medium’s potential content, but in how the reader interprets each panel as part of a sequence. As Marshall McLuhan said, “the medium is the message”.

One of the seminal media analyst’s best known concepts is that of a medium being either ‘hot’ or ‘cool’. In a broadcast on California’s CBC Television on June 22, 1965, McLuhan was asked to explain his concept:

“‘Cool’ is a slang term borrowed from the world of Jazz and popular music. The word ‘cool’ has a kind of mystical meaning that is not unlike the Hindu idea of detachment. The Hindu idea of detachment means complete involvement in an action and detachment in action. Whereas when people are merely involved in an action but not detached from the action, that’s ‘square’, or ‘hot’. Most people think of ‘cool’ as merely detached from action, but the word ‘cool’ as used in Jazz, and as I use it as sense for a medium, means a medium which uses all of you, but leaves you detached in the act of using you.”

(McLuhan, CBC.CA, 1965)

McLuhan exemplified hot media as: radio, print, photographs, movies and lectures; and cool media as: the telephone, speech, cartoons, TV and seminars. The comic book is another example of a cool medium, and to test McLuhan’s ‘Hot and Cool’ probe as a whole we will look at a comics series that typifies the medium; In the words of the great (but fictional) Professor Charles Xavier, “To me, my X-Men”.

For some, the X-Men comics are the epitomy of how all comics should be; fluid, exciting, intelligent, emotionally complex, relevant, and expressively drawn. Since their conception by Stan Lee in 1963, Marvel Comics X-Men have come very far indeed. Comparing the X-Men across the decades, panels of artwork today are richer with visual information. Comics in the early 1970s had a palette of just twelve colours. The reader had to use their mind’s eye to see the School for Gifted Youngsters that the series’ artists had in mind when drawing the series. Today, details such as old chalk-marks and lecture notes appear on the blackboard in Storm’s history classroom to give the reader a deeper sense that Mutant Academy is a real place. The characters’ facial expression are radically improved today, greater subtleties between Wolverine’s bad moods can now be conveyed, and fiery explosions are almost photo-realistic in terms of their adhering to physical law. There are now fewer gaps for the reader’s mind to fill in. Because of technological advancements and improvements in artistry, the X-Men and comic books as a whole have heated up. However, the images are still constrained within the borders of each frame. We must imagine what else is in the room when we see it from just one perspective, whereas in hotter media such as film the camera angles are generally wider, providing more information and therefore less detachment.

The panel makes up one essential part of the comic book’s construct. The second aspect is ‘The Gutter’ or the space between the panels. McCloud states that:

“Here in the limbo of the gutter, human imagination takes two separate images and transforms them into a single idea. Nothing is seen between the two panels, but experience tells you something must be there! Comic panels fracture both time and space, offering a jagged, staccato rhythm of unconnected moments. But closure allows us to connect these moments and mentally construct a continuous, unified reality. If visual iconography is the vocabulary of comics, closure is its grammar. And since our definition of comics hinges on the arrangement of elements then, in a very real sense, comics is closure!”

(McCloud, 1993, page 66-67)

Using the Gestalt definition of closure as “a principle of organization holding that there is an innate tendency to perceive incomplete objects as complete and to close or fill gaps” (TheFreeDictionary.com) and fusing it with McCloud’s concept of ‘the gutter’, in a McLuhanesque sense ‘the gutter’ becomes a very cool place indeed, completely devoid of content but for the reader’s own imaginings. There is actually an evolving art to reading or processing a comic as the guttering mechanisms become more and more sophisticated. In this way, comics could be said to be getting cooler. In some ways there’s a gutter happening whenever you turn the page too – a good example being that really ‘cool’ thing that happens when you turn the page in an action sequence (or somesuch) to a full or double page spread, sometimes with no words, and the reader really gets pulled in by the dramatic effect. Perhaps readers who are more brought up on squarer mediums, where they are more ‘spoon-fed’ with input have a harder time comprehending the narrative flow of certain comic books and so they are put off by the medium and never get to appreciate what it can offer. There are some types of comic book that require a pretty experienced reader, well versed in the ‘laws of the gutter’ and these are the coolest types of comics.

McLuhan’s thermometer is a tool for relativity, and works best when comparing multiple media. Luckily the X-Men do not appear solely in comics. They also appear in two animated television shows and have a highly successful movie franchise under their (spandex utility) belts. In a comic book, frames are viewed from up left, to bottom right. This requires active participation from the reader to help the narrative unfold. Panels can be ‘rewound’, ‘paused’ or skipped entirely. In cinema, frames on a film reel are shown to you in rapid succession and in the same place: the screen. All that the viewer has to do is sit back and let the medium wash over them. There is only a miniscule ‘gutter’ between the frames of a movie because the frame rate must be high enough to make still images on a movie reel appear to be moving fluidly. The closure between these frames is continuous, involuntary and imperceptible. The closure of frames in comics is reader dictated, involving, and necessary to the understanding of the book as a whole. This is what makes cinema hot, relative to comics’ cool.

Since VHS and, more recently, DVD became available; audiences can not only immerse themselves in a movie’s narrative from their home, but splash around a little too. Mystique and Wolverine’s fight scene cannot be rewound and played in slow motion in the cinema, but it is possible to do this at home. This makes home viewing a cooler immersive experience than cinema. The X-Men also have video games of their own, allowing a fully interactive, involving and entirely participatory immersion in the world of superheroes. In Activision’s 2004 game ‘X-Men Legends’ players can choose a team of their favourite heroes and work together to stop the psychotic Magneto. The game has its own scripted narrative, but allows real involvement that not even comic books can offer. Comics may be cool, but video games are cooler.

So what have we learned about the medium of comic books? We know that the medium consists purely of the panels in which content is placed, and spacing between the panels, which connects adjacent panels over space and time. We know now that although comics’ content is warming up as technology and craft allows, comics’ defining component, the ‘gutter’, must always remain an icy-cool permafrost desert of bleakness: Comic books would not exist without it. We have also learned that, in the Gestalt sense, comics can be seen as a configuration of elements so unified as a whole that a comic cannot be perceived by the reader as merely the sum of its parts, and that it is our own cognitive programming that turns a series of images into an involving narrative. We have learned that McLuhan’s ‘Hot and Cool’ probe still applies to varying media, but lacks a content analysis that might render it more relevant, especially with the dawn of video games as a highly prevalent medium. Hopefully though, the main lesson is that the true art of comics is not what we see on the page, but what we see in our mind’s eye.

1624 words

Bibliography

CBC.CA. (1965). “Marshall Mcluhan: A Pop Philosopher.” Retrieved 16/03/05, from http://archives.cbc.ca/IDC-1-69-342-1818/life_society/mcluhan/clip4.

McCloud, S. (1993). Understanding Comics: The Invisible Art. New York, HarperCollins Publishers, Inc.

TheFreeDictionary.com. “Closure.” Retrieved 16/03/05, from http://www.thefreedictionary.com/closure.

Further Reading

Barker, M. (1989). Comics: Ideology, Power and the Critics. Manchester, Manchester University Press.

Carrier, D. (2000). The Aesthetics of Comics. Pennsylvania, The Pennsylvania State University Press.

Levinson, P. (1999). Digital McLuhan: A Guide to the Information Age. London, Routledge.

McCloud, S. (2000). Reinventing Comics. New York, HarperCollins Publishers, Inc.

McLuhan, M. and W. T. Gordon (2003). Understanding media: the extensions of man. Corte Madera, CA, Gingko Press.

Stearn, G. E., Ed. (1967). McLuhan: Hot & Cool. New York, The Dial Press, Inc.