Screens

[box]This post originally appeared on the FTMF.info planning blog.[/box]

Within the pages of Watchmen, Adrian Veidt, the so-called “smartest man in the world”, esteemed business leader and founding member of the Crimebusters is shown at a wall of televisions, each tuned to a different channel. He uses this clatter of imagery, sound and motion to make sense of the current geopolitical and social climate and to act upon it:

Watchmen 10 - 08

Reads a bit like Social Media Monitoring, doesn’t it? But Adrian Veidt, AKA Ozymandias, was multi-screening before it was even a thing. Nowadays, we do it by default, up to 60% of the time, and in the age of 4.6 connected devices per household it just comes naturally.

Multi-screening can be simultaneous (same journey across devices, as in the above case), sequential (different journeys across devices simultaneously), or separate (different journeys across devices simultaneously) – but it’s an emergent behaviour that needs much further inquiry. There are few real thought leaders, except for SecondSync perhaps, or Microsoft, who so succincinctly define the terms I’ve used here.

One other thought leader is Kevin Kelly, co-founder of Wired, whose view is that as screens proliferate further into each aspect of our lives, their role becomes not just to display but also to help filter information – we’re literally ‘screening out’ the stuff we don’t want to see.

Watch his talk on ‘screening’ and five other ‘Verbs for the New Web’ below – it’s great:

And finally, screens can also be mirrors, lenses or even windows. Clever, aren’t they?!

Web Discoveries for May 27th

These are my del.icio.us links for May 27th

Introduction

Augmented Reality (AR) is a theme of computer research which deals with a combination of real world and computer generated data. AR is just one version of a Mixed Reality (MR) technology, where digital and real elements are mixed to create meaning. In essence AR is any live image that has an overlay of information that augments the meaning of these images.

Digital graphics are commonly put to work in the entertainment industry, and ‘mixing realities’ is a common motif for many of today’s media forms. There are varying degrees to which The Real and The Virtual can be combined. This is illustrated in my Mixed Reality Scale:

mixed-reality-scale
My Mixed Reality Scale, a simplified version of Milgram & Kishino’s (1994) Virtuality Continuum

This is a simplified version of Milgram and Kishino’s (1994) Virtuality Continuum; simplified, because their research is purely scientific, without an explicit interest in media theory or effects, therefore not wholly applicable to my analysis. At the far left of my Mixed Reality Scale lies The Real, or physical, every-day experiential reality. For the longest time we lived solely in this realm. Then, technological innovation gave rise to the cinema, and then television. These media are located one step removed from The Real, a step closer to The Virtual, and can be considered a window on another world. This world is visually similar to our own, a fact exploited by its author to narrate believable, somewhat immersive stories. If willing, the viewer is somewhat ‘removed’ from their grounding here in physical reality, allowing them to participate in the construction of a sculpted, yet static existence. The viewer can only observe this contained reality, and cannot interact with it, a function of the viewing apparatus.

Later advancements in screen media technologies allowed the superimposition of graphical information over moving images. These were the beginnings of AR, whereby most of what is seen is real with some digital elements supplementing the image. Indeed, this simple form of AR is still in wide use today, notably in cases where extra information is required to make sense of a subject. In the case of certain televised sports, for example, a clock and a scoreboard overlay a live football match, which provides additional information that is useful to the viewer. Television viewers are already accustomed to using information that is displayed in this way:

Simple Augmented Reality, televised football matches augment meaning with digital graphics
Simple Augmented Reality, televised football matches augment meaning with digital graphics

More recently, computing and graphical power gave designers the tools to build wholly virtual environments. The Virtual is a graphical representation of raw data, and the furthest removed from physical reality on my Mixed Reality Scale. Here lies the domain of Virtual Reality (VR), a technology that uses no real elements except for the user’s human senses. The user is submersed in a seemingly separate reality, where visual, acoustic and sometimes haptic feedback serve to transpose them into this artificial, yet highly immersive space. Notice the shift from viewer to user: this is a function of the interactivity offered by digital space. VR was the forerunner to current AR research, and remains an active realm of academic study.

Computer graphics also enhanced the possibilities offered by television and cinema, forging a new point on the Mixed Reality Scale. I refer to the Augmented Virtuality (AV) approach, which uses mainly digital graphics with some real elements superimposed. For example, a newsreader reporting from a virtual studio environment is one common application. I position AV one step closer towards The Virtual to reflect the ratio of real to virtual elements:

An Augmented Virtuality, the ITV newscasters sit at a real table in a virtual studio
An Augmented Virtuality, the ITV newscasters sit at a real table in a virtual studio

There is an expansive realm between AV and VR technologies, media which offer the user wholly virtual constructions that hold potential for immersion and interactivity. I refer to the media of video games and desktop computers. Here the user manipulates visually depicted information for a purpose. These media are diametrically opposed to their counterpart on my scale, the cinema and television, because they are windows this time into a virtual world, actively encouraging (rather than denying) user interactivity to perform their function. Though operating in virtuality, the user remains grounded in The Real due to apparatus constraints.

Now, further technological advancements allow the fusion of real and virtual elements in ways not previously possible. Having traversed our way from The Real to The Virtual, we have now begun to make our way back. We are making a return to Augmented Reality, taking with us the knowledge to manipulate wholly virtual 3D objects and the computing power to integrate digital information into live, real world imagery. AR is deservedly close to The Real on my scale, because it requires physicality to function. This exciting new medium has the potential to change the way we perceive our world, forging a closer integration between our two binary worlds. It is this potential as an exciting and entirely new medium that has driven me to carry out the following work.

To begin, I address the science behind AR and its current applications. Next, I exploit an industry connection to inform a discussion of AR’s development as an entertainment medium. Then, I construct a methodology for analysis from previous academic thought on emergent technologies, whilst addressing the problems of doing so. I use this methodology to locate AR in its wider technologic, academic, social and economic context. This discussion opens ground for a deeper analysis of AR’s potential socio-cultural impact, which makes use of theories of media and communication and spatial enquiry. I conclude with a final critique that holds implications for the further analysis of Mixed Reality technology.

Web Discoveries for December 4th

These are my del.icio.us links for December 4th

Gutter Talk

Gutter Talk: Hot & Cool in the World of Comic Books

The medium we call comics is based on a simple idea: the idea of placing one picture after another to show the passage of time. Legendary comic artist Will Eisner defines comics as ‘sequential art’. The idea that art can be ‘joined up’ to create a narrative can be seen in practice throughout the ages and is therefore not new. Examples from history include Egyptian hieroglyphics, European stained glass windows, Greek wall frescoes, Japanese scrolls and prehistoric wall paintings. In the year 1066 the Bayeux Tapestry, a 230 foot long piece of sequential art, was completed. Reading from left to right, it illustrates the events of the Norman Conquest of England unfolding in deliberate, chronological order before its viewers. Today the formula for making sequential art remains the same, though Scott McCloud’s definition is more relevant to the type of comics we have come to know. In his book ‘Understanding Comics’ McCloud states that comics are:

“Juxtaposed pictorial and other images in deliberate sequence, intended to convey information and/or to produce an aesthetic response in the viewer”

(McCloud, 1993, page 9)

This definition ignores single-panel comics such as Larson’s ‘The Far Side’, and other visual mediums like animation, but includes the use of the ‘speech bubble’ as a critical element in a comic’s narrative. McCloud’s definition lends itself to the most common forms of comics: newspaper strips, magazine-format comic books and graphic novels.

So how do modern comics work? Each panel in a comic is contained within a border. Each panel is a self-contained piece of art, and a fragment of a narrative. When read in a sequence, the gap between each panel is ‘filled in’ by the reader’s imagination. It is the reader that ‘animates’ these still pictures, not the medium itself. This happens because the mind is designed to close gaps in our awareness. Gestalt psychologists call this process ‘closure’. The real power of comics, then, lies not in the characters involved, nor what they are saying, or even the stories themselves, nor any of the medium’s potential content, but in how the reader interprets each panel as part of a sequence. As Marshall McLuhan said, “the medium is the message”.

One of the seminal media analyst’s best known concepts is that of a medium being either ‘hot’ or ‘cool’. In a broadcast on California’s CBC Television on June 22, 1965, McLuhan was asked to explain his concept:

“‘Cool’ is a slang term borrowed from the world of Jazz and popular music. The word ‘cool’ has a kind of mystical meaning that is not unlike the Hindu idea of detachment. The Hindu idea of detachment means complete involvement in an action and detachment in action. Whereas when people are merely involved in an action but not detached from the action, that’s ‘square’, or ‘hot’. Most people think of ‘cool’ as merely detached from action, but the word ‘cool’ as used in Jazz, and as I use it as sense for a medium, means a medium which uses all of you, but leaves you detached in the act of using you.”

(McLuhan, CBC.CA, 1965)

McLuhan exemplified hot media as: radio, print, photographs, movies and lectures; and cool media as: the telephone, speech, cartoons, TV and seminars. The comic book is another example of a cool medium, and to test McLuhan’s ‘Hot and Cool’ probe as a whole we will look at a comics series that typifies the medium; In the words of the great (but fictional) Professor Charles Xavier, “To me, my X-Men”.

For some, the X-Men comics are the epitomy of how all comics should be; fluid, exciting, intelligent, emotionally complex, relevant, and expressively drawn. Since their conception by Stan Lee in 1963, Marvel Comics X-Men have come very far indeed. Comparing the X-Men across the decades, panels of artwork today are richer with visual information. Comics in the early 1970s had a palette of just twelve colours. The reader had to use their mind’s eye to see the School for Gifted Youngsters that the series’ artists had in mind when drawing the series. Today, details such as old chalk-marks and lecture notes appear on the blackboard in Storm’s history classroom to give the reader a deeper sense that Mutant Academy is a real place. The characters’ facial expression are radically improved today, greater subtleties between Wolverine’s bad moods can now be conveyed, and fiery explosions are almost photo-realistic in terms of their adhering to physical law. There are now fewer gaps for the reader’s mind to fill in. Because of technological advancements and improvements in artistry, the X-Men and comic books as a whole have heated up. However, the images are still constrained within the borders of each frame. We must imagine what else is in the room when we see it from just one perspective, whereas in hotter media such as film the camera angles are generally wider, providing more information and therefore less detachment.

The panel makes up one essential part of the comic book’s construct. The second aspect is ‘The Gutter’ or the space between the panels. McCloud states that:

“Here in the limbo of the gutter, human imagination takes two separate images and transforms them into a single idea. Nothing is seen between the two panels, but experience tells you something must be there! Comic panels fracture both time and space, offering a jagged, staccato rhythm of unconnected moments. But closure allows us to connect these moments and mentally construct a continuous, unified reality. If visual iconography is the vocabulary of comics, closure is its grammar. And since our definition of comics hinges on the arrangement of elements then, in a very real sense, comics is closure!”

(McCloud, 1993, page 66-67)

Using the Gestalt definition of closure as “a principle of organization holding that there is an innate tendency to perceive incomplete objects as complete and to close or fill gaps” (TheFreeDictionary.com) and fusing it with McCloud’s concept of ‘the gutter’, in a McLuhanesque sense ‘the gutter’ becomes a very cool place indeed, completely devoid of content but for the reader’s own imaginings. There is actually an evolving art to reading or processing a comic as the guttering mechanisms become more and more sophisticated. In this way, comics could be said to be getting cooler. In some ways there’s a gutter happening whenever you turn the page too – a good example being that really ‘cool’ thing that happens when you turn the page in an action sequence (or somesuch) to a full or double page spread, sometimes with no words, and the reader really gets pulled in by the dramatic effect. Perhaps readers who are more brought up on squarer mediums, where they are more ‘spoon-fed’ with input have a harder time comprehending the narrative flow of certain comic books and so they are put off by the medium and never get to appreciate what it can offer. There are some types of comic book that require a pretty experienced reader, well versed in the ‘laws of the gutter’ and these are the coolest types of comics.

McLuhan’s thermometer is a tool for relativity, and works best when comparing multiple media. Luckily the X-Men do not appear solely in comics. They also appear in two animated television shows and have a highly successful movie franchise under their (spandex utility) belts. In a comic book, frames are viewed from up left, to bottom right. This requires active participation from the reader to help the narrative unfold. Panels can be ‘rewound’, ‘paused’ or skipped entirely. In cinema, frames on a film reel are shown to you in rapid succession and in the same place: the screen. All that the viewer has to do is sit back and let the medium wash over them. There is only a miniscule ‘gutter’ between the frames of a movie because the frame rate must be high enough to make still images on a movie reel appear to be moving fluidly. The closure between these frames is continuous, involuntary and imperceptible. The closure of frames in comics is reader dictated, involving, and necessary to the understanding of the book as a whole. This is what makes cinema hot, relative to comics’ cool.

Since VHS and, more recently, DVD became available; audiences can not only immerse themselves in a movie’s narrative from their home, but splash around a little too. Mystique and Wolverine’s fight scene cannot be rewound and played in slow motion in the cinema, but it is possible to do this at home. This makes home viewing a cooler immersive experience than cinema. The X-Men also have video games of their own, allowing a fully interactive, involving and entirely participatory immersion in the world of superheroes. In Activision’s 2004 game ‘X-Men Legends’ players can choose a team of their favourite heroes and work together to stop the psychotic Magneto. The game has its own scripted narrative, but allows real involvement that not even comic books can offer. Comics may be cool, but video games are cooler.

So what have we learned about the medium of comic books? We know that the medium consists purely of the panels in which content is placed, and spacing between the panels, which connects adjacent panels over space and time. We know now that although comics’ content is warming up as technology and craft allows, comics’ defining component, the ‘gutter’, must always remain an icy-cool permafrost desert of bleakness: Comic books would not exist without it. We have also learned that, in the Gestalt sense, comics can be seen as a configuration of elements so unified as a whole that a comic cannot be perceived by the reader as merely the sum of its parts, and that it is our own cognitive programming that turns a series of images into an involving narrative. We have learned that McLuhan’s ‘Hot and Cool’ probe still applies to varying media, but lacks a content analysis that might render it more relevant, especially with the dawn of video games as a highly prevalent medium. Hopefully though, the main lesson is that the true art of comics is not what we see on the page, but what we see in our mind’s eye.

1624 words

Bibliography

CBC.CA. (1965). “Marshall Mcluhan: A Pop Philosopher.” Retrieved 16/03/05, from http://archives.cbc.ca/IDC-1-69-342-1818/life_society/mcluhan/clip4.

McCloud, S. (1993). Understanding Comics: The Invisible Art. New York, HarperCollins Publishers, Inc.

TheFreeDictionary.com. “Closure.” Retrieved 16/03/05, from http://www.thefreedictionary.com/closure.

Further Reading

Barker, M. (1989). Comics: Ideology, Power and the Critics. Manchester, Manchester University Press.

Carrier, D. (2000). The Aesthetics of Comics. Pennsylvania, The Pennsylvania State University Press.

Levinson, P. (1999). Digital McLuhan: A Guide to the Information Age. London, Routledge.

McCloud, S. (2000). Reinventing Comics. New York, HarperCollins Publishers, Inc.

McLuhan, M. and W. T. Gordon (2003). Understanding media: the extensions of man. Corte Madera, CA, Gingko Press.

Stearn, G. E., Ed. (1967). McLuhan: Hot & Cool. New York, The Dial Press, Inc.