Creating Digital Value

I recently read an interesting book called Creating Customer Value on the Digital Frontier, which presents a model for considering ideas for creating products and services that deliver value by using digital technology to escape the boundaries of space, time, and matter. I realize how utterly hokey and overwrought that sounds, but bear with me ...

The author provides a model of eight “realms” of experience, that begin with the tangible real-world experience, consider how technology enables us to depart from real environments, physical objects, and linear time to overcome some of the limitations of real-world experience. It’s a different way of considering “waht can technology do for me” that is a bit more structured than the daydreaming approach to physical reality. A brief overview of this model follows:

1. Reality (actual time, space, and matter)

Reality is the real of physical spaces and objects and linear time, and is the space with which people are most familiar because it is where they live and have the majority of their experiences. Examples are numerous: go for a walk in the park, leaving your gadgets behind, and you are immersed in a real experience.

2. Mirrored Virtuality (virtual space, actual time and matter)

Mirrored virtuality represents actual time and matter in a virtual space. Consider the experience of watching a live sporting event on television: real people acting in real time, but the space in which you watch them (the television screen) is a virtual environment – you are not at the stadium watching the actual athletes, but see exactly what is happening through a device.

3. Augmented Reality (virtual matter, actual time and space)

Augmented reality layers virtual elements over a real environment in linear time. The purest example would be a cell phone application that enables you to view your actual environment, with an overlay of data: point it at a person, and see their name pop up beneath their face, with a link to view various details about them.

4. Warped Reality (virtual time, actual space and matter)

Warped reality depicts actual environments and things, but allows the user to play with the timeline. A good example of this might be online video, where you can play it in real time or slow motion, skip back and forward in the timeline, watch the sequence out of its normal order.

5. Augmented Virtuality (actual space, virtual time and matter)

Augmented virtuality depicts actual spaces, but virtual time and matter. Take the example of an application that demonstrates the construction of a building. The lot on which it will eventually stand is an actual space, but the depiction of the building that will someday be there is a virtual object that is depicted outside of linear time (in the future).

6. Physical Virtuality (actual matter, virtual time and space)

Physical virtuality depicts real objects in imaginary environments. An example might be an application used by a realtor to depict how your current furnishings would look in a home that has not yet been built.

7. Alternate Reality (actual time, virtual space and matter)

Alternate reality relies on a linear experience in a virtual environment filled with virtual things. Most video games offer a form of alternate reality, in which the only aspect that is locked to the real world is that your character progresses through a linear plot or progression.

8. Virtuality (virtual time, space, and matter)

Virtuality divorces itself from real life experience altogether: imaginary objects in an unreal setting that do not obey linear time. It’s difficult to describe a truly virtual experience – a depiction of a dream-like state in which objects and places are unreal and there is no logic to the sequence of events.

Nothing is Completely Real or Completely Virtual

My main problem with the examples, the book, and the theory in general is that it creates a false dichotomy between “real” and “virtual” time, space, and matter: neither of these extremes is ever fully realized.

What is depicted in the virtual extremes of time, space, and matter are often representations of reality: a completely imaginary “alien word” contains components taken from reality – the environment is a pastiche of elements from the real world: the rocks may be green, but the rocks are based on what we understand to be “a rock” based on our real experience.

What is experienced in the real world, restricted by time, space, and matter, is created by imagination. The experience of the user is all in the mind of the user: give any child a stick and a stack of cardboard boxes, and he becomes a knight in a castle fighting an imaginary dragon. This sense fades in adults, but is still present: who reads a book projects himself into an imaginary space and time.

***

In spite of the shortcomings and the blurred lines of theory, my sense is the book remains a good read, and provides some structure for brainstorming digital value: how could the experience of shopping for tires be improved if we removed the need to travel to a physical location (space), look at a limited selection of options in inventory (matter), and do so during store hours (time)?

If you consider escaping the constraints of the real world in one or two dimensions, a flood of ideas come to mind – more than if you simply asked the vague question “how could tire shopping be better?” So ultimately, it’s worth applying a theoretical model that may not be theoretically perfect.

Failing in Three Dimensions

Three-dimensional "virtual reality" experiences have often been suggested as the ultimate goal of the digital medium, but I've been dubious of their value. In particular, the notion of making the e-commerce experience similar to shopping in a physical store never has made much sense to me, and I figured the notion had long since been abandoned. Having seen a prototype of yet another attempt - and quite a good one from a technical standpoint (highly mimetic, seamless interaction, etc.) - I still predict that this approach will be a complete failure.

I mention graphic quality and load speed because those have been the straw-men of those who advocate 3D virtual worlds - they maintain that the concept is good, and the fact that people don't share their enthusiasm it is simply due to technical problems with the presentation: if they could make it less fake-looking and choppy, all the other complaints would go away.

Computer technology does a very poor job of simulating real life - or even providing ample evidence to overcome the willing suspension of disbelief for many users. It does this fairly badly, as it works with only two of our senses - however well it mimics the look and sound of a real environment, it offers nothing to smell and taste and touch. No matter how it good the audiovisual elements get (and they're remarkably good even today), it's still lacking. This is the chief reason that the Internet has not been a good venue for selling products that appeal to those senses. This was an obvious issue that was not overcome by the prototype, as it presented a very well-crafted audiovisual simulation of merchandise for which tactile sense is critical.

But even beyond that, the experience of shopping in a 3D environment is ill-suited even for products that sight alone, or even when just the product name without an image is sufficient. The reason for this is that one of the chief values of the Internet is overcoming the tedium of physical space, and virtual worlds bring that tedium right back to the fore.

An example might help: consider the experience of shopping in a bookstore. Even if you eliminate the need to travel to the store itself, the shopping experience involves navigating physical space: you have to walk through aisles and browse the shelves to find what you're looking for. If I want a cookbook, a reference book, and a novel, I have to go to three separate locations inside the store, walking through aisles of things I am not interested in buying, then visually scanning the shelves for the precise item I want - and picking up items one at a time to read the jacket copy and perhaps flip through some of the content. Im the real world, it's a tedious and time-consuming process.

Contrast that to shopping for a book online online: if I know the title or author, I can navigate to a specific item in seconds. If I have a topic in mind, I can run a search and retrieve a list of matching titles. Where I want to read a description or excerpts, it's a click away. Where I am not certain which of three similar books best matches my interests, I can open them in tabs or side-by-side windows for a detailed comparison. It's much faster and less tedious than real-world interaction.

In that sense, the 3D virtual store brings back the tedium of having to travel through a (simulated) physical environment. Even if the site is designed so that I could "warp" from one location to another, it's far more work than I am accustomed to doing. The "gee whiz" factor runs out very quickly, overshadowed by the tedious necessity that has been copied from reality, in a channel in which I have grown accustomed to doing things in a much less cumbersome manner.

This is why I still maintain that 3D virtual reality will continue to fail as a medium for merchandising, no matter how technology and design improve the graphic quality or reduce the load time - and even if they can bring back the senses that are currently muted.

What's more, this is a problem that I do not believe can be overcome. I'd love to be proven wrong by a clever designer who can create a virtual experience that eliminates the tedium ... but I don't think that will happen. While the advocates of 3D will focus on the technical issues, as they seem to have been doing all these years, they will likely continue to ignore that the task of interacting with a 3D world is a simulation of an inconvenient experience.

The Mimesis of Depth

I was reading a chapter in a collection of academic papers about the computer-generated 3D models, and my brain went on a tangent (as it's prone to do) about the value we seem to automatically attribute to the quality of mimesis.

Specific to 3D modeling, it's generally accepted that the computer-generated model of a 3D environment that is depicted on a "flat" monitor screen is a very poor imitation of a real-life environment., and the author's point was that much improvement (and special equipment) is needed before we can realistically depict an immersive environment to the user.

What struck me is the degree to which the user is able to accept a "fake" 3D environment, and I'd suggest it's not because we are applying the perceptual skills learned in the real world, but are transferring the cognitive interpretations learned from similar two-dimensional media: paintings, photographs, and television screens.

The artist learns tricks to "fool" the mind into accepting a flat representation as a three-dimensional one: an object that is smaller is further away, as is a one that lies further along diagonal lines that converges at a horizon. A closer object throws a more distinctive shadow, and an object further away isn't as clearly discernable. If an object overlaps another, it is nearer to us.

We learn these visual cues of "closeness" and "farness" specifically to the two-dimensional representation of depth - such that when we see a highly mimetic painting of an apple, we remark that we could almost reach into the painting and pick it up. I posit that the "almost" is because, in spite of all the visual trickery, there is still a quality that is missing that cues us that what we are viewing is, in fact, a clever forgery of depth rather than actual depth.

So, in the end, the question with which I'm grappling is whether, when we see a "3d" image on a flat screen, we are recognizing it as being three-dimensional from our experiences in the physical world, or because we recognize the conventions that are used to depict depth in a two-dimensional medium.

I tend to think that it's the latter, which may explain the reason why, in spite of all the advances in 3D presentation technology, we find ourselves unsatisfied with even the most detailed and attentive imitations. No matter the degree of effort the presenter undertakes to "fool" us, we recognize the tricks, and cannot accept the mimicry, though it might delight us, for a time, to engage in the willing suspension of disbelief.

An analogy comes to mind - the distortions to the sense of taste. In this culture, and at this time, it is probably reasonable to assume that more people have tasted artificially lime-flavored candy than have tasted actual limes. So a person who recognizes the lime flavor of candy is borrowing upon his sense memory of lime-flavored candy in the past, not from his memory of ever having tasted an actual lime. In fact, if you gave him a piece candy that tasted like a real lime, he might not be able to accurately identify the flavor. Such is the problem with mimesis in the virtual medium.

I generally accept, albeit with some degree of reservations, that a good approach to design in the online media is to mimic "real world" phenomena. The reservations I have is that the online medium has significantly different capabilities and qualities than the real-world medium.

An early example I recall was a video player that attempted to use an interface that was similar to a television remote control, to enable to user to "click" through video presentations in the same way that they surfed channels. This was well before a on-screen programming guides that allowed users to scroll through lists of programs by name, and even view descriptions before making a selection.

In that instance, mimesis has worked in reverse: the television medium, which was presumed to be more familiar to users, has actually adopted navigation controls from the online medium, rather than the other way around - for no other reason than it was easier to use, and just made sense.

It also occurs to me that I have never seen a cell phone interface with a rotary dial - which was at one time the most familiar paradigm for operating a telephone. And the reason is that a rotary dial, while necessary to a physical telephone until about twenty years ago, simply didn't make sense. Like the analogy to artificial "lime" flavoring, a person presented with a rotary dial would have no idea what it was, or how to use it.

And so after that long meander, I return to the notion of creating realistic 3D environments for the Web. Is it really more convenient? Does it really make sense? For the longest time, the answer seemed a self-evident "yes" ... but the more I think about it, the less plausible it seems.