Modes of Evaluation

Successful delivery of satisfactory customer experience has a critical dependency upon the degree to which the experience designed by the provider matches the experience received by the customer, and the overall satisfaction of the customer is generally an amalgam of the various touch-points he has with the brand in the course of his experience (shopping, purchasing, using, and disposal).

This seems straightforward enough until you consider the modes by which the customer assesses the experience he has received.  While there are at present many elements of customer experience that suffer from a lack of deliberate design, there are others that are deliberately designed, but the mode by which the designer expected the experience to be evaluated and the mode in which the customer actually did evaluate their experience are different.

The term “mode” is used because it is independent of the means, which would be the individual qualities by which the customer evaluates his experience.   The mode considers the method in which those qualities are assessed – so while it may be relatively simple to define the means by which satisfaction is achieved (meeting or exceeding expectations by comparison to expectations), the mode of evaluation can also be mismatched.

This is complicated by the fact that there is no single mode in which all customers evaluate experience – people have various preferences and they assess them in different ways at different times – but it is likely that achieving complete success depends on matching or at least accommodating the basic modalities of evaluation.    Said another way, I’m about to venture into a very abstract and conceptual mediation – and will likely leave the reader to his own devices in determining how this applies to the evaluation of various “real world” touch-points.

Binary

The binary mode of evaluation provides options by which an evaluator will consider the subject to fall into one or the other – the typical arrangement of black or white, A or B, and is or is-not, in which the evaluation chooses a single option and, in so doing, dismisses the other.   This mode can be seen in the traditional Greek philosophy or Socrates and Aristotle, and tends to predominate even to modern times – when someone insists that something is “either this or that” and must be evaluated to be one or the other.

The obvious flaw in the binary mode is the assumption that the two classifications defined are mutually exclusive and that no other classification exists.   To insist that something is black or white is to imply no shade of gray exists, which seems all the more foolish if you were to substitute any other two colors, such as stating everything must be evaluated as being blue or yellow.   What if it is a shade of green that is believed to be between blue and yellow?  Or what if it is red?   Such instances confound evaluation.

But more to the point, what occurs when a designer accepts a false dichotomy?   This will prevent him from accurately predicting the customer’s evaluation and, worse, being confident in pursuing options that seem sensible given his prediction, but are wholly inappropriate to the customer’s mode of evaluation.

Before leaving this, I should mention that the “binary” mode implies that there are just two options – and that there are also instances in which there are three or more to choose among.   This is similar to binary thinking in that the experience must be considered to be “in” or “out” of each option, and that being “in” one option means being “out” of all others.   Also, this leads to further confusion because there can be no fall back to “if it’s not one it’s the other” modes of conflict resolution.   But essentially, the problem of a multiple-choice mode is largely analogous to the binary mode.

Scalar

The scalar mode of evaluation provides an improvement over the binary mode, in that it provides a wide range of options between the two diametrically opposed extremes, and likely requires the evaluator to first consider whether the options he has selected are in fact both mutually exclusive and diametrically opposed in order to establish them as opposites on a scale.

However, scalar evaluation also carries with it the same fundamental flaws as binary evaluation: it provides two options that are presumed to be opposed to one another.   In the analogy to color distinction, a scalar representation enables the evaluation of a hue as blue, yellow, or somewhere in between.  While this accommodates shades of green (albeit in a clumsy and inaccurate way) it still prevents the proper evaluation of red.

It also presumes a conflict between the options provided: to suggest that something is more blue requires the evaluator to also assess it to be less yellow – when in reality it might have more of both qualities.   It might be necessary to depart from the analogy of color to better describe that: consider the emotions of happiness and sadness, and the middle state of bittersweetness that occurs when an event has elements of both happiness and sadness.   But being more happy doesn’t make one less sad as the two can be similar in their intensity – e.g., an event can be both very happy and very sad, and the poignancy of each does not diminish that of the other.

It is also little better for the designer in terms of accepting a false dichotomy, as it restricts the possibilities to the two options and a range in between.   And again, he becomes confident that no other quality could be considered and boldly sets sail in the wrong direction entirely, which will not be discovered until the mismatch between his assumption and the customer’s actual mode of evaluation results in a rejection of the schema.

Bidimensional

A dimensional mode of evaluation is the configuration of a matrix of two scalar evaluations, resulting in an evaluation along two different axes that determine a relative positioning of an event according to four qualities that are arranged as two diametrically opposed scales superimposed on one another.

For example, the blue-yellow scale is matrixed against a black-white scale to determine a shade of green that falls roughly into the quadrants of blue-black, blue-white, yellow-black, and yellow-white depending on its perception along each of the axes.

This is considered to be more informative than a scalar evaluation, but carries with it the same flaws as both the scalar and binary modes on which it is based.   But instead of making a single evaluation, the subject makes two separate evaluations to place it upon each of the two axes, which is confounding.   It is more likely that a general assessment is made first, as a single unit, which is then deconstructed to correspond to the axes of evaluation.

There can be some argument that such an evaluation gives the designer of an experience even greater latitude in his ability to make mistakes – to be correct about one axis but not the other, or even wrong about both, and to offer an experience that is horribly disjointed to the expectations and actual experience of the customer.  And when things go wrong, it becomes far more difficult to diagnose precisely which of the axial scales is to blame for the imprecision of his evaluation.

Multidimensional

The multidimensional mode of evaluation is the ultimate extension of binary, scalar, and bidimensional modes – but “ultimate” only in the sense that it is indefinite in its ability to accommodate a number of scalar evaluations from three to infinity.   It seems highly likely that, in the normal course of events, multidimensionality is likely most representative of the manner in which most phenomena are naturally evaluated, and any lesser form is an attempt to simplify.

This simplification comes at the cost of accuracy, but is likely necessary for the sole purpose of discussion.   That is to say that a person experiences a sensation such as he does, and it is only in the course of describing this sensation to another person (in words) that he begins to determine the individual qualities of that sensation and alight them along a series of scales in order to communicate that sensation to another person.   And most people do this very badly.

For example, consider the way in which a person might describe the color of a specific individual’s skin. It may seem to represent a point on the red-yellow continuum that is tinged with green (the blue-yellow continuum) and of a given level of darkness (the black-white continuum) – but ultimately words fail them and they cannot provide a sufficiently accurate description, even in a multidimensional evaluation, that would allow the other person to accurately understand.   It is the color it is, and words fail to describe it sufficiently.

In addition to the difficulties of accuracy and the manner in which multiple scales confound both the tasks of definition and diagnosis, the multidimensional mode of evaluation leads to a highly complex analysis that a designer would struggle to accommodate, even given a dynamic and flexible mechanism for customizing the experience to each customer.

This is likely where the role of a live service provider becomes critical: because the number, manner, and degree of scalar evaluations is for all intents infinite, it requires real-time adaptation to understand the various scales and degrees to which a customer might communicate their expectations of an experience – in a way that only a human mind is able to perceive and adapt as the idiosyncrasies of customer and product collide.

Conclusion

Given these four modes of evaluation have been presented in increasing order of complexity, there is likely the sense that after reviewing the most complex of them that predictive models of evaluation for experience design may be impossible to derive – so it’s necessary to back up a bit, because this was not at all the intent of this meditation.

Given the inability to articulate the qualities of experience, the customer likely avoids the complexities by considering only two or three of the criteria he applies in evaluation, and may consider them in a binary rather than scalar mode.   Again, it is specific to the individual, product, and buying situation but does not entirely avoid prediction.

However, it does suggest that there are areas that may be simplified, but other areas in which a great deal more deliberation and flexibility is required to arrive at an approximation of the customer’s mode of evaluation in a specific instance – as well as a willingness to reconsider the model when an experience design fails to a significant degree.

And in the end, the critical factor is identifying both the qualities and the mode of their evaluation – and not merely the first without the second – that becomes critical to success in customer experience design.