B. Concepts: HCI as the object of semiotics

Peirce distinguished between the object of a conception and the conception of an object. This distinction should allow us to proceed from a perspective of explanation (for which semiotic terminology is often used) to one of synthesis: How do we make interactions possible? In particular, how do we design user interfaces or any other entity that support HCI?

1. Human-computer interaction IS semiotics applied!

This statement, repeated many times by several designers attending CHI 2000 is probably the best description of what we are after.

Semiotics is the underpinning of human-computer interaction, as it is of traditional human-human interaction. WHY? Simply put, signs pertaining to our senses (visual, aural, oral, tactile, and even olfactory) are used in HCI. And semiotics focuses on signs. Signs and sign processes are its object domain.

What makes things a bit more complicated in comparison to the most common social forms of interfacing through the intermediary of natural language is that user interface is part of the computer system. (Nadin, 1988)

Given the fact that the computer is a semiotic machine, if you work in HCI, you cannot NOT be a semiotician. (In my preliminary statement to the Workshop Semiotic Approaches to User Interface Design, I preferred to work on the thesis that You cannot NOT interact, which puts the subject in a broader context.) Furthermore:

If there is a science of interface (computing interface or any other kind), then this science is semiotics (Nadin, 1981)

2. Some definitions

2.1. Semiotics:

  1. . . . the knowledge of signs (C.S. Peirce)
  2. . . . general theory of representation. (Charles Morris)
  3. . . . theory and practice of mediations (M. Nadin, 1981, Zeichen und Wert)

2.2. Semiology is used concurrently with semiotics to designate the theory of language and its linguistic applications to different ensembles of meaning. It was developed by the Swiss linguist, Ferdinand de Saussure. (See also point 6).

  1. . . . a science that studies the life of signs alongside and within social life; it would form part of social psychology and consequently of psychology in general; . . . It will teach us what signs consist of, what laws regulates them. Since it does not yet exist, one cannot say what it will be (de Saussure)
  2. . . . general theory of signs in all their forms and manifestations among man and animals, normal and pathological, linguistic or non-linguistic, social or individual (Umberto Eco)

2.3. Here are some other terms and definitions that you should find helpful:

2.3.1. Sign:

  1. Something that stands to someone for something in some respect or capacity (C.S. Peirce, Collected Papers, 2.228) (pict. a)
  2. . . . anything of whatsoever mode of being which mediates between an object and an interpretant, since it is both determined by the object relatively to the interpretant and determines the interpretant in reference to the object, in such wise as to cause the interpretant to be determined by the object through the mediation of this sign (Peirce, MS 318)
  3. . . . the unity between a signifier (the actual sign embodied in some material form, such as words, shapes) and the signified (what the sign is supposed to mean) (F. de Saussure) (pict. b)
  4. Something that directs behavior with respect to something that is not at the moment a stimulus (Charles Morris)
  5. A mediating entity in intellectual and material praxis; a memory device (we don’t remember things, but their signs) (Nadin, 1983, Lecture)
pict. a
pict. b

2.3.2. Semiosis (pl. semioses):

Sign process, that is, the process through which the representations of an object and the associated representation are brought together. (Nadin, 1984)

2.3.4. Interpretant:

2.3.4.1. Actually, the potentially infinite process of interpretations of a sign.

Each time someone interprets a sign, that person becomes part of the sign and thus of the process of its interpretation.

2.3.4.2. Peirce defined the term with more technicality:

My immediate interpretant is applied in the fact that each Sign must have its peculiar interpretability before it gets any Interpreter. My Dynamical Interpretant is that which is experienced in each act of Interpretation and is different in each from that of any other; and the final Interpretant is the one Interpretive result to which every Interpreter is destined to come if the Sign is sufficiently considered. The Immediate Interpretant is an abstraction, consisting of Possibility. The Dynamical Interpretant is a single actual event, a Final Interpretant is that toward which the actual tends. (C.F. Peirce, Letters to Victoria, Lady Welby)

3. Interpretation

Interpretation is not the same as Interpretant. It is rather a particular instantiation (manifestation) of the interpretant process.
There is no right or wrong semiotic interpretation—regardless of which sign definition is adopted. The premise implicit in the sign definition will be reflected in the semiotic inference. As with any definition, once applied, it identifies elements that fit the definition!

4. Sign as a construct

Not unlike the number or entities of logic, the sign is a construct. Accordingly, in order to draw any coherent and meaningful semiotic conclusions, one has to adhere to a strict discipline of terminology and method. You cannot change the definition of an integer as you advance in mathematics. You cannot take one aspect of the sign and declare it to be the whole sign.

 

5. Ethics of terminology

If semiotics qualifies as the logic of vagueness (Peirce), the "mathematics of the humanities" (a definition used at CHI 2000), or as a meta-language, then it has to be practiced within an ethics of terminology. Due to the current disregard for intellectual integrity, all kinds of sign definitions —some rooted in language theory, others in logic, some synchronic, others diachronic—have become part of the semiotic scene. This is the post-Morris syndrome (as in Charles Morris, who produced a simplified image of semiotics in 1938), an intellectual disaster from which semiotics does not seem to recover. The consequences are obvious: the outcome of applied semiotics rarely justifies expectation.

6. Problems with semiology-based definitions

People working in semiology—based on the sign-definition given by de Saussure—hold that each sign system—visual, aural, oral, tactile, olfactory—and their combination (multimedia) are shaped by verbal, or natural, language. They also claim that the definition of a sign is based on a static model: the so-called synchronic (time is frozen) view. Since language changes very slowly, once can consider a relatively permanent structure and describe how the signifier (i.e., the word) signifies, that is, makes possible a relatively permanent interpretation. The relationships between signified and signifier are dyadic.

7. Peirce’s Semiotics

This school of semiotics is based on a dynamic triadic model. The triad that constitutes the sign includes the interpretant, which is the infinite process of interpretation through time and changing circumstances (as opposed to a specific or ideal interpreter). Hence, this semiotics introduces an opening in time. Implicit also is the context, i.e., the circumstances of each interpretation.

"Nothing is a sign unless interpreted as a sign."

This corollary to Peirce’s definition of a sign (cf. above) means that once we take it upon ourselves to look at the world from a semiotic perspective, we contribute to the RELATION between what is represented (object), what represents the object (representamen, pl. representamina), and interpretation as one instantiation in an infinite series of possible interpretations. Nature, art, social interactions, fashion, food, sex, machines, interaction with technology, and practically everything else can be interpreted as signs. But nothing is a sign, not even a street sign, unless a person takes it upon him or herself to interpret it as such.