Readings' list

Paul Dourish, Where the Action Is: The Foundations of Embodied Interaction (Bradford Books) , MIT press, 2001.

1-A History of Interaction

Dourish presents a history of interaction from the point of view of the different sets of human skills the systems are designed to exploit.

Electrical: at the dawn of computer era computers were analog, used in the lab. To set up a new computer scientists had to reconfigure the computer through the incorporation of new circuits. A little later it involved dealing with plugboards and patch cables.

Symbolic: use of standard component such as registers, accumulators and so on. Then passage to numerical language, assembly language. Symbolic interaction is much more natural and we are able to exploit a greater range of skills, visual, cognitive.

Textual: textual interaction occurred when the user began to use video terminal instead of punchcards or other media to store the programs. He looks at the actual form of interaction. A grammar for interaction is formed, with commands and parameters. Textual interaction draws on linguistic skills.

Graphical: it opens up a new dimension. It is characterized by its use of the space. The move to graphical interaction made possible the use of other skills such as: peripheral attention, pattern recognition and spatial reasoning, information density, visual metaphors.

New models of interaction

Tangible computing: 

-distributes computation across a variety of devices which are spread across the environment (printer, PDA) and communicating

-adds computational power to everyday object

-creates environments in which we interact through physical artifact. 

Social Computing:

-Incorporating social understanding into the design of the interaction itself.

-Anthropological and sociological approaches are used to uncovering mechanism through which people organize their activities.

-How interaction with computer can be enhanced by incorporating info about other people and their activities.

Embodied interaction:

Hypothesis 1: Social and tangible interaction are based on the same principles: they both exploit our familiarity with the everyday world, physical or social. The ways in which we experience the world are through directly interacting with it, and we act in the world by exploring the opportunities for action that it provides to us.

Hypothesis 2: embodiment is what they have in common. Embodiment denotes a form of participative status.
    Embodiment is important for computer systems designers because 
    a) Interaction is intimately connected with the setting in which it occurs.
    b) Attention to details of interaction, real details instead of abstract models, field studies. So concern in how interaction is manifest in the interface.
    c) Artifact embodied in the world can carry different meaning (medical card): it is important to consider how computation participates in the world it represent. Importance of the duality: representation, participation.

Hypothesis 3: embodiment is based on phenomenology.

Hypothesis 4: we can build on the phenomenological understanding to create a foundational approach to embodied interaction.

2- Getting in touch

The personal computer was established as the model for everyday computing. There are other paradigm, one of those is ubiquitous computing.

In Xerox PARC at the beginning of the 1990s, Mark Weiser proposed the "Ubiquitous computing" where lots of low- power mobile devices will transform the computational landscape: instead of taking work to the computer, why not bringing computation wherever it may be needed? His model was also one of invisible computing.

In Xerox he introduced the "computation by the inch, by the foot and by the yard." Each element was interconnected to the others, and each user possessed more of these elements.

In EuroParc: Digital Desk: combine physical and digital to use documents both in paper and digitally.

Virtual Reality (as opposed to augmented reality): immerses user in a computationally generated environment.

 Design trends: marble answering machine and Live Wire. These designs shows us that communication is important (as opposed to invisible computing) and that the technology IS already in the world.

Tangible bits: Hiroshi Ishii incorporates aspects of ubiquitous and design. Physical objects as interfaces to virtual world. While physical and digital are informationally equivalent, they are not interactionally.

Interacting with tangible computing: no single focus, sequential ordering is lost, but we use the physical properties of the interfaces to suggest its use.

3- Social Computing

Social computing refers to the application of sociological understanding to the design of interactive systems: sociological methods, reasoning for the design, development and evaluation of interactive systems.

Of the different aspects of sociology Dourish is only interested in those that are related to the organization of social conduct, oriented toward real activities rather then abstraction or models, and they adopt anthropological perspective on collecting, interpreting and using field materials.

Anthropology: emerged in the mid-nineteenth century: it studies the cultural web of signification that give social structure and interaction meaning. Anthropology developed the ethnographic methods.

Ethnography places an emphasis on the detailed understanding of culture, through intensive, long term involvement and "thick" description. It is often based on participant observation.

The school of Sociology of Chicago was very important for HCI because of some studies during which they explored particular modes of work (nurses, medical students etc.): they introduced a concern with the details of how works gets done and they used ethnographic methods to study working practices. 

Sociology in HCI
The effect of the introduction of sociological techniques, such as ethnography, helped to show how the use of computer systems involved more than simply the user and the computer, but also the context of the activity that the user was engaged in.

Work process: formalized and regularized procedures by which work is conducted.

Work practice: informal but nonetheless routine mechanism by which these processes are put into practice and managed in the face of everyday contingencies.

The duality of processes - practices is inevitable, and should not be eliminated. The way in which people deviated from formalized procedures tend to reflect a better or more fruitful adaptation of the process to the specific circumstances in which the activity is carried out. It is important to study practice to understand the dynamic in which practice mediate between processes and the circumstances in which they are enacted.

An Analytical perspective

A turning point for sociology in HCI was Lucy Suchman's "Plans and situated actions".

In this book she did a detailed critique of the then dominant paradigm for modeling human behavior in AI. Focusing on the problem of human-machine communication she also provided the basis for a reorientation of the role that sociology could play in the analysis and development of interactive systems.

Suchman's prime critique is the notion of "plan" used in AI. For Suchman our actions are organized in response to the features of the setting in which they arise; actions are situated.

The planning model was at the base of the design of interactive systems: Suchman provided detailed analysis of the interaction problems arising from the mismatch between  the stale model that a system may have and the immediate and fluid circumstances in which the user find himself.

She brought in focus ethnomethodology.

Ethnometodology
Ethnometodology originated in the work of Garfinkel in the 1960s.
The main sociological problem of sociology at that time was the "problem of social order": how stable and orderly social facts can arise out of independent action of individuals. The theoretical model that dominated for decades in American Theory was the presence of social facts or realities and the role of sociology was to investigate them. To study the behavior to discover the rules underlining it. For Garfinkel the social facts were not a "facts" to be assumed as principle, but a phenomenon to study: the objective of sociology was to understand how social reality was achieved, how people made it work. He called this study of the common sense methods by which people manage and organize their behavior "Ethnomethodology."

Ethnography: methods to collect and arrange material.

Ethnomethodology: theory to inform the analysis of the data.

Technomethodology:
Dourish and Graham Button coined term "Technomethodology" (1998): deeper relationship between system design and ethnometodology. 

The relationship should draw from ethnomethodological insight about organization of action as being a moment to moment improvisational response to practical problems, and it should relate these understanding to the basic, fundamental principles around which software systems are developed (abstraction, function, substitution, identity, and representation.)

One relationship among the two is the relationship between "accountability" in ethnomethodology and "abstraction" in system design. In the difference between them lies some problems for interactions.

Accountability: Accountability means observable and reportable. Accountability is evident to members in a context. Accountability arises as a feature of social action, action and accountability cannot be separated. Members are those who share a common sense of understanding.

Abstraction: Software systems are built from abstraction. Abstraction hides information, hides how actions are organized.

Accountability and Interface Abstraction: How to introduce accountability in an interface? In order to be accountable it should show what is happening during an action or transformation. Maybe reflection? Reflection deals with two domains: the domain in which the application is dealing and the domain of the program itself. Can reflection be used to show accountability?

Space, Place and Locales

Another example where sociology is used in designing interactive system is in understanding collaborative setting in which social action takes place. This is a different aspect from technomethodology.

For example difference between "space" and "place". 
Space: concerned with physical proprieties.
Place: way that social understanding convey an appropriate behavioral framing for an environment.

It is important for design to look at "place" as opposed to space because: 1) turns attention to activities and not the structure of the space. 2) Place reflects the emergence of practice.3) Place is relative to a community of practice.

In some sociological work the idea of "local framework" is used in a similar way to "place".

 

4-Being in the world

"Embodiment: embodied phenomena are those that by their very nature occur in real time and real space."

Using this definition Dourish wants to include phenomena that occur in real world and not in a virtual world or simulation or metaphors of real world such as the desktop metaphor.

Phenomenology: originated in the last part of the nineteenth century.

Edmund Husserl (1859-1938)

Founder of the phenomenology tradition.

His primary criticism of his time science was that it had distanced science and mathematics from everyday world. It was too abstract. He envisioned a science that was firmly grounded on the phenomena of experience. He saw his science as an extension to Descartes' intent.

Phenomenology aims to uncover the relationship between the object of consciousness and our mental experiences of those objects. (The idea we have of an object, such as a rabbit, and our experience when we are observing one.)

Phenomenology rejected abstract and formalized reasoning, looking instead at the everyday experience. Many people later turned away from his work but it had considerable influence in turning attention to experience as a phenomenon to be studied in its own right.

Martin Heidegger (1889-1976)

Husserl's student: mental life and everyday experience are fundamentally interwined. He departed from Husserl's focus on cognitive, mental phenomena separated from the physical phenomena.
Husserl's duality came from Descartes: the world of reality and the world of mental experience.

For Heiddegger we need to be in order to think: the nature of being, how we exist in the world, shapes the way we understand the world. Instead of "How can we know about the world?" he asks "How does the world reveals itself to us through our encounters with it?" 

We encounter the world as a place in which we act.

The basis for Heidegger's phenomenology is that we interpret the meaning through the way we encounter the world. One way we encounter the world is to be able to use what we find in order to accomplish our goals.

The tools or instruments in the world disappear from our cognition when we use them to accomplish a goal: the goal is our focus and the tool a means, a ready to hand. A ready to hand is not grasped theoretically. The world occurs as an unconscious but accessible background to our activity.

Present-at-hand. When I am meanful about the object, it is present at hand (for example when there is a problem.)
Ready-at-hand.

So for Heidegger the central role is our engaged participation in the world.

Alfred Schultz (1899-1959)

Schultz extended phenomenology beyond the individual to encompass the social world. He concentrated on the problem of intersubjectivity: "Given that our experiences of the world are our own, how can we achieve, between different individual, a common experience of the world, and a shared framework for meaning?"

Shultz argue that the actions of others seem to us to be the actions of reasonable social actors because we assume them, in the first instance, to be so. So the intersubjectivity is the outcome of this assumption.

For Shultz social actors solve sociological problems as practical, mundane problems solved routinely in the course of their day to day routine.

Maurice Merleau-Ponty

Phenomenologist philosopher. He tried to reconcile Husserl's Philosophy with Heiddeger's by proposing the body as a third party between perception and acting in the world. Perception of an external reality comes about through and in relation to a sense of the body. 

 

Apart from phenomenological tradition, other approaches have dealt with cognition and action. (abstract vs body).

Gibson (visual perception, ecology of psychology, affordance.)

And others ...

Wittgenstein (1889-1951)

The Austrian language philosopher gains a position in Dourish's tale of embodiment tradition for his work on language.

After 1929 Wittgenstein reoriented his view of language from a logical construction of facts and truth statements to a set of loosely connected "language games", socially shared linguistic practices "consisting of language and the actions into which it is woven."

"The meaning of a word is its use in the language." Language is a form of action.

"To imagine a language it is to imagine a form of life."

 

So Dourish's new definition of embodiment is:

"Embodiment is the property of our engagement with the world that allows us to make it meaningful."
"Embodied interaction is the creation, manipulation, and sharing of meaning through engaged interaction with artifacts.
"

 

????

There is something not clear in this chapter. Dourish defines Embodiment as a phenomena. It means something that occurs in the real world. 

The phenomenological tradition deals with the knowledge of the world: how do humans know the world? They try to understand the mechanism by which humans perceive things, by which we enter in contact with an outside reality.

It looks to me like embodiment and phenomenology are at two different levels. Dourish embodiment has more to do with "we human are used to the world so we know how to deal with phenomena in the world;" while phenomenologists are dealing with epistemological issues, with issues to a deeper level.

Suchman's situated action is still close to the level of discourse of phenomenology: there she talks about human action, human planning. It is still at a fundamental level of human behavior.

Also technomethodology seems a bit skew: while ethnomethodology is a kind of study, or a method to study, technomethodology is a "relationship". It is misleading.

I think that the chapter four is the weakest of the book, because the phenomenological tradition isn't really a tradition for embodiment like Dourish claims. There is too much tangle between different levels, philosophical and sociological levels relative to perception and intention, to action-planning on one side and with Dourish's action - interaction in the world at a more higher level on the other side.

5-Foundation

Goals of this chapter:
1-explore in more detail the notion of embodiment and embodied interaction.
2-relate the notion of embodiment to interaction design.

Three aspects of Meaning

Ontology
In computer system design ontology usually means the internal representation structure of the system. It also used to refer to the elements of a user's conceptual model. Designer needs to design a system that will fit "the user's ontology."

Since a real ontology should form with time and practice, ontology is subject to continual revision, rather than being statically embedded in some artifact. A designer may reflect a particular ontology, but it cannot provide an ontology to the user.

The designer and users do not necessary share the same experiences and ontology, so to include deeply in a system an ontology may cause interaction problems. It can results in mismatch between the assumption encoded in the system and the expectation of the users.

Intersubjectivity
Intersubjectivity is about how the meaning can be shared.

There are two ways in which the meaning can be shared.
First: communication between the designer and the users. the system is the medium. Aspect of the designer model should be made available to users.
Second: communication between users through the system. By this is meant the way that people develop and communicate shared ways of using software systems and ways of doing their work with software systems. Systems comes to be appropriated.

Intentionality
Intentionality refers to the relationships between two entities, for example a real object and its representation inside a computer program. Computation is about representation, so it is an intentional phenomena.
If computation is intentional, so also interaction with those computational elements also carries with it intentional connotations. A key element of interaction with computation is how we act through it to achieve an effect in the world (click a button and receive a product by mail.)

 

Coupling

Coupling is how an intentional reference is made effective. In the physical world an action can have a remote effect, through a chain of coupling, from one thing to another. The idea of coupling is an intentional one too: the actions are directed outwardly through a chain of associations.

When we operate on a computer we deal with software abstractions: buttons, applications, cursor etc. We direct our attention back and for to different levels of abstraction (that are implemented in the system). While interacting with a computer, at any given moment my action may be directed toward one of any number of elements in the world of computer. I act upon some; I act through others. So my action is directed toward and assemble (coupling) of objects at different level of abstraction that together accomplish my action.
Coupling addresses how we assemble a set of abstract computational representations into a tool, and then act through that tool to achieve some end results.

There is a difference between the model in which the software is structured and the model for user activities. User activities are not structured after layered models.

Coupling allows us to select from a variety of entities offered the ones that are relevant to our activity and put them together in order to affect action.

Coupling and Metaphor

Metaphor, such as buttons or desktop, extend the intentional range of systems by providing new entities for us to be directed toward. Metaphors suggest actions or simplify an action.

 

Embodiment and Technological Practices

The ideas behind embodied interaction can be used in two ways:

1-as the basis for an approach to design
2-as a way of uncovering issues in the design and use of existing technologies.

6-Moving towards design

A common framework

The broader idea of embodied interaction points out that action and meaning arise in specific setting - physical, social, organizational, cultural and so forth.

Design principles

Computation is a medium

Computation rather than computers are a medium. Meaning is transmitted not only through the a system but also through practices. The meaning is not simply what the system conveys, but how it fits into a wider pattern of practice. 

In order to extract the meaning concepts of visibility, awareness, and feedback are important. Awareness of other persons activities but also making the system actions visible, and also the user actions on the system visible.

Meaning arise on multiple levels

Objects carries meaning on multiple levels: as elements on their own, as signifiers of social meaning, as elements in systems of practice, and so on.

Systems or artifacts supporting embodied interaction need to be designed with an orientation toward the multiplicity of meanings that may be conveyed through them.

Meanings can be conveided along different dimensions:
-iconic/symbolic (arrow, map, numbers)
-entities to which they refers: objects or actions (events, operations, behavior.)

What embodied interaction adds to existing representation practice is the understanding that representations are themselves artifacts.

Designers need to understand how the different levels of representation will be manipulated.

Users, not designers, create and communicate meaning
Users, not designers, manage coupling

How technology will feature as an aspect of working practice cannot be predetermined by the designer, but instead will emerge from the specific, situated activity in which the technology is incorporated.

The process of users adapting and incorporating systems is not widely studied. (but note Orlinkowski and Adaptive Structural theory.)

Embodied technology are used to create meaning, managed at multiple levels through selective coupling; because they can only have meaning through the way in which users incorporate them into working practice, then
1- the  manipulation of meaning and
2- coupling
is done by the users.

In the traditional approach the designer manage the interaction between user and artifact through control of the design parameters. Instead of designing ways for the artifact to be used, the designers need to focus on ways for the user to understand the tool and understand how to apply it to each situation.

The designer should provide the user with the following resources:

1-Ability to operate on entities at different levels.
2-Visibility: in CSCW: it means mainly awareness. Visibility of action: what other colleagues are doing. Or, related to a common artifact, visibility of its effects: my actions, or other people's actions, modify the artifact and all can see the transformations taking place.

Awareness allow people to coordinate their own actions, instead of them being dictated by the system. They can adapt their work to the immediate needs of the moment. The need to manage their own actions arises, in turn, from the fact that those actions take on meaning for the users around the work being done.

Embodied technologies participate in the world they represent

Embodied interaction rejects the separation between representation and object: both are entities that participate in a single coextensive reality. This extend not only to physical realities but to also to actions: embodied action participate in the world they represent.

In the example of the medical record, the words on the card "represent" information, but the object itself, the card, carried information. So the card represented information not only about the patient treatment, but also " about itself" and about the activities surrounding it.

The medical record is an example of a physical artifact, but there are also non physical entities: for example the conceptual organization of files in an organization, the categorization of structures of files not only represented the company work, but it also participate in it.

So representations work on multiple levels, and so interactive systems need to allow people to operate on them at multiple levels.

Embodied interaction turns action into meaning

Meaning does not reside in the system itself, , but in the ways in which it it used.

So there are design opportunities surrounding the ways in which technology might adapt to different patterns of activity, and those adaptation might be, themselves, ways in which a community of practice might establish and convey meaning.

6-Conclusion and Directions

How does embodied interaction see common themes in personal computers?

Information Appliances and Convergence

Information Appliances: specialized appliances containing computation.

Convergence: different media comes together, and internet will provide a common media framework.

The two concepts diverge but they may be brought together by embodied interaction: information appliances and convergence both establish boundaries between information streams or activities and tasks. But both concepts have to leave the users create boundaries, not the designers.

Invisible user interfaces

The movement towards ubiquitous computing spawned several ideas, among them the idea of invisible interfaces: an interface should disappear, get out of the way between the user and the task. But invisibility creates problems:
- In design there is aesthetic, holistic and expressive concerns. Some design attempt to make the interaction experience engaging. And invisibility is against such concerns.
- Embodiment is not possible if the objects disappears. Coupling at different levels is not possible.

Social and Technical

Embodied interaction gives a new way to bring together the design and the social science: the focus is on the concepts of practice and appropriation. Appropriation is a practice as well as being about the emergence and evolution of working practices, and their relation to the settings - technical, organizational, physical, etc. - in which they emerge. It relies on a variety of technological opportunities and design features. Looking at how appropriation happens is deeply related to the principle around which systems are developed, rather than specific system requirements.

Physical and Symbolic

Tangible computing deals with physical representation. But physical (tangible) artifact has a symbolic meaning too. When I act on them I manipulated also a symbolic entity, other ways tangible computing will not make sense. Since embodied interaction unifies action and meaning, it creates the bases for unification of the action on the physical levels and their meaning on the symbolical level.

This looks too stretched to me!

Radically embodied Cognition

Other approaches are for a more radical stance towards the concepts of non representation. Dourish is for non representational towards interaction and cognition. Others, such as Agre, Chalmers, Smith, Stein, Clark are for a "thesis of Radically Embodied Cognition": "Embodied cognition is best studied by means of nonrepresentational and non computational ideas and explanatory schemes involving e.g. the tools of dynamical system theory."

Comments

He talks about representation and objects represented: example, the medical card, the words on the card "represent" information, but the object itself, the card, represent information. Ok, that is meant to show a point, but somehow is up to us to decide what something represent: we can always say that the medical cards has several representations, beside the written test.

Embodied interaction is the way humans use to understand the world. This happens with every artifact, physical, abstract or a program. It doesn't necessary has to be physical. Embodied interaction is just the way we deal with the external world. If we deal with an artifact that has a bad design, we still interact with it. We don't get out of it what we could with an artifact that is better designed, but still we adopt the same way to deal with it than with any other object. 
Suchman study revealed the characteristics of a user interacting with an artifact imbued with a help in the form of an expert system. The study focalized on the dialog between human and the machine, considering the fact that the resources available to the user nad to the machine were limited in comparison to human-to-human communication.
Are there similar studies for interactions with an artifact that include computation?
Are there preliminaries studies of a person interacting with a non computation tool? About the methods people uses to understand how the tool works? Or how to use it? So it will be possible to compare it with the interaction with a "computational tools", even if in this case the effect will be the opposite of Suchman. In the study of Suchman the expert system was a simplification of a conversation with a human, in the case of a computational tool, we have interaction with a more complex tool.

In the principles for design, a very important tenet is that the user creates meanings, not the designer. This means that the system must be designed in such a way that it is open to be adapted and transformed  to the user practices. But still the artifact has a purpose. It is not a general tool. So ...

 

Disclaimer: I wrote these summaries to help me remember the content and the main ideas of the paper. Since I am interested in certain aspects, I may leave out others.

If you have any comment, contact me at