Readings' list

John M. Carroll, HCI Models, Theories, and Frameworks: Toward a Multidisciplinary Science (Interactive Technologies), Morgan Kaufmann, 2003

Table of Contents

1. Introduction: Toward a Multidisciplinary Science of Human-Computer Interaction

by John M. Carroll, Virginia Tech

1.1 The Golden Age
1.2 Let 100 Flowers Bloom
1.3 Scientific Fragmentation
1.4 Teaching and Learning References

2. Design as Applied Perception

by Colin Ware, University of New Hampshire

2.1 Motivation
2.2 Scientific Foundation
2.2.1 Stage 1: Features in Early Vision
2.2.2 Stage 2: Pattern Perception
2.2.3 Stage 3: Objects
2.2.4 Claims and Limitations
2.3 Case Study
2.4 Current Status of Theoretical Approach
2.4.1 Application References

3. Motor Behavior Models for Human-Computer Interaction

by I. Scott MacKenzie, York University, Toronto, Canada

3.1 Motivation
3.2 Overview: Models and Modeling
3.2.1 Predictive Models
3.2.2 Descriptive Models
3.3 Scientific Foundations and Model Descriptions
3.3.1 Fitts's Law
3.3.2 Guird's Model of Bimanual Skill
3.4 Case Studies
3.4.1 Case Study #1: Fitts's Law Prediction of Text-Entry Rates on Mobile Phones
3.4.2 Case Study #2: Bimanual Control and Desktop Computer Affordances
3.5 Current Status and Further Reading

4. Information Processing and Skilled Behavior

by Bonnie E. John, Carnegie Mellon University

4.1 Motivation for Using the Human Information Processing Theory in Human-Computer Interaction
4.2 Overview of GOMS
4.3 Scientific Foundations Underlying GOMS
4.3.1 Conceptual Frameworks
4.3.2 Computational Cognitive Architectures
4.3.3 Task-Analysis Techniques
4.4 Detailed Description
4.4.1 KLM
4.4.2 CMN-GOMS
4.4.3 CPM-GOMS
4.5 Case Study: Project Ernestine
4.5.1 Details of Project Ernestine's CPM-GOMS Modeling Effort
4.6 Current Status
4.6.1 GOMS in Particular
4.6.2 Human Information Processing in General
4.7 Further Reading
4.7.1 Seminal Text in Human Information
4.7.2 Human Information Processing in HCI
4.7.3 Human Information Processing Embodied in Computational Cognitive Architectures
4.7.4 ACT-R
4.7.5 EPIC

5. Notational Systems—The Cognitive Dimensions of Notations Framework

by Alan Blackwell and Thomas Green, Cambridge University, Cambridge, England

5.1 Motivation
5.1.1 Example
5.2 Overview
5.3 Scientific Foundations
5.4 Detailed Description
5.4.1 Activities
5.4.2 The Components of Notational Systems
5.4.3 Notational Dimensions
5.4.4 Profiles
5.4.5 Trade-Offs
5.4.6 Use by an Analyst
5.4.7 A Questionnaire Approach
5.4.8 Cognitive Dimensions of Interactive Devices
5.5 Case Study: Evaluating a Visual-Programming Language
5.5.1 Illustrating the Notation
5.5.2 Conclusions
5.6 Current Status
5.6.1 Dissemination
5.6.2 Clarification and Formalization
5.6.3 Coverage
5.6.4 Analysis Tools
5.6.5 Beyond CDs: Misfit Analysis
5.7 Further Reading

6. Users' Mental Models: The Very Ideas

by Stephen J. Payne, Cardiff University, Wales

6.1 Motivation
6.2 Scientific Foundations
6.2.1 Idea 1. Mental Content vs. Cognitive Architecture: Mental Models as Theories
6.2.2 Idea 2. Models vs. Methods: Mental Models as Problem Spaces
6.2.3 Idea 3. Models vs. Descriptions: Mental Models as Homomorphisms
6.2.4 Idea 4. Models of Representations: Mental Models Can Be Derived from Language, Perception, or Imagination
6.3 Detailed Description
6.3.1 Idea 1. Mental Representations of Representational Artifacts
6.3.2 Idea 2. Mental Models as Computationally Equivalent to External
6.4 Case Study
6.4.1 A Yoked State Spaces Analysis of Calendar Design
6.4.2 Experiments on Internalization of Device Instructions
6.5 Further Reading (ed—please confirm—this isn't in my notes) References

7. Exploring and Finding Information

by Peter Pirolli, Palo Alto Research Center

7.1 Introduction
7.2 Motivation: Man the Informavore
7.2.1 Emergence of the Global Information Ecology
7.3 Scientific Foundations
7.3.1 Influence of Evolutionary Theory: Adaptationist Approaches
7.3.2 Information-Foraging Theory
7.3.3 Optimal-Foraging Theory
7.4 Detailed Description: Scatter/Gather
7.4.1 Simulating Users
7.4.2 Information Scent
7.4.3 Information-Foraging Evaluations
7.4.4 Simulating Users and Evaluating Alternative Scatter/Gather Diagrams
7.5 Case Study: The World Wide Web
7.5.1 Information Scent as a Major Determinant of Web User Behavior
7.5.2 Simulated Users and Usability Evaluation
7.6 Current Status


8. Distributed Cognition

by Mark Perry, Brunel University, London, England

8.1 Motivation
8.1.1 Designing Collaborative Technologies
8.1.2 Distributed Cognition in Context
8.2 Overview
8.3 Scientific Foundations
8.3.1 External Support for Thought and Systems Perspectives in Cognition
8.4 Detailed Description
8.4.1 Computation and Cognition
8.4.2 The Social Organization of Group Problem Solving
8.4.3 Communication and Coordination of Distributed Knowledge
8.4.4 "Doing" DCog
8.5 Case Study: Engineering Design and Construction
8.5.1 Organizational Coordination and Control in Representation Transformation
8.5.2 Representational Transformations in Information Processing
8.5.3 Coordination of Representational Transformations
8.5.4 Summary
8.6 Current Status
Author Notes
Further Reading


9. Cognitive Work Analysis
by Penelope M. Sanderson, University of Queensland, Australia

9.1 Motivation
9.1.1 Connection of CWA with Other Areas
9.1.2 Designing for Unanticipated Events in First-of-a-Kind Systems
9.2 Overview of CWA
9.3 Scientific Foundations
9.3.1 A Systems Perspective
9.3.2 An Ecological Orientation
9.3.3 The Role of Cognition
9.3.4 Summary
9.4 Detailed Description
9.4.1 Overviews of CWA
9.4.2 Description of CWA Classes of Constraint
9.4.3 CWA and the System Life Cycle
9.5 Case Studies
9.5.1 Display Design
9.5.2 Systems Engineering and Human-System Integration
9.6 Current Status
9.7 Further Reading


10. Common Ground in Electronically Mediated Communication: Clark's Theory of Language Use
by Andrew Monk, University of York, England

10.1 Motivation
10.1.1 Production Plus Comprehension Multiplied by Communication
10.1. 2 Language Use as a Collaborative Activity
10.2 Overview
10.3 Scientific Foundations
10.4 Detailed Description
10.4.1 Fundamentals
10.4.2 Grounding, Levels, Layers, and Tracks
10.5 Case Studies—Applying the Theory to the Design of Technology for Communication
10.5.1 The Costs of Grounding (Clark & Brennan)
10.5.2 Why Cognoter Did Not Work (Tatar, Foster, & Bobrow)
10.5.3 Predicting the Peripherality of Peripheral Participants (Watts & Monk)
10.6 Current Status
10.7 Further Reading

11. Activity Theory

by Olav W. Bertelsen and Susanne Bødker, University of Aarhus, Denmark

12. Applying Social Psychological Theory to the Problems of Group Work

by Robert E. Kraut, Carnegie Mellon University

12.1 Motivation
12.2 An Overview of CSCW Research
12.3 Scientific Foundations
12.3.1 Input-Process-Output Models of Group Functioning
12.3.2 Process Losses
12.3.3 Social Loafing
12.4 Detailed Description—Explaining Productivity Loss in Brainstorming Teams
12.4.1 Application to System Design
12.5 Case Study: Applying Social-Psychological Theory to the Problem of Undercontribution to Online Groups
12.5.1 Social Loafing and Online Groups
12.6 Current Status

13. Studies of Work in Human-Computer Interaction

by Graham Button, Xerox Research Centre Europe, Grenoble, France

13.1 Motivation
13.2 Overview: A Paradigmatic
13.3. Scientific Foundations
13.3.1 Ethnography
13.3.2 Situated Action
13.3.3 Ethnomethodology
13.4 Detailed Description
13.4.1 Critique
13.4.2 Evaluation
13.4.3 Requirements
13.4.4 Foundational Reconceptualizations
13.5 Case Study
13.6 Current Status
13.7 Further Reading


14. Upside-Down Vs and Algorithms—Computational Formalisms and Theory

by Alan Dix, Lancaster University, England

14.1 Motivation
14.1.1 What Is Formal?
14.1.2 The Myth of Informality
14.1.3 Chapter Overview
14.2 First Steps
14.2.1 Two Examples
14.2.2 Lessons
14.3 Scientific Foundations
14.3.1 A Brief History of Formalism
14.3.2 The Limits of Knowledge
14.3.3 The Theory of Computing
14.3.4 Complexity
14.3.5 Good Enough
14.3.6 Agents and Interaction
14.3.7 Notations and Specifications
14.3.8 Kinds of Notation
14.4 Detailed Description
14.4.1 Two Plus Two—Using Simple Calculation
14.4.2 Detailed Specification
14.4.3 Modeling for Generic Issues
14.4.4 Computer-Supported Cooperative Work and Groupware
14.4.5 Time and Continuous Interaction
14.4.6 Paradigms and Inspiration
14.4.7 Socio-Organizational Church-Turing Hypothesis
14.5 Case Study—Dialogue Specification for Transaction Processing
14.5.1 Background—Transaction Processing
14.5.2 The Problem . . .
14.5.3 All About State
14.5.4 The Solution
14.5.5 Why It Worked . . .
14.6 Current Status
14.6.1 Retrospective—Formal Methods in Computing
14.6.2 Retrospective—Formal Methods in HCI
14.6.3 Prospective
14.7 Further Reading


15. Design Rationale as Theory

by John M. Carroll and Mary Beth Rosson, Virginia Polytechnic Institute



Ch. 11.     Activity Theory

by Olav W. Bertelsen and Susanne Bødker, University of Aarhus, Denmark

The rise of personal computer challenged mainframes systems for automation of existing work routine. Furth more it brought forth a need to focus on how to work on materials and objects through the computer.
In search for theoretical and methodical perspectives it seemed promising to turn towards the young HCI research tradition. But HCI was already facing problems: lack of consideration for other aspects of human behavior, for interaction with other people, for culture. Cognitive science-based theories lacked means to address several issues that came out of the empirical projects.

Because of the shortcomings it was necessary to move outside cognitive science based HCI. Activity theory in HCI took its inspiration from psychology: German work psychology and Scandinavian critical psychology.

Development of activity-theoretical angle: primarily Bødker and Kuutti, with inspiration from Scandinavian activity-theory groups from Scandinavian psychology.  Bannon and Grudin made significant contributions and introduced it to HCI. Kaptelinin connected it to the early Russian development.

Activity theoretical HCI has come to focus on:

It offers conceptual tools rather than a collection of tools and techniques.

Activity theory understands human conduct as anchored in collective, shared practice. Learning is not a matter of how the individual adapts to the artifacts; it is also a matter of how the collective practice develops. To design an artifacts means not only to design the "things" or device, which can be used by human beings as artifacts in a specific kind of activity. As the use of artifact is part of social activity, we design new conditions for collective activity, such as new division of labor and new ways of coordination, control, and communications.


The authors used as an example the study of Design/CPN. Design/CPN is an application that support the construction and evaluation of complex colored Petri nets.

Design/CPN is a mediating artifact that allows the user to produce CPNs. Such CPNs in turn are mediating how users verify alarm protocols. A particular artifact most often mediates a multitudes of activities and what is sometimes the objects of an activity is in other instances itself a mediator. To fully understand the use of an artifact we must find out which activities the artifact is used in and how these are connected. This is why they talk of web of activities.

An important concept for the analysis of the tools is the focus shifts. Sometimes the users' attention is drawn toward the object of the operation and sometimes towards the tool itself. For example is the tool has problems the user may shift the focus from the operation to the tool itself.

Scientific foundation

Action Theory is a psychology theory. It was developed by Vygotsky and his students in the Soviet Union at the beginning of 1900s.

As a psychological theory, it was aimed at understanding the mental capacity of a single human being. Activity theory rejects the isolated human being as an adequate unit of analysis, insisting on cultural and and technical mediation of human activity.

Historically activity theory is an answer to the problem of studying isolated isolated individual in the laboratory setting.

For Vygotsky human activity has three fundamental characteristics: 

- it is directed towards a material or ideal objects
- it is mediated by artifacts
- it is socially constituted within a culture.

A student of Vigotsky, Leontiev proposed a slightly different basic model in the analysis of cognition. For him socially mediated activity comes before activity mediated by artifacts. According to him activity can be analyzed according to three hierarchical levels:

-Activity: activity is directed to satisfy a need. The activity has a motive.
-Action: human activity is carried out through action.
-Operation: actions are realized through a series of operations.

The three levels of activities are not fixed an action can become an operation and an operation can become an action.

The fact that an act can belong to different levels according to the context is what differentiate this psychological theory from others.

Activity systems are fundamentally marked by contradictions: dynamics are understood as the eternal resolving of contradictions. Contradiction are driving forces in human learning and development. activity is constantly developing as a result of contradictions and instability, and as a result of the development of new needs.

Human being re-create their environment: activity is crystallized into artifacts in two ways:

-they are externalizations of operations with earlier artifacts
-they are representation of modes of acting in the given activity.

Detailed Description

Here they outline a series of principles of activity theory with emphasis on HCI. It is explained against the concrete example of the CPN2000 tools: study of the previous tools and practices and design of the new one.

In particular they instantiate concepts such as 

    -mediation: instruments or society mediate between subject and object)
    -internalization - externalization: procedures that becomes automatic, or need our attention if there are problems
    - web of activities, 
    -development: human activity constantly develops.

Activity theory in Practical Design and Evaluation. 

They summarize how to make an analysis. They proposed a checklist of an analysis situating an application in use, followed by a focus shift analysis. 

I prefer the points reported by Korpela (2000):

The concept of artifact in use as a tool in the redesign of the CPN tool: they look in their example the historical reasons behind some contrasts such as the data structure.

The user interface: understand the user interface elements such as buttons and scroll bars as mediators/artifacts.

Current status

 Nardi (1996) suggests that action theory is a powerful descriptive tools rather then a predictive theory.

Several researchers are trying to expand the view and deal with design issues. Others are map out the relationship of activity theory to most recent theoretical trends in social sciences. (Engestrom and Miettinen 1999) and other disciplines.

Activity theory is starting to get used for analysis of design processes.


In this paper I found very interesting the introduction, the overview of the philosophical theory and his historical development. Even if I think some aspects were given for granted: later in the paper they talk about important aspects but those aspect are not described anywhere. 

I found less convincing the application to HCI. In particular the "crystallization" of the checklist. On one side there is the philosophical theory, and on the other the application to a field: this is bound to create some tension. Nonetheless I think the application was too forced. Sometimes the application insisted on details such as button and scroll bars and their role as mediator. While mediation was  an important point, it should not be seen it as an item in itself but in the realm of the bigger picture. But it looked like the authors wanted to extract from the theory concrete little pieces to work on. Well in general my impression is that while trying to adapt it to HCI they distance themselves from the main points of the theory. The claims about the activity being object oriented are kind of unclear.

This is not to say that I don't find the theory applicable to HCI, but only the way the theory is applied in this paper doesn't convince me. Other authors may have different approaches.

15. Design Rationale as Theory

by John M. Carroll and Mary Beth Rosson, Virginia Polytechnic Institute

Design Rationale: documentation and analysis of specific designs in use. It describes the features of a design, the intended and possible use of those features, and the potential consequences of the use for people and their task. This involves observing or hypothesizing scenarios of user interaction, and describing their underlying design tradeoffs. User requirements, discussions, debates, reasons for particular features, reasons against particular features it doesn't have, the weighting for tradeoffs, and so forth.

This chapter shows how  documenting and analyzing design rationale can contribute the theory development in HCI.

15.1 Motivation

The 1980s produced a lot of theories for HCI. Their perspective were narrows though: they addresses isolated users working without errors on routine tasks. The limitation of these theories provoked the rise of other theories approaching error recognition, learning and so on and approaches crossing over from different disciplines (sociology, anthropology, etc.) and it also provoked debates about science and theory in HCI. In particular it provoked debates about HCI theories, their validity, their relevance, their boundary condition for applications, and so on.

These debates motivated "design rational" as theory. Design rational is documentation and analysis of specific designs in use.

Design rationales addresses three themes from the HCI theory debates:

  1. Context: context must be considered directly in formulating, assessing, and applying theories. Context includes contexts of uses and context of system development.
  2. Applicability: applying theories involves mapping concepts across different domains. For example from psychology to HCI. Design rationales helps make these transactions.
  3. Scope: theories may be too broad or too narrow. Design rationales helps integrate distinct sources of theory. Different aspects of different theories can be seen as tradeoffs. 

15.2 Overview

Task-artifact framework: there is a cycle between scenarios and development of technology.

1-Tasks are represented as scenarios of use. Scenarios include activity context, the actor's motivations, actions, and reactions during the episode.

        -> claims analysis: enumerates the features of a system that are hypothesized to have positive or negative consequences.

2-Design of new technology: claims analysis is used to guide the design of new technologies.

    -> new technologies raises new opportunities for human behavior

The cycle then continue by documenting the new behavior into new scenarios.

Design rationales for families of artifacts also overlap, leading to more abstract design rationale that creates a design space.

15.3 Scientific Foundations

15.3.1 Ecological Science

Principle of Ecological Science: systems in the natural and social world have evolved to exploit environmental regularities.

"Ecology" of HCI: everyday practices and materials of users and developers.

HCI science can be developed as ecological science at three levels:

  1. Taxonomic Science. Design rationale as design documentation. Design rationale provides an inventory of the components and relationships in system and software.
  2. Design Science. Descriptions of particular designs can be abstracted and generalized by categorizing designs, and the features that define the categories can be associated with general consequences for users and their tasks.
  3. Evolutionary Science. Tasks and the contexts of carrying out the tasks create needs and opportunity for new artifacts, but those new artifact consequently alter the tasks and eventually create new needs and opportunity for new designs. This is the task-artifact cycle. To pursue an ecological science it is important to also to project, understand, and manage the trajectories of change in the task-artifact cycle.

15.3.2 Action Science

If some sort of intervention is the ultimate objective of research, then the intervention itself should be a part of the research project.

This  is efficient and effective since reduces the transfer of knowledge. Making interventions is also a good method for understanding systems. Social and technical systems are constantly evolving. It is important to identify in an evolving situation the possible transformations of the situation and invariants in the situation with respect to transformations.

15.3.3 Synthetic Science

Design rationale allows the insights and predictions of diverse technical theories to be integrated within a coherent analysis context.

15.4 Detailed Description

Steps in constructing a design rationale:

  1. Identify a collection of typical and critical scenarios of user interaction. Ensure balance across different types of scenarios using several methods to identify scenarios.
  2. Recognize claims or tradeoffs.

15.5 Case Study

15.5.1 MOOsburg as a Case Study in Action Science
15.5.2 MOOsburg as a Case Study in Ecological Science
15.5.3 MOOsburg as a Case Study in Synthetic Science