Exploring Tangible AI with Theory Instruments
This interactive half-day workshop aims to explore what role anthropological theories may play in designing AI beyond the computer screen, like assistive devices, self-driving vehicles. Understanding how humans react to AI devices is a substantial challenge, but this is crucial knowledge for designers. Theories from the social sciences can help. Because theory discussions may be too ‘theoretical’ for fast-paced designers in industry, we have developed a set of Theory Instruments that turn discussions into playful, collaborative activities. The workshop investigates cases of Tangible AI that react and propose actions.
Figure 1. Six theory instruments.
Introduction
For humans it is a serious challenge to understand the workings and limitations of products equipped with Artificial Intelligence. This makes it difficult for designers to conceive and trial new proposals. While AI for text production and diagnostics receive much attention, there is less focus on ‘tangible AI’, i.e. physical products with embedded artificial intelligence; on ‘things’ beyond the computer screens. Such products and systems may look like something we know and understand, but they increasingly act with agency of their own. While this challenge may appear entirely new, there are fundamental mechanisms at play in how we as humans develop social relations, classify experiences, interact with tools etc. Anthropology and the social sciences have for a century developed theories to describe such mechanisms. Such theory perspectives can help understand Tangible AI challenges and opportunities. With base in studies of busy UX designers and design researchers in industry, we have developed ways of turning theory discussions into playful, tangible activities with Theory Instruments: Physical contraptions that challenge participants to collaboratively ‘build’ an understanding with everyday objects as representatives of theoretical concepts.
Workshop Focus
With this workshop we aim to initiate a conversation about what role anthropological theories may play in exploring how humans will relate to tangible forms of AI beyond the computer screen.
Tangible AI
Tangible AI ‘allows the user to experience and feel those systems’ [1]. Autonomous vehicles and human robot interaction [2] are dominant research fields, but there is an increasing focus also on more mundane products like robot cleaners [3]. Tangible AI borrows from the field of Tangible Interaction Design which has promoted the advantages of engaging touch and manipulation in the interaction with technical
systems. Originally, Tangible User Interfaces (TUI) were proposed as alternatives to Graphic User Interfaces (GUI): ways of turning Human Computer Interaction physical [4] with a particular focus on physicalizing data. Subsequently, interaction designers from an industrial design tradition expanded the concept of tangible interaction to innovate physical products like alarm clocks, photo cameras, lamps [5]. Here, the emphasis is on rich bodily movements in the interaction. Hornecker & Buur proposed a framework that brought together the different streams of Tangible Interaction research based on how users experience them [6].
Our focus within Tangible AI is on physical products with embedded artificial intelligence; on ‘things’ beyond the computer screens. Examples include assistive devices such as adjustable hospital beds, insulin pens, hearing aids. Technology that learns from human behaviours and adjusts, makes proposals etc. prosaically described as ‘human-in-the-loop systems’ [7]. Here, a core challenge is to understand not just how users experience the interaction, but how they conceptualise the technology and develop relations with artefacts through the interactions. The concept of Explainable AI is increasingly drawing attention [7].
Useful theories
In collaborations with industry partners, we have selected a set of theories that have the potential to stimulate discussions about connectivity, digitalization and AI, Table 1. Some of them are classics within anthropology, some describe relations between humans and technology. We have found that these theories had broad potential for instigating perspective shifts but lacked the accessibility to traverse the disciplinary gaps among practitioners [8]. This led to the development of playful Theory Instruments.
Table 1. Six selected theories, and how they may enhance a conversation about human perceptions of Tangible AI devices.
Theory | Perspectives | Proponents |
Rites of Passage | Identity formation and group membership: How do people become socially acknowledged as AI users? | Van Gennep [9] VictorTurner [10] |
Reciprocity & Exchange | Forming human relationships: How do users build relations with AI producers and technology | Claude Lévi-Strauss[11] Marcel Mauss [12] |
Classification | The conceptual power of language: Which terms do users employ to describe AI devices? | Mary Douglas [13] John Hendry [14] |
Forms of Capital | How agency depends on resources: Which assets do users bring in play when negotiating with AI devices? | Pierre Bourdieu [15]
|
Actor-Network Theory | The agency of ‘things’: Which agency do users ascribe to AI devices? | Bruno Latour [16]
|
Product Ecology | Products in emergent practices: How do users adopt and adapt AI devices in larger ecologies?
| Jodi Forlizzi [17] |
Theory Instruments
The idea of ‘Theory Instruments’ emerged from an experience in industry that theories about human behaviours do not easily find way into busy design research and UX practices, even if they have much to offer in terms of explanatory perspectives [8]. We designed these Theory Instruments with mundane, every-day objects like clothespins, rods, rings, dice, and boxes, Figure 1. The objects obtain specific meanings in the conversations between participants when they ‘play’ the instrument. For example, the clothespins become ‘actors’ and ‘actants’ in one instrument, and ‘gifts’ in another. In some cases, we use labels to highlight particular theoretical terms, like ‘liminality’ and ‘social capital’. Each instrument comes with step-by-step instructions for an activity of 15-20 min duration [18]: Things to build, things to discuss.
The use of Theory Instruments matures design research by making theoretical perspectives tangible, keeping theory active during all stages of design research, and integrating design and anthropology [19]. Like musical instruments, Theory Instruments allow for ‘playing different tunes’ with hands and body, rather than bringing about a foregone conclusion. As in an orchestra, Theory Instruments require players of diverse expertise, playing different instruments, to produce ‘harmonious music’. In this workshop we like to co-explore how Theory Instruments develop a theory-based sensitivity to the social world around us and, in particular, to the opportunities and challenges of Tangible AI. And, not least, to a critical reading of what AI means [20].
A Hands-On Workshop Format
Planned Activities
A half-day, face-to-face Workshop for sharing experiences with user perceptions of Tangible AI and hands-on work with ‘Theory Instruments’.
- An introduction round where attendants explain how they relate to Tangible AI and theory (45 min)
- Hands-on work with the Theory Instruments in groups: We will bring material from a case study (90 min)
- General discussion of the idea of employing theory to understand human perceptions of Tangible AI (30 min)
- Summary of learning points and new avenues for research (15 min)
Following the workshop we would like to write up the reflections drawing on participants’ examples, and list potential avenues for research and further collaborations.
Participants
This is a come-as-you-are activity. We hope for a group of 12-20 participants to form a cross-disciplinary group of:
- Academics with research in AI and design.
- Industrialists and designers curious about innovative perspectives on Tangible AI.
- Educators who like to explore new ways of teaching theory.
We will attract participants through our networks within design research and design anthropology.
Workshop organisers
We bring together competencies from design, anthropology and engineering, and years of experience with interacting with design researchers and UX designers in the manufacturing industry. We have successfully organised earlier versions of this workshop in a medical company in Denmark and a machine manufacturer in the USA.
Design, Media and Educational Science, University of Southern Denmark:
Jacob Buur is professor of User Centred Design, PhD in design engineering. He brings UCD management experience from manufacturing industry. He has organised hands-on workshops at conferences like DIS, CHI, PDC, EPIC, OzCHI, DRS, NORDES. Also initiator of the Participatory Innovation Conference series and the Student Interaction Design Research Conference. Contact: buur@sdu.dk
Mette Gislev Kjærsgaard is associate professor of Design Anthropology, PhD in visual anthropology. She has industry experience from design research in several companies. She is co-organiser of the Research Network for Design Anthropology. Contact: mgk@sdu.dk
Ona Pirol is programme coordinator for the IT Product Design graduate course and project manager for industry relations activities. She combines design with a degree in history. ona@sdu.dk
Department of Education, Aarhus University:
Jessica Sorenson is an anthropologist and postdoctoral researcher mapping the human-AI infrastructures of wastewater management. She brings experience of organizing workshops at HRI and IROS, as well as public workshops for policymakers and citizens. jeso@edu.au.dk
References
[1] Flemisch, F. et al. (2020). Let’s Get in Touch Again: Tangible AI and Tangible XR for a More Tangible, Balanced Human Systems Integration. In: Ahram, T., Karwowski, W., Vergnano, A., Leali, F., Taiar, R. (eds) Intelligent Human Systems Integration 2020. IHSI 2020. Advances in Intelligent Systems and Computing, vol 1131. Springer, Cham. https://doi.org/10.1007/978-3-030-39512-4_153
[2] Paci P. et al. “Who’s a Good Robot?!” Designing Human-Robot Teaching Interactions. P. Lukowicz et al. (Eds.) HHAI 2023: Augmenting Human Intellect
[3] Sung J, Grinter RE, Christensen HI. 2010. Domestic robot ecology. International Journal of Social Robotics. Dec;2(4): 417-29.
[4] Ishii H. and Ullmer, B. 1997. Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. 234–241.
[5] Overbeeke, CJ, Wensveen, SAG, Hummels, CCM, Frens, JW & Ross, PR 2010, DQI Interaction Research. in C Mareis, G Joost & K Kimpel (eds), Entwerfen, Wissen, Produzieren. Designforschung in Amwendungskontext.Transcript Verlag, Bielefeld, pp. 193-207.
[6] Hornecker E. and Buur, J. 2006. Getting a grip on tangible interaction: a framework on physical space and social interaction. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 437–446.
[7] Colley A, Väänänen K, Häkkilä J. Tangible Explainable AI-an Initial Conceptual Framework. InProceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia 2022 Nov 27 (pp. 22-27).
[8] Kjærsgaard M. G., Mosleh W. S., Buur J., Sorenson, J. 2021. Anticipating Connectivity in (UX) Design Practices: Reframing Challenges by Introducing Theory Cards. In Ethnographic Praxis in Industry Conference Proceedings Nov 1, 159-173
[9] Van Gennep A. 2013. The rites of passage. Routledge.
[10] Turner V. 1969. Liminality and communitas. The ritual process: Structure and anti-structure. 94(113): 125-30.
[11] Lévi-Strauss C. 1996. The principle of reciprocity. The gift: An interdisciplinary perspective. 18-26.
[12] Mauss M. 2002. The gift: The form and reason for exchange in archaic societies. Routledge.
[13] Douglas, M. 1984 [1966]. ”The Abominations of Leviticus.” In: Purity and Danger: An Analysis of the Concepts of Pollution and Taboo. London: Ark Paperbacks, 41-57.
[14] Hendry, J. 2008. Sharing our worlds – An introduction to cultural and social anthropology. New York University Press, 36-110.
[15] Bourdieu P. The forms of capital. 1986. Cultural theory: An anthology. 2011;1:81-93.
[16] Latour B. 1992. Where are the missing masses? The sociology of a few mundane artifacts. Shaping technology/building society: Studies in sociotechnical change. Jul 27;1: 225-58.
[17] Forlizzi J. 2008. The product ecology: Understanding social product use and supporting design culture. International Journal of Design. Apr 1;2(1).
[18] Buur, J., Karyda, M., Kjærsgaard, M.G., Sorenson, J., Ağça, A.Ö. and Antonelli, M., 2023, February. A Collection of Tangible Theory Instruments for Design Anthropology. In Proceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 1-12).
[19] Gunn, W., Donovan, J. 2012. Design and Anthropology. Surrey, England: Ashgate.
[20] Suchman, L. (2023). The uncontroversial ‘thingness’ of AI. Big Data & Society, 10(2)