srishti: expressive objects

2018 tagged with: teaching, experience design filed under: projects

teachers are facilitators: we provide a brief that stimulates students (so that they may, physically and mentally, be present); and then—as they navigate down paths of their own—we nudge them, gently, confidently, to help them stay with the brief.

i wanted machines to express themselves to humans with parts/materials they were already made of—using vibrations, textures, mechanical sounds, light, and whatever such mode they already possessed—, reducing the urge to strap-on a screen or speaker on every interactive machine/robot we create.

in 2015, i’d explored how a machine’s behaviour could win it a human’s trust, and how machines and humans could collaborate for expression. in 2016, lucio and i tried imagining a vocabulary for machine-behaviour : but the ocean was vast and deep, and our boat modest. in 2018, given the freedom to teach whatever i wanted for one month at srishti, i threw sixteen minds at the problem instead of one, in a project titled “expressive objects”.

the class was organised into a multi-disciplinary studio tasked with an open brief (and a sharp deadline). we moved quickly; worked iteratively and in short-bursts; distributed into fluid 1–3-person teams; overlapped often and fed off each other; and used prototyping as a tool for theory-formation in design. (srishti prepared a video advertising our work.)

this post documents the steps we went through together.

step 1: engage

an essay on little thoughts of thinking machines (1983); the first-half of a podcast on objectification and humanisation (2016); a video inciting anthropomorphisation (1944); a paper on theory-formation in design; group discussions; open crits.

step 2: observe

finding objects that seem expressive, based on how they look (appearance) or how they behave (movement).

a hairdryer that is perceived to be aggressive.

step 3: compartmentalise

when faced with a large complex question, at times it helps to approach it part-by-part — in this case: investigate ‘machines’ and ‘people’ separately. so i asked, “in a world filled only with objects living their own lives, how would they express things like disgust or excitement to each other?”

students learned how to frame a subject and work rapidly; and to appreciate subtle movements and the role of time in behaviour.

step 4: explore

bring people back into the picture, and imagine a world where objects express things like care and anger to humans.

students learned how to pay attention to light when working with a camera; draw and enact storyboards; make low- and high-fidelity prototypes; and plan workflows.

step 5: theory-making

we collected all our work, and looked for patterns in it. what common parameters had we played with while trying to make expressive objects? did changing those paramters let us change the expression in a predictable manner?

we deduced that an expression can be dissected into its speed, duration, direction and quality-of-a-movement; and drew-up a ‘framework of expressions’ to refer to as we moved forward. we supported this by animating simple things — a sphere, the sound from a musician’s metronome, a vibrating phone — to test and improve the framework.

step 6: apply the framework

next: we turned our attention to the ‘mode’ an object uses to express itself. a phone can use vibration as a mode; a fabric can change its shape; a medical device pasted on your body may use small electric shocks as a mode; a bulb can change its colour in different ways; etc.

so, we worked with simple objects and ‘moved’ their modes in various patterns to express different things. the hands on a clock ticked in different speeds; texture on wood changed as you moved your fingers over it; a surface vibrated stronger or gentler; a sound got shriller, louder, softer to convey anger or sadness; walls in a room changed their colours to convey calm or panic; and so on.

we weren’t designing useful objects : we were demonstrating the rich potential of different modes of communication, and urging designers\engineers to leverage this potential while building robots and other interactive machines.

we could create expressive sound-pieces by making simple adjustments to the frequency, tempo and amplitude of a metronomic sound.

mode: sound • expression: sadness/tiredness (↓)

mode: sound • expression: excitement (↓)

varying the intensity, frequency and distribution of vibrations along a surface could also express things like excitement, tiredness, etc. (this is, however, less straightforward to present on a website; so please make do with images.)

we took everyday objects and subverted them; if we showed them moving in interesting ways, perhaps designers may think of new uses for them (and engineers invent techniques for this to happen). the hands on a watch might slow down if it senses us enjoying ourselves; or time may freak out if we’re callous with it.

while these ‘animations’ paled in comparison to an animator’s work (à la aladdin’s carpet in a disney film), we reminded ourselves that our animations followed rules (the framework): hence, they could have just-as-well been created by algorithms embedded inside objects; and besides movement, they could have affected the sound or vibration or colour of the object, and eventually be a part of unfathomablly-complex behavioural systems.

recently fashionable ‘table-top connected devices’ (from apple, google, amazon, etc) tend to be cylindrical, static, and married to speech (whether heard or spoken). we wondered if adding a small amount of movement might make such objects more expressive, computationally cheaper than speech-based expressions for some conversations, and overall more diverse.

gently pushing into art, we also built a cubic room (in virtual reality) whose walls changed colour—akin to turrel’s ganzfelds, perhaps—or pattern to create a discomforting or calming atmosphere.

step 7: present & exhibit

we presented our work at a public exhibition (rangoli, bangalore; 201812).

step 8: verify

at the exhibition, we tested our ideas with a wider audience: were our framework-driven objects perceived expressive in the way we’d hoped?

visitors heard a sequence of sounds as if a machine was making them. then, they identified what emotion best described what that machine was feeling at that time. while the results vindicated our approach, they also elicited the limits of using ‘emotional words’ to describe machine-behaviour.

step 9: reflect

we identified that words like ‘anger’, ‘disgust’ or ‘sadness’ may not be suitable for describing machine expressions; instead, we could describe the state and behaviour of machines in terms of their energies: low, high, intensely transmitted, conserved, directed or not, etc.

assessments:
i assessed students on the following criteria: intellectual-engagement, technical-skills and professionalism; in other words, i expected them to be able to question, demonstrate and collaborate; and do all three equally well.

students:
advaita govindan, naomi menezes, shreya mishra, kavya murugan, nisha rangdal, tanay sharma, yohaan kukreja, rujuta sondur, vandana shetty, chinmayee murugkar, shiv dinesh, sucharita premchandar, sheethal jacob, srushti gangawanwale, calvin stanley. (all, bachelor students of design in their third year of study.)

with thanks to:
sadhvi, for introducing me to srishti.
geetha narayanan, meena vari, gautham, kumkum nadig (and srishti!), for trusting me with their students.
yadu rajiv, who worked with me on the course, and formed the bridge between srishti and me. sai krishna mulpuru, for his advice on mechatronics.
rohit, for telling me what a mode is.