What world do we experience through touch?

For Ted Adelson's class Touching and Grasping with Soft Fingers (9.357), I prepared a set of everyday items for haptic exploration so that we could get a phenomenological sense of the breadth of our ecological haptic world. We explored the objects together, sometimes in a freeform way and sometimes in response to particular challenges. As we shared our experiences, we noted the challenges for our respective fields (the class brought together roboticists and perception researchers, a great combination). All of these experiences were framed by some discussion of the perception literature (slides). By the end we felt as though we had touched a huge world, through just a few objects. Here I provide an outline of the class and some of the items we used, but please consider this a recipe and have fun!

2D versus 3D haptic recognition

Demo: Shortly after I got to grad school, one of my new friends gifted me a bunch of rubber ducks. I brought those to class, and I also used a glue-gun to create an ridged outline of a swan on pieces of paper (one for each person). I gave out the outlines and asked people to recognize the objects. Then we did the same with the figurines. The difference is stark and funny.

Literature: Here, our companion paper was Lederman, Klatzky, & Barber (1985). They showed that blindfolded people could identify everyday 3D objects (such as a whistle, pen, sweater) nearly completely accurately after touching them for only a couple seconds. They were responding to a consensus in the literature that the haptic system could not perceive spatial structure effectively (e.g., Magee & Kennedy, 1980); and this had been tested through getting participants to essentially trace raised line drawings. What does the 3D world open up?

Haptic subsystems

Demo: I brought in pencaps and toliet paper. Each person held out their hand and I put one of the objects in their palm. "What can you tell without moving?" Then I let them close their hand around the object (but not move otherwise). "What can you tell now?" And finally, I let them explore freely.

Literature: We talked a bit about the different thermo- and mechano-receptors in the skin, as well as the kinesthetic inputs. The idea is that the hand's motor abilities are inherent to its sensory abilities, drawing from J.J. Gibson's 1962 "Observations on active touch" and 1966 "Senses considered as perceptual systems".

Haptic qualia and exploratory procedures

Get opaque paper bags (one for each person in the class) and fill them with everyday objects. The bags don't have to be all the same. Also make sure you have a way for people to clean their hands. A diversity in the crowd of ordinary objects is the key thing. Watch out for safety of course.

Start the session by allowing everyone to explore the bags quietly for a moment (no looking), then have people start saying out loud what they're experiencing. What is the quality of exploration? What is the sensation, and what do they perceive? What is their hand doing? Then, guide them through a few particularly interesting examples by posing questions. One simple but effective question is "Can you find an object with [this quality]?". Here are some items we used for this first part (and some that got left out):

• Different shapes of dry pasta
• A slice of red pepper. "Name something in the bag that's wet."
• A few pretty long pieces of rope
• Surface textures: fabrics, sandpaper, velcro, etc.
• "3D textures": small plastic lunch bags, one filled with beans, one with rice, one with flour. Tighten the bag around the filling with an elastic band. "Find the bag with rice in it." What motion is used?
• Pliable objects like clay, bluetac, or hairtie.
• Marbles. "Find the smallest object in the bag."
• Wooden block. "Find something made of wood."
• Skewers. "Find something sharp."
• Leave 'til last: two plastic easter eggs, one filled half way with beans and another similar except also stuffed with cotton. Put the participants into pairs. "One partner should close their eyes. The other partner crumples the paper bag while the first partner tries to distinguish the two eggs." (An exploration of intuitively generating sensation and sound. The bag crumpling is for noise cover-up at first)
• It might have been nice to explore temperature in other ways: maybe with chilled coins or something that can melt like chocolate.

Literature: Lederman & Klatzky's seminal work on exploratory procedures (1987) is essential. Do these exploratory procedures describe what people are doing? What's missing? You'll find inspiration for your objects in here.

Adapting exploratory procedures to constraints

Include a number of ways to constrain the exploration process to illuminate the sensitivity of the hand. For example:

• Only use a single finger
• Chopsticks
• Put clay/styrofoam on the tips of the chopsticks
• Finger splints
• Gloves (of different materials and clunkiness)
• Hand cream
• Pipe-cleaner
• Duct-taped hands

In the following scenarios, we explored if we used the same exploratory procedures adapted to new tools, if we came up with new exploratory procedures, or if we just failed. I think that most of these were included in the bags in the first part:

• Two pieces of linguine, of different lengths. Tell them apart. How do you do it with two fingers? Chopsticks or a single finger? Two hands? (My answers respectively: wiggle the end, contour following, grabbing ends)
• Medicine bottle. What's the global shape? Use a pipe cleaner, chopstick, duct-taped hands, normal hands.
• Clay blocks on a smooth table vs. sandpaper. Which block is heavier? Use your chopsticks.
• Instant noodles. How do you know that they're brittle (that you can break them) with your hands? How about your chopsticks?
• Can you distinguish the 3D textures with gloves on? Can you open the bags tied with the elastic with chopsticks?
• Can you tell the hardness of any objects around you with the pipe-cleaner?

Literature: Lederman & Klatzky, 2004.

Notes from our exploration in 9.357

• When you're rooting around in the bag, how you feel without actively exploring any particliar object - even a hand brushing by gives vivid impressions. How is exploration different with the constraints?
• There are many interesting qualia left out of L&K, e.g. brittleness, bendiness of pasta, wetness, sliminess, stretchiness
• With just a single hand: stabilization, manipliation, sensing, comparisons. It's actually a pretty ineffective strategy to pick up one object at a time and explore.
• Just holding an object in your hand opens up a lot of structure - mlitiple contacts seem important. Sometimes K&L write off these simpler movements but the in-hand manipulations
• When you use chopsticks/feel something through anything else, where do you feel it?
• Mental model of constraints: we (incorrectly) think we won't be that impaired by the gloves vs. the immediate pipe-cleaner "oh shit." Still, whatever the constraint, you can often find many ways to work around them and use the environment as another hand part
• How do you tell that something in your hand is a single object? What's a "single object" experientially?

Thank you to Ted Adelson, Sarah Schwettmann and Luke Hewitt.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.