TactileLetters

Tactile Letters is a tangible tabletop that leverages the use of texture cues to help children with dyslexia aged 5-6 years to learn letter- sound correspondences.

Course Project: research, design, prototyping

My Role:  research, technical solution, UI and interaction design, prototype construction

Timeline: 2014.05-2014.06

Research

In education programs, researchers and practitioners often use the Object-Imaging-Projection method (OIP) to teach children letter-sound correspondences. This method associates letters/sounds with particular objects that have forms very similar to the letter shapes and whose beginning sounds are the letter sounds (e.g., a/a/ is often associated with “apple”).

Inspired by the OIP method, we want to explore if we could have a proof-of-concept design that leverages the use of various materials to help children to learn. 

Prototyping

We present the design of Tactile Letters. The prototype is comprised a digital display, an interactive table, 24 pieces of letter cards, and a set of 3D tangible letters. Each letter in the system is associated with a unique texture based on the OIP method.

Children can choose an appropriate level and learn about letter sounds by placing 3D textured tangible letter(s) or letter card(s) on an interactive table. Each tangible letter (sound) is associated with three letter cards with a pseudo-word on each. Visual-audio feedback is provided once children make correct/incorrect associations between 3D tangible letters and letter cards.

a-apple-shiny smooth paper; b-balloon-rubber; d-dog-fur; o-orange-leather; p-pillow-pillow cloth; q-quilt-quilt cloth

Each letter card contains a pseudo-word that helps children to practice the learnt sounds.

Each tangible letter or letter card is identified by the reacTIVision engine, which passes identification and location information to a custom application written in Processing.

This application includes two levels of learning contents.

The first level was designed to support learning of single letter-sound correspondence. When a user places one tangible letter and three letter cards on the interactive table, this action triggers the associated digital contents on the display (the letter sound, the object, and the 2D uppercase and lowercase letters).

The second level was designed to support learning of opaque letter-sound correspondences. If the two adjacent letters generate a new sound (e.g. oa/ei/), the system triggers the new sound and displays a red frame to indicate the two letters are grouped.

Original Study Plan

In order to explore whether adding texture cues in a tangible tabletop enables children to learn letter-sound correspondences better, we also created another set of 3D tangible letters.

In this set, all the letters have the same standard texture (wood with sticky paper). In order to control the possible influence of colour, all letters are a single colour – red.

Our next step is to conduct a small-scale user study to refine the prototype design. Then we will conduct a controlled comparative experiment with this prototype to investigate the role of texture cues of a tangible tabletop in supporting learning to read for children.

Adjusting Our Research Direction

We presented our Working-In-Process (WIP) work in the ACM TEI (Tangible, Embedded, Embodied, Interaction) Conference in 2015 and received valuable feedback from researchers and design experts. Many suggested that our design may particularly benefit for children who learn English as a second/foreign language(ESL/EFL) because these children usually need more cues to memorize letter-sound correspondences.

Before running the study withEFL children, we presented an exploratory study to investigate how EFL learners associated colours and materials to English letter-sound pairs. We focused on young EFL adults rather than EFL children because previous studies suggested letter-sound-colour mappings maybe influenced by previous language learning, so working with adults to understand their associations can then be transferred to design for children.

User Study

We conducted a mixed method study design with 18 Chinese college students (9 males and 9 females) with an average age of 20 years old.  The preliminary results indicate that

  • The most colour significant results were r/r/-red and b/b/- blue (p<0.001) mappings.
  • Other colour significant results were p/p/-purple (p<0.01), m/m/-pink and orange, d/d/-black, and g/g/-green (p<0.05) mappings.
  • Significant texture results were m/m/-velvet, n/n/-cotton, g/g/-linen, b/b/-plastic grid, d/d/- plastic grid, t/t/-metal foil paper, p-/p/-styro-foam, and o/o/-textured (dot) rubber (p<0.05) mappings.
  • The letter sound to colour mappings were influenced mainly by the literal meaning of the letters (e.g. b/b/-blue, d/d/-dark).
  • The letter-sound to texture mappings were strongly influenced by the characteristics of letter sounds (e.g. /m/ and /n/-soft sounds-fuzzy materials).
  • We found little influence of participants’ first language on their mapping choices.

Design Implications

  • Consistency in letter-sound to colour and texture mappings suggests that designers could use these relations to help learners better understand and remember letter-sound pairs or differentiate similar pairs by providing additional cues.
  • The potential influence of literal meanings suggests that designers could reinforce particular mappings through additional pictorial cues that mirror the mappings.

Publications