Learning to read can be difficult for any child, but for children who are profoundly deaf it can be an insurmountable challenge. Storysign was an AR app created to help deaf children overcome this difficulty and open up the world of books to them.

Children scan the pages of physical story books with their device to reveal animations of our character Star signing the relevant passage of the story with the correlating word highlighting on the page as she signs. 

Insofar as character design and animation designing for deaf children posed some unique challenges. Facial expression can account for a high percentage of sign language coupled with nuanced body movement and complex hand/finger movements.

Often with character design you’ll strip back features to their most basic, or drastically change proportions, but with Star we had to keep things humanistic but balancing that against making a character that kids would get excited by.  The design strives to position Star as an identifiable role model to a wide community of deaf children.  

When animating we motion captured real sign language performers in order to get the most accurate performance possible. The process involved suiting up our actors in a motion capture rig with a high fidelity marker set-up that recorded their full body and facial performance from over 70 infrared cameras placed around the studio.

These performances were then correlated with the books’ metadata, and therefore associated with the real books using a machine-learnt AI image to text system ensuring a seamless experience for young users.

Books are available across 14 languages and the app is supported by International Deaf Associations and charities around the world.  

Storysign is the proud winner of 4 gold, 1 silver and 2 bronze Cannes Lions awards, 3 D&AD awards, a Campaign Big award, 4 One Show awards, 10 LIA awards and been nominated for a Children’s BAFTA. 

 

Download from Google Play

Download from iTunes

Interested in commissioning content or have a project similar to this you’d like to discuss? Then drop us a line at interactive@aardman.com

Add clips to the playlist by clicking the stars