StorySign

Loading...

Learning to read can be difficult for any child, but for children who are profoundly deaf it can be an insurmountable challenge. StorySign is an AR app created to help deaf children overcome this difficulty and open up the world of books to them. Children scan the pages of physical story books with their device to reveal animations of our character, Star, signing the relevant passage of the story with the correlating word highlighting on the page as she signs.

We encountered challenges when considering character design and animation for deaf children; often in animation, we strip back features to their most basic, or drastically change proportions, but for this project it was important to ensure Star was both ‘humanistic’ but also a character kids would be excited by, and one we could position as an identifiable role model to a wide community of deaf children.

When animating, we motion-captured real sign language performers in order to get the most accurate performance possible. The process involved suiting up our actors in a motion-capture rig with a high fidelity marker set-up that recorded their full body and facial performance from over 70 infrared cameras placed around the studio. These performances were then correlated with the books’ metadata, and therefore associated with the real books using a machine-learnt AI image to text system, ensuring a seamless experience for young users.

Books are available across 14 languages and the app – available for both iOS and Android - is supported by International Deaf Associations and charities around the world.

Gallery

Drag me
  • 2019
    Cannes Lions
    Winner
    • Digital Craft
    • Design Lions
    • Health and Wellness Lions
    • Brand Experience/ Activation
    • Mobile - Social Purpose
    Winner
    • Mobile Utility App and Quality Education
  • 2019
    D&AD Awards
    Winner
    • Digital Design/Platforms and apps (Graphite Pencil)
    • Digital Design/Creative Use of Technology (Wood Pencil)
    • Educational & Financial Empowerment (White Pencil)
  • 2019
    Campaign Big Awards
    Winner
    • Original Ideas
  • 2019
    One Show
    Winner
    • Best Intellectual Property
    • Product
    • Digital/Use of Technology
    • Innovation in Health and Wellness
  • 2019
    LIA Awards
    Winner
    • Grand Brand Entertainment
    • Digital Apps
    • Digital Branded Content
    • Digital Innovative use of Content
    • Digital Public Service/ Social Awareness
    • Digital UX
  • 2019
    Campaigns for Good Awards
    Winner
    • Best Health Cause Campaign
  • 2019
    BAFTA Children's Awards
    Nominated
    • Digital

Get in touch

Rob Goodchild

Commercial Director

Rob is responsible for the overall rights and ancillaries programmes for Aardman IP, along with business development for the creative services offer across commercials and interactive.

Lorna Probert

Head of Interactive Production

Lorna is responsible for the studio’s Games and Interactive productions, generating business and overseeing the successful delivery of high quality games, apps and immersive experiences to support the strategies of both Aardman’s own IP and those of our external clients.