IBL News | New York
Nvidia unveiled an AI platform for teaching American Sign Language (ASL), the third most prevalent language in the country after English and Spanish. The goal is to break down communication barriers between the deaf and hearing communities.
In the U.S., around 11 million people are deaf or have significant hearing loss.
Developed in partnership with the American Society for Deaf Children and the creative agency Hello Monday, this interactive web platform, named Signs, supports ASL learning and the development of accessible AI applications.
A 3D avatar demonstrates signs and uses an AI tool that analyzes webcam footage to receive real-time feedback on their signing.
Users of any skill level can contribute by signing specific words to help build a video dataset for ASL.
Nvidia aims to grow this dataset to 400,000 video clips representing 1,000 signed words, aiming to have a high-quality visual dictionary and teaching tool. Fluent ASL users and interpreters are participating to ensure the accuracy of each sign.
This dataset, which is starting with initial set of 100 signs, will be used to further develop AI applications. It will be available to the public as a resource for building AI agents, digital applications and video conferencing tools.
The dataset behind Signs is planned for release later this year.
While Signs currently focuses on hand movements and finger positions for each sign, ASL also incorporates facial expressions and head movements to convey meaning. The Nvidia team behind Signs is exploring how these non-manual signals can be tracked and integrated into future platform versions.
This team is also investigating how other nuances, like regional variations and slang terms, can be represented in Signs to enrich its ASL database — and working with researchers at the Rochester Institute of Technology’s Center for Accessibility and Inclusion Research to evaluate and further improve the user experience of the Signs platform for deaf and hard-of-hearing users.
“Improving ASL accessibility is an ongoing effort,” said Anders Jessen, founding partner of Hello Monday/DEPT, which built the Signs web platform and previously worked with the American Society for Deaf Children on Fingerspelling.xyz. This application taught users the ASL alphabet. “Signs can serve the need for advanced AI tools that help transcend communication barriers between the deaf and hearing communities.”
• Start learning or contributing with Signs at signs-ai.com
• Nvidia’s trustworthy AI initiatives.
• Nvidia GTC, March 17-21 in San Jose, Signs live demo.
