nemozone

Project Shuwa's First Step Towards Universal Sign Language Recognition and Learning

Imagine a world where everyone understands sign language, and the deaf community feels more included in everyday conversations. That's the vision of Project Shuwa, which has taken its first step with SignTown, an online game designed to teach sign language to hearing individuals. This innovative approach aims to create a more inclusive and harmonious society, where everyone has the opportunity to communicate effectively regardless of their hearing abilities.

SignTown is a virtual environment where players are encouraged to use sign language to complete various tasks, such as packing for a trip, staying at a hotel, or ordering food at a café. By using their computer cameras to sign, players receive immediate feedback from an AI-powered recognition model that assesses their signing accuracy. This real-time feedback enables players to learn and improve their signing skills efficiently.

One of the major challenges in sign language recognition has been the lack of accuracy in previous models due to the complexity of visual-gestural language data. Sign languages encompass a wide range of gestural information, including hand, head, and body movements, facial expressions, and mouth shapes. Omitting these subtleties can result in grammatically incorrect or difficult-to-interpret messages.

To overcome this obstacle, Project Shuwa developed a machine learning model capable of recognizing, tracking, and analyzing 3D hand and body movements, as well as facial expressions, using a standard camera. This is a significant advancement compared to previous approaches that required complex equipment, such as 3D cameras and sensor-equipped gloves.

Currently, SignTown offers the choice between Japanese and Hong Kong Sign Language, with plans to expand the available options in the future. The next phase of Project Shuwa aims to create a sign language dictionary and an automatic translation model that recognizes sign language conversations and converts them into spoken language using computer and smartphone cameras.

Cheng Ka Yiu, a computer officer at CSLDS involved in the development of SignTown, acknowledges that there's still a long way to go. However, he believes that SignTown is opening the door for people worldwide to learn about sign language in real-life situations. By leveraging advanced technology, Project Shuwa is striving to build an inclusive and harmonious society where equal opportunities exist for everyone.

In conclusion, SignTown represents a groundbreaking step towards universal sign language recognition and learning. By engaging hearing individuals in a fun and interactive way, this online game fosters greater understanding and inclusivity between the deaf and hearing communities. As Project Shuwa continues to develop, the potential for a world where sign language is accessible to all becomes increasingly attainable.

https://sign.town/ https://projectshuwa.org/ https://www.asiaresearchnews.com/content/sign-language-all https://blog.google/outreach-initiatives/accessibility/ml-making-sign-language-more-accessible/

A similar project as a Play store App:

https://play.google.com/store/apps/details?gl=US&hl=en_US&id=edu.gatech.popsign