Translation device a sign of the times

Three college students in northeastern China have developed a two-way sign language translation technology to help people with hearing and speech disabilities communicate with others.

The technology uses wristband sensors that detect the muscle movements of the wearers' hands and arms. A hearing or speech-impaired person can sign while wearing the wristbands, and this will be translated into text and sounds through a mobile app.

The system works both ways, so those who cannot sign will have their spoken words converted into sign language and text, which is displayed on the screen of a mobile device.

In China, over 20 million people have hearing disabilities, and 1.3 million people have no ability to speak.

He Xiao, Li Rongjie and Zeng Zhen - three juniors at Northeastern University in Shenyang, Liaoning province - won the top award at the 2018 Enactus China National Social Innovation Competition on May 21 for their work in designing the system.

In October, they will visit Silicon Valley in the United States to compete with international candidates for the global Enactus award.

Wang Fei, who came back from Japan in 2005 after earning a doctorate in intelligent structures and mechanical systems engineering, has been leading and overseeing the sign language translation project at Northeastern University for more than 10 years.

"Japan has been dealing with an aging population far longer than China, so the whole community is used to paying close attention to the well-being of the elderly and people with disabilities," the professor said. "Japan has been a front-runner in terms of using artificial intelligence to solve problems for such groups."

Zeng, a computer science major, said he is grateful to be part of Wang's team. "I've met people with hearing disabilities who have trouble communicating with those who can't read sign language," he said. "It's my responsibility to use my expertise to bridge the gap."

In the past three years, the team has upgraded the algorithms that decode sign language users' movements into associated words in order to expand the data sets, thereby increasing accuracy. About 10,000 signing gestures have been fed into the system so far.

"The system is able to recognize and translate 50 signs into Chinese while maintaining a 98 percent accuracy rate," Zeng said. "We're aiming to hit 1,000 signs before October."

The team is also stepping up efforts to create a domestically made wristband with hardware that senses muscle movements. A version is currently available from Thelmic Labs, a Canadian company founded in 2012.

"The imported wristband costs over 1,000 yuan ($155). It's expected that the price can at least be halved to 500 yuan with a homegrown one, making it much more affordable for those in need." Zeng said.

Determined to deliver the product to the market, the team has recently reached out to investors, nonprofit institutions and research bodies to accelerate development and seek production venues with hopes of benefiting all sign language users in China.

Wang Xiaoyu contributed to this story.