American Sign Language Plug-in for Apple devices

This American Sign Language (ASL) Plugin is an innovative software solution designed to bridge the gap between the deaf community and digital communication platforms.

CONTEXT

4th Semester (B.A.)     Invention Design II

TIMELINE

February 2020 - May 2020

SUPERVISION

Ludwig Kannicht

TEAM

Kübra Terzi, Xinley Qi, Maren Schlatter

01 Research

Research/Interviews

In order to get a first overview, we investigated various problem areas and came across an extensive area that deaf people in particular are confronted with on a daily basis. Via different kinds of research and interviews with those involved, we found out that these are the following large problem areas for deaf people.

Keyfindings:

Communication Barriers

Limited access to sign language interpreters, lack of closed captions in public spaces, and inadequate technology contribute to the communication gap faced by the 
deaf community.

Limited Career Opportunity

The job market can be particularly challenging for deaf individuals, who are often confronted with preconceived notions about their abilities. Discrimination, inaccessible workplace environments, and the absence of effective communication tools hinder career growth and perpetuate unemployment rates higher than the national average.

Educational Inequality 

Deaf students encounter numerous obstacles in obtaining a quality education. Mainstream schools often struggle to provide specialized resources and accommodations, leaving deaf students feeling isolated and left behind.

Emotional and Mental Health 

Living with deafness not only impacts a person's daily life but also takes a toll on their emotional and mental well-being. Feelings of isolation, anxiety, and depression are prevalent among the deaf community due to the stigma associated with disability and the lack of support systems.

02 Ideation

How-Might-We Question

In order to collect the previously gathered insights, we formulate “How might we" questions to divide the central information into problem spaces. We focussed on one “How might we" question in particular.
How might we overcome the sign language/spoken language barrier by using an existing technology, which is usable by anyone to make online communication easier in any type of way.

Sketches

In order to record our initial ideas, we use the "Crazy eight" method, which enables us to visualise and differentiate initial concepts.

03 Concept

Technology

To make our vision of an translating tool come to life, we decide to use the motion tracking technology to track the movement of an object (or person), applying tracking data for specific movements on to another object. This results a composition in which images and effects follow the motion.

Description

With help of hand-motion tracking the sign language will get translated into grammatically correct written sentences. Other features, such as using sign language to write and send emails or to make phone calls via voice output are now possible. It is now possible for deaf people to write grammatically correct articles in any way. 
The user should be then able to make calls (customer service)/ online communicate with hearing people (translator)/ write emails and texts with correct grammar (education).

Features

The ASL plug-in can be used in different ways and offers   various features

  • The iPad's camera automatically recognises when the plugin should come into effect, because the camera is activated as soon as both hands are recognised by the iPad and the user can start signing.

  • The user can decide during or after signing which words he wants to replace with certain slang words. The plugin recognises grammatically incorrect sentences and automatically changes them.

  • The plugin translates in both directions! From sign language to textual output; from textual input to sign language output by using an avatar!