Reconocimiento y clasificación de lenguaje de señas usando Kinect e inteligencia artificial

Abstract

In the world there is about 466 millions of people with deafness, it represents more than 5% of global population . These people have several inconvenient to communicate with others who do not speak the sign language. This issue as social isolation and prevents equal growth. This project is based on the needs for people who do not speak this type of language, so they can understand the signs made by someone deaf through a software which uses artificial intelligence and a 3D sensor. In this research, the hand trajectories obtained with the help of the 3D sensor are used for training, validation and test of an artificial intelligence algorithm based on neural network. The network is responsible for recognition and classification of 5 basic signs determined with the help of an expert in this field. The neural network is a multilayer perceptron trained by back propagation. The training is done with 6 people and the test with 9 additional. In the simulation phase, the system has an effectiveness in recognition and classification of 99.6%; while, in the field test of 98.7%. In addition, an HMI was developed for data acquisition and the presentation of results.