Desarrollo de un intérprete de 20 frases en lengua de señas ecuatoriana a voz en tiempo real usando redes neuronales recurrentes.

Abstract

Nowadays, deaf signers have several problems to carry out their daily activities normally. The independence and accessibility of these people is affected as they usually need to be accompanied by a speaking person to help them in their daily life. This problem has negative effects on the quality of life of deaf signers. This project seeks to develop a real-time interpreter of 20 sentences of Ecuadorian sign language into speech using artificial intelligence to facilitate communication between deaf and hearing people. This interpreter will have a simple implementation so that it can be portable and, in the future, turn it into a mobile application. For the interpretation process in this work, a combination of a convolutional neural network and a recurrent neural network (CNN-LSTM) was used. The first one extracts the features from an input image that will be the region of interest obtained by Google's MediaPipe library. The second uses the feature vector as input to analyze and memorize the image sequences in the training videos to predict the signed sentence.