Talking with signs: a simple method to detect nouns and numbers in a non annotated signs language corpus

• Autor: Huiza Pereyra, Eric Raphael
• Formato: tesis de maestría
• Fecha de Publicación: 2020
• Institución: Pontificia Universidad Católica del Perú
• Repositorio: PUCP-Institucional
• Lenguaje: inglés
• OAI Identifier: oai:repositorio.pucp.edu.pe:20.500.14657/172359
• Enlace del recurso: http://hdl.handle.net/20.500.12404/16906
• Nivel de acceso: acceso abierto
• Materia: Redes neuronales (Computación); Algoritmos computacionales; Reconocimiento óptico de patrones; https://purl.org/pe-repo/ocde/ford#1.02.00

People with deafness or hearing disabilities who aim to use computer based systems rely on state-of-art video classification and human action recognition techniques that combine traditional movement pat-tern recognition and deep learning techniques. In this work we present a pipeline for semi-automatic video annotation applied to a non-annotated Peru-vian Signs Language (PSL) corpus along with a novel method for a progressive detection of PSL elements (nSDm). We produced a set of video annotations in-dicating signs appearances for a small set of nouns and numbers along with a labeled PSL dataset (PSL dataset). A model obtained after ensemble a 2D CNN trained with movement patterns extracted from the PSL dataset using Lucas Kanade Opticalflow, and a RNN with LSTM cells trained with raw RGB frames extracted from the PSL dataset reporting state-of-art results over the PSL dataset on signs classification tasks in terms of AUC, Precision and Recall.