Date of Publication :8th January 2024

Abstract: SignSpectra: Sign Language Translator is a multi-phase AI /ML project dedicated to bridging the communication gap between the deaf/mute community and the hearing population. This system utili zes Indian Sign Language (ISL) as the primary medium for translation, transforming sign language into spoken English, thereby fostering inclusive interactions. The project is organized into three developmental phases: Phase 1 (P1) involves training models on a dataset containing alphabets and numbers; Phase 2 (P2) expands on this foundation with a dataset of commonly used words; and Phase 3 (P3) further extends the system’s functionality by incorporating sentence structures. In Phase 1, AlexNet and VGG19 architectures were employed to classify alphabets and numbers. For Phase 2, AlexNet was used exclusively, leveraging its performance with word-level classification. Phase 3 adopts a more com plex approach, utilizing EfficientNet B0 and LSTM architectures to handle sentence level translations, achieving a higher degree of contextual accuracy for natural communication flow. Throughout the project, careful attention was paid to model overfitting and accuracy optimization, with techniques such as early stopping, splitting the dataset into train, test, validation, and limiting the number of files trained per epoch. The entire system was developed and tested across multiple ID Es, including Spyder, Google Colab, and Kaggle. Each phase has been completed successfully, demonstrating the system’s capability to translate ISL into spoken English, thereby enhancing accessibility for the deaf/mute community.

Reference :