AI-Based Real-Time American Sign Language Recognition System Using Jetson Nano with Alphabet, Word and Gesture Detection
Specifications
- Code
- RA2614
- Department
- DKR
- Technology
- NVIDIA Jetson
- Theme
- AI & Computer Vision · Health & Assistive Care
- Student
- Nurul Hanisah binti Mohd Heruwan
- Supervisor
- Noor Nadiah bt Mohd Azali
Objectives
To design and develop a real-time American Sign Language (ASL) recognition system using Jetson Nano and a webcam. 2. To implement computer vision and MediaPipe hand landmark detection for recognizing ASL alphabets, selected words, and common hand gestures. 3. To display the recognized output through a graphical user interface (GUI) and provide text-to-speech technology
Scope of project
To develop a real-time American Sign Language (ASL) recognition system using Jetson Nano, a USB webcam, and MediaPipe hand tracking to recognize statis ASL alphabets (a-z), selected ASL words (e.g.,hello, thankyou, help) and commonhand gestures such as ILY ( i love you) sign to display the recognized output through a graphical user interface (GUI) and convert it into speech using text-to-speech (TTS)
Project description
The AI-Based Real-Time American Sign Language Recognition System Using Jetson Nano with Alphabet, Word and Gesture Detection is developed to facilitate communication between individuals who use American Sign Language (ASL) and those who are unfamiliar with sign language, helping to reduce communication barriers and improve accessibility. The system’s key processing flow begins with a USB webcam capturing hand signs, followed by image processing using Python and MediaPipe hand landmark detection to recognize static ASL alphabets, selected words, and common hand gestures in real time. The recognized signs are then processed and displayed as text through a graphical user interface (GUI), allowing individual letters and words to be combined into readable sentences. Finally, the system incorporates Text-to-Speech (TTS) technology to convert the generated text into speech when the user presses the ‘F’ key, enabling users to communicate their messages verbally to individuals who may not understand sign language.