.st0{fill:#FFFFFF;}

American Sign Language Vowel Recognition using AlexNet in MATLAB 

 December 30, 2020

By  Kadeeja Tamanna

Among the deaf and vocally impaired community, sign language acts as a medium to know people, build relationships, understand technology, etc. The use of hands and face plays a primary role in this kind of community. There is no universal sign language. Different sign languages exist in various countries or regions. American Sign Language (ASL) is one of the most predominantly used sign languages in the world. Unlike the hand gestures used in the vocal community to augment speech, the hand gestures in sign language have a predefined highly formalized norm for each sign. Exactly as how spoken language has a vocabulary of words, sign language has a vocabulary of gestures. This allows the construction of sign-language dictionaries. Each sign of the language is equated to one or more words in a spoken language.  

Our objective is to use ASL alphabets, also known as the American Manual Alphabet (AMA) data, and build a recognition system capable of classifying the given ASL data into spoken language. 

You will learn to build a deep learning model capable of recognizing these alphabets using AlexNet in the coming sections. 

Input Data

The data set is a collection of alphabet images from A-Z from the American Sign Language, downloaded from Kaggle. The training data set contains images that are 200x200 pixels. We will be using only the vowel data set for our task that is A, E, I, O, and U. The hand gestures for vowels are as shown below.

Hand Gestures for Vowels in ASL

Source: Google

Each alphabet consisted of about 3000 images used for training the neural network. A relatively new function, ImageDatastore, was used to import this data into MATLAB. The advantage of using this function is it enables you to import or manage a collection of image files in batches that are too large to fit in memory.

Resizing Image 

Resizing an image is a part of image interpolation wherein you resize an image from the one-pixel grid to another without losing image quality. Interpolation works by using the known data to estimate values at unknown points.   

When you need to increase or decrease the total number of pixels in an image, the concept of image resizing comes into the picture. In MATLAB, we use the imresize function to do the following.   

As mentioned earlier, we are using an AlexNet designed for images of size 227x227x3. Hence, it would be ideal if the training images are of the same dimensions while using the network. 

AlexNet 

We will be using a deep neural network named AlexNet to recognize the vowel shown by sign language. To know what a deep neural network is and how it works, check our Blog of Object Recognition Using MATLAB.

AlexNet is the most commonly used Convolution Neural Network (CNN) for object recognition. The architecture consists of 5 convolutional layers, 3 max-pooling layers, 2 normalization layers, 2 fully connected layers, and 1 SoftMax layer. The network overall has 60 million parameters.  

AlexNet is a neural network, already trained on a million images. Hence, directly can be used for testing. But in our case, we are going to fine-tune a pre-trained AlexNet convolutional neural network to perform classification on a new collection of images. Let us see the steps in doing this,  

  • First, we begin with loading the pre-trained AlexNet neural network.  
  • The last three layers of the pre-trained network are configured for 1000 classes. But, for our classification problem, the 23rd and the 25th layer must be fine-tuned. Extract all layers, and then replace the following layers from the pre-trained network.  
  • The 23rd layer with a fully connected layer, and the 25th layer with the classification output layer. Set the fully connected layer to have the same size as the number of classes in the new data.  
  • Specify the training options. For transfer learning, keep the features from the early layers of the pre-trained network. Also, you do not need to train for as many epochs. Specify the mini-batch size and initial learn rate.  
  • Train the network using the transferred and new layers.  
  • Finally, save the trained network.   
Steps to reuse AlexNet

Source: MathWorks

Requirements 

You can use AlexNet to classify objects in any image. In this example, we'll use it to identify the sign language images saved in the ImageDatastore object. In addition to MATLAB, we'll be using the following: 

  • Deep Learning Toolbox 
  • Support package for using AlexNet 

MATLAB Code 

Get Access to
Codes, Training & Test Images

Build an Americal Sign Language, i.e., ASL Recognition system capable of classifying vowels using Deep Learning with AlexNet in MATLAB; Developed in MATLAB R2019a with Deep Learning Toolbox and AlexNet

Output

H9+XW2ozlQAAAAABJRU5ErkJggg== | American Sign Language Vowel Recognition using AlexNet in MATLAB | MATLAB Helper ®

Conclusion 

In this blog, we have presented a system that is capable of recognizing vowels in ASL. The code worked perfectly without any errors for all the test data. This task can be extended to wide-ranging applications using the ASL data, some of which are for you to try.  

  • American Sign Language alphabet Recognition using AlexNet in MATLAB.  
  • American Sign Language number Recognition using AlexNet in MATLAB.  
  • Build a system capable of converting a complete sentence word by word using AlexNet in MATLAB.  

Did you find some helpful content from our video or article and now looking for its code, model, or application? You can purchase the specific Title, if available, and instantly get the download link.

Thank you for reading this blog. Do share this blog if you found it helpful. If you have any queries, post them in the comments or contact us by emailing your questions to [email protected]. Follow us on LinkedIn Facebook, and Subscribe to our YouTube Channel. If you find any bug or error on this or any other page on our website, please inform us & we will correct it.

If you are looking for free help, you can post your comment below & wait for any community member to respond, which is not guaranteed. You can book Expert Help, a paid service, and get assistance in your requirement. If your timeline allows, we recommend you book the Research Assistance plan. If you want to get trained in MATLAB or Simulink, you may join one of our training modules. 

If you are ready for the paid service, share your requirement with necessary attachments & inform us about any Service preference along with the timeline. Once evaluated, we will revert to you with more details and the next suggested step.

Education is our future. MATLAB is our feature. Happy MATLABing!

About the author 

Kadeeja Tamanna

An engineering graduate, who is passionate to employ her knowledge and skills, in developing and providing solutions to technological requirements in the healthcare domain. Proficient in signal processing, data analysis, statistical analysis, computational modeling using MATLAB programming. Currently a MATLAB Developer intern at MATLAB Helper ®.

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}

MATLAB Helper ®

Follow: YouTube Channel, LinkedIn Company, Facebook Page, Instagram Page

Join Community of MATLAB Enthusiasts: Facebook Group, Telegram, LinkedIn Group

Use Website Chat or WhatsApp at +91-8104622179

>