Among the deaf and vocally impaired community, sign language acts as a medium to know people, build relationships, understand technology, etc. The use of hands and face plays a primary role in this kind of community. There is no universal sign language. Different sign languages exist in various countries or regions. American Sign Language (ASL) is one of the most predominantly used sign languages in the world. Unlike the hand gestures used in the vocal community to augment speech, the hand gestures in sign language have a predefined highly formalized norm for each sign. Exactly as how spoken language has a vocabulary of words, sign language has a vocabulary of gestures. This allows the construction of sign-language dictionaries. Each sign of the language is equated to one or more words in a spoken language.
Our objective is to use ASL alphabets, also known as the American Manual Alphabet (AMA) data, and build a recognition system capable of classifying the given ASL data into spoken language.
You will learn to build a deep learning model capable of recognizing these alphabets using AlexNet in the coming sections.
The data set is a collection of alphabet images from A-Z from the American Sign Language, downloaded from Kaggle. The training data set contains images that are 200x200 pixels. We will be using only the vowel data set for our task that is A, E, I, O, and U. The hand gestures for vowels are as shown below.
Each alphabet consisted of about 3000 images used for training the neural network. A relatively new function, ImageDatastore, was used to import this data into MATLAB. The advantage of using this function is it enables you to import or manage a collection of image files in batches that are too large to fit in memory.
Resizing an image is a part of image interpolation wherein you resize an image from the one-pixel grid to another without losing image quality. Interpolation works by using the known data to estimate values at unknown points.
When you need to increase or decrease the total number of pixels in an image, the concept of image resizing comes into the picture. In MATLAB, we use the imresize function to do the following.
As mentioned earlier, we are using an AlexNet designed for images of size 227x227x3. Hence, it would be ideal if the training images are of the same dimensions while using the network.
We will be using a deep neural network named AlexNet to recognize the vowel shown by sign language. To know what a deep neural network is and how it works, check our Blog of Object Recognition Using MATLAB.
AlexNet is the most commonly used Convolution Neural Network (CNN) for object recognition. The architecture consists of 5 convolutional layers, 3 max-pooling layers, 2 normalization layers, 2 fully connected layers, and 1 SoftMax layer. The network overall has 60 million parameters.
AlexNet is a neural network, already trained on a million images. Hence, directly can be used for testing. But in our case, we are going to fine-tune a pre-trained AlexNet convolutional neural network to perform classification on a new collection of images. Let us see the steps in doing this,
- First, we begin with loading the pre-trained AlexNet neural network.
- The last three layers of the pre-trained network are configured for 1000 classes. But, for our classification problem, the 23rd and the 25th layer must be fine-tuned. Extract all layers, and then replace the following layers from the pre-trained network.
- The 23rd layer with a fully connected layer, and the 25th layer with the classification output layer. Set the fully connected layer to have the same size as the number of classes in the new data.
- Specify the training options. For transfer learning, keep the features from the early layers of the pre-trained network. Also, you do not need to train for as many epochs. Specify the mini-batch size and initial learn rate.
- Train the network using the transferred and new layers.
- Finally, save the trained network.
You can use AlexNet to classify objects in any image. In this example, we'll use it to identify the sign language images saved in the ImageDatastore object. In addition to MATLAB, we'll be using the following:
- Deep Learning Toolbox
- Support package for using AlexNet
Get Access to
Codes, Training & Test Images
Build an Americal Sign Language, i.e., ASL Recognition system capable of classifying vowels using Deep Learning with AlexNet in MATLAB; Developed in MATLAB R2019a with Deep Learning Toolbox and AlexNet
In this blog, we have presented a system that is capable of recognizing vowels in ASL. The code worked perfectly without any errors for all the test data. This task can be extended to wide-ranging applications using the ASL data, some of which are for you to try.
- American Sign Language alphabet Recognition using AlexNet in MATLAB.
- American Sign Language number Recognition using AlexNet in MATLAB.
- Build a system capable of converting a complete sentence word by word using AlexNet in MATLAB.
Get instant access to the code, model, or application of the video or article you found helpful! Simply purchase the specific title, if available, and receive the download link right away! #MATLABHelper #CodeMadeEasy
Ready to take your MATLAB skills to the next level? Look no further! At MATLAB Helper, we've got you covered. From free community support to expert help and training, we've got all the resources you need to become a pro in no time. If you have any questions or queries, don't hesitate to reach out to us. Simply post a comment below or send us an email at [email protected].
And don't forget to connect with us on LinkedIn, Facebook, and Subscribe to our YouTube Channel! We're always sharing helpful tips and updates, so you can stay up-to-date on everything related to MATLAB. Plus, if you spot any bugs or errors on our website, just let us know and we'll make sure to fix it ASAP.
Ready to get started? Book your expert help with Research Assistance plan today and get personalized assistance tailored to your needs. Or, if you're looking for more comprehensive training, join one of our training modules and get hands-on experience with the latest techniques and technologies. The choice is yours – start learning and growing with MATLAB Helper today!
Education is our future. MATLAB is our feature. Happy MATLABing!