What Is Image Recognition?

3 things you need to know

Image recognition is the process of identifying an object or a feature in an image or video. It is used in many applications like defect detection, medical imaging, and security surveillance.

Why Image Recognition Matters

Image recognition can speed up tedious tasks and process images faster or more accurately than manual image inspection. Image recognition is a crucial technique in many applications, and is the main driver in deep learning applications like:

Visual Inspection: Identifying parts as defective or non-defective in manufacturing can quickly inspect thousands of parts on an assembly line.
Image Classification: Categorizing images based on the image content. This is especially useful in applications such as image retrieval and recommender systems in e-commerce.
Automated Driving: The ability to recognize a stop sign or a pedestrian in an image is crucial to autonomous driving applications.
Robotics: Image recognition can be used by robots to identify objects and enhance autonomous navigation by identifying locations or objects on their path.

Image recognition in a visual inspection application for part defects.

Image recognition is the core technology at the center of these applications. It identifies objects or scenes in images and uses that information to make decisions as part of a larger system. Image recognition is helping these systems become more aware, essentially enabling better decisions by providing insight to the system.

Image Recognition vs. Object Detection

Image recognition and object detection are similar techniques and are often used together. Image recognition identifies which object or scene is in an image; object detection finds instances and locations of those objects in images.

Common object detection techniques are Faster R-CNN and YOLOv3.

Image recognition (left) and object detection (right).

What is Object Detection? (3:20)

Getting Started with Object Detection Using Deep Learning

How It Works

Techniques for Image Recognition

There are many methods for image recognition, including machine learning and deep learning techniques. The technique you use depends on the application but, in general, the more complex the problem, the more likely you will want to explore deep learning techniques.

Image Recognition Using Deep Learning

A deep learning approach to image recognition can involve the use of a convolutional neural network to automatically learn relevant features from sample images and automatically identify those features in new images.

A typical deep learning workflow for image recognition:

Prepare Training Data: Start with a collection of images and compile them into their associated categories. This could also include any preprocessing steps to make the images more consistent for a more accurate model.

Create a Deep Learning Model: While you can build a deep learning model from scratch, it may be best to start with a pretrained model that you can use as a starting point for your application. (Learn more about pretrained models.

Train the Model: Model training involves presenting the test data to the model. The model then iterates over the data multiple times and automatically learns the most important features relevant to the images. As the training continues, more sophisticated features will be learned by the model, until the model can accurately decipher between the classes of images in the training set.

Test Data: Test on new data that the model hasn’t seen before to see what the model believes the image to be. If results are not what you are expecting, iterate on these four steps until the accuracy is more acceptable.

Deep learning techniques may sound complicated, but simple examples are a great way of getting started and learning more about the technology.

Introduction to Deep Learning: What Is Deep Learning? (3:33)

Introducing Deep Learning with MATLAB

Deep Learning with MATLAB: Deep Learning in 11 Lines of MATLAB Code (2:38)

Create Simple Image Classification Network

Tip: Deep learning techniques are popular for image recognition because they provide highly accurate and robust results. Deep learning tends to work best with a large amount of training data, and techniques such as transfer learning can simplify the image recognition workflow. Deep Learning Toolbox™ provides a framework for designing and implementing deep neural networks with algorithms, pretrained models, and apps.

Image Recognition Using Machine Learning

A machine learning approach to image recognition involves identifying and extracting key features from images and using them as input to a machine learning model.

Train Data: You start with a collection of images and compile them into their associated categories.
Extract Features: Select the relevant features in each image. A feature extraction algorithm might extract edge or corner features that can be used to differentiate between classes in your data.
Create a Machine Learning Model: These features are added to a machine learning model, which will separate these features into their distinct categories, and then use this information when analyzing and classifying new objects.

Machine learning workflow for image recognition.

You can use a variety of machine learning algorithms and feature extraction methods, which offer many combinations to create an accurate object recognition model.

Machine learning example with image recognition to classify digits using HOG features and an SVM classifier.

Tip: Using machine learning for object recognition offers the flexibility to choose the best combination of features and classifiers for learning. It can achieve accurate results with minimal data. Statistics and Machine Learning Toolbox™ provides functions and apps to describe, analyze, and model data.

Image Recognition: Deep Learning vs. Machine Learning

How do you know when to use deep learning or machine learning for image recognition? At a high level, the difference is manually choosing features with machine learning or automatically learning them with deep learning.

Deep Learning and Traditional Machine Learning: Choosing the Right Approach

Introduction to Deep Learning: Machine Learning vs. Deep Learning (3:47)

Image Recognition Using Traditional Image Processing Techniques

Aside from deep learning and machine learning, many classic image processing methods are very effective at image recognition for some applications. Image processing techniques tend to be well suited to “pixel-based” recognition applications such as:

Color-based image recognition: Often color can provide an excellent feature for image recognition. Features such as hue, saturation, and value (HSV) or red, green, blue (RGB) can provide insight into an image.
Template matching: This technique uses a small image, or template, to find matching regions in a larger image.
Image segmentation and blob analysis: This uses simple object properties, such as size, color, or shape.

Tip: Typically, if an object can be recognized using a simple approach like image segmentation, it’s best to start by using that approach. You may have a robust solution that does not require hundreds or thousands of training images or a complicated solution. Image Processing Toolbox™ may have all the tools you need.

Image Recognition with MATLAB

MATLAB^® simplifies the hard parts of image recognition.

Image labeling apps: Clean, preprocessed data ensures the best opportunity for success with image recognition. With the Image Labeler app, you can automate the process of cropping and labeling images.

Label images and videos interactively.

Exploring deep learning and machine learning algorithms: When first starting out, it may be unclear whether to use deep learning or machine learning techniques. MATLAB offers the ability to try all combinations of approaches. Explore pretrained models for deep learning, or machine learning classification algorithms.

You can interoperate with networks and network architectures from frameworks like TensorFlow™, Keras, PyTorch and Caffe2 using ONNX™ (Open Neural Network Exchange) import and export capabilities.

Integrate with Python-based frameworks.

Automatic code generation for deployment: Ultimately, your algorithm may need to live outside the desktop environment. MATLAB provides code generation tools to deploy your image recognition algorithm anywhere: the web, embedded hardware, or production servers.

After creating your algorithms, you can use automated workflows to generate TensorRT or CUDA^® code with GPU Coder™ for hardware-in-the-loop testing. The generated code can be integrated with existing projects and can be used to verify object detection algorithms on desktop GPUs or embedded GPUs such as the NVIDIA^® Jetson or NVIDIA Drive platform.