What Is Visual Inspection?

Visual inspection is a task intended to find surface failures or defects. It’s commonly applied in sectors such as manufacturing, construction, and aerospace.

Automated Visual Inspection Systems

In the manufacturing industry, automated visual inspection systems with high-resolution cameras efficiently detect microscale or even nanoscale defects that are difficult for human eyes to pick up. However, false detection sometimes happens when an unknown or a variety of defects are present, which is a major challenge. Hence, software with deep learning technology such as MATLAB^® and the Computer Vision Toolbox™ Automated Visual Inspection Library is playing a more important role.

Screenshots of a mounted board and anomaly detection output calculated and visualized with MATLAB. — Mounting board with abnormalities highlighted using deep learning in MATLAB.

Developing an Automated Visual Inspection System with MATLAB and Computer Vision Toolbox

With MATLAB and the Computer Vision Toolbox Automated Visual Inspection Library, you can develop visual inspection systems. MATLAB supports the full workflow, from image acquisition to algorithm development and then deployment. Interactive apps in MATLAB help engineers explore, iterate, and automate algorithms to improve productivity. Engineers apply these techniques for many industrial applications.

Success Stories: Automated Visual Inspection in Automotive and Aerospace Industries

For example, Musashi Seimitsu Industry, an automotive parts manufacturer, was inspecting approximately 1.3 million parts per month with manual visual inspection. Using MATLAB to develop deep learning–based approaches to detect and localize different types of anomalies, it built an automated visual inspection system for inspecting bevel gears. The updated approach is expected to considerably reduce the company’s workload as well as its costs.

An automotive component bevel gear and inspection system built with MATLAB. — Musashi Seimitsu Industry uses deep learning for automated visual inspection of automotive parts with MATLAB.

Similarly, Airbus built a robust visual inspection artificial intelligence (AI) model for automatically detecting any defects in multiple aircraft components to ensure its airplanes have no defects before entering service. Using MATLAB simplified the process by enabling Airbus engineers to interactively prototype algorithms and test for defects in a short amount of time.

Screenshots of a partial anomaly being found with object detection developed with MATLAB at Airbus. — Airbus used MATLAB and built a robust visual inspection AI model for automatically detecting any defects in multiple aircraft components.

Read story

The Defect Detection Process: Data Preparation, AI Modeling, and Deployment

The defect detection process can be broken down into three main stages: data preparation, AI modeling, and deployment.

A diagram showing an end-to-end visual inspection workflow with examples of data preparation, AI modeling, and deployment in MATLAB, with iteration and refinement occurring between steps. — End-to-end automated visual inspection workflow with MATLAB.

For visual inspection, anomalous images are sometimes rare or diverse. In that case, you can train an anomaly detector, unsupervised learning that only needs normal images on training. If there are a sufficient number of abnormal images, supervised learning should work.

In the following section, you can learn the necessary steps and practical MATLAB functionalities for anomaly detection methods as unsupervised learning and object detection as supervised learning.

Data Preparation for Visual Inspection

Data comes from multiple sources and is usually unstructured and noisy, making data preparation and management difficult and time-consuming. Preprocessing images in the data set will result in higher accuracy in detecting anomalies.

Image Preprocessing with MATLAB

MATLAB has several apps to support various preprocessing techniques. For example, the Registration Estimator app in MATLAB lets you explore various algorithms to register misaligned images, making it easier for AI models to detect defects.

A screenshot of the Registration Estimator app aligning a pair of images of hex bolts to improve the accuracy of visual inspection. — The Registration Estimator app in MATLAB helps make image preprocessing easy.

MATLAB provides automation capabilities to accelerate the labeling process. For example, the Image and Video Labeler app can apply custom semantic segmentation or object detection algorithms to label regions or objects in an image or video frames.

AI Modeling for Visual Inspection

Characteristics of Anomaly Detection Methods Used in Visual Inspection

Available anomaly detectors for visual inspection include:

Fully convolutional data description (FCDD)
FastFlow
PatchCore

The following table distinguishes the characteristics and performance of these anomaly detection methods available through the Computer Vision Toolbox Automated Visual Inspection Library during training and inference.

Training Aspect	PatchCore	FastFlow	FCDD
Input image size	Small to medium preferred (due to memory limitation for large images)	Small to medium preferred (due to memory limitation for large images)	Small to large (high-resolution images)
Model size	Moderate to large (can vary with value of the compression ratio)	Moderate to large	Small (lightest weight model)
Performance speed	Fast	Fast	Fastest
Low-shot training regime	Supported	Not Supported	Not Supported

Anomaly Threshold and Explainable AI in Visual Inspection

For anomaly detection methods, the automated calculation of the anomaly threshold helps to avoid human-dependent variations in judgment. Evaluation with the explainable AI view enables to explore and interpret the classification results.

Screenshot of the viewAnomalyDetectionResults function that helps people understand and use AI models in visual inspection. — The explainable AI view is a key to success in implementing AI in visual inspection.

Pretrained Networks Used in Supervised Learning in Visual Inspection

When applying deep learning for supervised learning, there are two approaches. One approach is to build and train a deep network from scratch. The other is to adjust and fine-tune a pretrained neural network, also known as transfer learning. Both approaches are easy to implement in MATLAB.

Using the Deep Network Designer App to Build, Edit, and Train Deep Learning Networks

MATLAB provides the Deep Network Designer app, which lets you build, visualize, edit, and train deep learning networks. You can also analyze the network to ensure that the network architecture is defined correctly and detect problems before training.

Using pretrained networks from external platforms in MATLAB, you can import networks and network architectures from TensorFlow™, PyTorch^®, and from and to the ONNX™ model format. You can use these pretrained and edit them for transfer learning.

Screenshot of the Deep Network Designer app showing pretrained neural networks that can be used for visual inspection. — Lots of pretrained models are available in the Deep Network Designer app.

Object Detection to Detect and Localize Defects

Object detection like YOLOX is used to detect, localize, and classify defects in an image. The YOLOX object detection model is a single-stage, anchor-free technique that helps with small object detection, significantly reduces the model size, and improves computation speed compared to previous YOLO models. The exported data labeled with rectangular regions of interest (ROIs) is used to train object detection models.

Screenshot of the results of anomaly detection in PCB using the object detection method YOLOX. — YOLOX object detection to find small anomalies in visual inspection.

Deployment of the Visual Inspection System

Code Generation and Deployment Framework

Deep learning models must be incorporated into a larger system to be useful. MATLAB offers a code generation framework that allows models developed in MATLAB to be deployed anywhere, without having to rewrite the original model. This gives you the ability to test and deploy the model within an entire system.

Deployment to Embedded Hardware Platforms First

MATLAB enables you to deploy your deep learning networks to various embedded hardware platforms, such as NVIDIA^® GPUs, Intel^® and ARM^® CPUs, and Xilinx^® and Intel SoCs and FPGAs. With the help of MATLAB, you can explore and target embedded hardware easily.