What do I see performance and numerical accuracy issues with quantized INT8 deep learning networks using GPU Coder in R2021a?

I’m generating code for a quantized deep learning network using GPU Coder but experiencing performance and numerical accuracy issues when using INT8 precision with cuDNN 8.

What versions of cuDNN are supported by GPU Coder in R2021a?

Answer 1

[Bill Chou]

In R2021a, GPU Coder supports cuDNN 8.1.0. For more information, see Installing Prerequisite Products (GPU Coder). It is recommended to use this version of cuDNN as other versions have significant performance and accuracy issues with INT8 workflows.

When using GPU Coder with cuDNN 8.0.x to generate CUDA code for a quantized deep learning network in INT8 precision, you may experience different issues depending on the version of cuDNN 8.0 used. The table below summarizes the issues you may experience.