How to Return a Opencv image cv::mat to Matlab with the Mex C++ API

Iam currently using the Mex C++ API to take a picture with a Basler camera (Pylon API) and send it back to Matlab. I have some issues with converting an opencv cv::mat data type to a suitable type for Matlab. The solutions provided by Matlab (https://de.mathworks.com/help/vision/ug/opencv-interface.html) for converting opencv data types unfortunately only work with the older mex c api, so I cannot use them.

I have managed the following workaround. This workaround uses two loops two assign the values to a matlab::data::TypedArray. This solution is very slow due to the two loops and the copy of the values. Can someone recommend another way which is faster or works without loops? See the extract code from my c++ mex file with ne mex c++ API:

#include <opencv2/core/core.hpp>
#include <opencv2/opencv.hpp>
#include <pylon/PylonIncludes.h>
#include <pylon/usb/PylonUsbIncludes.h>
#include <pylon/usb/BaslerUsbInstantCamera.h>
#include <pylon/PylonUtilityIncludes.h>
#include "mex.hpp"
#include "mexAdapter.hpp"
#include <chrono>
#include <string>

class MexFunction : public matlab::mex::Function{
public:
    MexFunction(){}
void operator()(ArgumentList outputs, ArgumentList inputs) {
CGrabResultPtr ptrGrabResult

... Here I take a picture with the camera and save it to "ptrGrabResult"...


Mat openCvImage; 
CImageFormatConverter formatConverter; 
CPylonImage pylonImage; 

// First convert the "ptrGrabResult" to a pylonImage 
formatConverter.Convert(pylonImage, ptrGrabResult); 

// Convert it to a openc picture 
Mat openCvImage = cv::Mat(ptrGrabResult->GetHeight(), ptrGrabResult->GetWidth(), CV_8UC1,(uint8_t *)pylonImage.GetBuffer(), Mat::AUTO_STEP); 

const size_t rows = openCvImage.rows; 
const size_t cols = openCvImage.cols; 

matlab::data::TypedArray<uint8_t> Yp = factory.createArray<uint8_t>({ rows, cols }); 

for(int i = 0 ;i < openCvImage.rows; ++i){ 
  for(int j = 0; j < openCvImage.cols; ++j){ 
    // [Row][Coloumn] = (row, column) 
    Yp[i][j] = openCvImage.at<uint8_t>(i,j);; 
  } 
} 

outputs[0] = Yp; 
}
}

The problem with this solution is that it takes way too long for me. Is there a faster and convenient method to cast the cv:mat type without a loop?? Or maybe it is possible to use pointer in this case, but I was not able to implement this...

Answer 1

[cui,xingxing]

you can use opencv ptr pointer to convert cv::Mat / matlab matrix each other, for example, Matlab matrix conert to opencv Mat type as follows:

 //  convert matlab matrix to opencv Mat，and vice versa, fishEyeImg is Matlab image matrix,oriImg is cv::Mat
        cv::Mat oriImg;
        bool is3Channels = fishEyeImg.getDimensions()[2] == 3;
        if (is3Channels)
        {
            oriImg = cv::Mat::zeros(oriImgS, CV_8UC3);
            for (size_t i = 0; i < fishEyeImg.getDimensions()[0]; i++) {
                cv::Vec3b* data = oriImg.ptr<cv::Vec3b>(i);
                for (size_t j = 0; j < fishEyeImg.getDimensions()[1]; j++) {
                    data[j] = cv::Vec3b((uchar)fishEyeImg[i][j][2], (uchar)fishEyeImg[i][j][1], (uchar)fishEyeImg[i][j][0]);
                }
            }
        }
        else
        {
            oriImg = cv::Mat::zeros(oriImgS, CV_8UC1);
            for (size_t i = 0; i < fishEyeImg.getDimensions()[0]; i++) {
                uchar* data = oriImg.ptr<uchar>(i);
                for (size_t j = 0; j < fishEyeImg.getDimensions()[1]; j++) {
                    data[j] = (uchar)fishEyeImg[i][j];
                }
            }
        } 
        

References:

https://www.mathworks.com/matlabcentral/answers/1806995-enhancement-build-in-function-detectcheckerboardpoints-unsatisfactory-corners-results#answer_1056045

-------------------------Off-topic interlude, 2024-------------------------------

I am currently looking for a job in the field of CV algorithm development, based in Shenzhen, Guangdong, China,or a remote support position. I would be very grateful if anyone is willing to offer me a job or make a recommendation. My preliminary resume can be found at: https://cuixing158.github.io/about/ . Thank you!

Email: [e-mail]

Answer 2

[cui,xingxing]

perhaps these can help you if you use MATLAB2022a or later.

uchar *data= openCvImage.data;
matlab::data::TypedArray<uint8_t> Yp = factory.createArray({rows, cols},data,data+rows*cols,MemoryLayout::ROW_MAJOR);

Since R2022a, the "inputLayout" parameter syntax has been added to create arrays.

template <typename ItType, typename T>

TypedArray<T> createArray(ArrayDimensions dims,

ItType begin,

ItType end,

InputLayout inputLayout)

References:

1、MATLAB Data API Random Access Iterators

2、matlab::data::ArrayFactory

-------------------------Off-topic interlude, 2024-------------------------------

I am currently looking for a job in the field of CV algorithm development, based in Shenzhen, Guangdong, China,or a remote support position. I would be very grateful if anyone is willing to offer me a job or make a recommendation. My preliminary resume can be found at: https://cuixing158.github.io/about/ . Thank you!

Email: [e-mail]