Matlab engine starting error in Python (docker)

Hi,

I am using Matlab with Python in a docker (cloud) environment. The installation was okay. But starting matlab engine (2022b) in Python (3.8) returns the following error:

>>> import matlab.engine
>>> eng = matlab.engine.start_matlab()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/matlab/.local/lib/python3.8/site-packages/matlab/engine/__init__.py", line 130, in start_matlab
    eng = future.result()
  File "/home/matlab/.local/lib/python3.8/site-packages/matlab/engine/futureresult.py", line 67, in result
    return self.__future.result(timeout)
  File "/home/matlab/.local/lib/python3.8/site-packages/matlab/engine/matlabfuture.py", line 87, in result
    handle = pythonengine.getMATLAB(self._future)
matlab.engine.EngineError: Connection to process with Exchange: " " was lost.

Using background = True didn't work as well:

>>> future = matlab.engine.start_matlab(background=True)
>>> eng = future.result()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/matlab/.local/lib/python3.8/site-packages/matlab/engine/futureresult.py", line 67, in result
    return self.__future.result(timeout)
  File "/home/matlab/.local/lib/python3.8/site-packages/matlab/engine/matlabfuture.py", line 87, in result
    handle = pythonengine.getMATLAB(self._future)
matlab.engine.EngineError: Connection to process with Exchange: " " was lost.

Can anyone help me identify the issue here? Thanks in advance!

Answer 1

[Rajeev]

The error message indicates that the connection to the MATLAB engine has been lost. This can happen for a number of reasons that include network issues, memory limitations, or other system constraints.

Here are some steps you can try to resolve this issue:

1. Increase the available memory for the Docker container. MATLAB can be memory-intensive, so if the container doesn't have enough memory allocated, it may cause the connection to be lost.
2. Check your network connection. Make sure the Docker container has access to the internet and that there are no network restrictions that may interfere with the MATLAB engine.
3. Try using a different version of MATLAB. Some versions of MATLAB may have compatibility issues with certain Python environments. You can refer to System Requirements for MATLAB Engine API for Python - MATLAB & Simulink (mathworks.com) and Versions of Python Compatible with MATLAB Products by Release - MATLAB & Simulink (mathworks.com)
4. Try starting the MATLAB engine with the -nojvm option. This will start the MATLAB engine without the Java Virtual Machine,that can help reduce memory usage and improve stability.
5. Try using a different Python version. Some users have reported issues with MATLAB engine in Python 3.8, so it may be worth trying a different version of Python, such as 3.7 or 3.9.

Have a nice day :)

[J.]

Hi @Rajeev thanks for the reply. I think I wil try 4 and 5. 2 and 3 should not be a problem in my case. Also, in 1, what does 'memory' mean here? It seems "By default, Docker doesn't set any resource constraints on a container. It allows the container to access all system resources." So it is not clear to me why it could be an issue. Can you give more explanation? Thanks a lot!

[Rajeev]

Hi @J.

You are correct that there is docker does not set any constraints on a container.

I meant the memory of your host system in this case.

You can refer to the docker documentation: https://docs.docker.com/config/containers/resource_constraints/#understand-the-risks-of-running-out-of-memory

and this stackoverflow answer: How to assign more memory to docker container - Stack Overflow for more info.

[J.]

This should not be a problem in my case. The host has 500+GB RAM.