Some built-in python modules are erased after call to clear classes

Hello,

I am trying to use a Python function from Matlab and I would like to put a Matlab structure in argument of the function.

MyStruct=struct('Test',1);
py.dict(MyStruct)

I have the following error:

Unable to resolve the name py.dict

I can create a numpy array in Matlab, or also import user defined module and it works. Python seems to be properly installed.

When I check Python environement I have the following:

          Version: "3.7"
       Executable: "...\Local\Programs\Python\Python37\python.EXE"
          Library: "...\Local\Programs\Python\Python37\python37.dll"
             Home: "...\Local\Programs\Python\Python37"
           Status: Loaded
    ExecutionMode: InProcess    

I am using Matlab R2019b.

Thank you for your help!

[Mikael Martin]

py.help('pandas')

This is also not working...

Unable to resolve the name py.help

Answer 1

[Mikael Martin]

I have just found a workaround.

It seems to work. My user psecific Python modules are updated and it does not erase some built-in classes.

I have created an intermediate function "Update_PythonModule()" and inside this function I call reloadPy().

function Update_PythonModule()
clear classes
P = py.sys.path;
% Add user specific python module
ModulePath='C:\...\Scripts';
% find python script in ModulePath
if count(P,ModulePath) == 0
    insert(P,int32(0),ModulePath);
end
reloadPy()
end

function reloadPy()
    warning('off','MATLAB:ClassInstanceExists')
    ScriptName='MyPythonScript';
    mod=py.importlib.import_module(ScriptName);
    py.importlib.reload(mod);
end

Answer 2

[Shubham]

Hi Mikael,

There is an update to the previous answer that I provided.

I am not encountering any errors when running the code snippet you have provided in MATLAB on my system.

The error message "Unable to resolve the name py.dict" suggests that Matlab is not able to find or import the dict function from Python correctly. This can be caused by a problem with the Python environment variables or path, or could be a compatibility issue with the version of Matlab and Python you are using.

To troubleshoot this issue, you may want to try the following:

1. Double-check that the correct version of Python is installed and listed in the Matlab environment variables, and that the Python executable path is set correctly.
2. Check the compatibility between the versions of Matlab and Python you are using. For example, not all Matlab versions may support the latest version of Python.

You can refer to this documentation to check the compatibility of python and MATLAB versions.

[Mikael Martin]

Hello Shubham,

Thank you very much for your answer and all your explanations.

I have now more info regarding this topic since I have in parallel contacted Matlab support.

To update any user specific Python script, we have to call the following function:

function reloadPy()
    warning('off','MATLAB:ClassInstanceExists')
    clear classes
    mod=py.importlib.import_module(MyModuleName); % call function using py.ML_NOx.xxx
    py.importlib.reload(mod);
end

The command

clear classes

is mandatory if you want to consider the latest version of your MyModuleName Python script.

But there is a bug in Matlab, still existing with the latest version. The "clear classes" clears some Python built-in module, which explains why after the call to reloadPy(), some python modules are not working anymore.

If I apply clear classes in Matlab workspace, python built-in modules are working again, but everything in the workspace is deleted.

A workaround provided by Matlab support is to run the command

terminate(pyenv)

after clear classes in the reloadPy() function. In addition to that, Python need to be executed in "OutOfProcess" Execution mode:

pyenv('ExecutionMode',"OutOfProcess")

I have tried this, but this is not solving my issue...

[Grace Kepler]

Hi Mikael,

The issue in this post was most likely caused by the behavior that you encountered with "clear classes", which left the py. package in an inconsistent state.

The behavior that you encountered with "clear classes" is not a bug and is expected, based on the way that you used the "clear classes" command.

The 'classes' section of the documentation, states that 'clear classes' issues a warning and does not clear a class of objects if any of those objects still exist after the workspace is cleared. This means, for instance, that objects can still exist in persistent variables of functions or figure windows.

In your case, you created a bool and complex objects in one function but used 'clear classes' in a different function, which means that the objects in the first function are not cleared before calling 'clear classes' in the second function. As a result, the 'clear classes' command is cut short, leaving the 'py' package in an inconsistent state.

To fix this, you need to clear the objects in the first function before calling 'clear classes' in the second function to ensure that the workflow is successful.

However, we realize that the affect of "clear classes" on the py. package may not be obvious from the documentation, and we will work to improve the documentation, so other users can avoid this confusion in the future.