Dependency management & package manager like pip in ML

30 visualizaciones (últimos 30 días)
Jan Kappen
Jan Kappen el 6 de Feb. de 2023
Comentada: Michelle Hirsch el 15 de Mzo. de 2024 a las 11:41
Hi all,
Matlab is officially still lacking a proper package manager like npm or pip. I do not understand how people structure projects with dependencies. I understand, there are
  • ML projects
  • ML toolboxes
  • several mpm versions
Toolboxes have a version number attached, projects have not. Both can be shared as a package. Projects can reference other projects, but that 1. requires the same folder structure on every developer PC and 2. requires that the correct version is checked out from the VCS (which is GIT in our case). There's no way to define "project abc@v1.2.0" as a dependency. Just folders/prj files.
How do you do complex app developments? Like a frontent GUI app that needs several self-created toolboxes and projects? The only way is to use GIT (submodules in worst case) to do dependency managment, and that's definitely a red flag.
Then, there's Matlab's mpm matlab-dockerfile/MPM.md at main · mathworks-ref-arch/matlab-dockerfile (github.com) but that's meant to be used to install different product versions and addons, and there's a third party mpm mobeets/mpm: Simple Matlab package management inspired by pip (github.com) which seems to be interesting, but there's no direct support from Mathworks.
Then, it seems like there's another mpm built-in in ML 2022b that mentions a Matlab -packages flag which is very interesting:
mpm
Error using mpm
This command is not supported when the packages feature is not enabled. Start MATLAB with the -packages command line switch.
There's no information about that. Starting ML with -packages reveals more:
Which seems to support different versions. But there's no documentation anywhere, and it's a built-in function © 2022.
What does that mean? Will there be a Mathworks' supported package manager? What exactly is a package by the way in ML? So far, packages were a synonym to namespaces, using the +package folders. How does that work together with ML projects?
I'm in the process of re-structuring code due to a migration from Bitbucket to Gitlab and I'm at a point where a proper dependency management must take place, and that must work in CI/CD, too.
Would be very happy to hear your thoughts.
Thanks,
Jan
  1 comentario
Jan Kappen
Jan Kappen el 9 de Ag. de 2023
Any updates from Mathworks?
I'd really like to know if theres something on its way or if I have to create my own dependency manager, probably in Python.
Thanks

Iniciar sesión para comentar.

Respuestas (1)

Michelle Hirsch
Michelle Hirsch el 9 de Ag. de 2023
We are working on a package management system for MATLAB. The mpm comand you found in MATLAB was from an early beta test we ran in R2022b to flesh out some of the basic principles of the system we are designing.
I think the best bet would be for us to connect you with an appropriate expert or two at MathWorks to talk through ideas of how to best approach meeting your needs now while setting yourself up to adopt our system when it is ready. I'll email you through your author profile to make the connection.
  2 comentarios
Ernst van der Pols
Ernst van der Pols el 15 de Mzo. de 2024 a las 9:51
Thanks Jan for raising this question and Michelle for giving an update.
We at KROHNE have a Nexus server for storage and distribution of software artifacts, and are still in the process of introducing this technology in the various software-creating communities within the company. Mainly .NET (nuget), and a little PowerShell (nuget) and Python (pip) are already there.
Leveraging this for our MATLAB software would mean a great boost for improving the quality and ease of sharing the abundance of MATLAB scripts and models.
I looked into some options to distribute MATLAB Toolbox packages via Nexus. Since a MATLAB Toolbox (.mltbx) is an OPC container, I tested the combination of cloaking an mltbx as NuGet package, which uses also OPC technology. With cloaking I mean building the package such that it can be handled by both MATLAB and NuGet, with at most only a manual fix of the package file name extension (from .nupkg to .mltbx vv.). Alas, this had no satisfactory results.
NuGet is already not only used for .NET packaging, but also for PowerShell packaging. Currently using the Semantic Versioning standard and a rather simpel Software-Bill-Of-Material (nuspec), it is easy to use, and to integrate in a build process.
Next best option would be to wrap the mltbx in a NuGet package, but before starting that up we first focus on the creation of a Toolbox in the first place. Hopefully we can expect some managed versioning solution that we can employ in-house soon.
Michelle Hirsch
Michelle Hirsch el 15 de Mzo. de 2024 a las 11:41
@David Sampson do you have any advice you could offer Ernst? I'm guessing you've supported others using NuGet.

Iniciar sesión para comentar.

Categorías

Más información sobre Startup and Shutdown en Help Center y File Exchange.

Productos


Versión

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by