How can I integrate the MATLAB testing framework with Team Foundation Server

We are attempting to introduce continuous integration into our MATLAB environment now that there is a full test suite of testing capabilities. Unfortunately, we are committed to using TFS as our CI tool. Using the TAPPlugin (or otherwise), is it possible to integrate with TFS such that builds automatically run the unit tests and fail the build if any test fails?

Answer 1

[Andy Campbell]

*** UPDATE ***

Another possibility you can try is setting up your own Jenkins Server (at least for simple cases this is not that hard). Then you can add the TFS plugin into your Jenkins server so that it polls TFS and runs your tests when the files change in TFS. The benefit here is that you can still automatically run all of your tests and be notified when they fail. The downside is that you have to work between these two systems.

The Jenkins TFS plugin is listed here .

*** END UPDATE ***

Hi Greg,

I am not sure if there exists a TFS plugin anywhere that enables TFS to read the TAP protocol.

This project looks promising, but it would require some development effort if it works. This project could potentially consume TestResults natively? Perhaps a plugin for the test framework could be written that uses the MATLAB .NET interface to talk to an assembly that implements these interfaces? Not sure whether that would work or not but maybe its a start.

I am definitely interested in whether this can be done so please let me know anything you find out.

-Andy

[Greg]

Hi Andy,

Thanks for the response. We did look into the possibility of using Jenkins as the CI server for our MATLAB environment, but we decided not to go down that route because all of our other internal development leverages TFS as the CI server and we don't want to introduce another system for a subset of our internal code base.

That being said, your link the to test automation plugin did indeed look interesting and led me down a different path. Rather than adding on to the project you referenced, I looked into leveraging built-in TFS extensibility. Turns out there are a set of interfaces that allow for the creation of custom test runners for Visual Studio and TFS. I followed the information contained in this blog post and was able to create a VERY basic test runner that will execute test functions in matlab and parse the .tap output. It is still very much a work in progress and will likely change as we learn more as an organization. I will update this post once I have some more formalized info to share :)

[Andy Campbell]

Thanks for sharing the info. Note that if you are doing your own analysis on the data you could also just reference the MATLAB TestResult properties. In other words, there may not be a need to actually write your own TAP Parser if you do not wish to. That being said, if you did invest in a TAP parser that could potentially allow other test frameworks to integrate into your CI system as well. Good luck and keep me posted. You've triggered and interest in TFS integration in myself so I will also continue to look for better solutions.

-Andy

Answer 2

[Swati Tiwari]

In R2015b, MATLAB has included a plugin for the matlab.unittest.TestRunner which produces test results in the JUnit-style XML format which can be integrated into continuous integration systems like Jenkins, Team Foundation Server and so on.

Hope that helps!