Exception Safety and Incremental Teardown for Unit Tests

Under help topic "Write Setup and Teardown Code Using Classes", it is recommended that instead of using teardown method, "addTeardown" should be used because (based on the help content) "This guarantees the Teardown is executed in the reverse order of the setup and also ensures that the test content is exception safe".

What can go wrong in case a single teardown method is used?

Answer 1

[MathWorks Support Team]

There are two ways to define Teardown code for MATLAB Unit Tests:

1. Creating a single "TestMethodTeardown" method
2. Calling the "addTeardown" method on individual fixtures

The "addTeardown" method has certain advantages related to Exception Safety and Incremental Teardown. Consider the following two examples which illustrates this advantage. The first one creates a single "TestMethodTeardown" method and the second one uses the "addTeardown" method.

classdef MyTestClass < matlab.unittest.TestCase    
    properties 
        MyMatrix
        MyFigureA
        MyFigureB
    end    
    methods (TestMethodSetup)        
        function setupMyTestClass(testCase)
            try      
                % Creating MyFigureA
                testCase.MyFigureA = figure(1);
                testCase.MyFigureA.Name = "Figure A";
                fprintf("Created %s.\n",testCase.MyFigureA.Name);

                  % Some error occurs here
                  testCase.MyMatrix = magic(2) * magic(3);  % Invalid Matrix Multiplication

                  % Creating MyFigureB
                  testCase.MyFigureB = figure(2);
                  testCase.MyFigureB.Name = "Figure B";
                  fprintf("Created %s.\n",testCase.MyFigureB.Name);                
              catch mExp
                  error(mExp.message);
              end
          end        
      end    
      methods (TestMethodTeardown)        
          function teardownMyTestClass(testCase)
              fprintf("Closing %s.\n",testCase.MyFigureA.Name);
              close(testCase.MyFigureA);
              fprintf("Closing %s.\n",testCase.MyFigureB.Name);                             
              close(testCase.MyFigureB);
          end        
      end
      methods (Test)        
          function startMyTest(testCase)
              fprintf("Running the Test.\n");
          end        
      end
  end

The code attempts to initialize a matrix and two figures before implementing the Unit Test and deletes the figures upon test completion. Test teardown is implemented by the method "teardownMyTestClass".

Although this is a straightforward method to define fixture teardown, certain problems may arise. If an exception is thrown in the middle of the "TestMethodSetup" method, the framework will attempt to execute the full content of the fixture teardown code. This may result in trying to teardown fixtures that were never setup, and unwanted errors could follow.

For this example, an error will occur when the program attempts to delete "MyFigureB" during teardown, because it does not exist. Also, if more than one "TestMethodSetup" and/or "TestMethodTeardown" methods are defined in the same class, then it is possible that the teardown does not occur in the opposite order of the setup.

To alleviate this issue, the "addTeardown" method can be utilized because it facilitates a deterministically ordered teardown process that is also exception safe. The order in which this teardown occurs is the opposite of the order the fixtures were added. This avoids executing teardown code for uninitialized class members.

classdef MyTestClass < matlab.unittest.TestCase    
    properties 
        MyMatrix
        MyFigureA
        MyFigureB
        MyFigureC
    end    
    methods (TestMethodSetup)        
        function setupMyTestClass(testCase)
            try                
                % Creating MyFigureA
                testCase.MyFigureA = figure(1);
                testCase.MyFigureA.Name = "Figure A";
                fprintf("Created %s.\n",testCase.MyFigureA.Name);
                testCase.addTeardown(@closeTheFigure, testCase, testCase.MyFigureA);

                  % Creating MyFigureB
                  testCase.MyFigureB = figure(2);
                  testCase.MyFigureB.Name = "Figure B";
                  fprintf("Created %s.\n",testCase.MyFigureB.Name);
                  testCase.addTeardown(@closeTheFigure, testCase, testCase.MyFigureB);

                  % Some error occurs here
                  testCase.MyMatrix = magic(2) * magic(3); % Invalid Matrix Multiplication

                  % Creating MyFigureC
                  testCase.MyFigureC = figure(3);
                  testCase.MyFigureC.Name = "Figure C";
                  fprintf("Created %s.\n",testCase.MyFigureC.Name);
                  testCase.addTeardown(@closeTheFigure, testCase, testCase.MyFigureC);
              catch mExp
                  error(mExp.message);
              end
          end        
      end
      methods        
          function closeTheFigure(testCase, fig)
              fprintf("Closing %s.\n",fig.Name);
              close(fig);
          end        
      end
      methods (Test)        
          function startMyTest(testCase)
              fprintf("Running the Test.\n");            
              % Code to run the test goes here     
          end        
      end    
  end

Execution occurs in the following order:

1. "MyFigureA" gets created

2. "MyFigureB" gets created

3. An error is thrown because of the invalid matrix multiplication

4. Teardown method for "MyFigureB" is called

5. Teardown method for "MyFigureA" is called

Note that the program does not attempt to close "MyFigureC" in this case.

Answer 2

[Andy Campbell]

There is a blog post that describes the addTeardown method and some of its benefits over other methods of managing test and test fixture state. Here is the link:

https://blogs.mathworks.com/developer/2015/07/27/addteardown/

Enjoy!