test parameters persist after suite ran

I am trying to run a parametrized test suite.

I have a myTestFactory class which uses myTest to generate a test suite, run it, and record the results.

myTest is parametrized at the class and at the test level.

• at the class level - I load data from a location determined by scene
• at the test level - I assert PARTS of the data based on f parameter

The reason this is setup is the way is because I have multiple scene values. each such value will generate a different sequence of f parameters.

I order to pass scene to the myTest, I write it to 's.mat' and read it into ClassSetupParameters.

The problem I am having is that for every iteration of the loop 1:N, fromClass method creates a suite with the exact same parameters as the previous iteration of the loop.

Meaning for i = 1: fromClass create a suite

• m Tests:
• 2 Parameters:
• scene = somethin
• f = i (in the range 1:m)

However, for i = 2 and onward, the scene paramter is the same.

Despite having loaded a different scene from 's.mat'.

In fact myTest.generate_s_parametes is NEVER executed for i = 2:N in the loop.

I want that for each run of the loop, myTest will be fully re-evaluated and scene parameter be obtained from 's.mat' every time.

classdef myTestFactory < handle
    properties
        test_results;
        seed;
    end
    methods
        function obj = myTestFactory(root_of_all_scenes)
            obj.root_of_all_scenes = root_of_all_scenes
        end
        function run(obj)
            for i = 1:N
                import matlab.unittest.TestSuite.fromClass;
                scene = select_a_scene(obj.root_of_all_scenes); % scene is a struct with a single field whos value is a local path on the machine
                save('s.mat', 'scene');
                suite = fromClass(?myTest);
                test_results = suite.run(); % test_results is accumulated in the loop - not included.
            end
        end
    end
end

classdef myTest < matlab.unittest.TestCase
    properties(ClassSetupParameter)
        scene = myTest.generate_s_parametes() % this changes 
    end
    properties(TestParamters)
        f = myTest.generate_f_parameters() % this is a struct('f1',1,'f2',2) whos range is determined 
    end
    properties
        data;
    end
    methods(TestClassSetup)
        function get_data(test_case, s)
            test_case.data = foo(s)
        end
    end
    methods(Test)
        function test_this(test_case, f)
            assert test_case.data(f) % f is used to select the data needed for this parametrized test
        end
    end
    methods (Static)
        function params = generate_f_paramters()
            load('s.mat')
            m = determine_f_range_based_on_scene(scene) % scene is loaded from s.mat
            for i = 1:m
                params.(['f' num2str(i)]) = i; % create struct('f1', 1, 'f2', 2, ..., 'fm', m)
            end
        end
        function params = generate_f_paramters()
            load('s.mat'); % loades a struct variable named 'scene'
            pararms = scene;
        end
    end
end

Answer 1

[Steven Lord]

If you're using release R2018b or later, you can use external parameters in your parameterized test. I believe that will allow you to avoid needing to pass data via the disk to the test class.

[Jack German]

Unfortunately, I'm using 2017a. 2018 is not an option at the moment.