Variation between data during simulation and that stored for use in second simulation

Hello,

I am working on a simulation for a system that uses a feedbackloop to correct for any system deviations. At each time step of the simulation the outout and input values are stored. I have another system wich is the exact same as the previous system just without the feedback loop. The data collected from the first system (with feedback loop) is then used to control the second system with almost no issue. After a while, however, an error starts to creap in and is notiacable at about 6 decimal places.

I have connected both systems so that the output from the system with the feedback loop is directly fed into the system without the feedback loop and there are no erroes biult up in this arrangment. If I then use that exact data again (stored from the first run with the feedback loop used) on the system without the feedback loop again an error builds up over time.

I am therefore wondering if anyone has any suggestions why this may be and how I can possibaly mitigate this issue.

Thanks

Kas

Answer 1

[Fangjun Jiang]

I think I understand your question. I will use this diagram from https://en.wikipedia.org/wiki/Control_theory to explain.

You have this diagram simulated in Simulink and recorded the data at "System input" and "System output".

Now you duplicate the block "System" to "System2" in the same model, feed the recorded data "System input" and record the output as "System Output2". You expect "System Output" and "System Output2" to match exactly but they don't.

The most likely cause is that the "System input" recorded is not exactly the same as the "System input" in the closed-loop. There will be truncated error (rounding error) when recording the data.

If you feed the closed-loop "System input" to both "System" and "System2", you might get "System output" and "System ouput2" to match. But they could still be different, due to the different execution order of the components inside "System" and "System2".

But the most important point I want to bring up here is this. If you have a fine-tuned closed-loop control system, don't expect to get the same ouput if you apply the recorded control input to the same system in open loop fashion. In theory, it should. But in real world, there will always be disturbance. In Simulation, data accuracy loss is sufficient to cause the difference, especially when "System" is un-stable.

This is the fundermentals of closed-loop control. With closed-loop, there is self-regulation, self-correction. Open-loop doesn't have it.

[KASRIEL]

Hello @Fangjun Jiang

Thank you for your responce, I did however have a few comments based on the response:

[1] At this point we are using perfect measurement so no noise or disturbance were included in the model.

[2] You had mentioned 'data accuracy loss' so the question is how to mitigate the data accuracy loss? Are there methods that allow to reduce the accuracy loss? For example capping the input/output data to x-significant figures or bits?

Thankfully,

Kas

[Paul]

I'm going to speculate that if you make an exact copy (i.e., same initial condtions, etc.) of System, call it System2, and run it in parallel with System in the same model during the same run, with System Input driving both System and System2, then the output of System and the output of System2 will be identical.

When you record System Input and then use that recorded data as input to System2 in a second simulation, the results will be different because you can only record System Input a fixed sample time and then the interpolation between samples of System Input as input to System2 is not going to be the same as what the values of System Input to System were at the minor time steps of the first simulation. This all assuming that System Input is a signal with continuous sample time that directly feeds blocks with continuous sample time in System.

[Fangjun Jiang]

@KASRIEL, both @Paul and I have explained the reason.

Point 1

@KASRIEL, in your statement, "I have connected both systems so that the output from the system with the feedback loop is directly fed into the system without the feedback loop and there are no erroes biult up in this arrangment.". Here "so that the output from the system with the feedback loop" should be "so that the input to the system with the feedback loop"

Both @Paul and I expressed the same. But even with this setup, I would say the matching system output is not guaranteed because "System" and "System2" would not be identical in terms of simulation. The execution order of the internal components (inside the system) could be different because the way that Simulink determines the execution order. With or without the loop could very much change the execution order. Again, in this case, the matching system output is very likely but not guaranteed.

Point 2

When feeding the recorded input, @paul has pointed out the possible sampling issues, where sampled signal is not the same as the continuous signal, or even with the same sample time, there are still the "minor time step" issue if you know what it means.

Point 3

Let's say your system is all discrete and single rate, it is possible or likely to get a matching system output with the recorded input. That doesn't really mean anything in this closed-loop, open-loop comparison. Don't get this idea that this approach can be applied in general. I see this happen quite often. People try to use this approach to debug without realizing the key roles of closed-loop control.

Final point

If you have a fine-tuned closed-loop control system, don't expect to get the same ouput if you apply the recorded control input to the same system in open loop fashion.

[Paul]

@Fangjun Jiang

I'm afraid I disagree on Point 1. If System and System2 are exact copies of each other, running in parallel in the same simulation, and being driven with the exact same input signal. I don't see any way that the ouputs could be different. Of course the excution order of the blocks in System and System 2 will be different relative to each other and the overall block execution order, but they better be identical realtive to the common block (single block in this example, could be multiple in general) that is driving both. Have you observed to the contrary? If so, that would be very bad for Simulink, IMO.

[Fangjun Jiang]

@Paul, I am pretty sure about this point. I drew to this point based on my discussion with a Mathworker. Assuming Controller, System and Sensor are all virtual subsystems and each subsystem contains hunderds of blocks, Simulink flattens all the blocks to one page and determines their execution order. There is no guarantee that the relative execution order inside System and System2 would be exactly the same. Keep in mind that, for the argument sake, I only need one counter-example to prove this point. I am pretty sure that Mathworker basically conveyed this point to me, that in this case, there is no guarantee that the relative execution order of the blocks inside System and System2 would be the same.

Not to make the case more complicated, if an algebric loop :) is formed by the feedback loop, then from the input to the System output, it will be very hard to infer the execution order, while from the input to the System2 out, there might be no difficulty at all.

Once you agree that the relative execution order inside System and System2 might not be the same, it is easy to prove that their outputs may not be the same with the same input.

[Paul]

If the relative execution order inside System and System are not the same, then for sure the outputs will be different. I'm skeptical that the execution order would ever be different, at least under typical use. As far as I understand it, which might not be much, the block execution order is determined by an algorithm. Why would that algorithm come up with a different answer when presented with the exact same configuration of blocks?

[Fangjun Jiang]

The "configuration of blocks" is not the same between System and System2, when System is in the loop with Controller and Sensor, while System2 is only connected to the output of Controller.

Imaging all the blocks are flatterned, due to the loop, the algorithm determines that it is best to start the execution order at the middle of System, going through Sensor, then Reference, then Controller, and then the left half of System. System2 is also in the model, but it is easy to infer the execution order would be from "System input" to the right end of System2. In this case, in the same time step, the relative execution order inside System and System2 is different.

The discussion I had with the Mathworker is about code generation, becaused we had cases where the generated codes are different for System and System2. The code differences are not about variable names, but rather, the different order of some lines of code. When the discussion is moved to the execution order of Simulink blocks, I basically asked this exact question. If I have two identical subsystems in the model, would the execution order of internal blocks be the same. The answer is no. Or "It depends", Or "Well, likely not if the system is complex."

[Paul]

Commenting only on simulation, not code generation, I'm having a hard time conceptually coming up with how that scenario (in the second paragraph) can happen. At the end of the day, System represents the mathematical relationship between its output and System Input. System2 represents the exact same mathematical relationship to the exact same input. So it's highly disturbing to me that Simulink would simulate the exact same mathematical relationships differently. If TMW says that can be the case, then I guess that can be the case. Doesn't make it any less troubling. But thanks for bringing it up. Definitely something to be aware of.

Answer 2

[Fangjun Jiang]

@Paul, I am determined to persuade you on this. Luckily, I was able to make an example quickly. See attached model in R2022b.

• Subsystem1 is copy and paste from Subsystem, Turn on Information Overlays, Execution Order.

• Inside Subsystem, the relative execution order, from top to bottom, is 2, 1, 4, 3

• Inside Subsystem1, the relative execution order, from top to bottom, is 1, 2, 4, 3

[Paul]

Thanks for putting in the effort. I apologize for steering the discussion to block execution order. I was thinking about the relationship between block exeuction order and the actual computational outputs of the blocks and didn't account for parrallel paths. In this examle, each subsystem has four blocks in parallel, so of course the outputs of the blocks will not be dependent on the execution order of the blocks. But this whole discussion was really about the values of the subsystem outputs.

What happens in this example if outputs 1-4 are each connected to a To Workspace block and then corresponding outputs from Subsystem and Subsystem 1 are compared after the simulation runs?

[Fangjun Jiang]

I hope this is sufficient. In fact, I was surprised that it was so easy to create this example. You can use the attached file or create your own.

• Subsystem1 is copy and paste from Subsystem, Turn on Information Overlays, Execution Order.

• Inside Subsystem

• Inside Subsystem1

[Paul]

Again, I think we should only be interested in the outputs at the Outports, not the block execution order in the subsystems (as I said above, I should not have steered the discussion that way). I created this same model, using the default block parameters for all blocks. I ran the model (with the algebraic loop warning) and then ran these commands at the command prompt:

>> figure
>> plot(out.yout{1}.Values)
>> figure
>> plot(out.yout{2}.Values-out.yout{1}.Values)

These commands produced two figures:

As we can see in the second figure, the outputs of Subsystem and Subsystem1 are identical.

Do you get a different result?

[Fangjun Jiang]

Now we've established that the Simulink execution order inside the identical System and System1 could be different, I don't think we need a numerical example to prove that the outputs of System and Sytem1 could be diffrent with the identical input.

Keep in mind that the difference is minor (OP's word, "notiacable at about 6 decimal places"). In most likely cases, the two ouptus are probably the same, to the naked eye, or to the precision of the double data type that Simulink uses.

But you have to admit, if System (and System1) contains thousands of blocks (operators/operations), many of the interim calculation results have to be stored in interim variables. If the execution order of all those operations could be different, then the final ouptut value could very much be diffrent.

A case in point is below. This might be due to minor precision loss. But if there is logic along the way to determine different downstream pathes, then the final otuputs could be significantly different.

a=1e30+2e40;
b=eps;
c=a+b;
A=1e30;
B=2e40+eps;
C=A+B;
a==A

ans = logical
   0

Again, we are not talking about theory or ideal case here. We are talking about in Simulink, duplicate a subsytem block, feed the same input, can we always expect an exact mathing ouputs.

[Paul]

I think you meant to compare c and C.

I'll give this more thought. But I don't find your code example convincing. From what I've seen so far, the effect of different block execution order would actually look like this:

% Subsystem 1, assume 2e40 is the input
a=1e30+2e40;  % block a
b=eps;        % block b
c=a+b;        % block c

% Subsystem 2, assume 2e40 is the input, the blocks have to have the same definitions, just a
% different sequence
B=eps;          % block b, same output, evaluated before block a
A=1e30 + 2e40;  % block a
C=A+B;          % block c

c == C

ans = logical
   1

Anyway, at this point I don't think we'll get any further absent a specific Simulink model that demonstrates the behavior, which I suspect would have to be quite complex as you say. So I guess I'll just leave it here and, as I said, continue to give the matter more thought.

[Fangjun Jiang]

Yes. I meant "c==C". But Yes, it is an invalid example. The operations were different.

I must have been thinking about code generation again. I remember we had cases where different codes were generated for the same subsystem, to the effect of, for System, it was "output=a+b+c". For System1, it was "interim=a+b;output=interim+c".