Closed-Loop Control: An Overview of Feedback Controllers
From the series: Control Systems in Practice
Brian Douglas, MathWorks
This tech talk covers an overview of many different feedback controller methods for closed-loop control. It starts with traditional methods, such as PID control, LQR, gain scheduling, H-infinity, and MPC, and then covers data-driven and AI methods such as MRAC, ADRC, fuzzy logic, extremum seeking, and reinforcement learning.
This introduction will give you an idea of how many different ways there are to close the loop between the outputs and inputs of a system.
Published: 22 Jan 2025
I think one of the most incredible and powerful concepts in technology is feedback. At its core, feedback is the idea that the output of a system influences its inputs, and this is a straightforward concept. For engineered systems you observe the output, and then a controller uses that information to adjust the inputs accordingly. And this process is known as closed loop control, as it closes the loop between the system's outputs and inputs.
And though it's simple, this concept is the foundation of all autonomy and automation. Now, there are several things that you have to design in order to achieve closed loop control. For example, you need sensors to measure some quantity about your system. You might need an observer to estimate states that aren't directly measured. You'll need actuators that can change the system state. You might need a way to generate what you want your system to do, like follow a path.
And then there's fault detection and responses and many, many others. But the heart of closed loop control is the controller. This is the algorithm that takes the outputs of the system, or the estimated states of your system, and then manipulates them in some way to generate the commands that go to the actuators. And as you might imagine, people have come up with 100 of different ways to do this, or maybe even thousands or tens of thousands. I'm not really sure, but there are a lot.
And we have MATLAB Tech Talks that go into depth into dozens of the most popular controller methods, and I've linked to them below if you want more details. But in this video, we're going to use this map of feedback control systems with MATLAB and Simulink to look at an overview of a few of these methods. And I think it's going to be pretty interesting to see how different some of these approaches are.
So I hope you stick around for it. I'm Brian, and welcome to a MATLAB Tech Talk. In the feedback control systems map, we tried to find a way to classify the different control approaches. And there are several ways that this could be done, but we divided them up into four different groups. Between the top and the bottom, we're differentiating between the controller method or the mathematical framework of the controller from the tuning algorithms, which is how we populate the parameters of the controller.
So, for example, one controller method, like PID control, could have several different ways to tune it, like using a model to calculate the optimal parameters, versus learning-based methods where the PID gains are tuned real time with hardware in the loop. And then between the left and the right, we're differentiating controllers that use data to learn the optimal solution in some way, versus what we're calling traditional methods, which doesn't really have a good definition, but we're saying that they are controllers that don't rely on data directly, but instead rely on a fundamental understanding of the physics of your problem.
Now, data driven controllers often still require some physical understanding of your system. And for traditional controllers, people still need data in some way. So it's not a perfect classification, but for the most part, the controllers on the right use data to learn the best solution. And the controllers on the left, the best solution is derived from knowledge.
Now, in this video, I'm only going to be showing a few control methods, the top half here. We're not going to get into different ways to tune them, but we will look at both traditional and data driven methods. And to start, let me show you some of the interesting ways that we have to incorporate physical knowledge of our system with traditional control methods.
And I think the first controller that most people think of is PID control. And this is where the output of the system is compared to a reference to get the error in the system. And this error is used in three different ways.
In one way, it's just scaled by the proportional gain. So the output of the controller is a multiple of the current error. The second path, the error, is summed over time through an integrator, and that value is scaled by the integral gain. And then in the third path, we take the rate of change of the error with the derivative, and then that output is scaled by the derivative gain. And the summation of all three of these paths produces the total output of the controller, which is the input into the system.
And this controller is nice because the gains are intuitive if you have a good physical understanding of your system, and therefore, they can be tuned in the field with some amount of experience. And it's also nice that you don't need a mathematical model of the system to use PID, although having a model can make the tuning process easier.
Now, PID control is usually reserved for single input and single output systems, and there are ways to do PID control on multiple variable systems, but they can get a bit clunky. So this is where other methods, like state feedback, comes in handy. In state feedback, you know, as you would infer from the name, the signal that is fed back to the controller is an estimate of every state in the system. So instead of having just three gains, like we did with PID, this controller has a matrix of gains with size equal to the number of inputs by the number of states.
So, for example, a system that has three states and two inputs, there would be six different gains. And in this way, every input into the system is just a linear combination of every single state. Now, to get the values of this matrix there's a number of different tuning methods. And for two examples, you could do pole placement where the gains are chosen to place the closed loop poles of the system in particular places, or LQR, which solves a quadratic cost function to optimize the trade off between performance and controller effort.
Now, PID and state feedback are linear controllers, and therefore they do best controlling systems that are linear or near linear. However, if the system is too non-linear across the entire operating range, then other methods have been developed, and one such method is gain scheduling. Gain scheduling is where you change the gains of a controller based on the current state of the system.
So, for example, you might have a PID controller with one set of gains for when the system is in one state, and then another set of gains for when the system crosses over into a different state. And instead of two sets of gains, you might have dozens of sets. You know, whatever you need to break up the state space into small enough chunks that your linear controllers can handle it.
And instead of discrete gain sets, you can smoothly interpolate between each set so that the gains are always continuously adjusting as the state of the system changes. In this way, you can have a linear controller that can operate robustly over the entire nonlinear range.
There are linear controllers that are designed specifically to be robust to uncertainties without having to schedule their gains, and two such methods are H infinity and Mu Synthesis. H infinity is a controller that's designed in such a way that minimizes the worst case gain from external disturbances to the system output, while driving the error signals to zero. And since it's explicitly trying to minimize the effect of any external disturbances into the system, there is an amount of robustness to these disturbances allowing you to meet system stability and performance requirements in their presence.
And Mu Synthesis takes this a step further and allows you to also specify structured uncertainties in your system itself, and then account for them. So basically, if you understand where uncertainties exist in your model and you can bound them within some limits and then include them in the tuning process so that the controller guarantees stability and performance across the entire uncertain range. In this way, Mu Synthesis is designed to be robust to both external disturbances and internal uncertainties.
Now, there is another class of controllers altogether, which is called Model Predictive Control, and MPC uses a model of the system to predict and optimize future behavior over a specific time horizon. And the basic approach is that you provide the controller with a model and an objective function that you want to maximize, and then also a set of constraints on the system.
And then at each time step, MPC uses the model to solve the optimization problem, to determine the best control actions that maximizes the objective while satisfying the constraints on the system. And MPC is really flexible because there are different optimization techniques that can handle different types of objective functions and different types of constraints, and even nonlinear system models. Plus, there is a data-driven MPC approach that creates and updates the system model from data as the controller runs.
All right, so these are just a few of the different control methods that we're calling traditional controllers. And hopefully, you have a sense of the breadth of options that we have just within this section of the map. So let's move on over to the data-driven section and see some of the different control methods that are available here.
So we're going to start with Active Disturbance Rejection Control, or ADRC. An ADRC uses an extended state observer to estimate both the system states and the total disturbance. And this is especially beneficial for systems that behave close to a pure first or second order system, but then have some additional unknown dynamics or unknown disturbances that impact it.
For example, a motor spinning a mass, like in a reaction wheel, behaves ideally like a double integrator. In ADRC, the extended state observer will estimate the two states of this system, and then feed those back through their own gain, like we had in full state feedback. However, the extended state of this observer lumps together all of the uncertain dynamics and disturbances of the system, and with data from the system, tries to estimate them so that it can mitigate those uncertainties by adjusting the input to counter them.
And if the estimated disturbances are equal to the actual disturbances, then they are canceled out and the system has that ideal second order closed loop behavior, which I think is a pretty neat idea. And another neat idea is Model Reference Adaptive Control, or MRAC.
MRAC is an adaptive control strategy, which means that it's changing over time to converge on the solution. And what it's converging to is that the output of your system follows the behavior of a predefined reference model. So instead of needing a model of your actual system, you define an ideal model that you want your system to behave like. An MRAC will adjust over time to try to match it.
And generally, how this works is that there is an adaptation mechanism which continuously modifies the controller parameters based on the discrepancy between the actual system output and the desired output. And once the two systems are in near agreement, the discrepancy is low and the controller parameters stop changing.
With a controller like this, you define the desired behavior in the form of a model. But there are other types of adaptive controllers. For example, with extremum seeking control, the goal is defined as an objective function that you want to maximize. So instead of changing the controller parameters based on the output of the system, extremum seeking control is trying to adjust the parameters to generate the largest objective value that it can, and it does this in a really ingenious way.
We place a perturbation on the system inputs, and then we observe how those perturbations change the output, and therefore how it changes the value of the objective function. If the perturbation increases the value, then extremum seeking control changes the input into the system slightly in that direction. And if the perturbation decreases the value, then it changes the input slightly in the opposite direction.
And we can use this for things like maximizing braking force in a vehicle. The objective in this case is to maximize the negative acceleration of the car. And as the brake is applied, acceleration increases up until the point where the tires start to skid, which lose their braking power. An extremum seeking could be applied to this system to lightly perturb the brakes and move the brake force in the direction that provides the most acceleration.
You know, it's kind of amazing how many wildly different methods there are in data-driven control. And to give you an example of one that isn't like any that we've talked about so far, let's look at fuzzy inference systems. These are a form of rule-based control that use fuzzy logic to model systems and make decisions based on imprecise or uncertain input data.
A fuzzy inference system consists of three main components, fuzzification, which takes the outputs of the system and converts them into fuzzy sets. And then there is a rule base which contains a set of if-then rules. You know, things like if a value is in this set, then the output should be this value. And depending on how much it's in that set, determines how much the output is applied.
And finally, there is defuzzification, which translates the outputs of the if-then rules back into crisp values that are fed back into the system. And this is a data-driven method, because we can use training data of the desired system behavior to tune the membership functions and rules to better match that behavior.
Now, the last data-driven method that I want to cover here is reinforcement learning. RL is a type of machine learning, but instead of learning a static data set, RL learns how to interact with a dynamic environment. And using RL terminology, an agent learns to make decisions by interacting with the environment to maximize a cumulative reward, and it relies on trial and error exploration to discover which actions lead the greatest long term benefits.
And the agent uses a policy, which is essentially the controller, to decide actions based on the current state, and then it receives feedback from the environment in the form of rewards or penalties, which it uses to update its policy and improve future decision making.
All right, so anyway, what I've showed you here is just a really small fraction of the many different controllers that exist. And hopefully, this brief overview gives you some idea about how many different ways people have come up with for controlling systems. You know, for closing the loop between the output and the input of a system. And there are really a lot of ingenious solutions out there.
And if you want more details about any of the methods on this map, which include more than what I've just talked about in this video, you can follow the QR code in the top corner, and it's going to take you to a page with links to examples and documents and videos that cover everything, and a link to the map is below.
Also, the last thing that I want to mention is that I know in this video I just listed a bunch of control methods without really comparing the pros and cons of each. And while we don't compare all of them, at the bottom of the what is optimal control page, we have a chart that lists out some of the features of several different optimal control methods. So at least you have some idea of what they're good for and when to use them.
And I've also left a link to this page below if you want to check it out. All right, so this is where I'm going to leave this video. Don't forget to subscribe to this channel so you don't miss any future videos. And you can find all of the Tech Talks videos across many different topics, nicely organized at mathworks.com. All right, thanks for watching and I'll see you next time.