Real-Time Simulation and Control of Hydrobatic AUVs | MATLAB Day for Marine Robotics & Autonomous Systems, Part 2

This is part of a big research project we have at KTH called SMaRC, Swedish Maritime Robotics Centre, consisting of a number of research projects and PhDs. And this is a presentation about one of those research projects. So if you can take the next slide, please.

And this project aims at looking at, so what can we do with more traditional AUVs in terms of making them more useful and agile in their maneuvering? And starting with looking at the traditional AUVs, they are oftentimes very slender, torpedo-like AUVs that requires speed for their maneuvering and diving.

And in the other end of world-class underwater vehicles, we have a type of vehicle called ROV, so remotely operated vehicles. And they are typically box-like vehicles, and designed for high maneuverability at low speed for manipulation maneuvers and those type of operations.

And what we are aiming to do in this project is see, how can we bring these slender flight-style AUVs to be closer to ROV-type performance in terms of maneuvering? And that's what this research project is about. And next slide, please.

And we can think of a lot of different useful impact areas for this behavior-- everything from ocean production, where we utilize the oceans for production of food, raw materials, and so on; environmental sensing is an important area where underwater robots can contribute with unique capabilities in terms of access environments that we can't access otherwise; and then lastly, in terms of security and defense, we see that there are a lot of useful aspects of utilizing underwater robots.

And if we can do more advanced maneuvers with these underwater vehicles, then we can utilize the sensor suit they have in a more efficient way. And we're thinking about smaller, cheaper, and more agile and efficient AUVs, which means that they typically can't have very advanced sensor suit.

But if we instead can point and direct the sensors in a desired direction, either if we're operating in something like an algae farm, where we have to maneuver in very confined spaces, or in an under-ice scenario, where you might want to explore some cavities or direct the sensor at the specific target-- or for that matter, in a security defense aspect, where it may be collaborations with submarines or other multi-agent systems that you have under the water. Next slide, please.

And for the purpose of all of our research, we have developed a small experimental platform, an autonomous underwater vehicle called SAM. And this vehicle consists of a number of trim systems that we are using to achieve the maneuvers that we want to achieve. It's steered by a thrust-vectoring system in the aft, where we are counter-rotating propellers to propel it forward and also steer the vehicle.

Otherwise, there are no external steering surfaces, so we have deliberately made the vehicle very slender, and with no antennas or fins or rudders sticking out from the vehicle that can get entangled in nets, algae, seaweed, and other things under the water. Instead, we have internal trimming systems that we want to utilize.

So we have a variable buoyancy system that we can use to pump in and out water, so we can essentially float up and down in the water column without speed. We can move the battery pack, so we can trim it a little bit aft and front. And we can also rotate weights on the vehicle to roll it around. So the idea is that you could essentially aim your sensors either towards the bottom, towards the side, or up on the other side of the ice or a cliff, or actually vary that angle during an operation. So next slide, please.

And what we face in this research is challenges like very advanced hydrodynamic flows that we need to encounter or that we need to predict for and be able to also model when designing the controllers. And we need to handle underactuated systems, meaning that we don't have actuators or thrusters in all degrees of freedom, but instead utilize that there is a coupling between the different degrees of freedom.

And if we can utilize that in a smart way with the controllers, then we can also, in the end, achieve a maneuver that we want to achieve, and in a very agile and energy-efficient way instead of as today's ROVs are using more or less brute force to counteract and achieve the desired maneuvers. Next slide, please.

And for that purpose, we have developed and worked on utilizing simulators to mimic or simulate the full 0-to-360-degree angle-of-attack range of the vehicles for maneuvers like rolling and also traversing perpendicularly in the stream. And I think this is where I hand over to Sriharsha, who will talk about the details of that research.

Thank you, Ivan. So as Ivan said, when we try to model these hydrobatic AUVs, we tend to get these very unsteady flows, especially at higher angles of attack or at higher Reynolds number. So one way to address, if we want to simulate an AUV or model it in real time, is to split it up into smaller pieces. So if you have different components in the AUV-- the hull, the nozzle, the different wings and control surfaces-- we can compute the hydrodynamics of each component separately and then add it all up. So that's the approach that we have been taking so far. And using this, what we have done is use the different toolboxes in Simulink to put together a hydrodynamic model of a robot.

So to build up the simulation model, we generate lookup tables for each component. And in these lookup tables, we calculate the hydrodynamic coefficients, like the drag coefficient and the lift coefficient, for different flow conditions. So where we know the flow quite well, like at low angles of attack and low Reynolds number, we can do computational fluid-dynamic simulations to populate this data set. And at higher angles of attack where we don't know it so well, we can use other methods which are semi-empirical. Like if it's a hull, we could use DATCOM or Jorgensen's formula. Or if it's a wing, we can use XFOIL and also flat-bullet theory.

So by putting together all of these data sets, we can create these lookup tables. And then based on the Reynolds number and angles of attack, you can get a specific drag or lift coefficient for that flow condition. Doing this, we don't do a computation in real time. But instead, we interpolate from a known database, for example. So that can allow us to do very fast simulations.

On the other side, we put all of this together in Simulink. This is like a cartoon of a Simulink model. And in that, we have the hull block, and then we have models for the actuators, the center-of-gravity trim systems, the buoyancy system, and the thrusters. We can also subscribe and publish to external topics-- example, to the robot operating system. We have some models for our sensor and state estimator. So we can add some error to the system if needed. And we use the ODE45 Runge-Kutta solver in this case.

So putting all of this together, I can show you what kind of simulations we perform. So this here is to perform a helix maneuver. It's part of a paper that Ivan and I wrote to the Journal of Ocean Engineering from IEEE. And here we're trying to inspect a target, for example, and to use a special kind of control strategy to do it.

And another thing is, we can validate our simulation model. So we have performed a compared the yaw rate between the simulator and the real vehicle. And you can see the dynamics are quite similar. And the qualitative behavior is very similar in both cases. And we've used the data from these experiments in order to tune the simulation model as well, so that we have good response in that sense.

And using this light simulation model, which is possible to run in real time, we can design new controllers. And we can perform this form of real-time control, given that we have this under-activated, yet quite complex system, and we can use different kinds of control strategies which take use of the couplings in the dynamics. For example, the depth and the pitch angle are very closely linked to each other. So if you try to only control the depth, you also affect the pitch. And then that causes a mess. So how do we address this? And the same applies to other degrees of freedom. So that's what I'm going to talk about now.

So we have integrated a lot of control strategies into the simulator. So thank you, MathWorks. You've got a lot of interesting programs there. We have made a setup where we can test controllers and simulation and then deploy them onto the hardware. So on one side, we can do controller management using these variant subsystems. So we had a very intelligent Master's thesis student, Shariklia, and she spent a lot of time putting this together.

So we can run model predictive control, linear-quadratic regulators, or PIDs, and we can switch between them quite easily. And like I said before, we can add different degrees of noise to the sensor feedback or add additional forces as disturbances to test the robustness of the system. And using the Robot Operating System Toolbox, we could also generate C-code and try to run it on the hardware. We are still doing trials on that.

One particular area where I'm working with in my research is of model predictive control. So when we have a system like SAM, it's quite complex. And when you have these couplings between the different states, we need to first get good sense of feedback, so we need a state estimator. We probably are following a certain objective in this mission that the robot is doing, so you have a supervisory controller, which in turn goes to a motion planner, which generates control objectives, like a set point to where you ought to go.

And we also have a certain degree of constraints-- maybe the dynamics of the system, maybe some unsafe states. So putting all of this together, the control objectives and the constraints go into a controller. And if we want to attack this kind of a problem, optimal control is a very nice area of interest. And within optimal control, model predictive control takes it one step further.

So in optimal control and model predictive control, we reformulate this control problem as an optimization problem. And we consider the dynamics of the nonlinear system-- in this case, our AUV-- as a constraint. So we have Newton's second law of motion here on the left side. We put that there, together with this C-effective, which is all the damping forces, which are modeled with our lookup tables.

And then we have an objective function. This objective function tells us that-- the first term is telling us to follow a reference trajectory; and the second term is telling us to use minimum control; and the third term is telling us to change the control as little as possible. So what a solver would do in optimization is to try to minimize this objective function, or maximize it if you put a minus sign here, while satisfying the constraints of the dynamics.

So if we do that in optimal control, we can find an open-loop solution to the system. And if you do model predictive control, we do this at every time step. So from now until the next 20 seconds, I predict what the dynamics will be and create a control. I apply the first step control and then again compute this. So in this way, you get this degree of robustness in the system. So using model predictive control, we can derive elegant controllers for this hydrobatics, because we take account of the couplings in the system through the model of the system.

We've been working quite hard with this. So these are images from Shariklia's Master's thesis project, where she used nonlinear MPC in MATLAB in order to control a model of SAM in the simulator to kind of go horizontally and vertically. On the vertical side, she's using the buoyancy system, and on the horizontal side, she's using the propellers and the thrust vectoring.

And here you can see that in preliminary results, we get reasonable performance. We can, of course, improve it. And this is the first step.

And then I took it a step further. I also used another software in Python called CVXPY, which is a convex optimization solver. And I ran a time-varying model predictive controller on the actual hardware. So we were controlling the buoyancy and the pitch at the same time, and while simultaneously holding a pitch angle and a depth.

So here you can see that when we were performing tests-- this is last December-- we got quite good performance. In fact, as compared to PID and linear-quadratic regulators, the MPC gave much better, what you call, long-term performance. Because it wasn't fighting against each other. When we use the PID, when we change the depth, the pitch changed. Then it had to compensate for the pitch. And then again the depth changed, and there was a lot of oscillation. But with MPC, the oscillations were much more reduced, because it accounted for this coupling.

That area that I was talking about so far was very specific to the controllers. And we also presented a simulator, which was mainly for the Simulink simulator. So when we put both of these things together, it's only a small part of this very complex system. If you want to have a real robot in the water, and if you want to work with robotics, there's a lot more aspects in it, too.

So we start first, maybe, if you're considering a survey operation, we start with a mission plan. So we have a mission planner called Neptus. From there, if the user tells the robot that I need to survey an area with these specific waypoints, then there's a mission execution tree called a behavior tree which takes these waypoints and provides actions. And based on these actions, a motion planner gives set points to these controllers. And the controllers give control signals.

However, the controllers are only as good as what state feedback you get. So based on the sensor data you get from the AUV, you have to perform some form of navigation and estimation. So in our case, we use dead reckoning. And with this pose, that is also fed back to the behavior tree. And in case you're in an emergency situation, the behavior tree has to account for that.

So these are the different software components. And in order to validate these software components, in addition to the controllers and Simulink, we've used another simulator called Stonefish, which gives us a virtual world-- something also like these game engines, like Unity or something of that sort-- where we can photo-realistically see what the robot sees, and validate all our software and how it interacts with the environment. And if our software works in both of these simulators, then we deploy it to the hardware. And most of the time, it's only minor tuning that we do once we go to the hardware.

And we put all of this together. And we did a lot of field trials over the past few years. So this year is an example of a waypoint mission. So instead of MPC, we're using PID controllers here, a bunch of cascaded controllers. And on the left side, you can see a behavior tree that we use where the first priority is to look for emergencies. If it's an emergency, it comes up to the surface. Otherwise, it follows the mission. And once the mission is complete, it comes back and takes the next mission. So that's the simple logic in the behavior tree.

So this here is a picture of SAM during a mission. And we performed the waypoint in a spiral to look for a submerged target. In this case, we were looking for this Mini Cooper car that we put in the water. And in the end, it was a success, that we were able to follow a trajectory and look at it.

And what I've been doing last week-- Yuhu mentioned that we were out at sea-- was we were trying to make SAM hold a 90-degree vertical balancing. So if you see here in the simulated figure, we wanted to use the trim systems and the propeller to balance SAM vertically at a specific depth and to slowly peek out of the water if possible.

So in simulation, it looked really nice. So on one side, I used the nonlinear model predictive control toolbox. And on the other side, I used a sequence of PID controllers together with some more bang-bang logics. So it was a kind of hybrid PID controller to hold the safest kind of behavior. So both of these things work nicely in simulation. And then we tried also on the actual hardware last week. And this here is a picture. So we're diving backwards in the water. And we're trying to hold a vertical state. Now it's reached almost vertical, and then it pushes forward.

One point to note here is that this is very sensitive. When you're trying to do such advanced maneuvers, it's very sensitive to the actual dynamics of the system. So in a simulated environment, our model of the AUV was much less stable than the one in real life. So the position of the center of gravity, the mass distribution, the weight distribution, even one small body put outside for the drag and the lift-- that changes the response of the system to disturbances, and also to control. And we have to account for that when we run for controllers.

So if you look at this image, you're going to see SAM jumping out of the water. And this was recorded last Thursday.

Today, in this presentation, you have learned a little bit about real-time simulation of the nonlinear dynamics of AUVs using a component build-up method. So we take the different parts of the AUV, we model each of them separately, and then superpose all of the components. And in order to get good hydrodynamics of each part, we consider these lookup tables, where we use the best available methods.

On the other side, we'll also talk to you about real-time control, focusing a lot on optimal control and model predictive control. We've shown you some results and videos. And then we have zoomed out a bit to talk about the cyber-physical systems that we see in real-world applications. We consider different software aspects, simulation environments, and hardware.

And building on this framework presented, today what we are doing is exploring new technologies as well. So on one side, on the modeling aspect, we're looking at system identification using different kinds of data-driven approaches in order to improve the models that we have got. And on the other side, to build on MPC, we are also working on reinforcement learning for hydrobatics. So a student of ours just finished his Master's thesis looking at reinforcement learning.

Using these kind of new techniques, we want to push SAM more and more into real-world applications. We already started doing it. We have been doing it for the past couple of years. But we want to push it to even more extreme areas. And we target to use AUVs in algae farms. So we're doing experiments there. We were there last week. And on the other side, also to take it out to the Arctic and the Antarctic, to calving glacier fronts.

And I'd like to thank you all for your attention. And I'm looking forward to your questions.