Towards a better PID controller.

[nhnifong]

The ASAS module in KSP is a basic PID controller. (with an excessively high D term ) but obviously a PID controller is not a great solution to the problem of controlling the attitude of the mass and spring model in space. You dont need to hunt around the target and waste fuel to stay pointed in the same direction. When I control it manually, I just use the RCS to thrust in the direction I need for a about a second, wait for it to rotate around, and thrust in the opposite direction for about a second. I can move to any target attitude and hold it manually using about 1/100th the amount of monopropellant that ASAS or mechjeb would use. Why is that?

What method does mechjeb use?

What do real spacecraft use to solve this problem?

[tavert]

I believe MechJeb is also a PID, but with a bit of compensation for the MoI of the craft in question.

In reality, PID is common. You can add a deadband to reduce low-level hunting. A lot of times thrusters need to be on-off, so various kinds of bang-bang or sliding mode control can also work. If you want to get fancy you take constraints into account and do Model Predictive Control.

[Bunsen]

In reality, PID is common. You can add a deadband to reduce low-level hunting. A lot of times thrusters need to be on-off, so various kinds of bang-bang or sliding mode control can also work. If you want to get fancy you take constraints into account and do Model Predictive Control.

I'm curious about the practical results of those schemes. As usual, the Wikipedia articles are a near-impenetrable quagmire of abstract math and precise but opaque jargon. I might be able to work through them in a week or so, but I'd rather just bother you instead. How deep into control theory would you have to go to find an algorithm that mimics nhnifong's approach? It looks to me like that involves some prediction of the system's behavior -- the first RCS burn places the system on a trajectory through state space that intersects the desired trajectory, then you wait for the intersection and burn again to stay on the desired path. But maybe you can approximate that with something really simple if you parameterize the problem the right way...

Dammit, I think I've just been nerd-sniped.

[K^2]

Actually a PID with critical damping is near optimal for fuel efficiency. Of course, that's provided that you also have a way to balance thrusters. I have illustrated elsewhere, probably in a thread that got "eaten", how to compute optimal I and D terms. I can also go over a balancing scheme.

I probably should sit down and write an ASAS module for KSP. I've done something very similar for missile guidance system for Garry's Mod. I had intercept missiles lead and hit most targets without any problems using all the same controls available to a rocket in KSP.

[tavert]

I'm curious about the practical results of those schemes. As usual, the Wikipedia articles are a near-impenetrable quagmire of abstract math and precise but opaque jargon. I might be able to work through them in a week or so, but I'd rather just bother you instead. How deep into control theory would you have to go to find an algorithm that mimics nhnifong's approach? It looks to me like that involves some prediction of the system's behavior -- the first RCS burn places the system on a trajectory through state space that intersects the desired trajectory, then you wait for the intersection and burn again to stay on the desired path. But maybe you can approximate that with something really simple if you parameterize the problem the right way...

Dammit, I think I've just been nerd-sniped.

Writing out the continuous-time constrained optimal control problem is pretty intuitive. Actually solving it is another story, you have to apply the Pontryagin maximum principle or the Hamilton-Jacobi-Bellman partial differential equation. It's messy and involves a lot of variational calculus. Luckily, many reasonably simple problems share a common solution pattern, which switches between minimum control input and maximum control input. For a rigid body tracking a step change in attitude reference, you can solve for how long the start and end burns need to be based on the total time you want the maneuver to take.

Practically speaking, to deal with more general types of problems we usually discretize the dynamics and use optimization algorithms to solve a finite-dimensional problem. Collocation and shooting methods are both popular here.

Actually a PID with critical damping is near optimal for fuel efficiency. Of course, that's provided that you also have a way to balance thrusters. I have illustrated elsewhere, probably in a thread that got "eaten", how to compute optimal I and D terms. I can also go over a balancing scheme.

I probably should sit down and write an ASAS module for KSP. I've done something very similar for missile guidance system for Garry's Mod. I had intercept missiles lead and hit most targets without any problems using all the same controls available to a rocket in KSP.

As long as you don't hit constraints. And neglect disturbances and model inaccuracies. Luckily rigid-body dynamics is a low-dimensional system, so you don't need a high-order controller.

Edited June 15, 2013 by tavert

[rkman]

Aside from PID parameter issues, craft can generally hold attitude by means of gyro wheels alone, without using any RCS. It would be nice if KSP would just not use RCS (even when it's on) when only holding attitude. Otoh gyro wheel force should be reduced so that RCS is really needed to turn all but the smallest craft.

[magnemoe]

Aside from PID parameter issues, craft can generally hold attitude by means of gyro wheels alone, without using any RCS. It would be nice if KSP would just not use RCS (even when it's on) when only holding attitude. Otoh gyro wheel force should be reduced so that RCS is really needed to turn all but the smallest craft.

You think heading not attitude? And yes the system should use gyros primary and rcs if it has to.

Wonder how hard it would be to add this to mechjeb, it would have the option to simply turn rcs on and off depending on need.

[K^2]

As long as you don't hit constraints. And neglect disturbances and model inaccuracies.

Hence the "nearly optimal". Point is, I don't think a more advanced controller is needed. The real issue is that KSP's PID controller has the one size fits all parameters. If parameters are going to be computed for each ship, it should be good enough.

[tavert]

It depends. We don't exactly have a rich set of control inputs available in stock. There is some linear programming-based RCS balancing work in MechJeb that is fairly interesting, I guess the plugin API lets you control the thrusters individually.

For tracking a transient like a step reference change, a PID doesn't know how to coast, the input of a PID is only zero along a single plane in the state space. And the cost function here isn't really L2 in the input like you see for most basic methods of control design, it's L1. A large input is actually better than a small input applied for a proportionally longer time. Applying anything less than maximum control input wastes time, and applying anything more than minimum control input wastes fuel. To reach the desired goal in a given amount of time, the optimum-fuel solution alternates between max and min input.

We'd have to start plugging in some numbers to quantify exactly how suboptimal a well-tuned PID would be. And optimization-based controllers are a bit much effort to worry about for KSP. If you ask me it's not actually all that much effort, but then again I don't think the KSP team has a control expert on hand.

[K^2]

I understand the whole L1/L2 difference, but it's really most relevant for big corrections. For a small enough correction, a controller that optimizes an L2 integral will optimize the L1 integral of the same quantity to within a desired epsilon. That's why I'm saying the PID is nearly optimal for what it needs to do. Most of the time, it only needs to make small adjustments. If your attitude is 90° off what you told ASAS to hold, you probably have bigger problems right now.

Similarly, control of the rotation of a rigid body is not a linear problem even if you are optimizing an L2 quantity. But again, it is only non-linear for large deviations. For small deviations, it is perfectly linear. So a PID controller should still be very good at optimizing fuel use for small corrections.

The other big issue is that right now the PID controller applies the same input on all the thrusters. That means it wastes a lot of fuel on accelerating the CoM, which it isn't meant to do. There should be a zero net force constraint on the input space. That's really the first thing I'd solve. It would improve fuel consumption and make docking easier.

[OdinYggd]

Mechjeb 2 has a tunable PID using the attitude adjustment feature.

What I've found is you need to vary Kp as your ship's tonnage changes, for instance a Kp of 10 at 30 tons while a Kp of 100 is better at 5 tons. Ki remains 0, setting it to anything else encourages overshooting due to integral windup.

Kd should be about 0.08 that of Kp.

When tuned well you end up with a ship that is wobble-free, but has good steering response. It will use the RCS when it has to without twitching back and forth wasting it while idle.

I'm currently working on an equation to relate tonnage to Kp which can be used as a guideline, however a compensation for control authority will be required in this equation to get good results.

[nhnifong]

Thanks for the balancing advice Odin

But no Ki term? that leaves you with the avionics nosecone basically, and it can drift off target slowly. definitely not good for planes which need to hold a heading for a while with no attention, or a ship that needs to perform a very long burn.

[OdinYggd]

Thanks for the balancing advice Odin

But no Ki term? that leaves you with the avionics nosecone basically, and it can drift off target slowly. definitely not good for planes which need to hold a heading for a while with no attention, or a ship that needs to perform a very long burn.

So far I have yet to find a good value for Ki that does not result in chronic overshooting and loss of heading. I'll continue experimenting and let you guys know if I find anything worthwhile.

The problem with having a Ki parameter is KSP craft tend to suffer from Integral Windup, leading to further overcorrection and loss of position. If they do need a Ki value, it will have to be very small.

Only way that we'll really be able to reliably do anything about integral windup is to have the PID behavior modified so that the integral factor is only applied when the steering error is very low.

Edited June 16, 2013 by OdinYggd

[K^2]

There is a way to derive all three terms based on ship's moment of inertia (roughly proportional to ML² of the ship) and desired response time. I can do a write-up, if you guys are interested.

[OdinYggd]

There is a way to derive all three terms based on ship's moment of inertia (roughly proportional to ML² of the ship) and desired response time. I can do a write-up, if you guys are interested.

That equation would yield Pu, the ultimate gain at which the system becomes self-resonant.

Kp would need to be 1/2 to 1/4 of that, in order to strive for a half-wave dampening.

One of the biggest issues with making a KSP ship self-tuning is there is no reliable way to account for control authority in order to mathmatically predict the correct PID values. Although you can ballpark it by mass and moment of inertia, it still would have to have a manual trim parameter to be user configured in flight.

[K^2]

You can chose Kd to give you critical dampening. That gives you optimal L2 efficiency, which as I've explained earlier, is as close as you are going to get to optimal fuel efficiency with PID.

And what do you mean you can't account for authority? You compute the moment of inertia and you compute the torques from thrusters. Then you can get optimal input vector with an SVD.

[tavert]

I understand the whole L1/L2 difference, but it's really most relevant for big corrections. For a small enough correction, a controller that optimizes an L2 integral will optimize the L1 integral of the same quantity to within a desired epsilon. That's why I'm saying the PID is nearly optimal for what it needs to do. Most of the time, it only needs to make small adjustments. If your attitude is 90° off what you told ASAS to hold, you probably have bigger problems right now.

Similarly, control of the rotation of a rigid body is not a linear problem even if you are optimizing an L2 quantity. But again, it is only non-linear for large deviations. For small deviations, it is perfectly linear. So a PID controller should still be very good at optimizing fuel use for small corrections.

The other big issue is that right now the PID controller applies the same input on all the thrusters. That means it wastes a lot of fuel on accelerating the CoM, which it isn't meant to do. There should be a zero net force constraint on the input space. That's really the first thing I'd solve. It would improve fuel consumption and make docking easier.

Anything is within optimal to epsilon, for sufficiently large epsilon. L1 tends to give much sparser control input, with significant regions of exactly zero control input, whereas L2 gives a dense least-squares solution that is only zero exactly at the origin.

The 90 degree transient is common when you're using MechJeb's Smart A.S.S, switching from prograde to normal for example.

I never said rotational dynamics was a linear problem. If you model the nonlinear dynamics, an optimization-based controller will take that into account just fine. I was mostly considering a principal-axis transient, in which case the dynamics are linear if you neglect the change in moment of inertia due to expending monopropellant.

Perhaps a more ASAS-applicable transient would be toggling ASAS with a substantial nonzero angular velocity. That could be interesting if the initial velocity is not aligned with a principal axis.

Yes, again, constraints are important and not easily handled by a PID controller (besides by saturation, which leads to windup issues). For the purposes of KSP it would be nice to just have that implemented behind the scenes, where RCS thrusters get throttled down based on distance from CoM, to impose zero net force for rotational input and zero net torque for translational input. This should be easy and linear, and MechJeb has some prototype work that does this IIRC.

Edited June 17, 2013 by tavert

[K^2]

Anything is within optimal to epsilon, for sufficiently large epsilon.

Arbitrary epsilon. Means you are free to pick epsilon as small as you like, and I can still find a patch of deviations where a PID controller will operate within that epsilon of L1 optimum. The patch might be very small, depending on how strict you are with the epsilon, but in practical sense, so long as you only need to maintain the attitude, PID controller is good enough.

The 90 degree transient is common when you're using MechJeb's Smart A.S.S, switching from prograde to normal for example.

But that's a completely different problem. We are talking stock ASAS here. Not an auto-pilot solution. Yes, you occasionally end up with ASAS having to fix a large deviation from course if, for example, you had RCS turned off and then you turned it back on. But this isn't really what ASAS is meant for. It is meant to maintain attitude, which means that if deviation is great, there is a more serious problem. And for small deviation we are back to not needing anything other than PID.

I was mostly considering a principal-axis transient, in which case the dynamics are linear if you neglect the change in moment of inertia due to expending monopropellant.

Rotational problem over principal axis with constant moment of inertia is still not linear. It's a problem with periodic condition. It cannot be linear by very definition. To get a linear problem you must assume that deviation never increases past pi. If you want to consider here the most general case, where desired state vector can change suddenly, you cannot guarantee that, and your controller must be capable of looking after the periodic condition.

Of course, if you agree with me that ASAS is never meant to do any of that, and it's only meant to look after small deviations, then we are back to a linear problem. Moreover, for small deviations, even if you aren't aligned with the principal axis and your moment of inertia tensor changes in time, the problem is almost linear. (Infinitesimal rotation operators commute, unlike finite rotations.)

For the purposes of KSP it would be nice to just have that implemented behind the scenes, where RCS thrusters get throttled down based on distance from CoM, to impose zero net force for rotational input and zero net torque for translational input.

It seems like that would cost you the maximum torque you could apply. I'd use SVD method to get the input vector of RCS thrust values from desired output vector of net torque/force. Again, this optimizes fuel in L2 sense, but if we are going to use it with a PID controller, that's precisely what you want.

[tavert]

Since L1 is non-smooth, I'm fairly sure the "patch of deviations" for small epsilon is not finite, it would only be the origin.

Most of the time, yes ASAS is designed for holding small deviations in attitude. But that doesn't mean you would want to ignore transient performance when you initially turn it on, as it tries to halt whatever current rotation exists then return to the setpoint where it was toggled.

It read to me as if nhnifong's initial question was regarding a step change in reference attitude.

Linear modulo 2 pi. As long as you unwrap the desired setpoint into the closest interval to the current state, you'll get the right behavior. You obviously want to take care that you don't oscillate between 0 and 2 pi (or +/- pi) as if there's a discontinuity there, but that's an easy preprocessing step that you don't really have to include as a nonlinearity in the dynamics.

You'd only get a different answer if you're commanding simultaneous rotation and translation. In that case you have coupled linear constraints, still not very hard.

[K^2]

Since L1 is non-smooth, I'm fairly sure the "patch of deviations" for small epsilon is not finite, it would only be the origin.

That is absolutely clearly wrong. As you said yourself, for a large enough epsilon, almost any controller will work well enough within the entire state space. In other words, for any patch of state space around the target, I can find an epsilon large enough so that PID gives me a solution that is within epsilon of the L1 optimal. If I reduce the patch around the target, I can get away with a smaller epsilon. The epsilon clearly goes to zero as the size of the patch goes to zero. That means that for any given epsilon, I can find a finite patch, including a very small epsilon.

The only thing that isn't smooth about the L1 case is the time profile of the output. In all other respects, it is well behaved, and therefore, linearizable.

Most of the time, yes ASAS is designed for holding small deviations in attitude. But that doesn't mean you would want to ignore transient performance when you initially turn it on, as it tries to halt whatever current rotation exists then return to the setpoint where it was toggled.

I'm not suggesting we ignore it. I'm suggesting we consider it and we find it insignificant. Compared to the fuel ASAS is going to burn through for attitude hold, the small overhead for initial correction is pretty much irrelevant. PID isn't going to perform immeasurably worse than a proper L1 controller. It will perform worse by some factor. (Would you like me to show the math?) In the grand scheme of things, PID will perform almost as well.

It read to me as if nhnifong's initial question was regarding a step change in reference attitude.

Except that you can't have a steep change in reference attitude. Once you turn on ASAS, your current attitude becomes your reference. That means that your initial condition already has correct orientation. Just not necessarily the correct derivative.

Linear modulo 2 pi.

Which is no longer linear.

As long as you unwrap the desired setpoint into the closest interval to the current state, you'll get the right behavior. You obviously want to take care that you don't oscillate between 0 and 2 pi (or +/- pi) as if there's a discontinuity there, but that's an easy preprocessing step that you don't really have to include as a nonlinearity in the dynamics.

Of course you do. Say your reference is almost pi from current attitude and you are rotating rapidly away from reference. Should you be trying to slow that rotation down? Of course not. You are much better off continuing with rotation you already have, rotating past pi, and wrapping around to reference. Linear controller cannot handle that sort of thing. That's why periodic state space automatically means a non-linear controller if you want to handle general case.

[tavert]

The math would be a good idea. I dispute your contention that epsilon goes to zero continuously, but I'm not sure we're talking about the same problem formulation. For very small deviations in state, L1 will apply zero input. PID will apply nonzero input. Nonzero input is infinitely worse than zero input, if we only consider the input's contribution to the cost. We will actually have a combined cost function of both input and state, probably L1 in input and L2 in state. The state deviation therefore has to reach a finite nonzero amount before any control input will be applied at all in the optimal solution. Considering the combined cost function, you're right that L1 control and PID will both have nonzero cost. But considering only the input, that is no longer true.

If you're moving "away from reference," you should have mapped that reference to the equivalent point that you are moving towards as a pre-processing step, then you'd have a linear problem. You can precompute this mapping offline.

Edited June 17, 2013 by tavert

[K^2]

First, lets look at response to initial velocity, which is what ASAS is really meant to deal with. Suppose, we want to hold x = 0, x' = 0. Instead, when ASAS is tripped, we start out with x(0) = 0, x'(0) = v₀. All the L1-optimal controller has to do is kill the velocity. This requires kv₀ of fuel for some k. Of course, unless the thrust is infinite, this will result in some deviation to be corrected, which will require a bit of extra fuel if you want it in finite time. I'll look at that in just a moment.

PID will follow x'' + 2ξÉ₀x' + É₀²x = 0. And since solution to L2 problem is critical damping, ξ = 1. Solution to the above IVP is x(t) = v₀t exp(-tÉ₀). If we integrate ||x''(t)||, we get (2+e²)/e² v₀. (Piecewise integration is left as exercise for the reader.) In other words, total fuel consumption is going to be roughly 1.27 kv₀, which is, as promised, only worse by a factor of a constant.

Now, lets suppose that we simply want to restore from a deviation. Suppose, x'(0) = 0, but x(0) = d. In principle, L1-optimal controller can recover from this using no fuel at all. But that's not useful, as that takes an infinite amount of time. Instead, lets require characteristic time of Ä so that x(Ä) is at most 2d/e. For L1-optimization, this really means that total recovery time is eÄ/(e-2) and, ignoring limitations of finite acceleration, the total fuel required is 2k(e-2)/(eÄ), or roughly 0.53kd/Ä.

This time, PID responds with d (1+tÉ₀) exp(-tÉ₀), where É₀ = 1/Ä. This gives total fuel consumption of 2kd/(eÄ) or roughly 0.74kd/Ä. Again, the consumption is higher by a constant factor. And since in both cases fuel requirement goes to zero as deviation goes to zero, for any finite epsilon there is a value for d which makes the difference between the two less than epsilon.

So all in all, PID will consume 39% more fuel for a given deviation in attitude and 27% more fuel for a given deviation in angular velocity. This is not all together insignificant, but either one is far, far better than what ASAS is doing right now. And while, yes, a well-programmed L1-optimal controller can work better than PID, I'm not sure it's worth the trouble, when simply tuning PID will improve things a whole lot.

[tavert]

Whoops, I was thinking comparing performance by ratio rather than comparing by difference. A small deadband's probably quite acceptable in most scenarios, so you can't beat zero. Your examples are good, and I would consider 27-39% fuel savings absolutely worth a more sophisticated controller. Maybe not for KSP, but in the real world where performance is critical (and reference tracking is the simplest of the many situations a controller will need to handle) and assuming you have the engineering resources to implement an optimization-based controller, it's the right way to go.

For the principal-axis dynamics, the linear-problem approximation only breaks down in corner cases like having very high initial velocity and not enough control authority to stop the first time around. But this nonlinearity isn't in the dynamics, you still have I*theta''(t) = B*u(t). It's in the cost function for reference tracking. The differential equation doesn't care that 0 and 2 pi are the same, the dynamics are still linear.

Edited June 17, 2013 by tavert

[Bunsen]

So all in all, PID will consume 39% more fuel for a given deviation in attitude and 27% more fuel for a given deviation in angular velocity. This is not all together insignificant, but either one is far, far better than what ASAS is doing right now.

Okay, now that we've scienced the living hell out of the idealized version of the problem (I love this thread, BTW -- ain't many games out there that inspire forum posts with differential equations), let's confront the reasons that the current ASAS behavior sucks so profoundly:

1. The PID gains are not optimized for each craft. This should be solvable with some computation of the moments of inertia and available control torques. Thrust vectoring is a complication, because it makes the control torque change with throttle setting. That still might be within reach of a well-programmed plugin.

2. The system is not a rigid body. Rockets bend. Rapid changes in control inputs excite vibrational modes of the craft, which cause the ASAS pod's measured state (IIRC, this is measured at the command pod) to deviate from the actual dynamics. That deviation tends to include phase lag, and when that gets between 1/4 and 3/4 of a cycle, control inputs intended to counteract motion amplify it instead, and things undergo rapid, unplanned deintegration. Or at least you waste craptons of RCS propellant.

The second one is, I think, the big reason we hate ASAS. A half-cycle lag for the fundamental bending mode is basically guaranteed when you have a command pod at the top of a long rocket and a vectoring engine at the bottom, or RCS clusters in their most efficient positions at the ends of the rocket. I see two obvious solutions, both of which kinda suck: Either smooth out the changes in control inputs enough that they don't significantly excite any vibrational modes (there goes any thought of bang-bang efficiency, and now your control is smeared out over a timescale longer than the rocket's fundamental vibrational period), or smooth out your state measurements enough to filter the vibrations out of the measurement (and again, you get sluggish response). The practical solution is to build stiffer rockets and make that fundamental frequency high enough not to cause problems, but that's only feasible for fairly small rockets, and small rockets bring shame to Jeb and the entire space program.

That seems like a pretty universal problem for control theory, though, so people have probably already thought up some incredibly clever ways to confront it. I hope.

[tavert]

I'd write a Kalman filter or moving horizon estimator to try to filter out the vibrational measurements. Then you could approximately separate the CoM dynamics from the vibrational disturbance, and apply control primarily based on the former. If necessary you could possibly also try to actively fight the vibration. But observing it can already be tricky if you don't have a good a-priori model of what you expect the vibrational modes to be, actively damping it would be as hard or harder.