I want to teach a rocket how to fly!

[Pds314]

On 6/13/2019 at 12:59 AM, Jirokoh said:

Well, I've been thinking about coding a "simple" KSP sim, but I'd like to try to see if I can't solve the problem in KSP first.

Because, while this solution might work for this easy problem, it's still only the first problem I want to try to solve. If I can make this work, I'd like to go further, and tackle more complicated problems. That would mean also making my simulation more complicated, to train it on those more complicated issues. But, maybe, why not? I think I'll have to wait to have more time on my hands to do that then, so maybe only in a few months!

 

While I am getting 1 data point for every run, each run is only 12 to 13 seconds long, and during 1 run I actually get 6 / 7 meaurement points. So it's not that bad. The real issue is the RAM leakage, meaning I can't scale my data scrapping on long periods of time. If I could, I'd just leave my PC to run during the night and get data like that. It would be long, but it would work.
Fixing this issue is what I'd like to do, because it would allow me to at least be able to then scale to more complicated problems using the same method, even if each run would be longer for more complicated problems.

Is the RAM leakage in your code or KSP?

[Jirokoh]

22 hours ago, Pds314 said:

Is the RAM leakage in your code or KSP?

KSP.
Apparently, if you revert to launch too many times, RAM starts getting saturated. That's just what I found online, but what I did see is that indeed, as I revert to launch more and more, my frame rate goes down. And I'm already at the lowest settings and lowest resolution to make this run as easily as possible for this.

[Pds314]

1 hour ago, Jirokoh said:

KSP.
Apparently, if you revert to launch too many times, RAM starts getting saturated. That's just what I found online, but what I did see is that indeed, as I revert to launch more and more, my frame rate goes down. And I'm already at the lowest settings and lowest resolution to make this run as easily as possible for this.

Does it supply the same to quicload/quiclsave?

Edited June 16, 2019 by Pds314

[Jirokoh]

7 hours ago, Pds314 said:

Does it supply the same to quicload/quiclsave?

Yep, I actually started with quickload, and in both cases I'm incounter the same issue.

But, when I have more time, there are a few tricks I'd like to test, that might allow me to solve, or at least work around that! We'll see!

[Jirokoh]

UPDATE: IT'S (sort of) WORKING!!

First of all, because that's the coolest thing, here's the video:

 

I've had a bit more time on my hands, since my final exams are finished. I still haven't completed my thesis yet, but I'm nearly done, so I can come back to KSP.
So, what I've done is basically train the neural network not only on the time segment where it currently is at, but look at the previous ones as well. When it's train and comes time to predict the action to take, Bertrand looks at the observations from the current time step as well as the previous one (so it looks at (t) and (t-1)), and makes a prediction based on that.

As you can see, it's not perfect, but it's doing (modestly) better than random already! In the example seen above, it takes in total 9 actions: it throttles down for the 3 first, but then for the following 6, it always go throttle up!
And the cool thing is, that was only trained on 30 examples which is a really, really small dataset. I'm way too excited that this (kinda) worked, so I shared it before trying to improve that with more training examples. The neural network itself can probably be improved with a bit of fine-tuning, so I still have a lot of work I can do.

So, yeah, there you go. I made an IA that can (sorta) learn totally on it's own to throttle up to go higher.
(well, technically it doesn't yet have time to go higher, since it always goes down for the first 3 steps, which makes it loose of thrust at the very beginning and there aren't enough time steps after to overcome that early loss. But what I mean is that it then just goes full throttle, which on the long term is what I want it to do).

Edited June 29, 2019 by Jirokoh
(video not showing)

[Dakitess]

Well you wanna it to go Full Throttle as long as drag losses remains below gravity losses : would your code lower the throttle past this point ?

How is it managed so far : does it check by reducing to 90% thrust and checking if the increment of the ratio DeltaV/FuelBurnedMass is still optimal at 100% ? (Edit : don't mind me, this is an algorithm approach here)

My own algorithm did it this way, even if never applied in KSP : checking this ratio, going up while the ratio is increasing. Once at 100% (cause you'll almost always be able to reach 100%), it'll check every 5s if the ratio is still the best at 100% by going down a little, say 10%. If not, it'll go back by adding 10% to the thrust value. If not, it would lower by another 10% the value, and again, and again, at each timestep. Only the 100% value would be a more "stable" position, with 5s to avoid that 100-90-100-90-100-90-100... Yoyo, but this is definitely needed to check if a lower value is not more optimal, sometimes. A better approach could be to interprate the "acceleration" of the ratio itself, to initiate the 5s check only when the ratio-acceleration seems to decrease faster and faster, meaning that drag is becoming an issue.

Anyway, this is algorithm and not learning, so i'm wondering if your model would interprate correctly from dataset, that past some velocity, excessive speed is not a good thing : is you rocket powerful enough to meet that specific point ?

Edited August 19, 2019 by Dakitess

[Jirokoh]

Just now, Dakitess said:

Anyway, this is algorithm and not learning, so i'm wondering if your model would interprate correctly from dataset, that past some velocity, excessive speed is not a good thing : is you rocket powerful enough to meet that specific point ?

That's the thing. What I would like to do, is provide zero domain specific knowledge to the algo. Asking it to check if the Thrust / wieght ratio is better at some stage is hard coding what we want it to do. Again, in this context, it is easier to solve the problem of flying like this, but I'm not going for the easy approach (I mean, where's the fun in that? :P)

What my algo would be able to learn, I would know once I'm able to get it working ^^ Which, for now isn't the case, because I haven't really gone back to it. I'm working on other projects at the moment, but this one is still somewhere in the backk of my mind, and I do plan on going back to it at some point!

At the moment, my rocket doesn't have enough fuel to go high enough for drag loss to be the most important ones. I'm only doing a 10 or so seconds flight for now. Hopefully in Bertrand 2.0 that would be implemented. But for now, Bertrand 1.0 isn't even here!