Jirokoh

That's the thing. What I would like to do, is provide zero domain specific knowledge to the algo. Asking it to check if the Thrust / wieght ratio is better at some stage is hard coding what we want it to do. Again, in this context, it is easier to solve the problem of flying like this, but I'm not going for the easy approach (I mean, where's the fun in that? :P) What my algo would be able to learn, I would know once I'm able to get it working ^^ Which, for now isn't the case, because I haven't really gone back to it. I'm working on other projects at the moment, but this one is still somewhere in the backk of my mind, and I do plan on going back to it at some point! At the moment, my rocket doesn't have enough fuel to go high enough for drag loss to be the most important ones. I'm only doing a 10 or so seconds flight for now. Hopefully in Bertrand 2.0 that would be implemented. But for now, Bertrand 1.0 isn't even here!

UPDATE: IT'S (sort of) WORKING!! First of all, because that's the coolest thing, here's the video: I've had a bit more time on my hands, since my final exams are finished. I still haven't completed my thesis yet, but I'm nearly done, so I can come back to KSP. So, what I've done is basically train the neural network not only on the time segment where it currently is at, but look at the previous ones as well. When it's train and comes time to predict the action to take, Bertrand looks at the observations from the current time step as well as the previous one (so it looks at (t) and (t-1)), and makes a prediction based on that. As you can see, it's not perfect, but it's doing (modestly) better than random already! In the example seen above, it takes in total 9 actions: it throttles down for the 3 first, but then for the following 6, it always go throttle up! And the cool thing is, that was only trained on 30 examples which is a really, really small dataset. I'm way too excited that this (kinda) worked, so I shared it before trying to improve that with more training examples. The neural network itself can probably be improved with a bit of fine-tuning, so I still have a lot of work I can do. So, yeah, there you go. I made an IA that can (sorta) learn totally on it's own to throttle up to go higher. (well, technically it doesn't yet have time to go higher, since it always goes down for the first 3 steps, which makes it loose of thrust at the very beginning and there aren't enough time steps after to overcome that early loss. But what I mean is that it then just goes full throttle, which on the long term is what I want it to do).

Yep, I actually started with quickload, and in both cases I'm incounter the same issue. But, when I have more time, there are a few tricks I'd like to test, that might allow me to solve, or at least work around that! We'll see!

KSP. Apparently, if you revert to launch too many times, RAM starts getting saturated. That's just what I found online, but what I did see is that indeed, as I revert to launch more and more, my frame rate goes down. And I'm already at the lowest settings and lowest resolution to make this run as easily as possible for this.

Well, I've been thinking about coding a "simple" KSP sim, but I'd like to try to see if I can't solve the problem in KSP first. Because, while this solution might work for this easy problem, it's still only the first problem I want to try to solve. If I can make this work, I'd like to go further, and tackle more complicated problems. That would mean also making my simulation more complicated, to train it on those more complicated issues. But, maybe, why not? I think I'll have to wait to have more time on my hands to do that then, so maybe only in a few months! While I am getting 1 data point for every run, each run is only 12 to 13 seconds long, and during 1 run I actually get 6 / 7 meaurement points. So it's not that bad. The real issue is the RAM leakage, meaning I can't scale my data scrapping on long periods of time. If I could, I'd just leave my PC to run during the night and get data like that. It would be long, but it would work. Fixing this issue is what I'd like to do, because it would allow me to at least be able to then scale to more complicated problems using the same method, even if each run would be longer for more complicated problems.

Thanks a lot to those how have been following this then! Hopefully in the next decade uh, weeks or months I'll actually have something working! I'm not sure I get your idea here? The goal is to have something learn to fly in KSP, not in the real world (though, if people have waaaay too much money and don't know what to do with it, my DM's open, just sayin'). So I would rather have to make an environment (understand code the said laws of physics, gravitation, atmospheric drag, intertia, you name it) that mimics KSP, but runs much, much faster, in order to speed the training process up, while still being in an environment that is close enough to then transfer the learning into KSP. No problem, all ideas are welcomed :-) Well, my bad then!

Thanks a lot! I'm very glad to see people have an interest in this! (and obvioulsy no, having an AI is not cheating, as long as it's mine) EDIT: damn, those other posts are impressive. I feel like I've got to raise the bar in order to match what the others are doing! Awesome to see so many different fun projects!

Again, I think we're going off track here. And, to be honest, I have some doubts about being instinctly afraid of a lion. I'm no expert, so I don't want to go deeper into this sort of discussion. It's only going to go more into what probably both of us don't know about. Thanks for the book recommendation though, I'll have a look! In other news, still pretty busy at the moment, so I haven't gone back to improving anything or making any progress. I'll keep you guys posted!

I'm not sure about that. From what I understood, it's rather that we have a hugely adaptable brain that can learn very efficiently and understand correlations very easily. It feels like the brain is an awesome correlation finding machine, which does allow us to in turn learn how our environment works, rather than that being hard-coded. That's why blind people, semi-paralized, or others can learn to interract with the world in a totally different way than you and me do, it's adaptation rather than rule based. Just like good code: it's scalable to new problems. But we kind of digress here. (still, super interesting topic, which I know way too little about compared to what people working in the field know. So, as the rest, don't trust what I write here )

Well, this analogy works but only up to a certain way. Humans are stupidly efficient in the way they learn compared to even the best of the articifical intelligence that we can make today. So, at the level where I am trying to make this, it's really super basic things. I think we should keep in mind these are analogies, that are good at explaining the big picture when first trying to understand these concepts, but that rapidly become ireelevant when trying to build or debug implementations of these algorithms. At the moment, the issue I have is more on getting enough data, and being sure that my model is actually learning what I want it to, so this starts really being outside of the boundaries of such an analogy. But again, these are great for explaining in simple terms how in general machine learning works! But we should be very careful not to push the analogy too far, that becomes anthropomorphism, which is starting to become irrelevant. There's still so much to do before we can even consider AIs to be really learning like humans. Even more in my case: this is really a dumb program, it's basically trying to find correlations and a local minima of 3 (for the moment) different data parameters.

Well, that's basically what I'm doing already, kRPC is controlling everything, and the only parameter is throttle for the moment. Pitch will come later, when I'll have something working for just thrust. This is more of a test bench to see if it can actually work. And, it's not *really* an evolutionary alogrithm, that works a bit differently, but the point about data is still pretty much the same. That is the biggest problem I have at the moment. Again, tha'ts why I'm only trying to teach it to throttle, going straight up, for only 7/8 seconds And I don't think I'll be able to tranfer the learning just as easily to be honest.

That did cross my mind at some point. There are multiple drawbacks to that: - Even if it's not *that* hard on paper, it's still going to take an aweful lot of time for me to do that. because, in programming, just like in KSP, things don't quite go well on the first try - It has to be very similar to the environement of KSP. The IA is going to learn how things are going in the environement it has trained in. But if the envrionement in then moves around in changes a bit, I'm afraid it would get totally lost That's more related to the way machine learning models work, they are very sensitive to this kind of change. Usually, we try to avoid this as much as possible. But, since I'm only using altitude, speed and throttle at this point, it might as well be possible. The big issue might be when trying to scale this up. Bascially I'm going to have to make KSP from scratch if I want to do this ^^ It could also be an intersting benchmark to test the machine learning model itself, not necessarily to train it for KSP, but just to see that in this environement, it does learn properly. Because right now, I don't know my problem actually is a lack of data. It might be, but it might also not be. But the big benefit is indeed that once it's working, it would speed up training by multiple orders of magnitude. (And not to say it probably would be a great excercise for me to try to implement that) The idea is quite tempting, at least for this easy first case. I'll have a more in depth look at what it would take to do this, because on paper it is a good idea, and I'll let you guys know. (I just have a thesis to finish at the moment, I'm not sure I'm going to have a lot of time to devote to this unfortunately)

Yeah, I've been following Code Bullet for a while! But, as you pointed out, KSP is quite different from Pacman, or the other games he plays, because he usually makes the environement himself, whereas I'm using one that already exists: KSP So I'm trying to work around the limitations of KSP, that clearly wasn't made with people trying to make IAs with it in mind. But it's an interesting project then! I did the live-stream, thanks for those that tuned in! I have a recording on it, but it's not on Twitch, I don't know if I can upload it once the stream is over. (Streamed for the first time, so of course a lot of things didn't go as expected!) Anyways, the take-away is that my overall program runs for the moment, but it's just not actually training the way I'd like it to. So there's going to be a lot of work to figure out what the issue is, and fixing it. I'll post a bit more later I think.

WOOHOO I'm not dead (and I've actually been working a bit) Just wanted to let you guys know, I think I have a little something that seems to be working. The end result is actually very bad, but I think that's mostly due to lack of data (and maybe of my skills too). That shouldn't be a problem too long (I hope). Anyways, if you guys are interested, I'm going to be streaming what I found and explaining it tomorrow on Twitch: So, on May 12th, at 1:00pm UTC time (I live in Taiwan at the moment, a lot of the people I know that play KSP are in France, so that's the time that most fits everyone). That makes it: 9am EDT and 6am PDT for the folks in the US. Hope you guys will be interested in this! Though, I should warn you: Don't get your hopes up, this is mostly a first test of linking KSP, Python and some machine learning together, it's faaaaaaaaaaaaar from perfect. (I promise I didn't harm any Kerba... Nah nobody believe that, you know how Kerbal science is done)

Just saw your post now! Thanks for sharing I didn't talk about that, but I'm most probably going to be using Tensorboard, sticking to what I know. It's a great tool for looking into models on TensorFlow / Keras. No news on this project for the moment, I have exams coming up, and also spend a lot of time preparing for interviews for internships. Hopefully when that's passed I can go back to KSP.