Search the Community

Hi everybody! (UPDATE on page 3, it's now sort of working) Hope I'm in the right section, I don't come very often to the forums, so just tell me if I got this wrong! So here's the idea: for a few months now I've been thinking about hooking a machine learning program to a rocket and pressing 'Launch' to see how bad things could go. I've looked it up, but it seems not much people have * actually* done it successfully. So HUGE disclaimer: I have no idea where this is going to go. Maybe I'll stop in 2 weeks, maybe I'll get super on board with the idea. But I think there's something really cool to try here and that's all I need. The main idea I have for how to do this is: - Control KSP through the kRPC mod, enabling us to run Python, which means we can import any machine / deep learning librairies alongside it. - Use reinforcement learning to learn from each launch, and try to improve the performances of the rocket. Note that here I'm not trying to generate a rocket design, I just start from a particular rocket and try to teach it to fly it. To put a bit of context around this, I have a few months of practice on neural networks (using mostly Keras over TenserFlow, which will most probably be the library used for running the basic neural network that will be trained) I have barely used kRPC for a hours the last few days to say what was possible with it, I just made a successful program that reloads a quicksave when a rocket after a certain time and looks at the altitude it's gone, to see if it was possible. I've also been through some of the documentation. Oh, and I've never done anything related to reinforcement learning, so just like when I started playing KSP a few years a go, I don't really know what I'm doing, but I have theoretical knowledge, and think this could be fun, that's good enough for me to start! I thought opening a new post here on the KSP forum could be a good place for me to first of all take the time to lay down what I want to do, and get some feedback from people that might have more experience than me, maybe some people might even want to follow progress on this! So to make things a little more interesting, I'm gonna give my IA a name, because I don't like how we just refer all the time to "IA". I'm going to call my 'IA' Bertrand. So, here are the initial ideas I have (this will be a bit technical as the trickiest part of this is to get a neural network running and interacting with KSP, while minimising the RUD counter). - In order to make any reinforcement learning, we need to define a loss function, this is what the neural network is going to have to minimize. Basically what is Bertrands purpose in life? What I would like to try for the very first stage is to get the rocket to a certain altitude. Say 50km to start with. That sounds super easy: just full throttle and nothing else. Yeah, but you haven't seen how dumb a untrained neural network is. So basically the reward function would be something like a function of the altitude, say Altitude*10 and you get -500 if you blow up (this is just a rough idea first, I have to figure out the specifics later). I think taking into the account the amount of fuel left when reaching the altitude is also important, as that's what we want it to do: be the best rocket out of all the self-teaching rockets out there, and make the best ascend possible, using the least amount of fuel possible. - Second, the controls: Bertrand, my IA, needs to be able to control the rocket to make it blow up fly it. But the thing is, with KSP, controls aren't really something we're short of. Remember, Bertrand is going to be really, really dumb at first, we're talking cat-walking-on-a-keyboard-level dumb here. I thought having control over yaw, pitch and thrust would be a good start. Since most rockets are cylindrically shaped, roll doesn't have much of an effect, and I think keeping just 6 buttons (2 for each control) is already going to be hard enough. I'm also afraid that if we start letting roll in, it will heavily confuse Bertrand, because it will change what yaw and pitch do. If this thing turns out to be a good idea, I might add multiple stages to my rockets and thus decoupling as a control. (I'm also pretty curious to see how it starts just randonly decoupling at any time and things just blowing up even more). But I'm worried that at first decoupling will make the capsule hop and give an easy win, so Bertrand keeps doing it without learning anything else. But we'll save that for later, let's not spoil Bertrand too much for now. - Third, this is going to need to run a loooot of times before Bertrand starts doing something not stupid and actually understand it has to throttle up and not do anything else to lift off. This is where my little kRPC program comes into play. I can make it reload the game as soon as the ship blows up, or reaches a certain altitude, and store the final altitude at which the game is reloaded. That's what I managed to do in a few hours of tinkering on kRPC, but I'm also going to need to register all the keystrokes Bertrand does, those are going to be the inputs of the neural network. That's a bit more tricky, but I don't think that going to be very hard. The hard part is going to be feeding that to the neural network. - Fourth I only have a Xiaomi Mi Notebook Pro, and that thing only has an 8th gen i5, with 8Gb of Ram and most laughable of all, an mx150 for a GPU. This bad boy doesn't have any issue running KSP at beautiful max settings and 60fps no problemo. But we're gonna have to train Bertrand too. And Bertrand is going to require some serious hoursepower to get better. This means I'm probably going to run KSP at a stupidly low resolution, and potato graphical settings to just even *try* to train the neural network. This is why it's going to be very important to keep the neaural network architecture as simple as possible! If that turns out to be really too long, I'll see if it isn't possible to use a Kaggle Kernel to get access to the free GPU and connect it to kRPC. Otherwise I might have to pay an online GPU, but I'd rather keep that as a last resort, I'm just a poor lonesome student with a bit too much time on his hands, but surely no money to rent a GPU. So I'll simply take this as the "optimization"' success trophy in my Bertrand plays KSP game. And if hooking a Kaggle Kernel to kRPC doesn't work, I'm not even sure it would with any other paid service; but that's for later anyways. - The idea I have, from this article from people that previously seemed to have relatively done what I want to do would be to use OpenAI's Gym, as a support for the reinforcement learning. OpenAI launched a thing called Universe a while back that was supposed to support KSP, but that got cancelled before we had time to see Terminator-powered rockets take over. But these guys (the ones from the article) made their own environment compatible with the Gym, which I will very happily clone, and tweak according to what I will be trying to do (after I eventually understand all of it). I'm basically going to use what these guys have done to try to get something working. I do think I should spend some time learning how to use OpenAI's gym on easier examples first before doing that though. So Bertrand isn't even going to see the world inside KSP first! That's basically how far my reflexion has gone for now. I've spend a good amount of time going through the internet looking for people trying this, and as I mentioned earlier, didn't seem to find anybody that did achieve this, apart from the guys who's work I shared in this last point. I also saw that a few years ago a guy streamed his neural network training, but that's long been finished, so I can't see that anymore Just to conclude here is what I have already done: Using kRPC, I made a program that goes on for 5 epoch (or 5 times) and each time makes the rocket turn after epoch_number seconds, then waits 5 seconds, gets the altitude the rocket got at after those 5 seconds, and reload to the last quicksave. These altitudes are stored in a dicitonnary called Dict_alt that is displayed at the end. This was my initial test to see if it was simply possible to be able to run multiple launches one after the other without me coming in. And storing in a dictionnary will probably need to be changed to a dataframe using a cuddly panda. TL;DR: I have no idea what I'm really doing, but I want to train an IA to fly a rocket, and I have a lot of work ahead of me for this to even remotely work. Anyways, if this is interesting to follow, well take a seat and prepare to wait a long time, and if you have any good ideas on how I could do this, things I have forgotten even if I haven't even started or improvements, please let me know! As I said, I don't really know what I'm doing with this, so all and any suggestions would be greatly appreciated! Thanks for having taken the time to read this! I hope this turns out to be a real project, maybe!

Hello everyone. First of all, I hope this is the correct subforum to post this, because I couldn't do it in another one. I'm an Electronic Engineering student and I'm a beginner in the aerospace world, but last two weeks I've been reading a lot (specially from NASA (earthobservatory.nasa)) about this. I have to make the final project of a course called "Neural Networks" and I decided to do something related with micro-satellites but I don't know what. I found two ideas: - A complete control project, by Schreiner (or something like that). It's too complicated to do that as my first project - An atmospheric density prediction, based on CHAMP measurements. I liked this project but I was told that it's not too much useful to predict the drag force for micro-satellites. Does anyone have any idea? Thanks!

This post will be very long, but I think it is important that we get involved, which does not mean that we can do something about it, but we can be more prepared to face that final step as human beings. (Sent me a pm if you find some English mistakes that makes a sentence unclear) Introduction: If I need to make a prediction, I like to include as many variables and data as my brain allows me. But there is always a particular variable which I choose to ignore, because if it is unleashed, it destroys any possibility of accuracy in the prediction. That variable is the moment when our technology escape from the limits imposed by our brain. In where the research and conclusion is done by the same technology which can improve itself in a positive feedback loop; generating an exponential explosion. To understand how big will be this change, we need to know first how our technology and human capacity evolved in the last 50000 years. Our brains hardly changed in this time frame. We already had language to help us to share discoveries, but it was not until the writing that we became more efficient in knowledge accumulation. Machines, population, cheap energy; all played an important role in transforming our linear slow growth in something more exponential. But our brain is still the same, our intelligence did not develop for visualize and understand complex concepts beyond our everyday reality, due this, we depend a lot on the experimentation to move on. The new age: We already enter in the age of self machine learning using neural networks and evolution principles. In case someone doesn't know, all the latest biggest software advances like speech recognition, image recognition, concept understanding, new search algorithms; between others, was achieved by these new neural networks structures. The trick was to mimic the things we know about real neurons and our way to learn, which is all based on how data are related. How neural networks work: Brain vs computers: In the past it took us a lot of code engineering and hundreds of experts working by many years, just to try to make an algorithm to identify objects in a picture or a song in the radio. They first started with 2 % of accuracy, then 5%, 8%.... many years later 25%, the first year Deep Learning go out (a new NeuNet algorithm that needs less human intervention and other characteristics) already achieve a 40% in a very short time without those hundred of engineers. Now the % of efficiency in any of its task was increased considerably. There are some small hardware chips which recreate the structure of an already trained NeuNet that can identify people, cars, and other objects in a video surveillance camera only consuming few miliwatt of power, whereas for the same task with normal programs would consume a lot of power. In 2011 IBM win the Jeopardy game using Watson, a supercomputer base in NeuNet, who was able to read Wikipedia and relate all its content, then it keep learning in other areas as Medicine, analytics, cooking, sport, advisor; helping to researches in a way that until now nobody could. Watson Links: Jeopardy, How it works?, as Advisor, Learning to see, General knowledge We can feed these algorithms with raw data as pixels in the screen, without teaching rules or nothing; the computer will learn what to do by itself just looking the screen. In this case; learning to play video games. There are two drawbacks with this technology. 1- Learns on its own, the acquired knowledge is not fully controlled by us. 2- We don’t really understand why it produces an outcome, because "it’s a complicated machine", then we cannot predict what it will do. Google in recent times bought the company DeepMind, which it has as goal to create a true AI in where our friend Elon Musk also invested some money to ensure that the necessary security measures are taken, this also gave him the opportunity to keep an eye on the development of this technology. Examples of today breakthrough with Deep Learning: The human conclusion To see if these neural networks really show intelligence traits in their results and what it is needed to achieve consciousness, first we need to find a better definition of what we call intelligence and consciousness. Michio Kaku makes a good job answering this question in the first 10 min of this video from a physicist's point of view, I recommend. Taking a look to deepmind papers and last breakthrough in neuroscience with the different brain mechanism, we are close to create an algorithm that would learn in a similar way as the brain; this does not mean that it needs to be equal, just needs to work. We can make airplanes that fly well without the need to imitate all the complex movements from birds. I realized that when looking for news in this field by selecting the option “this year” is not enough; you need to select months or even weeks given how fast it progresses. So this take us to our final question: -and then what? Well, we will reach the time when our brain is not longer the limit to our technology, our slow way to learn, test and developing will be over. Our technology at this point allow us to create a learning machine smart enough to improve its own design, this point in time is called “THE SINGULARITY”. Even today, it is becoming more difficult to make predictions, but once we're in the singularity, all our predictions collapse, we can no longer see the future, neither (to a certain degree) the machine that is driven this. At this point all resources are mostly focus to improve the power of this Hard IA, any other application of the new acquired technology will become in a waste of time and resources. Why? Because the knowledge will increase so fast, that any application that we might think of as useful, it will be outdated in few months by the new discoveries, a Hard IA does not need experimentation to prove new theories (which is something that consume us a lot of time), it can do it only by deduction. We will reach a time when we (or it / Hard IA) will double all human knowledge progress in just 1 year, then we double again in 1 month, then again in just a week. There is not hard to imagine no matter how complex the universe, all possible questions will be answered in a very short amount of time after the singularity, this means jump from a limited knowledge to a godlike knowledge without middle app steps. So, when will this happen? They made this same question to many scientists and people working in the field in 2012; the average answer was by 2040, but many of those specialist was not even able to predict the grade of success that deep learning has today in just a period of three years. Elon Musk said it may happen in 5 to 10 years. if I have to make a prediction, I would say that in 10 to 15 years, even 15 years looks like an eternity at this accelerating rate. We saw many signs like this in the past, but this certainly reflects the end of predictions, with respect to the Time magazine issue, that note was in fact about the singularity, it was released in 2011. So this make us think; what about all our silly predictions about in how long mankind will begin to colonize other worlds?, or the technology needed for a Von Neumann probe?, how long until we reach another star? what about our life plan of have grandchildren and die from old?, Global warming really matter? We were always so wrong to ignore this variable in all our predictions, but well, maybe now we are more prepared to explore and enjoy these last years of life as we know it.