Lisias

I got a misbehaviour that could be explained by skipped Updates on my old potato running Unity 2019 under heavy load - what's, frankly, all the time. I considered placing a log on the Update in question, but rapidly realised that doing disk I/O on it will induce huge latencies, screwing up the test I wanted to do at first place. Interestingly, things got incredibly smoother once I set MONO_THREADS_PER_CPU=1 on my rig. Now I can run KSP >= 1.8 on my old patato, and I can even enjoy it a bit as long the craft is under 40 or 50 parts. Without the MONO_THREADS_PER_CPU stunt, I got a slideshow already on the Main Menu, what to say flying something. My suspicion of Updates being skipped, however, is not due the stuttering, but due some exploratory code of mine failing under heavy load - and the most plausible explanation for it is the Update not being called. (and, yeah, the MonoBehaviour.enabled was set to true!). That was a major bork on mine. I had googled for the subject, ended up finding what I wanted to find and got myself trapped on confirmation bias. I should had read that thread way further than I did, but failed on it and ended up wasting your time. My apologies.

Don't bother, I already have the minimals. Now I need something concrete in order to know when I reach a good enough result - the usefulness of the minimals was expired. Whatever is the craft you are getting the misbehaviour, is the one I need now to use as benchmark. Don't mind the add'ons you used, this time I need to get my pawns dirty the hard way to get the results I aim. — — POST EDIT — — Oukey, I did what I do best: I tortured different versions of KSP about the issue until they talked. The whole test session starts with this comment: https://github.com/net-lisias-ksp/KSP-Recall/issues/60#issuecomment-1445226847 TL;DR: KSP 1.11.2 with Recall is recovering more funds than KSP 1.9.1 without, where there's no one trying to workaround the fund recovering problems. On 1.9.1, I lost 779 Funds on the sample craft. On 1.11.2 with Refunding, I got an extra 61. I wish I could install Refunding on my bank account. Serious now, it's really a float inaccuracy problem now. I put WarpPlugin in check, and it passed with flying colours - it is using doubles on the currency computing, what's really good enough for the amounts we are handling now (decimals would be the perfection, but double is good enough and, well, are faster). @GoAHead, I'm pretty convinced that you are being bitten by KSP now. Please just zip your savegame the way it is now, and then tell me the name of the craft where you are being screwed. I don't need to load the savegame, I know where to look now by screening the files. What I need to do is, as I said, get the current values being used by your savegame so I can benchmark them on my development rig. I will forge a simple savegame but using the values I find in yours, and then I will try to fix the float robbery by trying to steal back the error to your agency's account. As it happened on my tests using 1.11.2, I probably won't be able to steal back the right value because the float's mantissa will not be able to represent it, but a little extra funds are better than too much less, right? — — TECHINICAL MAMBO JAMBO — — To simplify things, I will explain how a floating point number works by using decimal numbers, way more easy to cope than binary or hexadecimals. A FP number has two numbers, the Exponent and the Mantissa - let's suppose we have a type of decimal floating points that can hold 4 decimal digits, 3 for the Mantissa, and 1 for the Exponent. The number one would be stored in memory like 001.0 - where the dot is just to separate the mantissa from the exponent so we can easily work with them. In Scientific Notation (science stuff!), we would write 1e0 , or one to the zero power. Interesting enough, since anything to the power of 0 is 1, the very same number can be represented in memory as: 111.0 100.0 010.0 etc So, we now had established that this representation can be somewhat wasteful. But it has its merits, because with only 4 digitis I can write some really big numbers using only 4 digits, as 1e9 or 1,000,000,000 and not using 10 digits as I did now. But there's a catch: I can't represent 1,000,000,001 !! Because I have only 3 digits for the Mantissa!! Using a 9 as exponent, I can only represent the following numbers: 001.9 = 1,000,000,000 002.9 = 2,000,000,000 ... 999.9 = 999,000,000,000 You see, everything under 999,999,999 is squashed - you lose precision. Now, on real word, since the floats uses 4 bytes in binary notation, the standard says that for 4 bytes floats (32 bits), we represent things as: 1 bit for the signal (positive or negative), 8 bit for the exponent (-127 to +128) and 23 bits for the mantissa. We use base 2 for this representation, i.e., our numbers have only 0 e 1. The same way we can count from 0 to 20 in decimal this way: In binary we count: Both lines are the same numbers, but on decimal we have more digits to play before a "carry 1" when we reach 9 (i.e., when we run out of symbols). With doubles we have more bits for exponent and mantissa, so bigger numbers with better precision. But since GPUs cards are optimized to handle floats e not doubles, Unity choose to stick with floats internally. KSP uses doubles for the physics - converting between them all the time can lead to problems, but this is another subject. Understanding floats in binary is a bit scary, but someone had this excellent idea: https://bartaz.github.io/ieee754-visualization/ You can flip the bits, or type numbers, and see how they are stored in memory. You will understand, by toying with it, why some numbers can't be accurately represented on floats. The way I solved the problem on Refunding, was by using a C# datatype called decimal. This thing is a huge bitstream with 128 bits, being 96 bits representing a binary integer value and the remaining 32 bits are flags and the Scaling Exponent. These numbers are different from the Mantissa/Exponent used on IEEE 754 (the IEEE standard almost everybody uses for floating points), but they play similar roles. The end result is a way slower computing, but with way better accuracy. Since the number ends up squashed to a float by the IPartCostModifier interface, Refunding can't assure the accurate value will propagate into KSP - but at least Refunding is not pilling up inaccuracies itself. It was using doubles before, but part of the computations were scaled down to floats due this begin the datatype used by the Resources's amount and cost, and this was an error (as I was pilling up inaccuracies while cimputing the resource's costs). Things are better now because the inaccuracy happens only once, when the final (good) result is squashed to a float by IPartCostModifier. But depending on the value, it will be enough to still screw up the recovery. I will try to calculate this loss, and then compensate for it on the process.

What doesn't changes the fact that all you need to check KSP1's assets is Windows Explorer and Notepad. Hard to say KSP2 made things easier for the beginners. If you need to be a professional on the field to mod KSP2, the franchise lost contact with its fanbase by a mile.

The Refunding stunt is, well, a bit fishy by itself. I'm (ab)using the resources to counter-screw the KSP's failure on correctly calling the iPartCostModifier interface of the PartModules on recovery - at the same time preventing screwing up with other code that is doing it, so when the Refunding resource is active, the PartModule's IPartCostModifier returns the negative value of it to prevent 3rd parties from being fooled about the cost of the craft. I'm, literally, leveraging the KSP's bug against itself. And over it, yes, we have this float on currency happening. What I can say for sure is that, now, Refunding is definitively out of the loop of the troublemakers. The only enemy left is the unhappy use of floats when handling IPartCostModifier. What doesn't means I can't try to do something else - but I need to do it carefully, because I may inject undesired side-effects on 3rds parties as it happened with MAS. I'm over my toes now. Well, back to the designing table. Please send me your craft file (you can zip it and drag'n'drop it on a post on the github issue, if you have a github account: https://github.com/net-lisias-ksp/KSP-Recall/issues/60 ) so I have something concrete to work with. I have some new crazy ideas… But I need to figure out what of them are just plain stupid.

ANNOUNCE KSP Recall 0.3.0.11 on the Wild, featuring: Rework Refunding to use decimal when handling currency to minimize losses due inaccuracies related to float point math. Kudos to GoAhead for the heads up! Closes issue: #60 Weird Misbehaviour on Interstellar Technologies This releases mitigates (besides not solving) a problem related to KSP using floating point math when handling currency, reported by the Fellow Kerbonaut @GoAHead(thanks a lot, dude!). The problem happens because floating point math induces inaccuracies when numbers are big (or small) enough and the whole number can't be stored into the Mantissa anymore. Refunding, the component working around a bug on KSP while recovering crafts, was updated to use decimal instead of doubles while handling currency, eliminating such problems internally. This makes things better (or less worse), but doesn't completely solve the problem as the IPartCostModifier, the interface used by KSP to ask the PartModule for its cost, is still using float - so, in the end, really big or really small values are still going to get screwed. But at least I could remove some inaccuracy, as they are not being pilled up inside Refunding anymore. — — — — — This Release will be published using the following Schedule: GitHub, reaching manual installers and users of KSP-AVC first. Right Now. CurseForge. Will not be published due a new release. SpaceDock (and CKAN users). Will not be published due a new release. I would like some early adopters to install this thing and use for some time to be sure the stunt I pulled while persisting the current PartModule cost on the Config File will really work in all the use cases.

Yep. Problem is that by doing it, they are risking creating the failure themselves. Most of the backslash is coming from the pricing being asked. Had they published a very minimalistic demo by 10 bucks, you can bet your mouse they would having way less criticism. Software Development is way counter intuitive - the current state of affairs doesn't means necessarily that they screwed up. It probably means that their current customers are the Content Creators and not us, but then some pointed egg head forced the hand on publishing the thing as is and Kraken help us all. The Content Creators need tools to do their job, and if the dev team spends time polishing the engine for us before the Creatives have their tools, we would have no Content at all.

Good to know, and it's fine to me. But I would prefer that the P/R guys didn't started this fire by enticing us with this "day one" buzzword because, hell, the only way to do that right now is by violating the forum rules!!! Source.

Moving back from the KSPIE thread. — — — the "IXSMainHullReactor" worked fine (or almost) on my rig, the refunding were happening almost perfectly. I'm recovering almost all the 44.5M Funds, with an error of less than 1K Funds (I started with 45.000K and ended with 44.999K). The problem appears to be related to the huge amounts of Funds involved, I think we may be handling a float rounding error on this one. In time, it's the reason using floats on currency in Real Life™. I also tried the "MuonCatFusionReactor2", but scaling it down to the 1.25M of the Command Pod Mk1. Interestingly, I got the same problem on this one - I started with 100.000F and ended up with 99.999F - an 1F error. @GoAHead I think this is not a "problem" neither on KSPIE, IT or even Recall - this appears to be related to using floats for currency calculations! I can mitigate a bit the problem on Recall (by using decimals while handling currency), but I can fix it 100% because KSP itself uses floats, so this is beyound my reach: I can't fix it, all I can do is to minimize my role on the problem. I'm working on it. My apologies to FreeThinker by polluting his thread, I genuinely though this was a IT problem. @GoAHead, lets keep this on the Recall thread, please!

Dude, you nailed it! Thank you very much! This is not a problem on KSPIE, IT or even Recall. This is happening because KSP uses floats for currency computing. Lets handle this on Recall thread, as any mitigation measures will be implemented there. My apologies to FreeThinger by polluting his thread, I genuinely though this could be something triggered by IT on one of your PartModules.

Well, not exactly "Not a Problem"… From the Add-on Posting Rules: So… Not so modding friendly after all. The very initial efforts for modding KSP2 is in violation of the Forum rules for modding...

So, no public API and we are back to hacking the game as we did in KSP1. Not a problem, but I remember Nate talking about a public API and dismissing Manley's concerning about decompilations et all.

As long you don't try to create threads for background processing - or, if you do, you don't use any Unity calls. I'm not talking about lack of parallelism on Unity. I'm talking about threading being not the preferable way to do it - to the point Unity itself is royally screwing you if you have too much cores with too many threads available (see that link of mine about MONO_THREADS_PER_CPU). My point on my previous post. It can be useful when you have a lot of crafts scattered in the Universe, but managing colliders for near crafts can be slightly more complicated. You will, at very least, need to synchronise the threads before checking for collisions - what probably will waste any savings you got with the stunt on a low craft count gaming. Now I'm guessing - so taking anything I say here with a considerable grain of (iodized) salt. It's my understanding that in the same way the Unity's Update callbacks can be skipped under heavy load, so do the Burst jobs. At least, is what some posts on Unity forum is telling us. If you built a job to handle the fuel system, and the job is skipped by some reason, well… You have a faulty fuel system with a misbehaviour really hard to diagnose...

It wasn't told to us that they were expecting modders to jump in on day 1? How?

Well… This is a collateral effect of Refunding (ab)using the Resources in order to correct the mistake Stock is doing while refunding your craft on recovery. The thing calculates how much you should be refunded, and then creates a "fake resource" that will add or subtract an amount of resources priced in a way that in the end you get the refunding you deserve. I had noticed that resources were being correctly handled by Stock, the sole problem is not calling the "IPartCostModifier" on recovery. I had set the Refunding "meta-resource" to be invisible, but on recovery the game show it the same.

Make no mistake, I believe you. I had dealt with similar problems before - what makes me very uncomfortable is that a few of them just vanished from the surface of Kerbin, and I never diagnosed the problem neither understood how it happened. This could be a race condition - but this is wild guessing right now. I'm rebuilding the test bed, I will come back to you soon. — — POST EDIT — — I **confirm** that on my rig, KSPIE is not being affected by the but - or, at least, the part I used. See https://github.com/net-lisias-ksp/KSP-Recall/issues/60 for the details, screenshots and zip files. @GoAHead, please try to use the same part I used ("Antimatter Catalyzed Fusion Reactor", the kspiWarpCoreUnit2 on WarpCoreUnit2.cfg.) and see what you get. I may be dismissed KSPIE too soon, but if by any reason the very same part that works for me () will bork on you, then we have an environmental problem - and this is going to hurt a bit to diagnose...

Yes. Up and down. Humm… So you are telling me that the behaviour had changed? This is really uncomfortable. I'm redoing the tests tonight, I will edit this post with the results.

TL;DR: reactors from Interstellar Technologies are being misrefunded, but reactors from KSPIE - at least the ones I used on my testings - weren't. The affected PartModule is IntegratedThermalElectricPowerGenerator. Again, only the part configs from IT were triggering the problem on my tests, KSPIE parts using this PartModule didn't triggered the problem for me. On one specific test (using the rbug.craft provided) reproduced the misbehaviour using Interstellar-Technologies-ICFSC-Reactor, but when I replaced it with kspiWarpCoreUnit2, the problem was gone for me.

I was talking about a hypothetical concurrent physics engine - if you have many threads simulating different partitions from the world, you need to syncronize them before detecting collisions, otherwise you will get some pretty hard to debug bogus collisions due race conditions.

Nope, I'm very up to date. See this thread (it is less than a year old). I'm talking about the GAME, not the engine. Additionally: If you create MonoBehaviour script and do things in any Fixed/Late/Update() function, this runs on main thread. Most of the Unity API's are not thread safe. Meaning you cannot create seperate thread and interact with the engine API's. Source (April 2022) Yep, physics is generally not well suited for multithreading, unless you stop thinking serially and start to think "distributively". Different crafts on different frames of reference don't interact with each other and, so, dont need to be simulated serially. Even two different crafts in the same frame of reference but not touching themselves can have their physics calculated in parallel - as long your data structures are tailored for that. Detecting collisions are still a problem, tough.

I agree. KSP2 is going to take some time to get traction - we will get a burst of downloads, then a lot of fuzz about, and then some (probably most) people will get bored and will probably go back to KSP1 for "real playing" - or play something else, if they perceive KSP1 as a "dead game". KSP1 is not dead yet. It eventually will be, but not yet. Until there, I think it's wise to give it some attention and care - it will be still be played by lots of people in the near future - or none of them will be played at all, to tell you the true. Keeping people interested on KSP1 will help KSP2 on the long run.

I can't say anything about KSP2, but I know that is shares some things with KSP1 - as Unity. Unity games usually relies heavily on single core performance - Unity, or at least the part of Unity that KSP1 heavily relies on, is heavily built [optimised] around a non threaded paradigm. As far as I know, the Unity's physics engine is not multi-thread capable - but this may be a weakness imposed by KSP1 itself, where all the Universe shares the same Frame of Reference at a given instant. The weird thing about Unity, mainly 2019 at least, is that the more cores you have, the worst the performance gets - believe it or not. So, high frequencies, low core count CPUs are the best hosts for playing KSP1 and, probably, KSP2 too. Memory is simple: the more, the best. I think that there's a point in which too much memory may choke the Garbage Collector (mainly due it's being hindered by the problem I described on the link above), but I didn't reached it yet. Your main problem is VRAM. From KSP 1.4.3 towards 1.12.5, the main factor that screwed performance on my rig was the high VRAM consumption. Even lightly modded KSP1s are getting screwed lately when using 6GB VRAM GPUs due the excessive use of VRAM by KSP1 and, lately, PD-Launcher. I got very promising results on my really old MacPotato by downsampling the DDS textures on KSP 1.12.x - not to the point in which it performed so well as on 1.4.3, but still better (the new ground rendering introduced on 1.8.0 is also pretty heavier on my rig). But, this is KSP1. Let's talk about what I hope to get on KSP2: Multiple Frames of Reference simultaneously. With each main body having its own origin, everything under its Frame of Reference can be calculated on a separate physics engine instance, each one on its own thread, and so we can have some very perceptible improvements on performance on savegames with multiple vessels scattered on different stellar bodies. Seweing the different Frames of Reference together will be pretty complex, but it will worth the pain IMHO Whatever Unity did screwed on 2019 that ended up getting worse performance as more cores you have, I really hope they had fixed it on Unity 2021. This is just abhorrent. I have better performance and lower CPU temperatures by reducing the number of threads per CPU on KSP1 to a single one. It's insane. Better Texturing Quality granularity than on KSP1 Way better VRAM management than on KSP1 Running on Windows 10. Microsoft is heavily pushing newer hardware for Windows 11, no matter the performance specs of the rig If KSP2 manages to tackle down at least 3 of the points above, we will have a better experience on KSP2 for sure - and if your rig can manage 1.12.5 at max settings, it should be able to run KSP2 in my humble opinion. Eventually, at least. — — POST EDIT — — I rephrased a statement above. I think it's more… faithfull… to what I'm meaning. I hope.

Nope. Windoze only, sorry. There were an official statement saying the game would be launched on 2020. This is one of the points on KSP2 in which I'm terribly pessimistic. Multiplatform support is something that should be worked on since the very first commit of a project. As much Unity tries to abstract things, abstractions leak, and they leak a lot - to a point that most development teams give up after a certain point of development time. As a proof os concept, take the Client Apps for Steam, Epic and GoG. Only GoG really works fine natively on MacOS - all the others need a loopback filesystem with the case sensitive filenames turned off. A pain in the back, I tell you - and this is only one of the problems to cope with. I hope, but I don't expect, a native Linux and/or MacOS for KSP2. Ever.

I can't say I'm disappointed because since the pandemics I'm resigned to cope to a lot of changes on Real Life™, most of them for the worst. And I'm convinced that this is not going to happen only to me, I don't expect my life to be better (or too much worse) than anyone else's. So I kinda knew that KSP2 would have a somewhat bleak launching - people had too much high expectations for it (and I think I can include even the developers on this wagon). And I think this is OK- lots of things on my life didn't gone as I wanted, but some of them gone as I needed, so I'm counting my blesses. Sometimes it's not evident, but I'm deeply grateful to a lot of things that I didn't wanted to happen, but it happened anyway and they ended up being what I had needed to happen. And I think KSP2 is going to be something like that - not what we want, but hopefully enough to fulfil what we need: consensus is not something that makes everybody happy, but something which makes nobody angry. KSP1 started as a very humble game. It grown a lot in the process - not without a lot of pain. I expect KSP2 will do the same - it will, eventually, be a hell of a game in the same sense KSP1 became a hell of game nowadays - there's still an incredibly marvellous game buried on all that collection of bugs, and that's the reason I'm still digging on it. “The sculpture is already complete within the marble block, before I start my work. It is already there, I just have to chisel away the superfluous material.” Michelangelo Coding is the same for me. It will eventually succeed, as long we keep digging. And hopefully modders too. The scene needs new blood, with more time to spend on a hobby and less desire on making this a "profession". KSP1 had a vibrant, colourful and intense modding scene in the early days - shifting to KSP2 without bringing all of that back will be way less attractive than otherwise. Is KSP2 going to be an elitist scene like X-Plane or DSC, with add'ons made by companies and not rarely costing more than the game itself? Or it's going to be fun and anarchic as KSP1 was in the beginning? What's going to lure add'on developers to KSP2? Fun? Amusement? Self accomplishment? Or money? (money changes everything). Are we going to need to pay money in order to publish something around here? This is concerning me more than anything else - the game itself, I KNOW will be fine as long the developers have time and resources enough - smaller teams with less resources managed to provide us with KSP1, I'm pretty sure these guys are more than skilled enough to do the same to KSP2. (mistakes will be made, the same way some already had been - but, hey, we are still alive and kicking, right?) I like graphics, but I like playing more. I'm very, very impressed with games like Tiny Combat Arena and Juno: New Origins - they are incredibly fun to play even on a crappy machine like mine. And, yeah, I spent more money on software than on hardware - it's one of the reasons I didn't upgraded my rig yet!! This came to my attention too. I'm guessing that these dudes took advantage of the initial momentum to get funding for things they knew will not be funded later once the money started to flow and the pointed egg heads smelled the incoming money. It's a sad state of the industry the one we have for decades already: the "fake it until you make it" mantra is still on the vogue. Theres a lot of things you need to get fast, or you will not get them at all. A fear the same. I really enjoy (most of) the videos, and I liked most of the things I saw on KSP2 until this moment - but as much as I think I will enjoy the game itself (eventually), I enjoy eating and paid bills more. I'm on a budget, and the machine used by SWDennis on his KSP2 test drive would cost around a whole year of the minimal wage on my country - and all of that to get 20 fps? I understand it was a beta (or even alpha) release of an early access game compiled in debug mode and all that jazz but… Gee… 20 fps on a machine that is way beyound the purchasing power of about 90% of the population of my country? This is not exactly the best way to get some money from the product right now: you can give the game for free, it will be of no use if people could not run it. I would deduce 18 months from it. Pandemics screwed everything, and on my line of duty, I closely overviewed the effects on a whole country. I have nightmares until nowadays due that (falling) numbers, wars didn't screwed things like that - and I know what I'm talking about. Hope. IMHO, this is the real threat to this game. Everything else is just life being life and people doing what people do. Manley doesn't talk anything that could detract the game since the developers complained to him in the past that he was making them look bad. Manley stopped to do things too bold since them (or, at least, it was what I perceived) to minimize crashes on his videos. So I pretty doubt he would criticise it openly on a video. Lowne, well, this guy is a grown "kid' - on the most positive sense of it. It's one of the main reasons I had enjoyed this videos in the past - he is enthusiastic, contagiously enthusiastic. I doubt he even noticed any problem on KSP2 while playing, he was enjoying himself. Check ShadowZone and SWDenis. These ones are being pretty critical on the matter (also, on the most positive sense). "The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time."

NOTAM It came to my attention that some users may be having problems with scaled parts on SubAssemblies. I could not reproduce the problem using the latest TweakScale (2.4.6.23 at this time) and KSP 1.12.5 - at least, using Stock parts only. Interesting enough, I got a single situation where scaling parts with variants left me with the wrong Attachment Nodes. But it happened only on the first craft I created on the first time I loaded Editor. Once I saved a craft and clicked on 'New', the problem didn't manifested itself anymore. Didn't tested yet if launching the craft or just quitting the Editor (or the savegame) and reloading Editor would make a difference. Something to be checked later. History suggests that we have something wrong on the Editor. Again. But I didn't zeroed in the root cause of this misbehaviour yet, so it may be something related to KSP-Recall. No (new) misbehaviours were detected on KSP 1.7.3, I will check other KSP releases (as 1.11.2) later. Any user experiencing this problem, please report. Cheers!

Approximately 50% of the TweakScale downloads come from SpaceDock nowadays, and I set up CKAN to download from there. I only wished things on CKAN would not take so much time to pass trough. It's a whole month already...