Win64 shouldnt be needed. Optimization time? I think so.

[Motokid600]

Yes, yes its 2015 and EVERTHING should be x64, bla, bla, bla... That doesn't change the situation. What we have is a great game running on a limited engine upon which the developers can only do so much with. So until that engine is more refined as Squad just said we must find a way to work around the memory limit. Win64 would be nice, but it shouldn't be needed. But as the stock KSP memory footprint grows with each update ( im sure 1.0 will give us the largest inflation yet ) the more and more people are going to be clamoring for their sixty four bits. This is where Squad comes in. The solution to that and 90% of the people who load their game up with mods lies in their hands. And its not Win64. Its this...

I think the answer here is, rather than try to fix something they do not control (Unity) Squad should fix something they do control, the loading system.

Right now, KSP loads every single texture, sound, etc in GameData during startup--regardless of when, or even whether it will ever be used by the game. That is the real source of the problem. If, instead, KSP switched to loading only those assets actually used in any given scene--during the Loading screen, it's not like this would somehow add loading screens, we already have them--then the memory problem would go a large way towards being solved.

Load on Demand--back when it worked, before KSP updates broke it--was a demonstration of this, and it worked wonderfully well. If modders can do it, I'm sure Squad can too.

Just, we, as fans, need to convince them that it's something we need. They've already said they'll focus on features relevant to all builds, and here's one of them. Let's persuade them to focus on this.

This is from Squads recent article on the subject. NathanKell hit the nail right on the head. That last sentence, "Let's persuade them to focus on this" inspired me to make this thread and get that ball rolling. Because in my opinion that is hands down the best solution we could all hope for and it'd be good for Squad in the long run. They can take their time and chip away at the x64 version with Unity until its ready and at the same time get most of the community off their backs about the memory limit.

Im one of the people that has to mod my game to the point of being unrecognizable from the stock version. So I am one of those people on Squads back , lol. Don't get me wrong. Ive spent countless hours performing my own optimization to cram as much as I can into GameData with that 3gb limit. You'd be surprised how much you can fit with a fair amount of work and subtlety. However if Squad were to work on such a thing I dare say I may not have to do that anymore. To an extent. Even with such a feature people will no doubt find ways to go over even those limits. RSS/RVE with a lot of big textures may still be exclusive to Linux, but as far as everything else goes you'd be hard pressed to hit the limits of a load on demand system.

And theres more. Pair that solution with OpenGL and that right there is all you'd need. The game im playing right now only runs in openGL. Ive got it modded to my upmost intent and its a stable beast. But no AA and a large performance hit is the trade off. Ive yet to try Directx11. Ive heard too many mixed comments. I would love for that to work instead of openGL. Dx11 would give you your performance and your memory savings. So instead of hoping for better x64 support from Unity. Would it not be better to hope for Dx11 support?

So what does everyone think? Is it time for Squad to begin this process? Is this the ultimate solution until x64 is straightened out? And what else can a player or.... Squad do to save memory? I say its time. 1.0 is five months out easy. Good. The longer the better. So now is the time to have these discussions. There is no way the game can release with an even larger vanilla memory footprint and not receive a significant amount of optimization. And in my opinion a load on demand system needs to be implemented by atleast the version after 1.0. Because once the masses start flooding the add-on release forum... its.. oh god...

Edited February 6, 2015 by Motokid600

[dlrk]

I agree. Squad needs to implement loading assets on demand....and they also need to stop pretending that mod users are in the minority.

[Red Iron Crown]

I agree. Squad needs to implement loading assets on demand....and they also need to stop pretending that mod users are in the minority.

I wish we had real numbers on what percentage of players mod, and to what degree. It's all guesswork at this point.

[Tuareg]

the problem is that they can't. what do you think. if they would have the ability, wouldn't they already have done it?

[cpast]

I agree. Squad needs to implement loading assets on demand....and they also need to stop pretending that mod users are in the minority.

I'm pretty sure mod users are in the minority, though. I have yet to see a credible claim that over 50% of people use mods (50% of forumers is not enough to conclude that, because that's a small part of the number of KSP players and an unrepresentative part). If you have a credible source for the claim, I'd like to see it. Otherwise, people on the forum should stop acting like the generic assumption is that someone uses mods.

[dlrk]

We've heard the 'vocal minority' thing from lazy devs forever. Given that KSP isn't much fun without mods(and that mods consistently implement features that Squad declared impossible), I'm guessing that it's just easier to develop for, and market to casuals.

[NathanKell]

the problem is that they can't. what do you think. if they would have the ability, wouldn't they already have done it?

I entirely agree! Good thing somebody already showed it was possible, eh?

More constructively: GameDatabase could be rewritten such that original load does two things: stores the URLs of all textures, and loads low-res imposters. Then, whenever a placed part calls on GDB for its texture reference, GDB stream-loads the texture at full res. When the part is destroyed, the texture is freed.

A similar method could be used for all the 2048x1024 textures celestial bodies use.

To speed things up, mbm could finally go the way of the dodo, and part textures (like all other KSP assets already are...) could be stored as DDS, thus significantly cutting down original load times and stream-load times since no compression (or, for PNG, first decompression) would be needed.

Edited February 6, 2015 by NathanKell

[longbyte1]

the problem is that they can't. what do you think. if they would have the ability, wouldn't they already have done it?

Their excuse is that their game is "still in production" so optimizations are their lowest priority.

[Motokid600]

I didn't want this to become a modded vs stock players discussion. Let's try to steer clear from that. The fact of the matter is people DO mod the game and that is going to contribute a significant amount to this games success. KSP has imo the greatest modding community I've ever seen in a video game. It's really something else and when the time comes the game will be praised for that. So to have this hard, set limit to how much you can expand the game will become more of a pain as time goes on. For both modders and Squad themselves. So wether you play stock or not this has to happen. You think it's bad now how people complain about the memory limit? Wait until summer this year it will be pandemonium.

Squad DOES have the power to alleviate that. Unity may not. Not for awhile, but right NOW... Squad can indeed do something.

Edited February 7, 2015 by Motokid600

[rkman]

the problem is that they can't. what do you think. if they would have the ability, wouldn't they already have done it?

If besides ability, time would not also be a factor...

re mods:

Squad went out of its way to make the game moddable. The idea that it should be optimized only to the point where stock runs well, does not mesh very well with that.

Edited February 7, 2015 by rkman

[Voidryder]

I wholeheartedly support what Nathan and others have proposed (and done) to optimize KSP's memory usage. Squad has stated that they will/are focusing on optimization, especially for 1.0. I hope this is looming large on their radar. All players be they vanilla or mod will benefit.

[DSKY]

I'm pretty sure mod users are in the minority, though. I have yet to see a credible claim that over 50% of people use mods (50% of forumers is not enough to conclude that, because that's a small part of the number of KSP players and an unrepresentative part). If you have a credible source for the claim, I'd like to see it. Otherwise, people on the forum should stop acting like the generic assumption is that someone uses mods.

The KSP mods page on Curse, which is the mods page linked on the official KSP site, lists 586,366 downloads for MechJeb as of this writing. The next five mods in the list are over 200,000 apiece, several more over 100,000, etc. Assuming the worst case, no one installs mods unless they also have MechJeb, then the minimum number of people running modded instals is nearly 600,000. Let's cut this in half, just to cover the possibility that users have downloaded twice to keep up with rev changes. That's 300,000. How many copies of KSP have been downloaded? I don't know. Don't know if that figure is public knowedge. But if it's less than 600K copies, well then the majority of copies are running with mods. Now, I've boiled the math down to a very conservative number. There are other sites that host mods, such as Kerbalstuff, and it seems likely that some people might be running other mods and don't have MechJeb installed.

What does this mean? Not a d@#$ thing. It doesn't matter one bit the percentage of users who run mods. It could be 10%, 1%, one single person. What matters is that Squad has designed the game to be moddable, actively encourages modding, and uses modability (if that's even a word) as a selling point. They have a link to the Curse mod page on the game's official web page. Their weekly blog dedicates one day a week Modding Monday, to modding.

Here's wher I'm going to make my point, so read slowly and carefully: Squad advertises KSP as a moddable game. A consumer who buys the game even partly for that reason and is subsequently disappointed in how the game performs with mods will be dissatisfied. If dissatisfaction occurs frequently or with enough severity, it will become public knowledge that will impact sales. Squad is under no moral, legal, or contractual obligations to any individual or group to make the game perform as advertised, but they are also not immune to marketplace reaction if they fail to do so.

I believe it is a safe assumption that despite sometimes ferocious disagreements, nearly everyone involved on these forums is a fan of KSP. Myself included. It is a wonderful, creative, addictive piece of software, and I want it and Squad to succeed wildly. So it is out of love for the game that we point out its shortcomings, potentially fatal (economically) ones, with the hopes that Squad will address the serious underlying issues that remain to be fixed.

[Beduino]

+1 to this I'm upvoting they fix it as soon as possible as described in this thread. They can probably do a better job than any mod by improving the base code.

Then we have:

Lower memory footprint

Reduced loading times

More mods+more parts+more fun

Maybe optimize the planets so we can have more, maybe even another star with little memory/performance hit.

Maybe they could even figure out a way to improve performance with increasing part count so we can use even more parts to build larger bases/ships.

[sumghai]

I definitely agree that Load on Demand should be prioritised for either KSP 1.0 or 1.1.

the problem is that they can't. what do you think. if they would have the ability, wouldn't they already have done it?

In software development, optimisation of features are often relegated to the end of the design cycle. This is because at the moment, SQUAD's focus is on making broad-stroke passes to get the core features into the 1.0 release, and given the interconnection nature of modern software, optimising a certain area prematurely may hurt the implementation of other areas down the line.

I know you've been an extremely vocal critic of SQUAD's development paradigm for a very long time, and while your frustration is understandable, lashing out at the developers by insulting them isn't going to speed things up.

[John FX]

I think for me, it does not matter how optimised the game is, I would want to run so many mods that I would break the 32 bit limits fairly quickly anyway, even if KSP used zero RAM.

Load on demand would be a very good start, as would plugging memory leaks.

Even better would be a separate thread for loading and unloading in anticipation of needed textures so there is no loading spike...

The game already can tell you that you will be entering a particular area years (literally if you do not warp) in advance so why not trickle load all the needed items before you hit the boundary?

I would suggest that the right time to implement this would be when you are overhauling the loading section anyway, for example if you are changing the code to allow directly loading DDS textures...

Like when I get my car fixed, I tell them to fix anything they find while they have things taken apart anyway. It is the most efficient way of fixing everything.

[Eric S]

Load on demand would be a very good start, as would plugging memory leaks.

I'd put fixing memory leaks as more important than load on demand, especially if you'll be unloading the assets when they're no longer needed. Repeatedly loading the same asset when every load leaks memory could be worse than what we've got now.

Personally, I'm interested in seeing the impact of the fixed memory leaks and possible switch to DDS textures before debating how beneficial/necessary load on demand and 64-bit memory addressing are. I won't deny that they'd be nice to have, but if the worst of the memory leaks are gone AND textures are only taking up 1/3 the memory they used to, then that's going to take a lot of the pressure off of this.

[Wolf Baginski]

I am just going to drop in a few thoughts, derived from experience with other software, but it is only Windows.

1: The usual advice is that you only need 64-bit Windows if you have more than 4GB of RAM. This is not in fact wholly correct. 32-bit Windows can't get at the full 4GB of RAM. Just as in the ancient days of MSDOS, you need address space for such as the video RAM. Using 64-bit Windows can make a big difference in the memory available, giving both the OS and the program enough memory space without paging to virtual memory.

2: One program I use can use up to 10GB of disk cache for data such as textures, which in that case would need to be downloaded over the internet. Most games have the texture data on a CD or DVD, but they're still slower than the hard drive. Something like load-on-demand that supports a disk cache which can be placed on any disk would open up the options, from slow hard drive via SSD to RAMdisk. And a cache system might be made more efficient than files on a disk. Again, experience elsewhere (and there is already a Mod which downsizes textures) suggests the cache might hold several versions of a texture, and only load the higher-res form of a planet if it is selected as a target.

3: A good starting default, which can be tuned by such people as heavy-Mod-users to suite what is on their PC, is tempting. But sometimes I have seen inadequate explanations of the options leaving them horribly misled.

[Whirligig Girl]

I think KSP would increase in awesomeness hugely if, when 64x Windows is stable, they dropped primary support for windows 32x. No one should be using a 32 bit computer by now, when 64-bit computers are ubiquitous. Obviously, don't just stop having a 32-bit version, but if we can get huge amounts of extra memory from 64-bit, so we could have awesome new parts or features and even new planets, then this is progress.

But if loading textures as required can also do this with no major downsides, do that. Especially if you do have a 64-bit and you want to keep 32-bit.

[Fel]

I entirely agree! Good thing somebody already showed it was possible, eh?

So, to confirm... you found someone who was exceedingly more familiar with Unity and Game Programming / Programming in General than SQUAD. (Really, not looking too deeply into this at the moment)... he also seems to have a C++ wrapper to give him direct control over the hardware because.... C# sucks.

I think when you say "someone proved it" you should really say "someone who is much more skilled than SQUAD did it." Which really gets back to what I was trying to say before: Unity sucks, and SQUAD isn't a collection of the absolute best programmers. You can scream all you want at them to optimize the code but it is never going to change the fact that they aren't skilled in doing so (and that C# and unity sucks).

Huh... this really would be a very very very effective way of bypassing C#'s stupidity and patching memory leaks.

The more I think about this... I mean, I doubt he didn't know anything about what he was doing... at that level he really must have... but this is very impressive what he probably was doing.

I wonder how hard it would be to get it working again; I won't deny that I am hardly experienced with the methodology behind the system, but the methodology isn't what would have broken, just the hooks. Even, possibly, recompiling with the updated DLL to link to.

More constructively: GameDatabase could be rewritten such that original load does two things: stores the URLs of all textures, and loads low-res imposters.

Please.... stop abusing WWW to load local files. I showed a long time ago that you can use the binary reader "as you're suppose to." WWW is such a dirty hack. Use file paths, and even better, use relative file paths... but not localhost.

Edited April 10, 2015 by Fel

[NathanKell]

As for the first point, I apparently have a much higher opinion of Squad's skills than you do--and a much lower opinion of how hard a streaming asset loader actually is to write (IMO, not that hard--and certainly doesn't require using native code or DX-only wrappers or any of that).

As to the second point, GameDatabase URLs was what I meant, not WWW URLs (though yes, the loader does have its WWW issues).

[Solarspaceprobe]

Aside from being able to spawn Kerbals into command seats from the SPH, this is one of the features I must see before I can consider KSP anything NEAR a complete game.