How to play KSP with Unity 2019 on Old Potatoes!! Part II

[Lisias]

While on a discussion about KSP for Mobiles, I remembered how well KSP 1.4.3 (as well KSP 1.3.1) was running on my old MacCrap, a Mid 2011 MacMini 5.1 - i5-2415M (2.3GHz dual-core Intel Core i5 with 3MB on-chip shared L3 cache), 8GB RAM and HD3000 with shared memory, max at 386MB VRAM (i think).

Now, on a less older MacCrap, a Late 2012 MacMini 6.2 -  I7-3615QM (2.3GHz quad-core Intel Core i7 (Turbo Boost up to 3.3GHz) with 6MB L3 cache), 16GB RAM and HD4000 with shared memory, max at 1536MB IIRC), KSP 1.4.3 runs magnificently. Really, really good.

Since I'm running on MacOS, I'm using OpenGL or Metal - and not DirectX.

However, this same configuration is just plain terrible for KSP 1.12.3 (besides being not that bad on 1.8.1, just saying).

Thinking about, I realised that once you work around Unity's… less then smart  … decisions, any minimally decent budget PC nowadays should be able to run an KSP - as long you restrain yourself from building crafts with hundreds and hundreds of parts, of course.  Probably even beefy tablets should do.

Problem: memory and loading times.

In the same way Unity is sabotaging your CPU by wasting processing on useless loops (spin locks), having 4096x4096 textures when you are using 720p or 1024p tops on your rig is, well, plain waste. But how wasteful is being KSP nowadays for dudes like me playing KSP on Potatoes? Well, someone has to calculate it, and I already spent a good fraction of my day debugging KSP, why not me and today? The whole Circus is already set, anyway.  So...

The following table depicts the data gathered on a MacMini 6.2, i7, 16GB RAM and HD4000 (1.5GB VRAM shared) under MacOS Mojave (10.14.6). The machine is purposely overloaded, simulating the load I get on KSP when I have that urge to waste a bit of time playing KSP during the day - or just doing a long mission on the second monitor while I work, as it would be a fancy screensaver . Really, whatever I could had done to make the rig's life a misery, I did.

All the test beds are running from spinning disks, the times I got from the best of two consecutive runs (to take advantage of the disk caches), absolutely pristine installment. Not even MM is there.

The Spread sheet is to big to be pasted here, so here goes the link on Google Docs.

And here is the summary:

Resolution	KSP Version
	1.3.1	1.4.3	1.4.5	1.5.1	1.6.1	1.7.3	1.8.1	1.9.1	1.10.1	1.11.2	1.12.3

Part Count:	298	364	364	364	364	414	426	427	438	453	461
DDS Count:	582	705	705	759	804	892	908	848	965	1014	1061
Size onDisk MB:	203	346	347	428	505	605	659	620	755	798	866
Proc Mem GB:	2.37	3.23	3.73	3.69	3.73	6.21	7.1	7.2	7.2	9.87	10.59
Time to Load:	0:01:29	0:01:38	0:01:37	0:01:33	0:01:34	0:01:47	0:01:59	0:01:54	0:02:21	0:05:00	0:05:29

And on the spoiler the values (badly formatted, sorry) for people unwilling to feed Google's craving for personal data.  

Spoiler

Resolution	KSP Version
	1.3.1	1.4.3	1.4.5	1.5.1	1.6.1	1.7.3	1.8.1	1.9.1	1.10.1	1.11.2	1.12.3
4x4	1	1	1	4	9	9	9	9	9	9	9
16x16	10	10	10	15	15	15	17	17	19	19	19
16x32	0	0	0	0	0	1	1	0	1	1	1
32x32	20	20	20	20	22	22	22	22	22	22	22
64x64	16	16	16	16	16	16	16	16	16	16	22
64x96	1	1	1	1	1	1	1	1	1	1	1
92x32	0	1	1	1	1	1	1	1	1	1	1
92x244	2	4	4	4	2	2	2	2	2	2	2
128x64	3	4	4	4	4	4	4	4	4	4	4
128x128	22	22	22	22	22	29	31	24	30	31	31
128x256	4	4	4	4	4	5	5	4	5	5	5
128x512	1	1	1	1	1	1	1	1	1	1	1
256x64	2	2	2	2	2	2	2	2	2	2	2
256x128	8	9	9	9	9	9	9	9	9	9	9
256x160	27	27	27	27	27	27	27	27	100	0	0
256x256	86	89	89	89	90	97	96	94	12	106	106
256x512	6	6	6	6	6	13	13	13	0	12	12
512x128	3	3	3	3	3	4	4	3	4	4	4
512x256	30	31	31	31	30	30	30	30	57	57	57
512x512	152	163	163	174	178	194	198	184	212	234	241
512x1024	6	6	6	6	6	13	16	9	14	14	14
1024x128	0	0	0	0	0	1	1	0	1	1	1
1024x256	3	3	3	3	3	3	3	3	3	3	3
1024x512	8	14	14	17	17	17	17	17	17	17	17
1024x1024	153	203	203	218	245	271	270	250	300	312	324
1024x2048	0	6	6	19	18	22	22	18	22	22	22
1024x4096	0	0	0	0	0	0	0	0	0	4	4
2048x1024	4	9	9	9	9	9	9	9	11	11	11
2048x2048	11	47	47	51	61	69	70	70	79	83	105
2048x4096	0	0	0	0	0	0	3	3	3	3	3
2560x1600	3	3	3	3	3	3	3	3	3	3	3
4096x4096	0	0	0	0	0	2	5	3	5	5	5

There're a lot of conclusions to be taken from this data set - in particular, the 1.12.3 numbers are terrible (not to mention almost 3 times the disk space for essentially the same material).

The measures were made from firing up KSP until the Main Menu is shown by checking the timestamps from KSP.log, from first entry to the timestamp for the ""Scene Change : From LOADING to MAINMENU. Terrain Detail, Scatter Density, Render and Texture Quality are all at tops on the Settings/Graphics.

Looking into these numbers, it's perfectly understandable why KSP 1.4.3 and 1.4.5 were my favorites on my older rig.

I only managed to play 1.7.3 on the less old one, used to gather these data. I will not even try to get the numbers from my older rig (still available as my secondary professional rig). And as a matter of fact, KSP 1.8.1 once it got stabilised, it's pretty good!! Too bad that at that time I didn't knew about the MONO_THREADS_PER_CPU=1 stunt yet, as this made KSP 1.8.1. and KSP 1.9.1 useable on this rig - but not so much as 1.7.3. These numbers also explain why I essentially waved most of the Add'Ons I was used to use on KSP 1.4.5 era when I jumped over to 1.7.3.  

Keep in mind that these numbers are for the graphical settings maxed out. Reasonable people will downgrade the Texture Quality to save memory - but the infinite KSP's bugs will bite you if you downgrade too much, as everything is downgraded including the User Interface, and you will get Icons and widgets blurred. Plain terrible, and unacceptable to me, being the reason I intent to use these data I intent to rework the Stock textures to save loading times and memory without using the Texture Quality setting.

If I manage to bring KSP 1.12.3 to the same level as KSP 1.7.3 more or less, I may be able to finally start playing something on it - otherwise, I will probably be stuck with 1.9.1 (once I make some optimisations on the damned thing, as I like to use Add'Ons, damn it!).

Wish me luck!  

— — ADDENDUM — — 

The file system used is Apple's APFS, no deduplication were applied and all relevant directories and their contents were compressed using:

find . -name . -type d -exec afsctool -c -v -9 "{}" \;

 

The numbers for the DDS files sorted by resolution were obtained by running this UNIX command on the (clean) GameData of each KSP test bed used:

# This one was being fooled by spaces on the pathname!
#find . -name "*.dds" -exec identify {} \; | cut -d " " -f 3 | sort -n | uniq -c

# This one works as expected!
find . -name "*.dds" -exec identify {} \; | sed -nr "s/^(.+) DDS ([0123456789x]+) .*$/\2/p" | sort -n | uniq -c

 

The disk space was calculated subtracting the values given by the following two UNIX commands:

du -a

du -a -I *.dds

There's no way to use du to get the sizes only from files specified by the mask, the mask can be used only to ignore some files. So I got the total value for the whole GameData, and then subtracted from the value given but everything but the DDSs - resulting on the number for the DDSs themselves.

The zDeprecated assets were included on the batch. They are not big enough to significantly affect the numbers, and I have a lot of Add'Ons that need them anyway. So I consider them Stock for the purposes of this essay.

— — Addendum's Addendum — — 

DU has royally screwed me up. Using the '-h' (humanise) option is was giving me scrambled numbers, and this invalided all the disk size info above [EDIT: Already fixed]. I'm re-running these values and I'm updating the spreadsheet and later this post.

— — addendum's addendum's addendum — — 

I was fooled by some spaces on the pathnames - I completely forgot Windows programs are used to have it (way less on Unix, but MacOS is a mess on this).

I fixed the command lines for a more robust one, and fixed the values. Nothing really changed, but an error is an error - and should be fixed.

 

------

Note: This is a follow up from this thread, but addressing different problems!

Edited October 23, 2022 by Lisias
note

[Gotmachine]

3 hours ago, Lisias said:

everything is downgraded including the User Interface

This issue has been fixed since KSP 1.8

3 hours ago, Lisias said:

almost 3 times the disk space for essentially the same material

You need to exclude all files lying in the zDepreciated folder. Those aren't loaded at all.

Also, your stats absolutely don't match mines. Considering only the Squad folder (not the DLCs) :
- On 1.12.3, there is 898 MB of dds texture (excluding 45 MB of zDepreciated textures)
- On 1.11.1, there is 782 MB of dds texture (excluding 23 MB of zDepreciated textures)
- On 1.7.3, there is 519 MB of dds texture (excluding 22 MB of zDepreciated textures)
- On 1.2.2, there is 302 MB of dds texture

As for load times, for a non-DLC install, I'm getting 41s for 1.2.2, 48s for 1.12.3, so I'd say that given the 3x inflation in texture size and all the added/revamped parts , modern KSP is actually more efficient in that regard.

Concerning memory usage, doing a repeatable test (new career save, swapping through the tracking station and editor scene, then launching a stock Dynawing, then teleporting it in LKO) gives me :
- 3.4 GB RAM usage on 1.12.3
- 1.9 GB RAM usage on 1.2.2

My 7 years old midrange Windows 10 PC is very far from being a decent machine by today standards, but it has a discrete DX11 GPU (GTX 750Ti 4GB).
I'm not an expert on the matter, but it is likely that your antique shared memory GPU is having trouble compressing textures on OSX/OpenGL,  resulting in vastly inflated memory usage.

The point is that you're getting to general conclusions based on a very specific setup (MacOS/OpenGL/shared memory crap GPU) that has very specific issues, and that is absolutely not representative of the majority of (potatoes) users.

1.12 runs quite decently on a my antique (13 years old) desktop with a core i5-670, 8GB of RAM and GTX 460 1GB, certainly not any significantly worse than what I remember from the KSP 1.2/1.3 times when I was playing daily on that machine.
However, 1.2/1.3/1.4 was (vaguely) playable on my 10 years old discrete GPU with 512MB VRAM  and 4GB RAM mid-range-for-the-era laptop, but 1.12 is a really bad experience in terms of FPS and load times due to RAM starvation.

Modern KSP does indeed require a bit more RAM than Unity 5 era KSP, and it also does require a bit more GPU power (but still nothing compared to any modern 3D game).

My experience (on Windows) is that as long as you have a discrete GPU that is less than 10 years old, and at least 6GB of RAM, modern KSP runs better than Unity 5 era (ie. 1.1 to 1.7) KSP.
On very low spec potatoes (no discrete GPU and/or 4GB RAM), things are a bit different indeed.

Note that a good way to alleviate VRAM starvation issues on iGPUs and old GPUs is to install the ReStock mod, which manage to reduce texture size quite a bit compared to stock (and while looking better, so win-win situation there).

Edited April 29, 2022 by Gotmachine

[Lisias]

6 hours ago, Gotmachine said:

This issue has been fixed since KSP 1.8

Really? Toolbar Icons still get blurry to me… But my memory can playing me tricks, I will double check this now.

 

6 hours ago, Gotmachine said:

You need to exclude all files lying in the zDepreciated folder. Those aren't loaded at all.

I had thought on doing it, but concluded that they would not made any significantly diference on the stats - and I like them, as I have some older Add'Ons that rely on them, so it's technically "stock" for me. But I should had explained that on the main text, fixing it.

 

6 hours ago, Gotmachine said:

As for load times, for a non-DLC install, I'm getting 41s for 1.2.2, 48s for 1.12.3, so I'd say that given the 3x inflation in texture size and all the added/revamped parts , modern KSP is actually more efficient in that regard.

The size of the textures, per se, is not the problem. The medium in which they are stored (spinning disks on my case), the memory available and the CPU are the main bottlenecks. And since my rig is already bottlenecked by dozens of Safari pages, Visual Studio, a bunch of terminals et all, my loadings were (purposely) hindered - my goal is to more or less benchmark KSP on real life conditions. Our PCs are not videogames, we don't reboot the machine just to play a videogame, we just open it the no matter what's already opened on the rig and and that's it.

And the current state of the memory pool also tax the process: with so much loads and reloads of KSP, I triggered one of the numerous annoying Kernel Panics on MacOS (seriously, my Mac is never turned off - I reboot it only when there're critical updates, power failures and Kernel Panics - these last ones the more frequent, I really abuse my rig). KSP 1.12.3 was being loaded at 8 minutes more or less, after the reboot it clocked 5! So I had to redo all the timings to keep consistency.

On the long run: the timing that matters is the one on your rig. I don't care about how fast you load KSP on your machine, is the loading times on mine that burdens me.

 

6 hours ago, Gotmachine said:

Concerning memory usage, doing a repeatable test (new career save, swapping through the tracking station and editor scene, then launching a stock Dynawing, then teleporting it in LKO) gives me :
- 3.4 GB RAM usage on 1.12.3
- 1.9 GB RAM usage on 1.2.2

I maxed out everything!   For all KSPs. All the tests were made with all the graphical settings at max, in all KSP versions.

Using Texture Quality at minimum, I think I remember getting similar values as your.

The "problem" I was going to measure later (and that you had caught already) is that most people can't use KSP with the Max Settings nowadays, they are intended to 4K GPUs. Nothing wrong on it, absolutely not wrong on it - except by the loading times and storage. IMHO the 4K textures should be offered as a downloadable DLC for people that effectively have 4K GPUs and would make good use of them.

In the end, what I intent to do is to write a simple proof of concept by just using ImageMagick to reduce the DDS sizes to more or less what was done on KSP 1.4.5 (really impressed optimisation work, apparently — see below)

 

6 hours ago, Gotmachine said:

Also, your stats absolutely don't match mines. Considering only the Squad folder (not the DLCs) :
- On 1.12.3, there is 898 MB of dds texture (excluding 45 MB of zDepreciated textures)
- On 1.11.1, there is 782 MB of dds texture (excluding 23 MB of zDepreciated textures)
- On 1.7.3, there is 519 MB of dds texture (excluding 22 MB of zDepreciated textures)
- On 1.2.2, there is 302 MB of dds texture

My stats is always with the full DLCs (when available). Because it's how I use them (this was specified somewhere on that wall of text).

But since we're here, let's go for it. Using the same command I did above (du -ah [-I *.dds]):

• On 1.12.3 I got ~949 MB
• On 1.11.2 I got ~289 MB
• On 1.7.3 I got ~192 MB
• On 1.2.2 I got ~96 MB

We have some serious discrepancies here!!!! Let's dig on the 1.2.2  as it's the smaller of the gang. This is how I got into these numbers:

Spoiler

cd ~/Workspaces/KSP/runtime/1.2.2/GameData/Squad
du -ah
<2 thousand entries!>
908K	./Parts/Utility/mk3CargoBay/Mk3CargoBay.dds
128K	./Parts/Utility/mk3CargoBay/short.mu
192K	./Parts/Utility/mk3CargoBay/long.mu
1.6M	./Parts/Utility/mk3CargoBay
 16M	./Parts/Utility
 91M	./Parts
255M	.

du -ah -I *.dds
<1.5K entries>  
  0B	./Parts/Utility/mk3CargoBay/medium.cfg
  0B	./Parts/Utility/mk3CargoBay/ramp.cfg
  0B	./Parts/Utility/mk3CargoBay/long.cfg
128K	./Parts/Utility/mk3CargoBay/short.mu
192K	./Parts/Utility/mk3CargoBay/long.mu
704K	./Parts/Utility/mk3CargoBay
6.2M	./Parts/Utility
 26M	./Parts
159M	.

So 255 - 159 = ~96

HOWEVER… You didn't pulled your numbers from the hat, you calculated it somehow on Windows. And these assets are the same for everybody, so what in hell we are getting here?

Looking on that listings again I found some 0B entries. Normal, DU works on disk blocks, and since I'm using APFS that minimises block waste by grouping smaller files into a single block (don't ask how they handle the mess), some files ends up using "0 block", as they were stored on a shared blocked (and only the first entry uses a block on the report).

Anyway, let's calculate this crap again, but without the -h option to see the numbers "in raw"

Spoiler

du -a
<2k entries>
1816	./Parts/Utility/mk3CargoBay/Mk3CargoBay.dds
256	./Parts/Utility/mk3CargoBay/short.mu
384	./Parts/Utility/mk3CargoBay/long.mu
3224	./Parts/Utility/mk3CargoBay
32136	./Parts/Utility
186688	./Parts
522800	.
  
du -a -I *.dds
0	./Parts/Utility/mk3CargoBay/medium.cfg
0	./Parts/Utility/mk3CargoBay/ramp.cfg
0	./Parts/Utility/mk3CargoBay/long.cfg
256	./Parts/Utility/mk3CargoBay/short.mu
384	./Parts/Utility/mk3CargoBay/long.mu
1408	./Parts/Utility/mk3CargoBay
12600	./Parts/Utility
53600	./Parts
325712	.

DAMN!!

So 522800K - 325712K = 197088K = ~192 MB !!!!! 

DU is screwing me up!!! DAMN!!!

I'm fixing that numbers, and then I'll come back to you!

Edited April 29, 2022 by Lisias
some entertaining grammars made less entertaining.

[darthgently]

45 minutes ago, Lisias said:

du -a -I *.dds

What does "-I" do?  Does nothing in my version of du

[Gotmachine]

1 hour ago, Lisias said:

I maxed out everything!

So do I.

But again, there are significant differences between running the game with OpenGL, DX9 and DX11, and even within OpenGL, things depend on what OpenGL features your GPU actually support.
On DX11, textures don't stay in RAM unless you don't have enough VRAM (and 1GB VRAM is enough for 1.12).

As for loading times, even with both DLCs 1.12 load in about 70s on my machine. Arguably, comparing 1.3 loading times with 1.12 with both DLCs is comparing apples to oranges.
I recently did quite a bit of testing on loading times, and KSP 1.12+2DLCs still load in under 2 minutes on a 5400tr/min 6 years old "storage oriented" 4TB HDD, so something else is at play on your end.

Again, I'm no expert on the matter, but from your RAM usage and loading time, this feels like old-time DX9 Unity 5 KSP where textures were kept in RAM forever, maybe because your iGPU doesn't support the texture compression format through OpenGL.
In modern KSP under DX11, as long as you have a GPU with enough VRAM, you can add tons of parts and that doesn't has any significant impact on RAM usage.

Edit : To give you another figure, my heavily modded 1.12 JNSQ install with 6.3GB of DDS textures and 1075 loaded parts uses ~7.2GB of RAM and ~3.5GB of VRAM after playing for 2 hours or so.

Edited April 29, 2022 by Gotmachine

[darthgently]

47 minutes ago, Lisias said:

du -a -I *.dds

I think you are trying to only include *.dds?  I think you want to us -X to exclude *.dds then subtract to get the dds total

[Lisias]

2 hours ago, darthgently said:

I think you are trying to only include *.dds?  I think you want to us -X to exclude *.dds then subtract to get the dds total

No. On DU, there's no way to only "include" files, only to exclude them. And on the BSD/MacOS, the option to exclude files is "-I". Don't ask, just accept it.

So the way to quickly compute the number of blocks used to store the data (that it's what really matters on time spent on I/O) is to calculate all blocks used on a clean GameData, and then exclude the number of blocks used to store everything but the DDS - and the resulting number is the number of blocks used by the DDS!

My initial numbers were completely screwed up because, somehow, the… less then smart… dude that coded the -h (humanise) option thought it would be a good idea to humanising the accumulator itself, and not only the numbers being displayed. This leaded to rounding errors being piled up on the accumulator! Without the -h option I got the real numbers in KBytes, the unit of measure used on the storage blocks. Then I "humanised" the final result myself.

 

2 hours ago, darthgently said:

What does "-I" do?  Does nothing in my version of du

I'm o MacOS. On Linux where I think you are working the option is a more reasonable one, "-X".

 

2 hours ago, Gotmachine said:

So do I.

But again, there are significant differences between running the game with OpenGL, DX9 and DX11, and even within OpenGL, things depend on what OpenGL features your GPU actually support.

On DX11, textures don't stay in RAM unless you don't have enough VRAM (and 1GB VRAM is enough for 1.12).

What's a huge problem on MacOS! Safari, Chrome, Firefox… All of these makes heavy use of the VRAM for everything on MacOS, as the whole Aqua GUI makes heavy use of GPU acceleration. So it's best to keep the textures on RAM and upload to VRAM only the ones needed on the current scene - otherwise the whole rig will start to stutter due VRAM dispute - watching tutorial videos while playing KSP can be a challenge due this.

 

2 hours ago, Gotmachine said:

As for loading times, even with both DLCs 1.12 load in about 70s on my machine. Arguably, comparing 1.3 loading times with 1.12 with both DLCs is comparing apples to oranges.

What's not a problem when you are selling Fruit Salads.

I play KSP, the full package. I don't play "vanilla" KSP. This is not a technical benchmark, this is a study to make KSP more bearable on crappy MacPotatoes and, perhaps, Linux and Windows too.

Newer (full) KSP have more features? Yes, they have. But they also loads slower and eat more memory - and these are the constraints I intent to work out.

Additionally, with or without more features, what's matter in essence to the problem at hands is not the number of features or the KSP version itself, but the total number of Textures, how much space they use and how much time they need to load. Analysing these data for the best TexCount vs TexSize vs Time To Load ratio, we will find the best compromise possible between all the KSP versions we have around, and I will have a guideline to resize 1.12.3 ones and still keep the Look and Feel I have on the older ones (i.e., the textures will be so crappy as the ones I use now, not crappier).

 

2 hours ago, Gotmachine said:

Again, I'm no expert on the matter, but from your RAM usage and loading time, this feels like old-time DX9 Unity 5 KSP where textures were kept in RAM forever, maybe because your iGPU doesn't support the texture compression format through OpenGL.
In modern KSP under DX11, as long as you have a GPU with enough VRAM, you can add tons of parts and that doesn't has any significant impact on RAM usage.

Edit : To give you another figure, my heavily modded 1.12 JNSQ install with 6.3GB of DDS textures and 1075 loaded parts uses ~7.2GB of RAM and ~3.5GB of VRAM after playing for 2 hours or so.

Yes, it's exactly this. The difference is that I'm not using DX11, I'm running Metal or OpenGL (in MacOS). Linux users should be having similar problems, I think - but I don't know Vulkan enough to be sure about.

These constrains were implied when I described the rigs being used on these tests - I will detail it further on the OP.

Now that I fixed the Disk Space numbers above, let's go back to the original discussion:

10 hours ago, Gotmachine said:

The point is that you're getting to general conclusions based on a very specific setup (MacOS/OpenGL/shared memory crap GPU) that has very specific issues, and that is absolutely not representative of the majority of (potatoes) users.

Nope. You are.

I described exactly the environment in which these values are valid. I'm working with the hardware I have at hands and, if I manage to cut it, we can then check if the same measures would be beneficial for Linux users - or perhaps older Window rigs with crappy GPUs (there're a lot of old notebook users around).

Linux users can be potentially be beneficed by using BTRFS with transparent compression for storing the KSP files (as I do on APFS). Windows users too, as NTFS also has transparent compression (and it's even easier to set it up). So we already have a common ground to work with.

I expect that Linux users may have more or less the same GPU problems I have. On the other hand, Windows users with DX11 will not for sure, as you explained above.

 

10 hours ago, Gotmachine said:

1.12 runs quite decently on a my antique (13 years old) desktop with a core i5-670, 8GB of RAM and GTX 460 1GB, certainly not any significantly worse than what I remember from the KSP 1.2/1.3 times when I was playing daily on that machine.
However, 1.2/1.3/1.4 was (vaguely) playable on my 10 years old discrete GPU with 512MB VRAM  and 4GB RAM mid-range-for-the-era laptop, but 1.12 is a really bad experience in terms of FPS and load times due to RAM starvation.

On MacOS (and I think Linux too), KSP 1.8.x was an absolute nightmare for performance - but this was due Unity's stupidity. See this thread for the reason - once I applied this trick on my rig, KSP 1.8.1 not only started to perform pretty decently, but also the CPU runs it cooler!! Faster and cooler (at least 2ºC cooler, more on heavy loads).

Interestingly enough (and au countraire to what I was thinking at that time), RAM starvation is less than a problem on MacOS due transparent memory compression. As long the current Scene doesn't needs more RAM/VRAM than reasonably available on the rig at the current load, things are working fine here even with a swapfile eating 4 to 6GB of disk space (implying a over commit at least twice that, due transparent memory compression).

Switching Scenes can be a pain, though.

 

10 hours ago, Gotmachine said:

Note that a good way to alleviate VRAM starvation issues on iGPUs and old GPUs is to install the ReStock mod, which manage to reduce texture size quite a bit compared to stock (and while looking better, so win-win situation there).

ReStock is a hell of a nice job. When I added DOE support for it (before they added new PartModules, that then broke the support), I had to check exactly how they were building the parts, and the reuse of meshes on the thing is laudable. It's a class on how to build objects in KSP.

But it also changes a lot of things on KSP that I'm not willing to cope right now - not to mention that I still use a lot of add'on that looks better with Stock.

Even even ReStock also imposes a lot of overhead on my rig the same Stock does. So instead of doubling my amount of work by trying both Stock and ReStock (as I need to keep that add'ons running too), I choose to go to the easier path and rework Stock textures and call it a day.

I can extend the work later for ReStock if someone asks for - assuming the dude/gal will not do it him/herself as I will publish the script that will do that.

Edited April 29, 2022 by Lisias
Hit Save too soon.

[darthgently]

25 minutes ago, Lisias said:

And on the BSD/MacOS, the option to exclude files is "-I".

Ok, yes I knew you wanted to exclude, but never saw the "-I" before.  That explains it.  Dang BSD heretical stuff with that MacOS reformation nonsense, lol

[Lisias]

12 minutes ago, darthgently said:

Ok, yes I knew you wanted to exclude, but never saw the "-I" before.  That explains it.  Dang BSD heretical stuff with that MacOS reformation nonsense, lol

Not to mention that Kraken damned "-h" option, man. Holly Krakens!!!

 

9 hours ago, Gotmachine said:

This issue has been fixed since KSP 1.8

Nope. At least for MacOS, both 1.8.1 and 1.12.3 still present the problem.

The GUI gets unbearable by setting the Texture Quality on the Main Menu / Settings / Graphical page.

 

Edited April 29, 2022 by Lisias
Tyops while fixing tyops….

[Gotmachine]

59 minutes ago, Lisias said:

At least for MacOS, both 1.8.1 and 1.12.3 still present the problem.

The GUI gets unbearable by setting the Texture Quality on the Main Menu / Settings / Graphical page.

It's not "the GUI". It's only the debug menu, and that happen because they somehow forgot to disable mipmaps  on the debug menu assets.
The actual "blurry UI textures" issue was because prior to 1.8, KSP was applying mipmaps to *.png textures when loading them.
I doubt you will find any other place where this actually happen.

Edit : somehow missed your big post above

1 hour ago, Lisias said:

I'm working with the hardware I have at hands and, if I manage to cut it, we can then check if the same measures would be beneficial for Linux users - or perhaps older Window rigs with crappy GPUs (there're a lot of old notebook users around).

Well, no.
Linux "older rigs" will still likely have a discrete GPU that has much better OpenGL support than your HD4000.
My 10 years old GTX 460 supports OpenGL 4.6 and performs just as well in that mode as my more recent DX11 card in terms of memory usage.
Even if they don't, they might have an AMD iGPU which were vastly superior and better supported than Intel at that time.

Again, your memory usage is not a normal situation, something is going wrong somewhere.

Edited April 29, 2022 by Gotmachine

[Lisias]

44 minutes ago, Gotmachine said:

It's not "the GUI". It's only the debug menu, and that happen because they somehow forgot to disable mipmaps  on the debug menu assets.
The actual "blurry UI textures" issue was because prior to 1.8, KSP was applying mipmaps to *.png textures when loading them.

Well, the debug menu uses the GUI. So it's the GUI.

 

44 minutes ago, Gotmachine said:

I doubt you will find any other place where this actually happen.

Loading Screens, User Loading Screens, 3rd Parties icons on the Toolbar - you name it.

Do what you do, stay away from Vegas. Your guessing skills are not up to the task.

[Gotmachine]

Interesting.
The debug menu has a specific issue with having mipmaps enabled on its texture, that's a fact. It will become blurry on any graphic backend when you lower texture resolution.

Your issue is weird, and a further indication that something is going quite wrong.
Something is applying mipmaps to every texture loaded by the game, even textures that don't have them initially, which mean all textures are getting converted at some point (which could explain why loading is so slow too).
Anyway, I'm indeed guessing here, that's about the extent of my GPU stuff knowledge.

It's just that what you're showing (as well as your memory usage stats) isn't something I've ever seen reported, be it with OpenGL or DirectX, Windows, Linux or OSX. Again, the "blurry icons/ui" issue that is common knowledge around here is specific to png textures.

[Lisias]

54 minutes ago, Gotmachine said:

Your issue is weird, and a further indication that something is going quite wrong.
Something is applying mipmaps to every texture loaded by the game, even textures that don't have them initially, which mean all textures are getting converted at some point (which could explain why loading is so slow too).

The way I choose to cope with the problem is to load my assets myself and being explicit to do not calculate mipmaps. None of my code load glyphs from the GameDatabase - and I move the assets to PluginData to prevent them from being automatically loaded. It's the reason that ugly TweakScale button on Editor is never blurred on my rig (besides being blurred would improve a bit that crap ).

What I think it's happening is that EVERYTHING those width is a power of 2 is being mipmapped (now and then I find a KSP.log entry complaining about not being able to compress the image due that), and perhaps (and now I am the one risking kiss Vegas byebye) the way Squad choose to prevent the problem was simply avoiding images with power of 2 width images. 16, 32 and 64 wide icons get screwed but 24, 48 and 96 ones doesn't.

 

54 minutes ago, Gotmachine said:

It's just that what you're showing (as well as your memory usage stats) isn't something I've ever seen reported, be it with OpenGL or DirectX, Windows, Linux or OSX. 

It was something known enough that someone wrote a thingy called Unblurr on the 1.7 times IIRC. It appeared to solve the problem by using a list of assets to be reloaded by brute force without the mimap. It surely would extend a bit the loading times, but it was a way to fix the toolbar icons for legacy addons without recoding them.

The thing was suddenly abandoned so I wonder that someone had found a nasty flaw on the process rendering it umviable.

 

54 minutes ago, Gotmachine said:

Again, the "blurry icons/ui" issue that is common knowledge around here is specific to png textures.

I think, but need to check to be sure, that DDS images with  power of 2 width are also affected. I will check this on this weekend.

Edited April 29, 2022 by Lisias
Tyops, what else?

[Lisias]

This is a preliminary study about the best resolution/compression scheme to be applied to KSP 1.12.3 (see OP for the link to the Google Sheet):

Ratio	KSP Version
	1.3.1	1.4.3	1.4.5	1.5.1	1.6.1	1.7.3	1.8.1	1.9.1	1.10.1	1.11.2	1.12.3
DDS/sec load	6.54	7.16	7.30	8.17	8.52	8.33	7.65	7.43	6.84	3.39	3.23
DDS/GB RAM	245.57	218.27	189.01	205.69	215.55	143.64	127.89	117.78	134.03	102.74	100.19
DDS/MB Disk	2.87	2.04	2.03	1.77	1.59	1.47	1.38	1.37	1.28	1.27	1.23

 

Curiously, whatever was being done on KSP 1.6.1, appears to be the way to go for the fastest load times possible. But the density of DDS per Gigabyte of RAM appears to pinpoint KSP 1.3.1 instead. 

Since KSP >= 1.4.3 has Making History (that consumes 1/2 GB of memory by itself) and KSP >= 17.3 also adds Breaking Ground over it, perhaps I can't trust this metric for 1.3.1. But the DDS density per Megabyte of disk also favours 1.3.1 by a mile. In a way or another, I'm ignoring that KSP >= 1.4 should have implemented some optimisations, as 1.6.1 wins from it by a long shot.

So my best guess is to go 1.6.1 style. I will check the sizes of the textures on and then apply them into 1.12.3 to see what I get.

Tomorrow.  

 

Edited May 5, 2022 by Lisias
Updated values for the tables, once I corrected the table on the OP

[Gotmachine]

21 hours ago, Lisias said:

It was something known enough that someone wrote a thingy called Unblurr on the 1.7 times IIRC

Yes, and the issue is perfectly known, it was fixed in 1.8 and latter and it only concerned *.png files, not *.dds ones.
This happened because when possible (ie, when a texture width and height is a power of 2), KSP convert uncompressed ARGB textures loaded from png files to dxt5 to save on VRAM usage.
However, prior to 1.8, KSP also added mipmaps while doing that conversion, resulting in ui textures loaded from png becoming blurry when lowering texture quality.
This was changed in 1.8 and latter (they basically just flipped the "updateMipmaps" boolean in the Texture2D.Apply() call).

Also, [snip]
I just looked up, and ONLY the two loading screen images (out of more than a dozen) you took screenshot of happen to have been exported with mipmaps, which is why they are affected by the texture quality setting.
I can't find anything else that has a such a mipmap issue, and although it's possible that similar mistakes exists here and there, that's hardly a problem.
Unless you have proof that "everything" is blurry and that there is actually an issue with KSP 1.8+ wrongly autogenerating mipmaps, stop [snip]

Edited April 30, 2022 by Vanamonde

[Lisias]

[snip]

Rewrote TweakScale's toolbar support to use the GameDatabase to load textures and shoved my icons outside the PluginData. Then I fired up the test bed, and found this on the KSP.log:

[LOG 07:21:17.763] Load(Texture): TweakScale/Icons/Scale_Auto
[WRN 07:21:17.785] Texture resolution is not valid for compression: '/Users/lisias/Workspaces/KSP/runtime/1.8.1/GameData/TweakScale/Icons/Scale_Auto.png' - consider changing the image's width and height to enable compression
[LOG 07:21:17.785] Load(Texture): TweakScale/Icons/Scale_Off
[LOG 07:21:17.858] Load(Texture): TweakScale/Icons/Scale_On
[LOG 07:21:17.875] Load(Texture): TweakScale/Icons/Scale_Unsupported
[LOG 07:21:17.891] Load(Model): Squad/FX/IonPlume

The gotcha? I rescaled the Scale_Auto image to 127x127.  

Enough is enough. I have more things to do now, I may go back to this issue later.

In the mean time, EVERYWHERE I open the Debug screen I got this:

I don't care if the damned thing is fixed on some places, I need this fixed on every place in order to be comfortable playing it.

— — POST EDIT — — 

A quick search on the bug track gave me https://bugs.kerbalspaceprogram.com/issues/24489

Quote

Blurry mod icons on the stock toolbar

Description

In 1.8.1 the mod application textures are blurry as if they were resized by something (even 38x38 textures as recommended in the API)

Note the date, note the KSP version. And the issue is still open.

I can only wonder the reason this is not a problem anymore nowadays (assuming it's really not happening anymore), but on a quick search on GitHub I found that a huge lot of add'ons are handling Toolbar Icons the same way I do, shoving them on the PluginData and loading them themselves. So, even if the problem would had indeed be solved on KSP 1.8.0 (what the issue above clearly proves otherwise), the sad true is that nobody would notice it as most authors had moved on from the issue by their own. Like I did.

[snip]

Edited April 30, 2022 by Vanamonde
post edit

[Vanamonde]

A number of personal remarks and accusations have been removed. Please make a case for you views without insulting the other party in the discussion. 

[Gotmachine]

7 hours ago, Lisias said:

Note the date, note the KSP version. And the issue is still open.

My bad, this was actually fixed in KSP 1.10, not 1.8.

Also note that prior to 1.10, you could place *.png files for which you didn't want mipmaps in any subfolder named "Icons", in a mechanism similar to the "PluginData" special folder name.
But unfortunately this has never become widespread knowledge and indeed everyone re-implemented manual png loading (or even worse, re-reloading of existing textures).

[Lisias]

Working on the data above, I had decided to go "1.6.1" style, i.e., I will try to follow whatever was the sizes of the textures that were used by it - since it has the fastest load ratio and one of the better DDS density on memory.

So I started by analysing the differences between 1.6.1 and 1.12.3 on the texture set:

Resolution
	1.6.1	1.12.3	Diff
4x4	9	9	0
16x16	15	19	4
16x32	0	1	1
32x32	22	22	0
64x64	16	22	6
64x96	1	1	0
92x32	1	1	0
92x244	2	2	0
128x64	4	4	0
128x128	22	31	9
128x256	4	5	1
128x512	1	1	0
256x64	2	2	0
256x128	9	9	0
256x160	27	0	-27
256x256	90	106	16
256x512	6	12	6
512x128	3	4	1
512x256	30	57	27
512x512	178	241	63
512x1024	6	14	8
1024x128	0	1	1
1024x256	3	3	0
1024x512	17	17	0
1024x1024	245	324	79
1024x2048	18	22	4
1024x4096	0	4	4
2048x1024	9	11	2
2048x2048	61	105	44
2048x4096	0	3	3
2560x1600	3	3	0
4096x4096	0	5	5

Obviously that 1.12.3 has more way parts, so has way more textures to cope with. But we will find that a good part of these new textures gone to the higher resolutions side of the force. Additionally, some smaller res ones were removed (512x160) and suspiciously the same number of higher res textures  (27) were added. Depending of the size of these textures, however, it may not worth the pain to downsize.

What I think it's going to worth is to downsize the 4096x4096 textures to half (or perhaps 2560x1600, depending what these textures does). I'm also considering downsize all the 2048x2048 ones to 1024x1024 - or perhaps to 512x512 (as well the 1024x1024 ones) as I'm run KSP on a 1600x900 monitor - I really don't need all that detail and I doubt I will find any difference (unless while zooming the parts, what's rarely done - but yet, some textures may be left alone if things get too ugly). There're some savings on downgrading the 256x256 to 256x160 - if things don't get too ugly.

The next step is the tedious one: enumarating all the textures and their respectives sizes and start to pick the ones to be downsized.

Edited May 5, 2022 by Lisias
Updated values for the tables, once I corrected the table on the OP

[Lisias]

Well, I build a table with all textures from 1.6.1 and 1.12.3 with the respective sizes and compared.

For the UNIX freaks, the following command line does the trick for one GameData:

find . -name "*.dds" -exec identify {} \; | sed -nr "s/^(.+) DDS ([0123456789x]+) .*$/\2 \1/p"

These are the conclusions at this moment:

• 618 textures didn't changed size!!
• 40 textures from 1.12.3 are "easy picks". They are direct related to lower resolution textures from 1.6.1 and will be just downgraded to the "original" resolution.
• 278 textures are shining new, and I will need to manually eye ball the respective parts on 1.6.1 (when revamps) or compare with some other similar 1.6.1 part to decide the new resolution.
  • I really didn't understood why some textures are 4096x4096 (~22MB on disk!), as the Normal for the WeatherStation from Serenity...

Spread sheet on Google Drive. See Sheet 4. The zDeprecated ones were not included on the main analysis, but they are available o the Sheet 5 - interesting enough, or I made a mistake somewhere, or some deprecated textures were downgraded on KSP 1.12.3… 

I will not copy & paste the data this time, as the sheet got almost 1200 lines and this will clog Forum.

[Krazy1]

I had been using full res textures and my VRAM was around 88%. Setting 1/4 res gives about 58% (using xbox game bar). I can see the same issues with the checkboxes and sliders Lisias showed above. The only mod icon that's affected is Trajectories v2.4.3... it's totally blurry.

GTX1060 3 GB, 1920x1200, Win10, KSP 1.12.3, visual mods: Scatterer, Parallax, Spectra

[Lisias]

Yep, I'm moving the discussion from Recall thread as this appears to be a better place.

 

On 7/2/2022 at 8:22 PM, Krazy1 said:

  Reveal hidden contents

I do have Scatterer and Parallax and the settings cranked up. It still shows some free VRAM though at 1920x1200. I'll try lower texture setting to see if it matters. 

. 

It still seems that part count is the most significant performance hit. I have 296 parts for this snapshot. Single-process physics engine just sucks.  Those numbers are so aggravating... 12 FPS because it's CPU limited at 15%. 

Right now video card prices are insane. Who pays >$1K for a video card? I'll wait 6 months, hoping the chip shortage will be better. I think I'll skip 2K and get a 4K. 

Humm…. Yes, in your case you appears to be right - the part count appears to be the issue for you.

How much is your FPS on a new savegame with no crafts flying?

On the other hand, you are also demonstrating that GPUs with 2GB or less are not suitable anymore on KSP 1.12, unless some heavy texturing reducing (the original intent of this current thread).

Perhaps one way to improve the situation for you would be to reduce the mesh quality instead? Blind guess, but it worths a try.

On KSP, we have two time critical loops. The Update and the FixedUpdate one.

The FixedUpdate, as the name implies, tries to be run on fixed time intervals. Shoving too much parts on your rig will overload it, and you will notice that your Mission Elapsed Time will start to flash in yellow or even red. IIRC, this thingy is aimed to be called 50 times per second, and the MET color changes when it drops below that.

The Update thingy is tied to the frame, you get one of that for each frame. The GPU frame rate drops, this one starts to be called less time per second (or even not called at all!! And this can be terrible for most Add'Ons).

When you have a clogged VRAM (apparently not your problem), the contention will affect the CPU (as the GPU, by it's own time-critical nature, have precedence over the CPU on the memory bus), and so you will have drop outs on the Update, but also will get the FixedUpdate affected too.

In your case, I think that a good way to detect exactly what's happening is to launch a new savegame without any crafts flying, and on KSC check the FPS. It must be way better than the 12 fps shown on the screenshot you posted. If it is only marginally better, than I'm betting that your GPU is dragging the whole system down.

 

On 7/2/2022 at 10:52 PM, Krazy1 said:

I had been using full res textures and my VRAM was around 88%. Setting 1/4 res gives about 58% (using xbox game bar). I can see the same issues with the checkboxes and sliders Lisias showed above. The only mod icon that's affected is Trajectories v2.4.3... it's totally blurry.

GTX1060 3 GB, 1920x1200, Win10, KSP 1.12.3, visual mods: Scatterer, Parallax, Spectra

Most add'on Authors (including me) gave up loading Icons or anything used for U.I. from the GameDatabase and started to load the glyphs from the filesystem themselves, literally passing over the KSP Texture Quality problems. You will find that the add'ons not blurring will probably have their icons inside a PluginData directory (a place KSP ignores). I had found a few that even shoved the images as a Resource into the DLLs.

I was told that moving the glyphs to folders called "Icons" would also help, but by now people had already moved away from this.

Edited July 17, 2022 by Lisias
better phrasing

[Lisias]

Oukey, now it's not only a memory issue anymore. We have a serious visual issue to cope with, and this affects everybody - and not only Mac (and Linux?) users.

There's no doubt that the memory footprint for the textures increased a lot on the latest versions - while hitting hard Mac (and Linux?) users, the VRAM usage increased for everybody.

While a 3GB GPU card is still enough for a pure stock or low mod count (as can be verified above), 2GB is not enough anymore for sure. Our fellow Kerbonaut above reported a 88% VRAM usage on his 3GB GPU card, or 2.64GB. By dropping the Texture Quality to ¼ he got 58% of VRAM available, or 1.74GB. And this explains why I have to play KSP on ⅛ Texture Quality, as my rig has a maximum of 1.7GB for VRAM. For everything, not only KSP.

This will hit hard people with 2GB VRAM GPU cards, or a very good part of the entry level or older notebooks. Granted, the most recents ones allow up to half the CPU RAM to be used as VRAM, but - obviously - at the expenses of having less RAM for the whole operating system and KSP itself. Notebooks with 16GB are not that uncommon anymore, but most of the ones I know of still have 8GB tops. For them, the less RAM the GPU steal from the CPU, the better.
So lowering the Texture's Quality significantly is mandatory for a meaningful part of the user base - this is not a Mac/Linux specific issue anymore. In order to quantitise the problem, the following tests were made with a minimally modded KSP:

• Firespitter (compiled properly to 1.12.3)
• HLAirships (Core and the other one)
• KAX
• TweakScale (and the respective TS Companion for Firespitter)
• KSP-Recall on KSP 1.12.3 only
• All DLCs available for the target
  • 1.6.1 doesn't have Serenity.

Since I selected 1.6.1 as the target, I'm benchmarking it against the latest, 1.12.3. I also added 1.7.3 as a middle ground between these releases. I'm not optimising the system for gaming, I'm just firing up KSP with whatever is running on it by now (perfomance is not  being checked right now). The machine is the same specified on OP, the memory values were fetched from the Activity Monitor as soon as the Main Menu is shown.

This is the screenshot with the graphics settings used on KSP 1.6.1 and 1.7.3 for the "1/1" test run:

And this is the screenshot for KSP 1.12.3 using the minimal graphical settings:

(similar settings were used to check the lowest memory footprint on 1.7.3 and 1.6.1)

And these are the numbers I got:

KSP version / Texture Quality	1/1	1/8
1.6.1	4.35 GB	2.56 GB
1.7.3	6.56 GB	3.13 GB
1.12.3	11.44 GB	4.85 GB

As we can see, the memory footprint increase is staggering on 1.12.3. Absolutely unusable on a 8GB machine on default.

So I got rid of all DLCs on all test beds and redid the test to see what I get:

KSP version / Texture Quality	1/1	1/8
1.6.1	3.46 GB	2.11 GB
1.7.3	4.74 GB	2.05 GB
1.12.3	9.47 GB	3.17 GB

So, yeah. If you have a Mac (or a Linux?) machine with less than 16GB RAM, you will probably need to trim down the Texture Quality in order to be able to load the damned thing on 1.12.3 - unless you close pretty much all the other programs. Additionally, if you have a GPU card with less than 3GB of VRAM (no matter where it runs), you will also probably need to trim down the Texture Quality - or you will have a terrible experience on trying to play KSP.

And now we get to the point: lowering the Texture Quality on modern KSP give us a terrible user experience. Not all textures are equally in higher resolution, so we end up with an unbalanced visual with some parts looking acceptably and others absolutely crap - not to mention all our Add'Ons still using their original textures that looks good on 1/1 but once downed to 1/8 are absolutely garbage. 

But how bad things get on ⅛?

Dude, a lot.  Check the following screenshots:

Sample Craft 1 (left, 1.6.1 on full quality; right, 1.12.3 at minimum quality). Click on the image for a full size image.
  

And also this other sample craft using HLAirships (same thing):
 

Pretty terrible, uh? KSP 1.12.3 is giving some users a way worse visual experience while demanding more memory - and that's the reason some users choose to stay behind (like me, still doing my playing on 1.7.3).

But there's a fix for this.

Edited July 18, 2022 by Lisias
the article. finally!