question about image compression and file reduction

[Jesusthebird]

so recently ive figured out you can reduce image quality to help save on ram and increased preformance; I'm assuming it's mostly related to part/world textures, but not all image files are nessisarily part/world textures. i downloaded KER and noticed they use a PNG image file. guessing its for the GUI or toolbar icon. anyways, im wondering if KSP automatically knows how to treat all image file formats? I read yesterday that .dds files run more efficiently than the others, so is there a reason why modders use png or mbm formats? could i use a converter to turn them into dds files and ksp would still know how to treat them without much hicup? or would i have to edit some files to point them to the right direction? would it break the mod in any way? Thank you, just trying to save some ram is all, im just about at my limit. and I do understand that in most cases, the images resolution is reduced.

 

Ive seen some threads about ram reduction, but as far as i know its in relation to actual part textures and things as such, not necessarily the other objects people might overlook like toolbar icons, or ingame GUIs, maybe it wont make much difference, but im willing to try and find out lol, unless of course someone  already knows the answer :D. Thanks again guys!

 

please correct any misinformation.(i say this because ive read the squad folder had some performance/size issues, but that may have been because the image files were not in dds format? which a recent patch corrected? basically dds files would be a better choice for performance correct?) THANK-YA! THANK-YA!

[Kerbital]

Take the following with a grain of salt. I may be wrong on many accounts.

As far as I can tell those PNGs may be uncompressed. I may be wrong though. You can compress a PNG but don't have to. Loading lots of images that are compressed takes more CPU cycles than loading uncompressed images. This is true for anything including graphics programs such a Photoshop. JPG compression is very efficient and compressed JPGs are the fastest to decompress but they have compression artifacts that makes them unsuitable for texturing and game graphics. PNG also uses something called lossless compression which preserves the quality of the original image. So even if compressed, PNG would still be a better choice when quality is desired. JPGs use lossy compression that degrade the image.

Oh, and the big one: PNG supports transparency while JPG does not. One more reason to use it for computer game graphics. This and quality is probably why the are the choice for game graphics.

Once an image is decompressed and loaded to RAM it takes the same amount of RAM whether it was originally compressed or uncompressed before as a file on the disk. So changing image file format will not save any RAM.

But then video cards (GPU) use texture compression when textures are loaded to their VRAM which I know next to nothing about. Textures are loaded to video card's VRAM not the host computer RAM and things like GUI graphics, loading screens are loaded to computer RAM. Textures are many and larger than misc game graphics so there is no need to worry about game graphics. .

Both textures and game graphics are not necessarily loaded to VRAM and RAM for the duration of the game either. Unused textures and graphics are flushed when used infrequently. The game does that depending on available GPU VRAM. That's why video cards with more VRAM are often faster because there is less texture loading and unloading.

Then there is the DDS compressed texture format which is decompressed by the GPU, not CPU. This makes decompression faster as there is less data to read from the hard drive and video cards (GPU) are a lot more faster at decompressing this type of file than the CPU as the format was specifically deigned to be decompressed by the GPU. DDS is proprietary Microsoft file format for Direct X textures. You wont use DDS for things like GUI graphics, it is specifically designed for textures and GPU decompression. Compression process lends itself to massive multi-threading. CPUs are poor at compression because they have fewer cores than GPUs that may have thousands of cores. Even though Hz by Hz a GPU may be slower than a CPU, the massive number of cores make GPUs monsters of compression and decompression.

You can't replace PNGs with DDS then.

Finally, I believe games and mods reference graphic files based on their name. If they can't find a samplefile.png bad things will happen.

The bottom line is that there is more risk in messing with game graphics that there are chances of any performance gain.

[Snark]

17 hours ago, Jesusthebird said:

i downloaded KER and noticed they use a PNG image file. guessing its for the GUI or toolbar icon. anyways, im wondering if KSP automatically knows how to treat all image file formats?

Basically, what @Kerbital said above, with respect to .dds files being better for in-game textures than .png files are.  As for GUI graphics like toolbar buttons... I don't have the specific technical knowledge to be able to confirm whether there's an advantage or not for some other-than-PNG file there... but I expect it doesn't matter much in that case.

Why?  Because they're tiny, generally speaking.  The file format (e.g. ".dds or .png?") really matters for in-game textures (of parts, etc.), because they're huge and there are lots of them.  There are many parts in the game, and they generally need to have really big, high-resolution texture files so that the parts don't look like crap when the player zooms up close to them.  So you're looking at a serious megabyte load on your system, there (not just holding them in RAM, but also in doing all the run-time manipulations to display them at various sizes and zoom levels).  Going with .dds instead of .png makes great sense there.

GUI graphics, on the other hand, are generally microscopic by comparison; they're generally just a few kilobytes in size (not the goodness-knows-how-many-megabytes of all the part textures).  So even if there were any advantage to switching to some format other than .png, it would be tiny and probably not worth the bother.

Worrying about the format of those graphics would be kind of like trying to save money on your electric bill by worrying about which toothbrush is the most energy efficient:  it's tiny to the point of irrelevance, because what really matters is the insulation on your house, and which-toothbrush-you-use basically is just lost in the noise.

[Corona688]

Compression is not helpful for in-game textures.  It reduces quality, and in the end, the game has to decompress them to use them anyway, so they still take the same amount of video memory.

KSP's choice of png for screenshots is regrettable though.  Almost nobody takes the time to convert to jpg, so threads of screenshots can take many minutes to load.

Edited November 29, 2017 by Corona688

[HebaruSan]

To illustrate Snark's point, here are the PNGs from KER mentioned in the OP:

They're about 4 KB total. At that size, your system probably reads the entire file into the various layers of cache as soon as the first byte is requested. Messing with the format of these files will have no effect on system performance.

[klesh]

35 minutes ago, Corona688 said:

KSP's choice of png for screenshots is regrettable though.  Almost nobody takes the time to convert to jpg, so threads of screenshots can take many minutes to load.

 

The Seam functionality for screenshots is great.  I have mine saving an uncompressed version as well as the standard .jpg one.  Built in uploader, and the display page has a comments section.

[eddiew]

35 minutes ago, Corona688 said:

Compression is not helpful for in-game textures.  It reduces quality, and in the end, the game has to decompress them to use them anyway, so they still take the same amount of video memory.

KSP's choice of png for screenshots is regrettable though.  Almost nobody takes the time to convert to jpg, so threads of screenshots can take many minutes to load.

Pretty sure this is the right answer. Every image format ends up the same when it's in GPU memory anyway.

That said... I also think that in d3d9 mode, KSP does a stupid thing whereby the images are loaded into system memory (as files) and also copied to the GPU. IIRC this is why running in OpenGL mode reduces memory footprint, because it dumps the redundant cached file once it's in the graphics memory.

I think  

[Corona688]

5 hours ago, eddiew said:

That said... I also think that in d3d9 mode, KSP does a stupid thing whereby the images are loaded into system memory (as files) and also copied to the GPU.

I think I can explain that.  Direct3D 9 couldn't guarantee that textures put in video memory would stay there, so the software copies were needed as fallback.

Stupid, but not Squad's stupid.

Edited November 29, 2017 by Corona688

[Jesusthebird]

I see, thanks for the schooling peeps. and ive started to wonder myself why running in open-gl reduces the footprint a tad. neat concept

 

so in the end, it all gets decompressed no matter what regardless of format, with certain formats being optimized towards certain tasks(dds for part textures, png for toolbar buttons, etc). i suppose the only real way to get a substantial footprint reduction is to simply remove unused parts so they dont get loaded into any ram. ohh well

 

as far as clutter cleanup for the game to keep running smoothly,  and in reference to the person who mentioned the loading and unloading of textures to the gpu; would it be worth it/possible to increase the frequency the game does rinse the ram of unused things? i dont really use ram optimizing programs, so lets avoid that. ive witnessed the formentioned screen freeze when the game does do a quick dump, half of me thinks this might not be such a noticable freeze if theres less clutter to actually dump, the other half thinks it wont matter at all. just thinking out loud here.

im not a software\hardware engineer, or any kind of IT professional. ive learnt what i know all by google and self teaching and forums such as these. so i appologize if im giving yall cancer of the eyes lol

Edited November 30, 2017 by Jesusthebird

[Shadowmage]

On 11/29/2017 at 9:21 AM, Corona688 said:

Compression is not helpful for in-game textures.  It reduces quality, and in the end, the game has to decompress them to use them anyway, so they still take the same amount of video memory.

This is incorrect, at least in regards to .DDS.

While it is true that DDS is a 'compressed' texture format -- it is not decompressed into GPU memory.  It is stored in GPU memory -as-is-, and decoded on the fly (during texture sampling).  Modern graphics hardware has special texture samplers that handle recreating the output from a much smaller amount of input, and results in a net savings on GPU memory bandwidth during sampling of DDS compressed textures.  If a DDS is 1.5mb on disk, it will be 1.5mb in RAM, and 1.5mb in VRAM.

As a result, DDS textures use less RAM, less VRAM, -and- are faster to render.  The downside?  As they are lossy compressed textures, you can get artifacts in rendering.  This is why you don't use DDS compressed textures for GUI's -- you want your GUI's to be nice and crisp, not suffering from compression artifacts.

Now other texture formats (.png, .tga, etc) -- those are decompressed into memory.  A .png may only be 31kb on disk, but decompressed might be 5mb> in RAM/VRAM.  (KSP also does not create/load MIP-maps for .png textures... which can make parts using them seem pixelated;  MIP's are not needed for GUI textures... so again... .png is a good choice for GUI).

[helaeon]

Pro graphic designer and tech guy... Shadowmage has it. DDS is special as far as file compression goes because the GPU has special hardware functions to make it do magical things (in effect).

Any standard decompression uses CPU cycles on load and then the true file size in RAM (which will be the same as if it were an uncompressed TIF or TGA).

Generally compression is lossy, you're approximating the file format for in order to use a mathematical function to describe it that takes up less space on disk. When your CPU reads the file it essentially re-draws the pixels from that math.
I have a special hate for JPG because of that lossiness. Most of my work involves large format printing (things that are feet wide and high), so the flaws of compression are something I deal with daily.
Here are some videos about what your computer is doing if you want to know more:

https://www.youtube.com/watch?v=AqscP7rc8_M - Resizing Images

https://www.youtube.com/watch?v=n_uNPbdenRs - JPG and Color

https://www.youtube.com/watch?v=Q2aEzeMDHMA - More on JPG Compression

[Corona688]

2 hours ago, helaeon said:

I have a special hate for JPG because of that lossiness.

Oh god.

I briefly worked in (well... got pushed into kicking-and-screaming) small desktop publishing and understand all too well.  "Why does our ad look bad", because you sent us a 500x200 gif, your website had nothing better, and you ignored us for two months straight.  Too late now.