blackrack

I use large arrays as a preliminary step to get data from disk into a 32-bit per channel rendertexture. I noticed that large color and float arrays don't seem to be freed after they are destroyed or go out of scope and the collector is called, I made sure there were no remaining references or anything. I only need to keep the 16mb rendertexture after processing but every time the memory usage would balloon up by 300mb or so. I'm not sure what exactly was going on, some people will tell you that the garbage collector is itself garbage and doesn't manage arrays well etc etc... in the end the only thing that helped was using System.Runtime.InteropServices to allocate some unmanaged memory, and then manage it manually as a float or color buffer and free it when I'm done, works like a charm (sorry source code is a bit of a mess right now as I mostly left he old code commented and there are bits and pieces everywhere). I've also had to avoid functions that return arrays in general, for example BitConverter.getBytes() was also causing some leaks along a few other things, when used in loops those were pretty bad. One peculiar behaviour that I observed is that these leaks don't go on forever with every scene change or operation but seem to top out at 500 mb or so, I've read somewhere that arrays are managed differently in memory, so maybe 500 mb is some kind of limit at which the collector starts doing something for the arrays but 500 mb is still way too much for KSP. Don't know, maybe you could shed some light on this. You could isolate the color arrays and check their impact on memory, and whether destroying and creating new ones will keep inflating the memory usage. In my case the combined size of temporary arrays was a bit big so having them keep leaking on scene changes was really bad. You're welcome

It's probably your memory usage going over the 32bit limit, try to remove all other mods and see if it works, also try the new .dll linked above, it does wonders for the memory usage. Of course when 1.1 releases nothing will work until I update the mod, but I'm not expecting any significant difficulties in making it run on 1.1 Great, thank you both! Now all I need is some reports that it also works fine on linux and mac.

It's updated for 1.0.5, I don't know, maybe CKAN isn't catching up.

For the godrays it's a limitation of the current method, check the first entry in the known issues section in the OP. Thanks for the test results. Edited: I pushed the memory optimizations even further, I believe the numbers speak for themselves. As with the previous one, this might be a bit dodgy so please test it out and report on whether it works or not and what kind of memory savings you get from it. If you have the 64bit workaround on windows also do test with it. https://github.com/LGhassen/Scatterer/raw/improvedMemoryManagement/scatterer/bin/Release/scatterer.dll Paging the people who tested the previous version: @Brigadier @SkyKaptn @selfish_meme I believe the loading process should be near optimal now, there might still be some "per-frame" leaks which I will check out next. If this works for everyone this should be great news for people with a memory budget and people on 32bit.

I create a big color array and a bunch of other float arrays as a preliminary step to loading a floating point texture from disk into a rendertexture. Since unity 4 doesn't support 32-bit per channel floating point textures outside of rendertextures the only way to do this is to read the file and encode it into 4 separate 8-bit per channel texture2D textures and then use a shader to read everything and decode it back to a 32-bit rendertexture. This is only done when switching scenes and after that I need all the arrays and texture2D textures gone ASAP. The issue is that the collector never gets rid of the Color and float arrays so although Color32 arrays would help it would still leak a bunch of memory. To get around this I used the Marshal class from System.Runtime.InteropServices to allocate some memory and then manage and free it manually, it seems to work quite nicely and saves up a lot of memory. Although using a Color32 array could limit the size of the "memory spike" during loading by a bit so I'll do this. For your tip about SetPixels32, I would indeed need to convert float values to Color32 (or 1 byte floating point number, what's that called? a minifloat?) so in the end probably like you said there wouldn't be a speed advantage. To clarify the source is not Color values but rather their float components, calculated by the encode function. You could download EVE which adds clouds or get one of the visual packs (e.g SVE, KSPRC) which include EVE with custom cloud configs and other custom textures and miscellaneous visual mods. Finally, a screenshot with something other than lens flare, thank you

I just tried //sunFlare.png flareSettings = 0.45,1,0.085 //sunSpikes.png spikesSettings = 0.6,1,0.1 Literally just made it 10 times bigger by adding a zero to the last value in the first line and a decimal point to the last value in the second line And the results:

Great, I might be able to save even more memory.

Great, do you notice longer load times on switching scenes?

I shaved down the memory usage by about 200-300 mb by managing the memory myself, I only tested in 32bits on windows and dx11 and it seems to work well for me, but I'm going to need you guys to test it out and confirm whether it works or not on your platform and if the savings are indeed real. https://mega.nz/#!HQI2TTqB!dCS6guq2vDyb0c3QVtG33nR6KMAn0T6CnITf_CqTg7w It seems to actually work, at the KSC I now get about 1450-1500 mb memory usage instead of 1700 mb and at the runway 1750 or so instead of 1980 mb, I posted the new .dll above and I'm waiting for more people to report on how (or if) it works on their platform.

Check if removing scatterer fixes it. Switch to dx11 or disable terrain shadows.

Yes, it's literally 5 lines of code. QualitySettings.shadowDistance = 100000; //or a variable foreach (CelestialBody _sc in CelestialBodies) { if (_sc.pqsController) { _sc.pqsController.meshCastShadows = true; _sc.pqsController.meshRecieveShadows = true; } } I guess I could make it into a separate mod too, if someone else wants to do it don't wait on me though.

Are you sure you have the latest version? Did this happen before? Are you using any rescale mods? Information is key here. Yes I knew this for a while but people always associate it with scatterer regardless so I put it in the known issues sections. Also in the previous version (last week) there was a very similar bug with the view stuck but also with a blue background behind KSC that was indeed caused by scatterer so it was a bit confusing.

Well, I might still do it one day, but nothing planned at the moment.

You can reconfigure everything to work just fine for rescaled planets, only issue that may crop up with bigger planets is z-fighting.

I see, however in that ink I posted earlier the poster reports the memory successfully being freed. So I don't really know what's happening behind the scenes but I thought it might still be useful to try.

Well it seems using Marshal.FreeHGlobal() should work, or so the internet says, I was just wondering if there wouldn't be issues in runtime, or on linux and mac when using unsafe code.

Quick question to anyone who would know, I have an issue with large float and Color arrays never getting garbage collected despite leaving no references or anything behind and the arrays going out of scope. Should I use unsafe code like this to manage that memory myself?

I could give it a try, one problem is that the water illumination, sun reflection color/intensity and other information like that come from the precomputed sky irradiance table, which is part of atmospheric scattering. Anyway I suppose I could frankenstein it, however I really don't see the point of having the water shaders with the stock sky, I can't imagine it looking anything other than horrible. Perhaps you could explain what you see in that possibility.

Silly me, I forgot to hit the "publish mod" button yesterday on spacedock, it's fixed now.

I've put a new download link for this and made the settings persistent. Sorry about that, I tend to completely forget about this mod.

Are there no issues in the debug log? The terrain shadows issue is known as "shadow acne" and is out of my hands, it's more of an issue with how unity does shadows, might be a reason why SQUAD chose to not have terrain shadows.

Could you test if the performance issue persists with stock only? This also doesn't make sense but whatever works...

Really, you could just ignore it for now, it's certainly still easier on the eyes than the stock ocean.

Could you try capping the framerate at 60? I'm assuming you don't have a 100hz monitor. I won't get into technical details, but I would ve thought it's framerate related, specifically getting a higher framerate than your refresh rate, but you said it isn't the case so no idea. Well you kept mentioning 0.0223 which doesn't exist so I assumed you misspelled 0.0232

I may have an idea on why this happens, but it's also not fixable most likely, another quick question, do you have an fps cap in game and what kind of framerate do you get?