ISA MapSat "Legacy" discussion

[Tiron]

Please stop editing your old posts to add new stuff to them, its annoying. And to answer your question, both maps were 2048x1024 when I cropped them. They were both scaled up x2 so you could actually see the differences better.

Oh, I see. You didn't mention the part where you scaled it down after making the 16k version.

And replying to your own posts in series is, as a general rule, against the rules of most forums and apt to get you banned. I've seen mods on here chastise people for it and tell them to edit their previous post instead.

In short, to add more information when nobody's replied, editing is the REQUIRED behavior.

I sometimes think of things faster than other people reply (bloody ADHD), so I end up doing that rather than waiting for someone to reply. I stop editing it and carry on below when someone else DOES finally reply.

Which means, in short, the only way it could change on you is if: A.) You opened the thread a substantial time before replying and didn't refresh, in which case missing edits is the least of your worries. B.) I happen to do an edit in the middle of you posting something, which isn't particularly any different than if I made a new post while you were typing.

As a general rule I mark my edits with headings for just this reason, to make it obvious which is which. Except for a short period right after I post, which is mostly just grammatical corrections I didn't catch initially.

[Majiir]

I meant a lot of CELLS. The current ISA maps are 2048x1024 (although the polars map obviously has substantial areas that aren't part of the actual map because it has two round bits in a rectangular texture.) Easy enough to set up logically, but I can just imagine that setting it up graphically could be a real pain.

I don't understand what you mean by cells... are you saying we'd need a large array? Yes, but likely smaller than the one required to store a texture.

At this point, I don't think memory consumption is really the greatest concern, but I think it's worth mentioning what techniques exist for milking the most out of the memory we have because it opens up other possibilities.

It's the method he uses for his Kethane Hex Map. It stores the scan data basically as a mask in the savefile, with a single character indicating if each segment's been scanned or not. There's some other data in there which I'm not sure exactly what it's for, but it's a really simple solution that only requires 1 byte per cell. He's also got it set up to have the cell colors be based on the amount of Kethane in the cell, and there's no actual on-disk textures for any of it. So far as I can tell, he generates them on the fly too. It's very slick.

Each character actually contains data for six cells. Each grid cell has a unique index, so the scan flags are concatenated as 10,242 bits which are then Base64-encoded. My local copy uses Deflate compression to make the whole thing even smaller on disk. They're not compressed in main memory, but the entire solar system of scan masks only uses about 21 kilobytes in RAM.

If I manage to get my new geodesic code working, I'll probably want to crank up the resolution to the point where partial loading will be necessary. With Kethane, I decided to be incredibly careful with what I store, and that affords me a lot of flexibility in how much of it I store. All the data on every planet remains in memory at all times because it all adds up to just a couple megabytes.

[Tiron]

I don't understand what you mean by cells... are you saying we'd need a large array? Yes, but likely smaller than the one required to store a texture.

At this point, I don't think memory consumption is really the greatest concern, but I think it's worth mentioning what techniques exist for milking the most out of the memory we have because it opens up other possibilities.

Each character actually contains data for six cells. Each grid cell has a unique index, so the scan flags are concatenated as 10,242 bits which are then Base64-encoded. My local copy uses Deflate compression to make the whole thing even smaller on disk. They're not compressed in main memory, but the entire solar system of scan masks only uses about 21 kilobytes in RAM.

If I manage to get my new geodesic code working, I'll probably want to crank up the resolution to the point where partial loading will be necessary. With Kethane, I decided to be incredibly careful with what I store, and that affords me a lot of flexibility in how much of it I store. All the data on every planet remains in memory at all times because it all adds up to just a couple megabytes.

Ahhh, I WONDERED why 'fully scanned' showed up as a long list of As. Makes sense. Very shiny.

And what I really was referring to was the fact that getting such a large array to show up nicely as a rectangular grid would probably be a pain, just because you're basically substituting entries in the array for pixels. While you could easily use the array indexes to figure out which goes where, getting the entire thing to display and line up properly, particularly in a movable UI item, would probably bit a bit of a bother, just because of the sheer complexity.

I have no doubt whatsoever it'd use a LOT less memory, simply because you'd only have to load a series of single-pixel textures that are used over and over, as opposed to 30 very large textures that are each used once.

As for getting the memory usage down not being a huge concern, I tend to agree...but it seems to be the number 1 complaint against the current version of mapsat, and the primary reason the topic that became this thread came up in the first place: People with a lot of mods have problems with crashes due to out of memory errors, and identify Mapsat as being a huge memory hog(which it is, although I think it's only because the game's caching all the intermediate steps in the 'get it to the vram' pipeline unnecessarily) and the cause of their problems.

That misidentification as being the cause of crashes seems to stem from one thing: They have more problems when they use a ship with a Mapsat part on it than they do if they don't use the mapsat parts. Which ultimately seems to stem from the fact that it doesn't even load the maps unless the mapsat module is loaded, which it obviously isn't if none of the parts that use it are loaded. This causes the total working set to be about 600MB higher when using a craft with mapsat than without it, and that pushes their already marginal memory usage right up to or over the edge.

Edited September 3, 2013 by Tiron

[Torminator]

I'm still trying to figure out what a "Class" is and where they put my beloved functions in c#, but if I'm interpreting Majiir correctly, here's what I think might be a good way to do it.

Mapsat scans the planet, by whatever means, and returns an elevation value for each point, depending on selected resolution. This information is stored as plain-text, or something similarly simple. I.E. lat/long/elv.

When you choose to view a map in, let's say a 512x512 window, the plugin generates (is this even possible?) a texture file that is only 512x512 for the region you want to view, based on the relevant data points in the plain-text file (which could be unloaded from memory when not needed, no?). The detail is less, but since you couldn't see more detail anyway, it doesn't matter. If you zoom in, it dumps the previous image and reconstructs a new one at a higher level of detail. If you zoom out, it does the same thing, but regenerating a less detailed map. In this way, the "maps" are never stored as images, only their raw information, and the textures that are generated are fully procedural and much smaller than a full map image. Perhaps prevent the plugin from "Releasing" the data file unless there have been no user inputs for ~2 seconds or so, to prevent jerkiness while panning/scrolling.

So, vector images, more or less. I'll admit to not QUITE being able to keep up with the coding talk, so maybe somebody already said exactly this, but hey.

[Tiron]

I'm still trying to figure out what a "Class" is and where they put my beloved functions in c#, but if I'm interpreting Majiir correctly, here's what I think might be a good way to do it.

Mapsat scans the planet, by whatever means, and returns an elevation value for each point, depending on selected resolution. This information is stored as plain-text, or something similarly simple. I.E. lat/long/elv.

It kinda can already do that. There's an option that allows you to save the raw data, which is lumped into a CSV file, which you could use an included program on to generate a map of any size you wanted, with configurable options for the colorscheme et-al that was used. The dev version still includes the option and still generates a file out of it, but didn't include the program to use the data. I forget why, need to re-read Innsewerants' post on the subject, heh. It was never a feature I paid much mind to.

When you choose to view a map in, let's say a 512x512 window, the plugin generates (is this even possible?) a texture file that is only 512x512 for the region you want to view, based on the relevant data points in the plain-text file (which could be unloaded from memory when not needed, no?). The detail is less, but since you couldn't see more detail anyway, it doesn't matter. If you zoom in, it dumps the previous image and reconstructs a new one at a higher level of detail. If you zoom out, it does the same thing, but regenerating a less detailed map. In this way, the "maps" are never stored as images, only their raw information, and the textures that are generated are fully procedural and much smaller than a full map image. Perhaps prevent the plugin from "Releasing" the data file unless there have been no user inputs for ~2 seconds or so, to prevent jerkiness while panning/scrolling.

So, vector images, more or less. I'll admit to not QUITE being able to keep up with the coding talk, so maybe somebody already said exactly this, but hey.

Well what Majiir was talking about more or less was storing the data as a grid instead of an image. You don't even really create a full map texture at all: You create a whole series of 1 pixel tiny ones that are shown for each point in the grid. This is actually exactly how an image actually works: Each pixel is a point with a defined color. In this case you'd be reconstructing the image with just the control data for it. The memory savings would be because you'd only have to generate a single 1x1 texture of each color you were using, and then re-use that for each grid point/pixel where that color occurs. It's the same basic method he uses for the current incarnation of the Kethane map, just a lot higher resolution and with a different layout.

I suspect it'd be a pain to get it working properly, in terms of time and effort, but there's no reason it couldn't be done (as he's amply demonstrated).

[macegee]

Any news on the subject, guys?

[Benzschwagel]

so, been in the field doing training exercises, what's been happening while I've been away? Someone mentioned something about having the maps loaded outside of KSP and the Unity engine, is this even possible? it would definitely help, unless the unloading issue gets solved. by the way, did innsewerants ever return? and I hope we can all get along, we are trying to make something cool to benefit everyone.

[ThirdHorseman]

I've been trying to simply replace the images in the PluginData folder with high-res maps, since I'm more concerned about getting my current position using the GPS part rather than scanning the terrain. Every time I replace the topo images with my own, they get all corrupted. I have scanning turned off on all my mapsats, but it seems like as soon as I toggle on the mod using the GUI it tries to alter the maps.

Is there a simple way to replace the maps?

I'm using the old 3.3.4 version BTW...

[Benzschwagel]

I think the mapsat mod built maps off the planet's current surface as soon as the scanning sat was within a certain range. or maybe I have that a little messed up, but it's basically like that. I forget how it worked, but it basically loaded up a map it had created. It would create the map at some point, whether as needed or all at once I'm not sure, but when the planet was being scanned it would list areas as "scanned". I guess if there was a way to just tell it, "stop building your own map, use mine instead," you could make something work. It's main problem was the fact that it wouldn't unload any maps from memory, causing it to crash KSP because it would reach Unity's 32-bit memory limit. I heard that Unity was planning on looking at the viability of a 64-bit version, but not sure if anything actually happened...

So, if you made some really low memory footprint maps, and forced mapsat to use those instead of creating new ones and overwriting yours, and then edited something to say that all maps were already fully scanned (like some sort of on/off or true/false feature) you could make something work. don't forget about that hilo.dat file, I believe that does something with reloading maps after a planet change. there may not be a simple way to replace the maps, then again, look through the code if you know how, maybe he set up a way to use manual maps.

Edited October 10, 2013 by Benzschwagel

[Benzschwagel]

bump, new info in first post

[zalzane]

wait so let me get this straight - he's keeping the mod open source, but with all rights reserved, and doesn't want anyone to build on it?

jesus christ it's like a high school programming course

[sal_vager]

It's his right, have to accept it I'm afraid.

[Tiron]

wait so let me get this straight - he's keeping the mod open source, but with all rights reserved, and doesn't want anyone to build on it?

Releasing the source is a Squad Requirement to post it on the forums or Spaceport.

Note that making the source available is not the same thing as 'Open Source'. 'Open Source' implies that the rights to modify and redistribute are included. Fact is though, there's two different, unrelated things and you can do one without the other.

He's not hardly the only modder that released the source to satisfy Squad's requirements but reserved all rights.

[John FX]

So we are back where we were in the middle of the thread with having to come up with something NOT based on the code for MapSat.

So, if the actual scanning is CPU intensive and in game we want to scan without disrupting our framerate, why not write a little bit of code that runs outside of KSP that `scans` all the planets and generates files which can be `scanned` in-game quickly? Then it can run in the background if you have spare cores+ram causing no slowdown in KSP and still full sized maps (or maps that can be scaled to match your memory profile)

This solves the issue of finding anomalies, waiting for a patch for the current maps, CPU intense scanning, and memory.

Ancient and majiir, have you had any progress?

[OrtwinS]

So we are back where we were in the middle of the thread with having to come up with something NOT based on the code for MapSat.

-snip-

Ancient and majiir, have you had any progress?

The tech part of this thread died 2 months ago.

In an attempt to refocus the discussion from 'ISA Support' or 'ISA legacy' towards 'new mapping plugin' I created a new thread:

http://forum.kerbalspaceprogram.com/threads/54422-REQUEST-ISA-Mapsat-replacement-with-area-indication?p=717870#post717870