[DEV][0.24.x][Jul 24] SCANsat v7.0rc4 -- Real Scanning, Real Science, at Warp Speed!

[DMagic]

Well fudge. This method of using a non-bitmask for the resource scanner type won't work.

When the system registers a scan it looks at a position on the planet and the scanner type. It says at Long X and Lat Y the scanner type 2 (hi res altimetry) has scanned this area. So the array gets set to: [x,y] = 0000 0010. Now when scanner type 8 (biome) comes along to the same position the array gets set to: [x,y] = 0000 1010. So when we want to check if a position has been scanned we take our scanner type and check if it matches the array, position [x,y] = 0000 1010 so both the biome and hi res altimetry bits have been set, but the low res altimetry (type 1, 0000 0001) hasn't been set.

When we use a non-bitmask enumeration to set the array this all breaks down. When resource sensor type 21 (uranium) passes over long x and lat y the array gets set to: [x,y] = 0001 0101. Now let's say we want to check if resource scanner type 1 (Kethane) has passed over this same location; we look at the array: [x,y] = 0001 0101 and see that the Kethane byte: 0000 0001 has been set, so we can say that the Kethane scanner has also scanned this area. This is obviously a problem.

With the regular SCANsat sensors every sensor is assigned its own bit in a single byte. So we can unambiguously determine which sensor has passed over which location. With the resource sensors that doesn't work, each sensor is assigned more than one bit and so we can no longer determine exactly what has been recorded.

I can try setting the standard coverage map to be a short or an integer array, that would give us 16 or 32 bits (meaning 16 or 32 different scanner types). For now there are five SCANsat sensors, two Kethane resources and eight ORS resources. So I'm thinking that a short would not be very wise. I'm not sure how much using an integer array will blow up the data stored in the persistent file. But I'm already storing two coverage maps, so maybe this won't be so bad. Hopefully the fact that many of the potential bits remain unused will keep the size of the string stored in the persistent file to a reasonable size (you can easily see the difference between the string for an unscanned vs. a scanned planet already).

Of course, this could also simplify some things. I think doing it this way should get around some of the weirdness I've been seeing when trying to use a separate scanner type.

[Morgan]

Hi techi, great job!

But sadly I ran into 2 bugs with 7.0.

1st: No BigMap

(No idea why)

2nd:

(maybe a problem with one of the visual mods, testing it now)

Edit1:

Ok, these bugs belong together. The navball thingy reappears if I "close" the invisible Big Map.

Edit2:

Same goes for Settings and Instrument Windows of the Small Map. They disappear if I try to open the Big Map.

Edited May 22, 2014 by Morgan

[technogeeky]

Hi techi, great job!

But sadly I ran into 2 bugs with 7.0.

1st: No BigMap

(No idea why)

2nd:

http://i.imgur.com/VMfhpUw.jpg

(maybe a problem with one of the visual mods, testing it now)

Edit1:

Ok, these bugs belong together. The navball thingy reappears if I "close" the invisible Big Map.

Edit2:

Same goes for Settings and Instrument Windows of the Small Map. They disappear if I try to open the Big Map.

Yes. I ran into this bug today. This is a pretty serious one, but it's intermittent. Restarting KSP seems to sometimes get rid of it.

It seems the 'big map' button in the toolbar will open/close a variety of windows. Earlier, my game was such that clicking the Big Map button opened/closed two BOSS (Screenshot) windows AND the SCANsat Instruments window.

The problem seems to be in SCANui.gui_show(). I'll take a look. DMagic: can you also take a look?

In my output_log.txt, I am seeing lots and lots of this:

[MechJeb2] Adding button for Vessel_Info

(Filename: /BuildAgent/work/d3d49558e4d408f4/artifacts/LinuxStandalonePlayerGenerated/UnityEngineDebug.cpp Line: 53)

Unpacking Scanner 3

(Filename: /BuildAgent/work/d3d49558e4d408f4/artifacts/LinuxStandalonePlayerGenerated/UnityEngineDebug.cpp Line: 53)

ArgumentOutOfRangeException: Argument is out of range.
Parameter name: index
  at System.Collections.Generic.List`1[System.String].get_Item (Int32 index) [0x00000] in <filename unknown>:0 
  at SCANsat.SCANui.gui_show () [0x00000] in <filename unknown>:0 
  at (wrapper delegate-invoke) Callback:invoke_void__this__ ()
  at RenderingManager.OnGUI () [0x00000] in <filename unknown>:0 

(Filename:  Line: 4294967295)

ArgumentOutOfRangeException: Argument is out of range.
Parameter name: index
  at System.Collections.Generic.List`1[System.String].get_Item (Int32 index) [0x00000] in <filename unknown>:0 
  at SCANsat.SCANui.gui_show () [0x00000] in <filename unknown>:0 
  at (wrapper delegate-invoke) Callback:invoke_void__this__ ()
  at RenderingManager.OnGUI () [0x00000] in <filename unknown>:0 

(Filename:  Line: 4294967295)

I talked to blizzy78 and he suggests that it really is in gui_show and is probably not toolbar related.

DMagic:

Also, the dev version seems to have some issues big map rendering. All of these images are rendered with the branch 'dev' -- except the one that looks correct. When I checkout 'master' and/or 'master' with cherry-picked palette changes, it works fine.

Javascript is disabled. View full album

[technogeeky]

Well fudge. This method of using a non-bitmask for the resource scanner type won't work.

When the system registers a scan it looks at a position on the planet and the scanner type. It says at Long X and Lat Y the scanner type 2 (hi res altimetry) has scanned this area. So the array gets set to: [x,y] = 0000 0010. Now when scanner type 8 (biome) comes along to the same position the array gets set to: [x,y] = 0000 1010. So when we want to check if a position has been scanned we take our scanner type and check if it matches the array, position [x,y] = 0000 1010 so both the biome and hi res altimetry bits have been set, but the low res altimetry (type 1, 0000 0001) hasn't been set.

When we use a non-bitmask enumeration to set the array this all breaks down. When resource sensor type 21 (uranium) passes over long x and lat y the array gets set to: [x,y] = 0001 0101. Now let's say we want to check if resource scanner type 1 (Kethane) has passed over this same location; we look at the array: [x,y] = 0001 0101 and see that the Kethane byte: 0000 0001 has been set, so we can say that the Kethane scanner has also scanned this area. This is obviously a problem.

With the regular SCANsat sensors every sensor is assigned its own bit in a single byte. So we can unambiguously determine which sensor has passed over which location. With the resource sensors that doesn't work, each sensor is assigned more than one bit and so we can no longer determine exactly what has been recorded.

I can try setting the standard coverage map to be a short or an integer array, that would give us 16 or 32 bits (meaning 16 or 32 different scanner types). For now there are five SCANsat sensors, two Kethane resources and eight ORS resources. So I'm thinking that a short would not be very wise. I'm not sure how much using an integer array will blow up the data stored in the persistent file. But I'm already storing two coverage maps, so maybe this won't be so bad. Hopefully the fact that many of the potential bits remain unused will keep the size of the string stored in the persistent file to a reasonable size (you can easily see the difference between the string for an unscanned vs. a scanned planet already).

Of course, this could also simplify some things. I think doing it this way should get around some of the weirdness I've been seeing when trying to use a separate scanner type.

Why can't we just move 'everything' in the bitmask way, way up to to 2^30 (from 2^8) or something -- and use all the values in between? Would this break existing persistent.sfs files?

[technogeeky]

Unfortunately not true. The colour altitude map kinda falls apart. That website you linked is interesting, though I wouldn't trust it completely. The best way to test for colour-blindness support is to actually proof the colours with an example image. Photoshop has two proof modes for colour blindness, and there are piles of simulators that can do it.

Here's an online simulator you can use: http://www.color-blindness.com/coblis-color-blindness-simulator/

Last night I extracted all of the colors SCANsat uses anywhere into a new file called SCANpalette.cs:

All of the colors that start with cb_ are the colorblind safe ones. Clearly, these do not comprise a majority of color usage in SCANsat, however. And it's also clear they are not used for map drawing (the heightGradient colors are all xkcd colors).

And I would trust the site I linked because someone's PhD thesis depended on its correctness

[Morgan]

Yes. I ran into this bug today. This is a pretty serious one, but it's intermittent. Restarting KSP seems to sometimes get rid of it.

It seems the 'big map' button in the toolbar will open/close a variety of windows. Earlier, my game was such that clicking the Big Map button opened/closed two BOSS (Screenshot) windows AND the SCANsat Instruments window.

The problem seems to be in SCANui.gui_show(). I'll take a look. DMagic: can you also take a look?

In my output_log.txt, I am seeing lots and lots of this:

[MechJeb2] Adding button for Vessel_Info

(Filename: /BuildAgent/work/d3d49558e4d408f4/artifacts/LinuxStandalonePlayerGenerated/UnityEngineDebug.cpp Line: 53)

Unpacking Scanner 3

(Filename: /BuildAgent/work/d3d49558e4d408f4/artifacts/LinuxStandalonePlayerGenerated/UnityEngineDebug.cpp Line: 53)

ArgumentOutOfRangeException: Argument is out of range.
Parameter name: index
  at System.Collections.Generic.List`1[System.String].get_Item (Int32 index) [0x00000] in <filename unknown>:0 
  at SCANsat.SCANui.gui_show () [0x00000] in <filename unknown>:0 
  at (wrapper delegate-invoke) Callback:invoke_void__this__ ()
  at RenderingManager.OnGUI () [0x00000] in <filename unknown>:0 

(Filename:  Line: 4294967295)

ArgumentOutOfRangeException: Argument is out of range.
Parameter name: index
  at System.Collections.Generic.List`1[System.String].get_Item (Int32 index) [0x00000] in <filename unknown>:0 
  at SCANsat.SCANui.gui_show () [0x00000] in <filename unknown>:0 
  at (wrapper delegate-invoke) Callback:invoke_void__this__ ()
  at RenderingManager.OnGUI () [0x00000] in <filename unknown>:0 

(Filename:  Line: 4294967295)

I talked to blizzy78 and he suggests that it really is in gui_show and is probably not toolbar related.

Resetting Window Positions fixed it in my career savegame, but not in my sandbox one.

[diomedea]

Why can't we just move 'everything' in the bitmask way, way up to to 2^30 (from 2^8) or something -- and use all the values in between? Would this break existing persistent.sfs files?

Just thinking aloud, but another solution could be to keep the "normal" SCANsat bitmask at 2^8, to deal with SCANsat sensors and Kethane resources, and add another "ORS resources" bitmask, minimum at 2^8 but you may want to enlarge that. Both bitmasks written in the persistent.sfs for each body, and managed the same way (LZF compression and all, you know that much better than me). That would minimize impact on existing saves, possibly avoiding you to build a conversion method from old to new bitmasks.

[DMagic]

Yes. I ran into this bug today. This is a pretty serious one, but it's intermittent. Restarting KSP seems to sometimes get rid of it.

I've seen versions of this too. I don't think it has anything to do with the toolbar or any other mod, it's just SCANsat's GUI screwing with others (in a similar way that other mods can reset the GUI to the KSP style buttons rather than the Unity style). And gui_show is the all-purpose GUI handler for SCANsat, it's where everything seems to get collected before being sent off to the rendering manager.

Are the people who are seeing this using both Kethane and ORS resources? Does it happen right away when opening the big map, or only after changing things in the settings menu?

The one sure way that I know of to cause it (at least when I tried it) is to select "Open Resources" button, then select one of the last resources on the list, say Alumina, then change to the "Kethane Resources" button. Because Kethane only has one or two options something is getting confused and thinking that the third option is selected when it doesn't actually exist (presumably the same would happen in the reverse, but there are generally more ORS resources than Kethane resources). If you don't have Kethane resources installed then you'll notice that selecting the "Kethane Resources" button just shuts everything off, the toggle button above it disappears along with the overlay buttons on the big map. Doing something similar should be fairly simple when you have both resource types installed, probably just resetting the selection to the first resource option when changing types. And these UI elements are all temporary, I'm certain there are better ways of handling which resource you want to display.

But if people are seeing this happen in other cases something else may be going on. I'd be interested to know if everyone else is seeing the same error in the debug log, that "system.collections.generic.list" line at the top may be the important part, the list in question being the list of available resources to display and the integer being which member of the list is selected.

Did the map distortion weirdness persist through re-sizing it? I noticed that once, but just changed the size, and since I was checking something else didn't pay much attention to it. It's also possible that I didn't copy something right into that branch. You might want to try the v7 branch and see if the same thing happens, it doesn't include any of the color changes, but has all of my other updates.

[technogeeky]

I've seen versions of this too. I don't think it has anything to do with the toolbar or any other mod, it's just SCANsat's GUI screwing with others (in a similar way that other mods can reset the GUI to the KSP style buttons rather than the Unity style). And gui_show is the all-purpose GUI handler for SCANsat, it's where everything seems to get collected before being sent off to the rendering manager.

Are the people who are seeing this using both Kethane and ORS resources? Does it happen right away when opening the big map, or only after changing things in the settings menu?

The one sure way that I know of to cause it (at least when I tried it) is to select "Open Resources" button, then select one of the last resources on the list, say Alumina, then change to the "Kethane Resources" button. Because Kethane only has one or two options something is getting confused and thinking that the third option is selected when it doesn't actually exist (presumably the same would happen in the reverse, but there are generally more ORS resources than Kethane resources). If you don't have Kethane resources installed then you'll notice that selecting the "Kethane Resources" button just shuts everything off, the toggle button above it disappears along with the overlay buttons on the big map. Doing something similar should be fairly simple when you have both resource types installed, probably just resetting the selection to the first resource option when changing types. And these UI elements are all temporary, I'm certain there are better ways of handling which resource you want to display.

But if people are seeing this happen in other cases something else may be going on. I'd be interested to know if everyone else is seeing the same error in the debug log, that "system.collections.generic.list" line at the top may be the important part, the list in question being the list of available resources to display and the integer being which member of the list is selected.

Did the map distortion weirdness persist through re-sizing it? I noticed that once, but just changed the size, and since I was checking something else didn't pay much attention to it. It's also possible that I didn't copy something right into that branch. You might want to try the v7 branch and see if the same thing happens, it doesn't include any of the color changes, but has all of my other updates.

Yes, it did persist through resizing. I'll look further.

edit:

Yes, it persists through resizing. Yes, it happens on v7-Resource-Overlay. And v7.0.

Edited May 22, 2014 by technogeeky

[DMagic]

Why can't we just move 'everything' in the bitmask way, way up to to 2^30 (from 2^8) or something -- and use all the values in between? Would this break existing persistent.sfs files?

This is what I was thinking. This would get around any issues of dealing with two coverage maps, and I think 32 sensor types is more than enough. I've been working on converting an Int32 array into a byte array that the compression algorithm can handle, I seem to have dealt with that, I just need to handle the reverse now. It's possible that it may affect existing data, I can imagine that in converting from a byte array to an integer array it might get confused about which values go where (ie the first four bytes get added to the first element of the array, instead of getting added to four consecutive elements).

Using a second 2 or 4 byte array might better option for preserving existing data.

[Morgan]

Are the people who are seeing this using both Kethane and ORS resources? Does it happen right away when opening the big map, or only after changing things in the settings menu?

BigMap Bug right after install and yes, Kethane! And I can reproduce it, clicking Kethane is the fix (not "reset window position" as I mentioned before)

[technogeeky]

This is what I was thinking. This would get around any issues of dealing with two coverage maps, and I think 32 sensor types is more than enough. I've been working on converting an Int32 array into a byte array that the compression algorithm can handle, I seem to have dealt with that, I just need to handle the reverse now. It's possible that it may affect existing data, I can imagine that in converting from a byte array to an integer array it might get confused about which values go where (ie the first four bytes get added to the first element of the array, instead of getting added to four consecutive elements).

Using a second 2 or 4 byte array might better option for preserving existing data.

I'll just have to get better at testing before we release something like that.

Can you get on IRC this week? I'm set up to be in there whenever now (using irssi). But if you just want to get on there temporarily, the mibbit client isn't so bad.

[technogeeky]

DMagic: there is something else interesting about this bug (the bigmap-click-button bug): it causes the smallmap scanline speed to be much faster than usual.

It makes me question several things, one of which being the effect that framerate limiting would have on SCANsat (right now I have it set to unlimited).

[DMagic]

I'll just have to get better at testing before we release something like that.

Can you get on IRC this week? I'm set up to be in there whenever now (using irssi). But if you just want to get on there temporarily, the mibbit client isn't so bad.

I'll get on tonight.

I've found a way to load up and convert the byte array stored in previous versions to a usable integer array (I think something like this was done in a previous SCANsat update), it's pretty simple, but it may need to be some kind of one-time thing. SCANsat can store the required information for this in its scenario module, but I'll need to find a way to handle new vs old saves.

[technogeeky]

I'll get on tonight.

I've found a way to load up and convert the byte array stored in previous versions to a usable integer array (I think something like this was done in a previous SCANsat update), it's pretty simple, but it may need to be some kind of one-time thing. SCANsat can store the required information for this in its scenario module, but I'll need to find a way to handle new vs old saves.

In order for me to work on it, and in order to debug (if needed) the Settings window, I broke it out from being one function and made it eight.

        public static void gui_settings_build ( int wid ) {
            GUI.skin = null;

            GUILayout.BeginVertical ();

            GUILayout.Space (8);
            gui_settings_xmarks(wid);                 /* X marker selection */
            GUILayout.Space(16);
            gui_settings_resources(wid);                /* resource details sub-window */
            GUILayout.Space (16);
            gui_settings_toggle_body_scanning(wid);        /* background and body scanning toggles */
            GUILayout.Space (16);
            gui_settings_timewarp (wid);                /* time warp resolution settings */
            GUILayout.Space (8);
            GUILayout.Label (gui_settings_numbers (wid));/* sensor/scanning statistics */
            GUILayout.Space (16);
            gui_settings_data_resets (wid);            /* reset data and/or reset resources */
            GUILayout.Space (8);
            gui_settings_window_resets (wid);            /* reset windows and positions */
            GUILayout.Space (16);
            if (GUILayout.Button ("Close")) settings_visible = false;    /* close settings */

            GUILayout.EndVertical ();
            GUI.DragWindow ();
        }

None of them actually need the window id, but I didn't know better at the time.

[DMagic]

I've reverted the changes I made to v7 for the Kethane side of things and fixed a number of bugs. It should initialize properly now, checking for the presence of any resources, and disabling the overlay if none are found. This will prevent the big map from freaking out if you don't have any ORS resources installed (it defaults to ORS resources, not Kethane). I've also fixed the problem I described above about switch from ORS to Kethane resources causing the same problem. I also fixed the screwed up map, the pixel color just needs to be re-initialized for each pixel, not each line.

Most of these changes are separate from the problems I discussed about using different scanner types, so these things needed to be fixed anyway. This version still doesn't have any support for Kethane resources.

TG: You can rebuild the v7-Resource-Overlay branch, leave out SCANsatKethane, like last time. It doesn't have any of the color palette changes, I just updated directly from the released version of the code.

Changelog:

- Big map should no longer break if no ORS resources are installed
- Switching from a high-number ORS resource to Kethane should not break the big map
- Incomplete big map should render properly
- Spottiness of resource scanners may be improved, it's hard to say, maybe ok up to 100-1000X timewarp
- Setting menu buttons don't change color anymore

[BananaDealer]

Huh? SCANsat got updated? Why did I not hear of this...

(a.k.a shameless sub post)

[DMagic]

Huh? SCANsat got updated? Why did I not hear of this...

(a.k.a shameless sub post)

You can try out v7, but it is primarily for testing, and the released version has a number of bugs, only some of which were fixed in the update described above.

[BananaDealer]

You can try out v7, but it is primarily for testing, and the released version has a number of bugs, only some of which were fixed in the update described above.

I am used to bugs... This is the KSP modding community we are it, after all...

Also, on a related note- Updating my 50-odd mods is tedious...

I leave KSP alone for a month and y'all decide to have a coding party...

[DMagic]

On the topic of colors, I'm wondering if the overlay should always have a background shade to help differentiate resource areas from empty areas.

I think this would be helpful, not only for color-blind support, but for making it easier to see what areas have and haven't been scanned. The ORS resources overlays currently don't do this; I think those need a better method for assigning a resource concentration cutoff first. Right now the cutoff is just the scaleFactor * scaleMultiplier and doesn't consider the display threshold value.

I'm also thinking that the grey scale map should stick with one color/shade for all resources, and only the color map should use different colors. Using a grey background for empty areas would probably help in finding compatible colors for that map too.

[phoenix_ca]

Those are good ideas. I rather like the one on the right. For colour-blind support, one colour for all resources wouldn't do it. Ideally you'd have a choice of about three colours to use, one for each primary type. More ideally, just grab the colour from a user-editable text file in some format like nodes we have now, or XML, or JSON, etc., so that colour-blind support would be as easy as creating a new colour profile for all colours in SCANsat. I'd happily volunteer making such configs if you need them.

[technogeeky]

Those are good ideas. I rather like the one on the right. For colour-blind support, one colour for all resources wouldn't do it. Ideally you'd have a choice of about three colours to use, one for each primary type. More ideally, just grab the colour from a user-editable text file in some format like nodes we have now, or XML, or JSON, etc., so that colour-blind support would be as easy as creating a new colour profile for all colours in SCANsat. I'd happily volunteer making such configs if you need them.

Slow your roll!

I'm working on color palettes as a whole package. Everything will be editable in-game. The primary unit of color currency will be a palette, not a color.

• The default color palette should be colorblind friendly.
  
• There should be three or four color palettes available for choosing on the right hand side (laid out vertically) of the Big Map.
  
• These three or four should be selectable from a larger set on some yet-to-be-named palette settings window.
  
• You should be able to add your own palette, or copy existing palettes and make changes to them.
  
• There should be smaller palettes for overlays (two or three colors).
  

[rottielover]

Earlier in the thread you mentioned that you were still considering more idea's for extending SCANsat's functions, so I'll re-suggest something I put in the original SCANsat thread back when it was in the 2nd or 3rd release (if I remember right).

"Landing Zone Assistant"

Highlight "good landing zone candidate spots" based on user selected criteria. Examples would include a slope range selection (IE I would like to land on level ground, or I'll tolerate x% slope). I would like to land in a spot that is within Y number of kilometers from Z number of biomes.

Thoughts?

[phoenix_ca]

Sounds good. I'll just resort to breaking it later then.

[technogeeky]

I've updated the first post to reflect the release of v7.0rc1. This includes several bug fixes with resource scanning. If you are using the DEV version and having bugs, we recommend you upgrade.