ISA MapSat "Legacy" discussion

[somnambulist]

The biggest limitation is actually PQS (the game's terrain system). It's computationally expensive to query for terrain details, so it's important to be careful with storing and caching that data

Would it be possible (and non-EULA-breaking) to extract the terrain data ahead of time and then generate maps on that preexisiting data set?

[johnsonwax]

I think the whole idea of using a map made up of hexagons for the purpose of creating an actual height map is insane. It's not intuitive and at all and would feel terribly realistic. It's different for resources, because you're not trying to create an exact map of the resources--you just want to know where they are, and where they aren't.

But you haven't addressed the problem of USING the maps, which a lot of us do. Lets say you want to land on the dark side of a body. The maps allow you to pick your landing spot even when you can't see the ground, you can get a set of coordinates, but then what? The in-game HUD works on the actual body - you can adjust your maneuver node to hit a spot on the ground, but there's no way to hit a set of coordinates unless you plug them into MechJeb. Since the in-game navigation tools work on the bodes themselves, it makes sense to put the map there. Pick a spot on the map, adjust your maneuver node and land. And the old system made it really hard to see if a given landing spot had kethane on it, and as new features get added, we'll have the same problem. If I can find a promising spot, hit a button and have it switch to kethane, then zoom in on that hex and fine tune my location, that would be fantastic. You can't do that now - not even close. And the web based maps don't help with that either.

For ISA, mapping precisely and completely is an art; that's the fun in it.

For you. But most players aren't trying to replicate the iridium network. You are. You like that stuff. That's cool, but that's not everyone. And like I said (and Majiir reiterates) there's no reason why the hex system couldn't still work where it requires that kind of effort with a setting to make it simpler for players that want a less detailed solution. The current system doesn't allow that.

[Benzschwagel]

about multitasking mapsat data acquisition, I understand that when you switch to a different location / session, the object basically disappears and is replaced by some sort of placeholder to display it's position, so no physics are being acted upon it. what if there was a way to simply use it's current position to find the terrain directly below it? you're not doing anything special, just waiting something to collect info on the surface below it, which could be saved to a file to save on memory, all while you're launching another mission. Maybe some sort of on/off tag associated to each hex to determine whether the mapsat had passed over it yet, allowing the data to be call up for that hex. I think this might actually be similar to kethane.

[Benzschwagel]

a possibility could be a type of overlay system that displays multiple types of info (terrain, kethane, topography, etc...) possibly superimposed on one another to allow for near pinpoint accuracy.

[drtedastro]

I like the idea of using 'hex' bins for elevation and all sorts of different attributes. Hexagonal binning was used in the mid 90's as an alternative to standard square and rectangular binning in the oil and gas industry for data analysis. As Majiir says, this would be a very interesting way to look at the data, especially if we can maintain the ability for course, medium and fine sampling. It certainly makes for far better mapping when we do not have to take datum, projection and such into account.

Alas, we may have to fully accept that the world is round some day....... But until then, stretch the hell out of the poles for a Mercator map......

[GavinZac]

about multitasking mapsat data acquisition, I understand that when you switch to a different location / session, the object basically disappears and is replaced by some sort of placeholder to display it's position, so no physics are being acted upon it. what if there was a way to simply use it's current position to find the terrain directly below it? you're not doing anything special, just waiting something to collect info on the surface below it, which could be saved to a file to save on memory, all while you're launching another mission. Maybe some sort of on/off tag associated to each hex to determine whether the mapsat had passed over it yet, allowing the data to be call up for that hex. I think this might actually be similar to kethane.

What I was thinking about, is that the RemoteTech system actually tracks where the crafts are in space, and works out stuff based on that. If the planet's data is preloaded but hidden (as opposed to really, actually scanned for the first time by the satellite as in ISAMapSat), the plugin could figure out what to mark as 'revealed' based on the satellite's behaviour, so long as the plugin is loaded at all times (perhaps included in every command core, I'm not sure if it can work by having it as a part-less plugin).

Just to correct someone previously, ISAMapSat makes very detailed maps. You may have missed this, but they can be zoomed in pretty far. That, in fact, is the biggest issue, it scans all of that and loads it all into memory at the same time.

A problem for the idea of concurrent scanning, particularly with the scanning as Majiir has described, is sheer complexity. What happens when we have 8 or 9 satellites in orbit around different bodies? How will the CPU handle that, how will memory handle that?

[Benzschwagel]

hmm, that would take some thought as to what options to offer. Offering a toggle switch to turn sat on or off, using a plugin to offer a gui to enable selection of which sat's to pull data from. I'm not sure about what would be required for ram and processing power, but to track a few sets of numbers, would it require that much?

[jrandom]

How will the CPU handle that, how will memory handle that?

That's dirt-simple: break the sphere into segments for the high-res data (like the kethane hexes) and only have one high-res segment per scanner in memory at any given time. Full display maps can be created for display-resolution (eg. much smaller) in a separate thread by reading in each segment the map window covers, running through all the data to generate the visuals, render to map window, toss away that segment and read in the next. If handled in a single separate thread this will run fairly smoothly and have nearly zero impact on the rest of the game. Memory footprint would be tiny. The OS will cache any segments only used for reading so you're not constantly hitting the disk re-displaying maps you've just looked at.

[GavinZac]

If we're assuming that the data is pre-existing and all we're doing is marking the nearest grid hex within range of each satellite 'revealed', it should be even easier, right? I mean, complex to code but not too taxing on the computer. Remember, while 'real' scanning is impressive, it's a game and the end user shouldn't see the difference anyway. So, like in Kethane, there would be a simple scanned/not scanned flag and the extra information (capacity remaining, in Kethane, the hex's map, in the theoretical Gridmapper) can be called on when it is needed, i.e. producing a single large scale planet map or viewing a particular hex in detail ([i would really love if we could see potential landing sites in the kind of detail that TakeOnMars shows you on its mission screen](http://i.imgur.com/4p1zpFV.jpg)).

I think this fits with something - and he can correct me if I misremember - that Majiir once said about how one of the points of making Kethane is to give people a reason to select their landing site. If it becomes easy (once scanned) to pick an awesome landing site, I'm far more likely to try that and risk oblivion than I am to, (as I do at the moment) say 'well, that bit look bumpy and I have no idea exactly what I'm landing on, so I'll just land on a flat bit'. It would be awesome, too, if anomalies could be literally seen on the maps, rather than represented as a red dot. Something that might require manual editing of the data before it is 'revealed'.

As an aside, something I've actually never thought about until this point: In Kethane, does the grid scale with the planet, or does it add more hexes for bigger planets?

[Crater]

Guys, as Majiir pointed out, the data is not pre-existing, and is in fact CPU intensive to discover. The reason that mapsat scans and builds the map (and CSV) is to actually find the data and cache it for future use. It doesn't flag that you "know" this bit now, so you can use it, it really does scan the terrain and find its height for mapping purposes.

This means that you can't just flag a hex as "known", unless you also perform the actual scans of that hex once you enter it. Someone suggested that a hex could be a full-screen map if you zoomed (which sounds good to me). That would mean something in the region of 1000px square, which is 1 million scan-point. Majiir quoted that in testing he has done, you can perform around 10,000 height queries in 200ms, which would mean that it would take a full 2 seconds to scan a single hex at that resolution. That would mean ZERO frames per second for a 2 second lag-spike, every time your sat entered a new hex.

In principal, I love the idea, but that would be WAY too stuttery for me, and probably many others, which would mean that you're back to really scanning actual over-passed areas in each frame as your sat moves, rather than whole hexes, for smoothness of game.

The same problem comes in if you enable non-focussed scanning. The issue isn't keeping all that data in memory, it is having 30 times more sats doing terrain queries in each frame than if you only have it for the focussed vessel. Either framerates would drop to nothing, or scan resolution would have to be terrible. You can't get anything for nothing.

[GavinZac]

Guys, as Majiir pointed out, the data is not pre-existing, and is in fact CPU intensive to discover. The reason that mapsat scans and builds the map (and CSV) is to actually find the data and cache it for future use. It doesn't flag that you "know" this bit now, so you can use it, it really does scan the terrain and find its height for mapping purposes.

I know that is true of ISAMapSat, I stated that above.

This means that you can't just flag a hex as "known", unless you also perform the actual scans of that hex once you enter it. Someone suggested that a hex could be a full-screen map if you zoomed (which sounds good to me). Your Duna is the same as my Duna. That would mean something in the region of 1000px square, which is 1 million scan-point. Majiir quoted that in testing he has done, you can perform around 10,000 height queries in 200ms, which would mean that it would take a full 2 seconds to scan a single hex at that resolution. That would mean ZERO frames per second for a 2 second lag-spike, every time your sat entered a new hex.

In principal, I love the idea, but that would be WAY too stuttery for me, and probably many others, which would mean that you're back to really scanning actual over-passed areas in each frame as your sat moves, rather than whole hexes, for smoothness of game.

The scans only have to be done once per planet-surface-update, and distributed with the mod. The same was true of ISAMapSat, simply dropping in someone else's scans gave you perfectly usable maps if you both were using the same version.

The same problem comes in if you enable non-focussed scanning. The issue isn't keeping all that data in memory, it is having 30 times more sats doing terrain queries in each frame than if you only have it for the focussed vessel. Either framerates would drop to nothing, or scan resolution would have to be terrible. You can't get anything for nothing.

Solved by above.

[Cykyrios]

These posts have been split from the ISA MapSat thread, as they are not discussing the mod itself, but ways to improve upon it.

Feel free to tell me if the title I gave that thread is inappropriate.

[Majiir]

The system GavinSac proposes is similar to what I'd do. I'd probably increment a number (representing scanned detail, more on how this might work another time) instead of changing a flag, but it's the same idea. Simply updating information in a grid structure isn't very CPU or memory intensive; you could run dozens of background satellites without impacting performance. When you view a cell, you can query PQS and progressively update the view for a smooth experience.

[Kimberly]

If you were to use a hexagonal scanning method, how would you handle differing resolutions and level of detail? Frankly, Kethane scanning is very simplistic--and that's not necessarily a bad thing, because mapping is not the central focus of the mod. But it would certainly devalue Mapsat if you went with the Kethane paradigm of "lower orbital period = better", with a ton of scanners being necessary for scanning at warp instead of being able to adjust resolution and orbit to something that will work at a high level of warp and produce a high-detail result.

(On a related note, if you reveal a pre-generated image instead of doing actual scanning, would the mod still be compatible after terrain changes like Mapsat is now?)

[MOARdV]

If you were to use a hexagonal scanning method, how would you handle differing resolutions and level of detail? Frankly, Kethane scanning is very simplistic--and that's not necessarily a bad thing, because mapping is not the central focus of the mod. But it would certainly devalue Mapsat if you went with the Kethane paradigm of "lower orbital period = better", with a ton of scanners being necessary for scanning at warp instead of being able to adjust resolution and orbit to something that will work at a high level of warp and produce a high-detail result.

(On a related note, if you reveal a pre-generated image instead of doing actual scanning, would the mod still be compatible after terrain changes like Mapsat is now?)

The LOD changes could be handled by counting how many scans a hex has received. More scans = more detail (the exact details to be worked out).

Using pre-generated images only works as long as the world doesn't change. I haven't looked at the data used in game, but the mapper could conceivably detect "changed" planets and invalidate its maps, maybe with a UI prompt warning that it has to regenerate the maps in question. If Majir is right and the PQS queries can run in a separate thread, those maps could regenerate in the background while the player is doing other stuff.

[Kimberly]

I wouldn't make it dependent on the amount of scans. That would encourage a Kethane-like scanning pattern, where you want to have as low an orbital period as possible because you can only scan one hex at a time anyway.

[GavinZac]

Using pre-generated images only works as long as the world doesn't change. I haven't looked at the data used in game, but the mapper could conceivably detect "changed" planets and invalidate its maps, maybe with a UI prompt warning that it has to regenerate the maps in question. If Majir is right and the PQS queries can run in a separate thread, those maps could regenerate in the background while the player is doing other stuff.

I still think that, while it'd force a re-download every time the planets change, supply all the data to the end user without having their machines generate the maps at all is a more foolproof solution, particularly once KSP hits beta and features begin to be locked.

[Schmonzo]

I still think that, while it'd force a re-download every time the planets change, supply all the data to the end user without having their machines generate the maps at all is a more foolproof solution, particularly once KSP hits beta and features begin to be locked.

I'm quite sure that KSP will not have a 'normal' beta. We will get new features (alpha) and test them (beta) and then repeat the process until we have the final version of KSP. That's how I understood Harvester anyway...

So I am not sure if features will be locked soon. Also: What happens when we get user created planets, wouldn't it e cool if the maping mod could map them without any additional work?

[Majiir]

If you were to use a hexagonal scanning method, how would you handle differing resolutions and level of detail?

Icosahedral grids are extraordinarily well-suited to this. They start as an icosahedron, which is twelve vertices and twenty faces, and then those faces are subdivided until you get the desired level of detail. You can even subdivide some faces more than others. Level of detail is essentially built into the design of the grid. Currently, the Kethane grid code isn't well-suited to varying subdivisions, but I've been working on an indexing method that more closely represents the recursive nature of the grid.

This means a mapping satellite would never scan the point directly below the spacecraft; instead, it would scan a point on the grid at the current level of detail. I doubt anyone would ever notice the difference, but it means you aren't generating a constant stream of potentially redundant data like ISA. Instead, you're constantly increasing the level of detail at areas around the planet, generating a grid that reflects those levels of detail, and then sampling terrain at those grid points.

It's hard to see why such a system works elegantly for mapping if you've only played with the Kethane grid, which appears to be a set of discrete same-sized cells. A subdivided grid (of any sort) affords a lot of flexibility with level of detail. Perfect example: the KSP terrain system uses subdivided quads. The geodesic grid is just a subdividing grid that works well for evenly distributing data on a sphere.

Frankly, Kethane scanning is very simplistic--and that's not necessarily a bad thing, because mapping is not the central focus of the mod. But it would certainly devalue Mapsat if you went with the Kethane paradigm of "lower orbital period = better", with a ton of scanners being necessary for scanning at warp instead of being able to adjust resolution and orbit to something that will work at a high level of warp and produce a high-detail result.

I wrote earlier that you should not jump to conclusions about how such a system would work just because some of the underlying code comes from Kethane. The details of Kethane scanning have little to do with the geodesic grid. The behavior of a mapping satellite can (and if I have my way, will) be very different.

On gathering PQS data: I'd suggest caching terrain data (and storing which game version the data comes from so it can be invalidated on an update), but I wouldn't ship the mod with data. There's not much benefit to be had from that. A background task (even if on the same thread) can populate terrain data for cells that the plugin expects to eventually render.

[DerekL1963]

But you haven't addressed the problem of USING the maps, which a lot of us do. Lets say you want to land on the dark side of a body. The maps allow you to pick your landing spot even when you can't see the ground, you can get a set of coordinates, but then what? The in-game HUD works on the actual body - you can adjust your maneuver node to hit a spot on the ground, but there's no way to hit a set of coordinates unless you plug them into MechJeb. Since the in-game navigation tools work on the bodes themselves, it makes sense to put the map there. Pick a spot on the map, adjust your maneuver node and land.

Yes, it works very well - for than *one specific purpose*. For pretty much everything else, like actually being able to examine the planet in detail, the hex grid system is an enormous step backwards. (You can't zoom in to anything even remotely approaching the same level of detail from the orbit window as you can in the Mapsat window.) It sucks for being able to examine fine scale details. (Because the huge size of the hexes reduce resolution.*) It sucks for trying to examine the dark side of the planet. etc... etc...

Graphical overlays are not maps.

*See this article on Wikipedia for information on this effect.

[TeddyDD]

In my view, all mods should be CC licensed. It would not be abandoned good mods :/

[GavinZac]

In my view, all mods should be CC licensed. It would not be abandoned good mods :/

It would be good, but I guess end-users should learn not to get too attached to 'private' mods made before the game it's modifying is even released.

[Britich]

In my view, all mods should be CC licensed. It would not be abandoned good mods :/

Fun Fact, according to real copyright laws any mod is owned by the IP holder, in this case its Squad.

So someone could take this mod over without even needing to ask.

[Crater]

Uhhhh, in this case, I'm pretty sure that there is Intellectual Property in the mapsat code, as well as the base game, and that IP is still owned by Innsewerants.

Otherwise, you could start to argue that Microsoft owned all the Excel addins out there, and given that some of the mathematical solver ones cost tens of thousands of dollars, I can't see them passing up a chance to take them over if the law supported it.

[Capt'n Skunky]

Fun Fact, according to real copyright laws any mod is owned by the IP holder, in this case its Squad.

So someone could take this mod over without even needing to ask.

That may be a fun fact, but the real fact is that Innsewerants still retains the rights to his code. And in the absence of specific permission, no permission is the default. So until Innsewerants says otherwise, we will not allow any derivatives of his work on our network.

This is not to say someone can't create and build a similar add-on, only that it can't contain code from ISA MapSat.

Cheers!

Capt'n Skunky

KSP Community Manager