Jump to content

Kerbin Geographic And Science Society

togfox

By togfox
in KSP Fan Works

 Share

Recommended Posts

togfox

togfox

Posted
  • Author
Posted

Thanks for the kind words. :). We have one or two part time 'artists' at the moment but if i include you in the rotation then that spreads the load. Thx. :)

If anyone feels like writing a technical article for the journal then pm me your idea. :)

Link to comment
Share on other sites
phoenix_ca

phoenix_ca

Posted
Posted (edited)

I'm looking to compile as much data as possible. Using the method outlined earlier to handle duplicate entries, and joining all of our data together, this should allow us to create very high resolution maps. When I get a decent CSV file-size, I'll redistribute them.

What I need from you:

Raw CSV data files generated by ISA MapSat v3.3.0+ in KSP 0.17+ (older versions have different elevation values and will likely seriously damage the master file when they are merged; do NOT send old data files). Upload them and PM me with a link to the file (compressed as best you can, in whatever format you want).

What I'll be doing:

Collecting all of these files and then smushing them all together, deleting duplicates, and redistributing them as CSV files (comma-separated, not semi-colon-separated) compressed somehow (I'll do a bit of testing to see what the most efficient compression format is for the data and use that).

I'll also use the data to create high-res raster maps with contour-lines and other stuff using GRASS GIS. Given that processing a single csv of 300MB takes about ten minutes, and also given resource limitations on my end (that is to say, I'm running a powerful consumer desktop, not a super-computer or say, Mac Pro with all the trimmings), it'll probably take at least a few days to complete, say, a map of Kerbin.

Once we have raster maps in-hand, it should be relatively trivial to upload them somewhere for people to use. (The generated images will be massive, assuming we collect all possible data-points.)

For now I'm going to say the initial target should be Kerbin (since we don't yet have a high-res data set for it). Just leave KSP running at 50x with your mapping satellite active, collecting raw data, and send it to me when it's reasonable (say, 300MB - 1GB? Ish?). If you *really* want to help me, then move the original out of the folder and send it to me, so that ISA MapSat creates a new file as it collects new data, then send me that second file later after it's grown. This will prevent me from having to reprocess the same data multiple times, and will save considerable effort on my end.

That said, I'm certainly willing to accept whatever data anyone wants to throw my way that conforms to the aforementioned standard (ISA MapSat 3.3.0+, KSP 0.17+).

Edit: Just to put things in perspective, the max-res file would be about 15GB. At a data collection rate of ~767KB/s (which is what I got by watching the csv file size over three minutes), it'll take 41 days of constant data collection to finish it. That's not accounting for the time lost generating duplicate data and, removing it, and hoping for non-duplicate data to be recorded (the rate at which new data can be collected is roughly asymptotic, approaching zero new points per time as the file size reaches the maximum possible; this means that the estimate of 41 days is very, very conservative).

Edited by phoenix_ca
Link to comment
Share on other sites
ramipro

ramipro

Posted
Posted

Phoenix, I'll upload whatever you ask me to. I can intensively record any plate's height. As of now, I have 2 GB of raw data from five different objects, including a 566MB one of Duna. If you need any other body, tell me which one to start with. I'll be mapping Kerbin until I hear from you.

Also, I think this is a great way to do what the KGSS is supposed to do.

Link to comment
Share on other sites
togfox

togfox

Posted
  • Author
Posted
Thank you togfox. Are there any maps (the new planets and moons) on which there are names of geological formations?

There arent any maps other than kerbin/mun/minmus with names as few ppl have named features outsde the kerbin system.

Link to comment
Share on other sites
Tommygun

Tommygun

Posted
Posted

I was wondering if anyone knows how much each pixel represents in kilometers on the Satmaps.

Can you take the equatorial diameter of a body and divide it by the pixels to get the size of each pixel at the equator?

For sizes like 1600x800 and Kerbin, it would be 600,000km divided by 1600 pixels= 375 kilometers?

Also does it change as you approach the poles?

Link to comment
Share on other sites
phoenix_ca

phoenix_ca

Posted
Posted (edited)

Assuming an equal-area map projection, that'd be right. Unfortunately, ISA MapSat projects maps using a Mercator projection, so while your calculation would be accurate for the equator, it would be increasingly inaccurate as you move away from it.

However, it should work (roughly) for all areas of the polar map projections. Ish.

For those offering to help, go ahead and send me your Kerbin data for now. I'll mush it all together with my own (like soup!) and see what we get.

Edited by phoenix_ca
Link to comment
Share on other sites
Tommygun

Tommygun

Posted
Posted

Thanks, I was trying to find out how big each of those black areas of missing info are on the maps.

Even on one of my big ones at 6400x3200 that would be about 93km.

I'm doing a long scan right now and I'm at 328MB so far for Kerbin.

When I get to 400MB or so, I'll upload it for you.

Link to comment
Share on other sites
phoenix_ca

phoenix_ca

Posted
Posted

Cool beans. I've been scanning Kerbin and have about 700MB now. Things might behave oddly when I combine the data, because of the potential for duplicate data points. It's a bit of a crap shoot. If we're lucky, we'll have all collected a lot of unique data points. If we're unlucky, there will be lots of duplicates and only a bit of unique data. We'll see.

Link to comment
Share on other sites
phoenix_ca

phoenix_ca

Posted
Posted

Great news. Tommygun sent me his csv data earlier today, and through a combination of sed, sort, and uniq, I've merged it with my own data. The result (1.09GB from me and 724MB from him) is a whopping 1.79GB csv master file that I'll merge all future Kerbin elevation data into. That's a mere 17MB of duplicate data tossed in the process.

So keep the data coming guys. Anyone who wants to contribute should contact me or let it be known here. I'll send you a link to a folder on Google Drive so that you can upload your (compressed!) csv files. :D

Link to comment
Share on other sites
phoenix_ca

phoenix_ca

Posted
Posted
I would really like to get into this. But it is all so confusing. I don't even know how to get the maps working and they don't seem to have any named places on them :(

What problems are you having, specifically, with ISA MapSat? And are you running it on OS X?

Link to comment
Share on other sites
AmpsterMan

AmpsterMan

Posted
Posted
What problems are you having, specifically, with ISA MapSat? And are you running it on OS X?

Lol no. I meant i don't really know what is going on with the KGS. I would like to help you guys in naming landmarks but I don't know how to even get the maps working or what they mean. I download the Excel files that are linked in the beginning. Then I import them using the metric import option and then I get a completely imcomprehensible hodge podge of lines.... Lol. Sorry for my imcompetence.

Link to comment
Share on other sites
togfox

togfox

Posted
  • Author
Posted

The excel files are actually text files (try opening in notepad for fun) and is raw data. You need another program to turn that raw data into a map. I use quikgrid because its free but its not the most user friendly program.

Link to comment
Share on other sites
phoenix_ca

phoenix_ca

Posted
Posted

Tbh I suggest creating your own maps for now, it's a bit easier (and the maps in the OP have incorrect elevation values for the oceans). Just toss an ISA MapSat satellite up into polar orbit and watch it go. When we have a significant bit of data going, I'll write-up a guide of some sort on how to generate ISA MapSat rasters from the data. Though the goal is to provide high-resolution contour and elevation maps (perhaps even voxel maps of particularly relevant areas, though considering how much time it takes to create those, I'll probably leave that as a special project), so that more people can help with landmarks and so on. Or at least have a good reference to go on.

Link to comment
Share on other sites
phoenix_ca

phoenix_ca

Posted
Posted
PHoenix, what size do you want the raw file I send you to be? I am at 400MB, but if you want more, I can easily get to 1GB in a couple days

It doesn't really matter. If you wish to send me 400MB now that's fine. You can upload updated versions later as you get more data. There's no restrictions, I just suggest larger amounts because small files (like say 60MB) are a little less efficient on time. Whatever you send me, whenever you send it (with multiple updates even), I'll figure-out the most efficient method of merging it with the master data file.

So really, it's up to you. If you want to send a bit now, and a larger file later, that's cool. Or send a lot more later, that's fine too. :) There's so much data to be collected, any contribution of any size will likely be helpful right now. I'd say the bottom cutoff is about 15MB for Kerbin elevation maps, at this time. With sizes that small, most are liable to be duplicates. I'd still process and merge them, but it's not the best use of anyone's time. (If I wasn't clear, sending 400MB now and more data later would be awesome.

Aside: To reiterate for everyone's benefit, please compresses raw data before seeing it along. Raw csv data compresses really well, to even less that 1/5th the original size in some cases. It saves both of us bandwidth and time.

Link to comment
Share on other sites
AmpsterMan

AmpsterMan

Posted
Posted

Okay guys, would sending some data about Mun help? It , and Minmus, are currently the only places I can reliably get to. If you like, I can put a satellite up in orbit around it and start-a-mappin'. Also, do I need to have KSP running continuously? or can I turn it off and continue at a later point?

Link to comment
Share on other sites
This thread is quite old. Please consider starting a new thread rather than reviving this one.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing
×
×
  • Create New...