Kerbin Geographic And Science Society

[togfox]

csv files can be zipped down into a very small file and hosted quite easily.

Mr Amazing - send me a forum private message outlining your favourite KSP Sciencey topic you think you could write well about.

bitbucker: +1 rep for you.

[phoenix_ca]

csv files can be zipped down into a very small file and hosted quite easily.

...wow. I'm an idiot. I'm so used to dealing with data formats that don't compress well, that I guess it just doesn't occur to me. D'oh! >.<

[Tommygun]

A 1600x800 Duna map made from a 337MB Duna Elevation file.

[phoenix_ca]

Man...300MB and only a mere 1600x800. Yeepers. O.o

Still, that's enough data to upsample it in photoshop easily with very little loss in fidelity. (I actually have a PS extension that can easily expand that to something insane like 10,000 pixels across...you know...if you wanted to make a whole floor out of it or something.)

[Tommygun]

Well I also have a 6400x3200 of Duna (24.5MB after rendering), but the forum rescales it and you really can't see much difference.

[phoenix_ca]

And...that'd be enough to print the side of a building at 300 dpi, if one wanted to...O.o

[ramipro]

Map of Eve 5000x2500 out of a single file of 460MB:

(Right click -> Open image in a new tab to see it full size)

Just a quick question, is anyone mapping the planets for the KGSS? I know the they did this for Kerbin, Mun and Minmus, but it seems like they're not really interested in the new planets

Edited September 30, 2012 by ramipro

[TheCardinal]

Map of Eve 5000x2500 out of a single file of 460MB:

...

I guess you must have made a typing error or something like that. No image available (error 403 when trying to access it directly)

[ramipro]

OH, right, i forgot to put it in the public folder in dropbox, my bad. Editing now.

[Rich]

I mapped ALL the things.

Awesome work

I gladly volunteer myself as a writer, if and when you need one!

If you have any articles you wish to write, just contact togfox

[Wrench Head]

Just a bit of info, I was amazed at the speed I sit.

[phoenix_ca]

Oh good gods. (Edit: Signs of the zodiac. Sorry, brain was way ahead of my typing.) My interest dropped rapidly at that point.

The question is, as always, relative to what? The Sun? Your chair? The centre of the galaxy? The centre of the universe? Jupiter? Alpha Centauri?

Since the video doesn't even establish the goal, that is, relative to what, what's the point?

They also use miles/hour, not the SI standard. Sooo...yeah. I have no idea what the point they were driving at was. O.o

Edited October 2, 2012 by phoenix_ca

[phoenix_ca]

Has anyone tried using some more advanced software for mapping? Like GRASS GIS? I'm trying to finagle it to accept the data output from ISA MapSat using r.surf.xyz (a module that generates a raster map based on sparse XYZ data points, which is what ISA MapSat generates), but it's kinda wacky. If anyone here has more experience than I using GRASS GIS, some pointers would be appreciated (heck, maybe even a guide...if no one knows how and I figure it out, maybe I'll write a guide, so Mac and Linux users have options).

[togfox]

I investigated a few of those GIS months ago when the KGSS was first established. I found that most of them assumed your current planet of interest was Earth. Bizarre idea! Anyway, I gave up on them because most of them assumed you're on Earth and automatically loads terrain profiles from a database somewhere. I never had the patience to work out how to override that.

[phoenix_ca]

I think I've found a successful way to get data into GRASS GIS. It's a little kludgy, but it works. You set it up as a region with an arbitrary XY coordinate system (so all it is is an XY region that can be modified). Then use g.region to set-up the region size, which thankfully, is very predictable here. The values will always be thus:

res=0.01
n=90
s=-90
e=180
w=-180
x=2
y=1

Easy peasy. There isn't even really a need to generate a raster map to analyze for region information, since we know beforehand what it will always be. The only issue I've run into now is dealing with these large volumes of data (I had NO idea just how much data scientists can end-up dealing with...it's just a little crazy O.O ).

At the settings above, the region is 648,000,000 data points. Presumably that could be the max number of data points for any planet/moon in the game. That...is a lot of data. My Kerbin_elevation.csv takes up 250MB (with r.in.xyz eating around 1.5GB of RAM to import it, for each pass, with 10 passes, meaning if I had loaded the whole thing it'd need 15GB...crazy). So presumably a file that had all possible data points would be, erm...lemme go do maths *pause*

Data points in import (i) = 13,303,561
Full set number of points (f) = 648,000,000

i / f = 0.02053018673

File size of imported data set (Fs) = 250.9MB = 2.1047e9 bits

So file size of a full data set from ISA MapSat:

2.1047e9 bits / 0.02053018673 = 102,517,333,509 bits = 11.93458837328944 GB

Yikes. O.O

In any case, the thing I can't seem to do is interpolate from that set, the rest of it. It's not that I can't make it work. It's that it will take a loooooong time. r.surf.idw2 (raster interpolation for scattered data points) works fine, but will take way too long. Simpler to just collect more data. (I started it off on its merry way but the progress bar didn't move, yet it used a single core at 100%, and it's a 64-bit module...yeah...too much data.)

And then there's the slight kludge solution of simply rendering the region at lower resolution. >.> Yeah. Gonna be kludgy here.

Edited October 2, 2012 by phoenix_ca

[togfox]

Nice. You'll probably find that your data can be de-duplicated quite significantly. Unfortunately there is no quick and easy way to do that and it requires specialist database tools/skills.

[phoenix_ca]

Hmmm, I didn't take into account the duplication that ISA MapSat does. Though it might be possible (if somewhat calculation intensive) to have a Python script load the CSV into a list with lists, search for duplicates, and delete them. The huge problem is that it has to read through the entire file once for every line. It's a real brute-force way of doing it but can be done...I just don't see any point, other than reducing the filesize of a CSV for upload. Maybe I'll write a cleaning script that can do this at some point, when it suits. Python can be pretty speedy, and compiled (well "compiled" in the case of Python) can be speedier, but...yeah. I wouldn't know how to limit the amount of data loaded into memory without digging through some of the GRASS GIS modules to see how they are importing the files in chunks.

Since once it's imported as a raster map to GRASS GIS any duplication is moot, it wouldn't help with things like interpolation of the raster map.

Edited October 2, 2012 by phoenix_ca

[Elembis]

Very nice! Mind uploading this to the wiki?

[phoenix_ca]

Very nice! Mind uploading this to the wiki?

Are you talking to me? Because if you are...not yet. I'm not settled on a good way to go about this, and I'm planning on writing a Python script that can clean-up the ISA MapSat files (I hope, if I don't completely destroy them first while figuring thing out, which is quite likely).

Though once I do get that sorted I think it'd be cool to start compiling multiple raw data files to produce the highest-res maps possible. (Though I only have 8GB of RAM, so I don't think I'll be able to process full-size files without some extra pythonic voodoo.)

Edit: Fighting to optimize a python script doing this so brute-forcely like on 100million+ rows in a csv file ain't pretty.

Edited October 3, 2012 by phoenix_ca

[phoenix_ca]

Lmao. So apparently over the past few days I had a total brain-fart and forgot that I'm on a Unix-like system (meaning, not Windows, meaning life isn't terrible), OS X.

After a bit more research, deleting duplicate entries from a csv file is as easy as this:

(source)

awk '!x[$0]++' file > file.new

Bam. A little while later, new file is produced that's a little smaller. Unfortunately, I only gained a mere 0.4 MB (still, that's probably around, what, a few thousand rows?). Ah well, writing a Python script that can do it was a good exercise for me.

It's little things like this that make me remember why I don't, ever, do serious work on the Windows OS. >.< For anyone who's looking to use this little gem, you can either install cygwin, or download a copy of Ubuntu, and boot that.

[togfox]

400k of text is quite a bit but i was expecting much more savings than that. Nice job but.

[Gray Death]

Lmao. So apparently over the past few days I had a total brain-fart and forgot that I'm on a Unix-like system (meaning, not Windows, meaning life isn't terrible), OS X.

Mac OS, including OSX isn't "Unix-like" It IS UNIX. Just like MS Windows grew from DOS, MAC OS grew from Unix. The only viable alternative to these two giants of Software families is teh Linux Software family. Very similar to Unix in the Command Line, but the Guis range from Unix-Like, to Windows-Like, to completely unique.

Now. On topic, has anyone considered building bases on Kerbin and beyond? By landing somewhere, one can claim something. By staying on something, one can KEEP it. I don't honestly know if v 0.17 will support bases because I'm temporarily stuck with only 0.13.3. (Come on, cash flow.) But, if it does, it'd be awesome to build some cities so that the Kerbals would have more of a presence on their world than just the KSC.

[Chebalu]

Hello Kerbonauts!

I think I am the first (with evidence:)) who have landed on this island, so I would like to name it Vivien Island!

Here are the evidences:

Vivien Island

Edited October 3, 2012 by Chebalu

[phoenix_ca]

Gray, I said "unix-like" precisely because it evolved from Unix. It ain't Unix anymore, though it is a -nix/FreeBSD hybrid of awesome. Still, that family of OS' is by far the best to use for most anything that isn't gaming. O.o

And hey, we still use Forth sometimes, right? >.> (Stack-based programming is so wacky and different compared to OO it makes my brain hurt sometimes.)

As for bases, not formally no. You can land an active flight there and call it a base, but it's technically just another ship that happens to be landed. There's not hint special about it saying "this a new ground base and not a "flight").

[Gray Death]

As for bases, not formally no. You can land an active flight there and call it a base, but it's technically just another ship that happens to be landed. There's not hint special about it saying "this a new ground base and not a "flight").

Well... Bugger! Any Dev input about the possibility in the future?