Ksp api

[DrDam]

Hi everyone.

Last week, I have rework some of my little web-caculators for KSP and I have a interrogation.

There is some of ksp-helpers on the web, and many of them use part configuration ( weight, drag, engin ISP/Thrust ... ), I thinks there all are using custom database of ksp parts data.

So I want to know if I work on a "web api" which parse all config parts files ( stock or mods) and expose a web services of this data, that can be usefull for someone ?

If it's usefull, Please tell me what do you need, or idea for devs.

Thanks

DrDam

[ratchet freak]

isn't that all already in the gameData folder, you can extract the data from there yourself

[DrDam]

isn't that all already in the gameData folder, you can extract the data from there yourself

The global idea will be to use this data for web-based-application.

I'll imagine making a parser will extract data from conf files and save this datas to a database web-disponible.

[katateochi]

I'm interested!

So if I understand it, you want to provide a web service that provides information about stock and mod parts? If that is the case I could find that very useful.

I'm developing http://KerbalX.com, a craft sharing site that can identify the parts that craft use and thereby know what mods they require. KerbalX has a knowledge base of parts and which mods they belong to and users can contribute to the knowledge base by running a "PartMapper" tool. That basically scans their GameData folder, finds all .cfg files and produces a list of what parts exists and what mods they belong to. That data is then uploaded to KerbalX and is used to influence the associations in the knowledge base (the associations are weighted, so it adapts by associations being voted up or down).

For a number of reasons the PartMapper only transmits the names of parts and mods, so all that KerbalX knows is what parts are called and what mod they belong to. This is sufficient info to detect the parts in craft files and determine what mods a craft uses, but it would be nice to have additional information about the parts. For example I don't have enough information to calculate dV from a craft file, but if I could use your service to calculate craft stats that would be amazing!!

questions;

How are you planning to mine the data about mod parts? Will you have to manually install and scan them?

How are you planning on dealing with different versions of mods?

[DrDam]

How are you planning to mine the data about mod parts? Will you have to manually install and scan them?

Actually, I plan to parse part directories and scan part files. So I imagine processing mod parts in different directories at the same. The goal is to use "complete" mod game_data folder because I'll would check updates.

How are you planning on dealing with different versions of mods?

I think deal with the last version of mod, but I don't see any problems too versionning mods.

thanks for your message, and "stay tuned"

PS: I'm not english-native ... but you'll have already seen it.

[katateochi]

Actually, I plan to parse part directories and scan part files. So I imagine processing mod parts in different directories at the same. The goal is to use "complete" mod game_data folder because I'll would check updates.

I do the same, the PartMapper tool for KerbalX parses the directories and scans all the .cfg files (for stock and mod parts), then sends the info to KerbalX.

If you're interested in my code for the partmapper it's here on github. It is written in Ruby, which is probably not the language your using, but ruby is easy to read and it might help. I've been developing the PartMapper for almost a year, first as part of another project and now as part of KerbalX, so it's had some good field testing and I've overcome some of the more querky edge cases (like parts that use module manager), so if you want any help on this do give me a yell.

My question was more to do with how you plan to discover new mods. So if I understand your plan, you'll have a tool that can parse the directories of a "complete" mod GameData folder, but how do you define what "complete" is. New mods come out all the time and while there are the "main stream" ones, we all have different definitions of what those are. So I guess my question is more; how do you define what a "complete mod game_data folder" is?

I think deal with the last version of mod, but I don't see any problems too versionning mods.

I think that using the last version of the mod is the only way to go (for now).

The problem I see with tracking mod versions is that there is no standard format that mods use to display their version info. So it's very hard (or impossible) to determine the version in code when scanning a GameData folder (it's slightly more possible if you still have the zips for each mod, but still there's no standard so its not reliable).

The other problem is there is no way to determine the version when reading a craft file. So for example you have a craft with a part called engine_1. engine_1 comes from a mod that has several versions and in each version engine_1 has a different ISP. From the craft file there is no way to tell which version of engine_1 was used when building it, so the only thing to do is use the most current version and assume that's right.

thanks for your message, and "stay tuned"

PS: I'm not english-native ... but you'll have already seen it.

I certainly shall stay tuned! (and your english is perfectly understandable).

[DrDam]

If you're interested in my code for the partmapper it's here on github. It is written in Ruby, which is probably not the language your using, but ruby is easy to read and it might help. I've been developing the PartMapper for almost a year, first as part of another project and now as part of KerbalX, so it's had some good field testing and I've overcome some of the more querky edge cases (like parts that use module manager), so if you want any help on this do give me a yell.

Thanks, I'll check it

My question was more to do with how you plan to discover new mods. So if I understand your plan, you'll have a tool that can parse the directories of a "complete" mod GameData folder, but how do you define what "complete" is. New mods come out all the time and while there are the "main stream" ones, we all have different definitions of what those are. So I guess my question is more; how do you define what a "complete mod game_data folder" is?

I already don't know how. I imagine something like a md5 sum of "datas" folder, and if it differ than a reference, I check all folder contents in it ( or maybe a md5 sub-tree for each subfolder. If a sum differ or not exist, the folder must be parsed).

I think that using the last version of the mod is the only way to go (for now).

The problem I see with tracking mod versions is that there is no standard format that mods use to display their version info. So it's very hard (or impossible) to determine the version in code when scanning a GameData folder (it's slightly more possible if you still have the zips for each mod, but still there's no standard so its not reliable).

The other problem is there is no way to determine the version when reading a craft file. So for example you have a craft with a part called engine_1. engine_1 comes from a mod that has several versions and in each version engine_1 has a different ISP. From the craft file there is no way to tell which version of engine_1 was used when building it, so the only thing to do is use the most current version and assume that's right.

That are the reason while I'll work on the stocke parts, and when it stable, on mods parts, and only after think about versionning

[DrDam]

Hi everybody,

I update this topic with the first test version of the API.

Please note that is a 0.0.0.0 version.

Actualy, the parsing & conservation of parsed datas are stable, and the API propose many lists out and a url for taking datas of parts.

The output format are JSON, and a XML format are on developement.

If you want test it, please first read README.md file in github : https://github.com/DrDam/KSP-API

And try it on kspapi.chezdamien.net

Thanks and I waiting for yours question and critics.

DrDam

[katateochi]

Hey DrDam, just checking in to see how this project is going. I see you've got it working for stock parts and deadly re-entry. Basic API seems fine, but if you don't mind I'd suggest you simplify the response structure.

So for example if I go to http://kspapi.chezdamien.net/get/parts?partname=Mark2Cockpit what I get back has a structure that has the data inside two hashes that only have 1 key each (so they're not really doing anything). It would be nicer if when I go to a specific part I just get straight to the data

In other words; rather than needing to do this;

  string = get "http://kspapi.chezdamien.net/get/parts?partname=Mark2Cockpit"
  JSON.parse(string)["Parts"]["Mark2Cockpit"][<the_thing_I_want>]

I do this;

  string = get "http://kspapi.chezdamien.net/get/parts?partname=Mark2Cockpit"
  JSON.parse(string)[<the_thing_I_want>]

Also it would be nice to not have an array of hashes in the response for get/list/parts.

So if I parse the response from get/list/parts, then step into "Parts" in the outer hash and then into "Squad" I then get an array and each element is a hash with 1 key (part_name) which entails a hash with 1 key (title) which returns the full name. What I'd find more useful is if it was either just an array of part names inside each mod, or a hash of partname => full_partname. It just means I don't have to process the response to extract a list of parts names from /get/list/parts, its either just ["Parts"][<mod_name>] or ["Parts"][<mod_name>].keys to get an array-of-partnames.

So I could really put this service to use with what I'm doing on KerbalX, but I really need more mod info than you have right now. Have you heard of CKAN? I only recently heard about it, but it provides an API to a repository of meta-data about mods, name, author, download_url etc. Unfortunately their API (http://api.ksp-ckan.org/) seems to be down right now, but you can still see the data on github https://github.com/KSP-CKAN/CKAN-meta (that's a-lotta-mods you gotta process!)

If you want to see a list of the mods that I need info on currently you can see that here - http://kerbalx.com/mods

Or you can read that via a JSON API (that I put in literally last night) GET http://kerbalx.com/mods.json and you'll get a hash of mod_name => database_id You can then go to http://kerbalx.com/mods/<database_id>.json for a hash of info for the given id. Some of the mods listed on KerbalX now have download urls, but that is only a few and that's sourced by users adding them in so its not 100% reliable.

But CKAN is all about download urls! So I'm planning to have KerbalX scan CKAN from time to time and get updated download urls. I'm also planning on writing a script that I'll run on my local machine rather than on the site's server, that will go through CKAN and follow each download url, download and unpack the mod, read the mod info for KerbalX and send it to the site. That way KerbalX will know about everything that CKAN knows about. But I wouldn't want to run that very often. Could you also do something similar, where you scan CKAN and download the mods so you can scan them with your tool? And then also maybe have some way of tracking when a mod on CKAN updates so keeping up-to-date won't be so much overhead?

Anyway, if you do get this tracking lots of mods then it would solve my problem of needing to know specific part details in order to calc detlaV, TWR etc for craft on KerbalX, so it would be extremely useful to me!

[hakan]

One tiny nitpick: Make sure to send your output with an HTTP type of "application/json" instead of "text/html". Make life much easier for parser clients

[DrDam]

@hakan : oups ... I'll correct that ASAP

@katateochi : Thanks for your return. I'll note all and try to make a 0.0.0.1 version.

For CKAN, I check.

My primary objectiv are to "make more visible" some data for parts :

- Mass

- ISP curve

- Thrust curve

- Volume of "ressource"

Stay tuned and Thanks

[hakan]

I only recently heard about it, but it provides an API to a repository of meta-data about mods, name, author, download_url etc. Unfortunately their API (http://api.ksp-ckan.org/) seems to be down right now, but you can still see the data on github https://github.com/KSP-CKAN/CKAN-meta (that's a-lotta-mods you gotta process!)

I will look into what happened to the API generator... Seems like another one of our daemons breaks things left and right...