[DEAD] Kerbal Modification Catalog

[TheAlmightyOS]

OP has become a mess. Going to move everything but a shortened About, Proof of concept and links. Everything else is going off site, if I keep it at all.

[TheAlmightyOS]

Sorry for the lack of updates. Was looking into CKAN and what it would provide the user in a completed state. I am now satisfied that the functionality of KMC and CKAN do not overlap to an extent that warrants the discontinuing of KMC.

Still waiting for user / modder input on the project. I can make this as big or as small as you guys want. The framework is done. New features would be easy to add at this point

[TheAlmightyOS]

• Justin Kerbice is working to bring a very important part of KMC to realization and that is update notifications. This is not a notification for you, this is a notification for the database itself, a flag that indicates "hay! this mod has been updated! you might need to update the db!". When we get that functionality it will be much easier to keep track of updates.
  I am noticing more authors than I realized keep all their mods in one thread. Will need to seriously think about this, maybe even change the db.
  

[TheAlmightyOS]

Ok, so for any mod authors out there that want to update their mod on KMC themselves (or any volunteers that want to help out the cause) Send me a text file named with EXACT NAME of the mod (find it in the db list in the first post) and say what changed. Anything that is not mentioned in the file is not changed That simple. Here is the syntax:

Database_Key='your new value'
Database_Key2='another new value'

And if you want to remove something, lets say a curse url:

Database_Key3=NULL

That is it. Nothing else to it. Below is a list of database keys and what they mean. Mod authors are open to change any value. Volunteers may only change the KSP Version, AVC participation and add/change locations.

ALL changes are vetted. If someone submits a file saying saying Farram made DangIt!, I am going to be sending an PM to Ippo and Farram, as well as an PM to the mods to report possible malicious behavior. Also, if one of the mods is called ;drop tables; or the such (not that it would work) I will send a request to have your account terminated. I don't play. This is my personal server and I never intended for it to be used this way. Authors and volunteers, please contact me via PM to submit a file. There will be a more automated submission process later, however I want PM's so I know there is no impersonation going on.

Database Keys:

KSP_V_Major

KSP_V_Minor

KSP_V_Rev

KSP_V_Build

These keys indicate the most recent version of KSP the mod is compatible with. It is separated out into Major, Minor, Revision and Build. At this time Build is not used (thankfully).

Mod_Type

Usually "Plugin", "Part" or "Craft", however can be whatever describes what the mod "is"

Mod_Author

The PRIMARY author of the mod. Can be a team name. Sorry, no lists. Looking into multi-authored parts in future enhancements.

Mod_Comments

A summery of what the mod does. No formatting. Try to keep it short, maybe a paragraph.

License

What is the License for the mod. Be specific. Don't just say GPL. Put the version. Example "GPLv3". Also, if using a custom License, just put "Custom". I will be adding a sub-table of licenses in one of the enhancements.

Forum_Thread

Curse_Site

KStuff_Site

GitHub_Site

Other_Site

These are all url's, the first is the forum thread, the others are other url's the mod can be found at. This is the only time I will ever change the information as shown on the thread. The mod author will often direct link to a download. I change the link to go to the mod's page on these sites without the direct download. Private sites like private servers or gdrive and media fire are simply not added.

Other_Name

Name of the other site listed in the section above

KSP_AVC

Does the mod participate in KSP_AVC? YES/NO

If there is any questions about any part of the process I am more than happy to answer them.

[Dunbaratu]

There seems to be a problem with duplicate naming of mods confusing the data mining.

kOS, for example, was once maintained by Kevin Laity, who became utterly unreachable and out of contact (nobody has any idea what happened to him - e-mail, google+, twitter, none of these methods to reach him worked). Eventually the fact that it was licensed with GPL3 was taken advantage of, and it was taken over by erendrake after enough months of zero activity and inability to contact the author to determine his wishes.

It's been under very active development, with several new releases... however... it was done by starting a brand new thread.

So there are now two threads both saying they are for kOS.

The data shown here seems to be only using the older defunct thread and not the newer one. So it's advertising to the world the false fact that the kOS mod is dead and out of date when it isn't.

It would have been better to have just picked a different name for the resurrected mod, but that wasn't how it was done.

Can something be done to the data mining algorithm to handle this case?

[TheAlmightyOS]

No data mining is going on. Just a guy with a database . When the system works as designed, people come together and find and fix little mistakes like this one. Right now, the DB is just a rough copy of the Community mods plugins library. Link in sig. We should tell them too.

As for using mod names as the primary keys I struggled with this during the design phase. I do not know what to use as a proper identifier. Any suggestions?

As for kOS, I will update it with the new information when I get a chance

There seems to be a problem with duplicate naming of mods confusing the data mining.

kOS, for example, was once maintained by Kevin Laity, who became utterly unreachable and out of contact (nobody has any idea what happened to him - e-mail, google+, twitter, none of these methods to reach him worked). Eventually the fact that it was licensed with GPL3 was taken advantage of, and it was taken over by erendrake after enough months of zero activity and inability to contact the author to determine his wishes.

It's been under very active development, with several new releases... however... it was done by starting a brand new thread.

So there are now two threads both saying they are for kOS.

The data shown here seems to be only using the older defunct thread and not the newer one. So it's advertising to the world the false fact that the kOS mod is dead and out of date when it isn't.

It would have been better to have just picked a different name for the resurrected mod, but that wasn't how it was done.

Can something be done to the data mining algorithm to handle this case?

[Dunbaratu]

As for using mod names as the primary keys I struggled with this during the design phase. I do not know what to use as a proper identifier. Any suggestions?

As for kOS, I will update it with the new information when I get a chance

Okay, thanks!

The key problem is probably that it should have had a name change when it was resurrected under new management (i.e. like when KMP became DMP). We've also been plagued by the fact the Google searches keep finding the old dead thread and not the new one when someone looks for "KOS KSP" or "KOS Kerbal".

[Justin Kerbice]

Maybe you could normalize your DB design, ie spread unique data into their own tables, it will improve the overall size (no duplicate) but it's not really a big deal here.

It would lead to tables: authors, ksp_version, license, mod type, ... each one with its own unique index.

tables:

author (pk id, string name) + enventually team_id for group

=> team table (id, author_id), with non unique id

ex:

team_id auth_id

1 5

1 7

1 15

2 3

2 33

so authors 5,7 and 15 are on team 1, etc

license (pk id, license name, full license text/link)

modtype (pk id, string type) (1 plugin, 2 part, 3 misc, 4 mixed=parts+plugin) misc for texture replacement/MM config, ...

The overall table may use its own index, not mod name, internally.

The tool I'm working on could also update one column of your DB itself ("mod update notice") so you can just do a simple select to be aware of potential update.

(I also think sending email and/or syslog entry).

[TheAlmightyOS]

I know where you are going with this. I did think about this during the design phase. I came to the conclusion that I could always build up with a database so to start small with the necessities.

Issue I am having is how to uniquely identify mods within the database. Right now I am using mod name and that, as we see in the case of kOS causes issues. I can't simply auto-generate an alpha-numeric because I want the public to be able to identify the mod by that ID and submit change requests. Just like Steven Mading did.

I have a plan much like what you are talking about here floating around in my laptop somewhere. When this concept db has been proven to be maintainable we can scale up.

Maybe you could normalize your DB design, ie spread unique data into their own tables, it will improve the overall size (no duplicate) but it's not really a big deal here.

It would lead to tables: authors, ksp_version, license, mod type, ... each one with its own unique index.

tables:

author (pk id, string name) + enventually team_id for group

=> team table (id, author_id), with non unique id

ex:

team_id auth_id

1 5

1 7

1 15

2 3

2 33

so authors 5,7 and 15 are on team 1, etc

license (pk id, license name, full license text/link)

modtype (pk id, string type) (1 plugin, 2 part, 3 misc, 4 mixed=parts+plugin) misc for texture replacement/MM config, ...

The overall table may use its own index, not mod name, internally.

The tool I'm working on could also update one column of your DB itself ("mod update notice") so you can just do a simple select to be aware of potential update.

(I also think sending email and/or syslog entry).

[TheAlmightyOS]

Do you have the information for the new kOS? If it is too much trouble I will search when I get home but right now I am at work and time is limited

There seems to be a problem with duplicate naming of mods confusing the data mining.

kOS, for example, was once maintained by Kevin Laity, who became utterly unreachable and out of contact (nobody has any idea what happened to him - e-mail, google+, twitter, none of these methods to reach him worked). Eventually the fact that it was licensed with GPL3 was taken advantage of, and it was taken over by erendrake after enough months of zero activity and inability to contact the author to determine his wishes.

It's been under very active development, with several new releases... however... it was done by starting a brand new thread.

So there are now two threads both saying they are for kOS.

The data shown here seems to be only using the older defunct thread and not the newer one. So it's advertising to the world the false fact that the kOS mod is dead and out of date when it isn't.

It would have been better to have just picked a different name for the resurrected mod, but that wasn't how it was done.

Can something be done to the data mining algorithm to handle this case?

[Dunbaratu]

Quick response: this is the thread for the newer kOS:

http://forum.kerbalspaceprogram.com/threads/68089-0-25-kOS-Scriptable-Autopilot-System-v0-14-2-2014-10-08

Longer Response: I may get back to you later if that's not enough. I am headed out the door and have to make this post quick.

[TheAlmightyOS]

Quick response: it's enough for now. Sending the data to the database

Longer response: we will fix it up proper later with all the correct information. Thank you for the info

[TheAlmightyOS]

kOS has bee updated based on info from the link provided.

New problem. I was a complete and put the links into the database complete with HTML tags. I thought it was brilliant at the time. Just realized they store in the database something like this:

<a href>fake.link</a> I should have known better. You never add formatting marks to a database.

Working on a way to fix that now. Going to need to change both the code on the main sites and my extractor as well as bulk edit those database entries.

[Justin Kerbice]

kOS has bee updated based on info from the link provided.

New problem. I was a complete and put the links into the database complete with HTML tags. I thought it was brilliant at the time. Just realized they store in the database something like this:

<a href>fake.link</a> I should have known better. You never add formatting marks to a database.

Working on a way to fix that now. Going to need to change both the code on the main sites and my extractor as well as bulk edit those database entries.

you can use HTML::Strip perl module, very simple:

my $stripper = HTML::Strip->new ();
my $line_no_html = $stripper->parse ($line);

($line can also contains more than one single line).

[TheAlmightyOS]

ah, not that simple my friend.

First, I need to edit all the HTML links inside the database. I suppose I could do some sort of extract and remove the junk with your stripper and then do a bulk update but I would have to be careful.

Second, I need to update the websites to see cells with web addresses as links. As they are almost purely coded in php I need to investigate how to conditionally format cells that have "*HTTP*" content. Since I was already going to need to look into conditional formatting this just accelerates the need.

Last, the DB_Extractor I created that makes the BBCode table in the first post needs to be edited. Of all of them, this might be the easiest. I think I might be able to use sed

[TheAlmightyOS]

some <longstringcurse> tried to hack me over at http://www.phphelp.com/. Thought he was slick "you're using outdated code, run this". I can read SQL. It drops a table called users (which, I have one with over 300 entries, so, um, NO) and then makes a users table. Possibly as back-doors. Supposedly it is all to install PDO which I only know two ways to install that: apt-get install php-pdo OR yum install php-pdo. So complete hacks over there. STAY AWAY!

According to w3c, MySQLi is still an up to date and valid way to call for database information but everyone is so stuck on this PDO crap. And I do mean crap. It is a horribly inefficient. Why write for all database types at once when you are only ever going to use SQL? it makes no sense!

I am going to see what I can do with both, hopefully I can make something that looks good.

[TheAlmightyOS]

removed the html code bits from the database. Clean data, but now you can't click on any links. Apologies. This is what I am working on currently

[linuxgurugamer]

Are you maintaining this again, or do you still need/want someone to take it over?

I may have resources available, and have the skills (db & programming) to work with it. I gather you are using Ubuntu, I would have to check with my hosting company as to whether they use Ubuntu or CentOS, either way, it would be doable.

Let me know, I PM'd you yesterday about it

Edited November 28, 2014 by linuxgurugamer
Added more comments

[Stone Blue]

I just read thru all 5 pages of this thread....Thats too bad you got stopped cold by hardware, of all things...

I hope someone gets SOMETHING up and running soon....I've been struggling with manually editing a spreadsheet of over 300 mods....I'm sure I've spent almost as much time working on that as I have actually PLAYING Ksp...

I hope you can pick this up again in the future. I know there doesnt seem to be much interest or feedback, but "...if you build it; .... They will come....."

I know when I first made a copy of my spreadsheet available publicly, there were only 3 or 4 replies to the thread over a week....But i had it available on GoogleDocs, and I had over 1300 hits in just the first 2 or 3 DAYS.... ???

Then when nismo started "THE" thread, at first, IIRC, response was slow and mixed....Now its a hit...

I'm sure if you happen to get this up and successfully working, it will become quite popular quite quickly... :thumbsup:

[TheAlmightyOS]

Are you maintaining this again, or do you still need/want someone to take it over?

I may have resources available, and have the skills (db & programming) to work with it. I gather you are using Ubuntu, I would have to check with my hosting company as to whether they use Ubuntu or CentOS, either way, it would be doable.

Let me know, I PM'd you yesterday about it

I am (was) using CentOS7. I would like to maintain it but I put the WTFPL license on it so anyone can pick it up if they want or fork it. Currently I just don't have the resources. Like I put in the first post, anyone want anything I got so far I will provide if available. Would take time to pull the database (data would be stale anyway) but I I have my php files.

If I got my hands on some stable hardware I could make this work. I think I saw a pm about hosting. Need to read those still. Thank you for the support

[TheAlmightyOS]

I just read thru all 5 pages of this thread....Thats too bad you got stopped cold by hardware, of all things...

I hope someone gets SOMETHING up and running soon....I've been struggling with manually editing a spreadsheet of over 300 mods....I'm sure I've spent almost as much time working on that as I have actually PLAYING Ksp...

I hope you can pick this up again in the future. I know there doesnt seem to be much interest or feedback, but "...if you build it; .... They will come....."

I know when I first made a copy of my spreadsheet available publicly, there were only 3 or 4 replies to the thread over a week....But i had it available on GoogleDocs, and I had over 1300 hits in just the first 2 or 3 DAYS.... ???

Then when nismo started "THE" thread, at first, IIRC, response was slow and mixed....Now its a hit...

I'm sure if you happen to get this up and successfully working, it will become quite popular quite quickly... :thumbsup:

Thank you for the support. It will happen. I just need something more stable than a 6+ year old custom rig. Tested everything I could possibly test. Has to be CPU or mobo. Been wanting a blade server setup. Maybe I will save my money up for something like that. Stuff like that is on craigslist all the time here. Till then, I will work on how to identify changes in FP of threads and cleaning up the php pages.

[VITAS]

electricity must be very cheap where you life