[AnyOS] KSP Mod Admin v2 - Mod install with a few clicks

[4o66]

OOps...Sorry...I been away from KSP awhile, and I didnt realise there even WAS a 2.0... lol

I'm running 1.4.0 PR15

2.0 is in closed beta, PM MacTee to get a copy. (Supports only CurseForge for now.)

1.4 is no longer being maintained actively by MacTee, I am keeping it alive on GitHub until 2.0 goes live.

I have release up to a version 1.4.0.17, but mod updating is currently broken in both versions. (Curse changed the way their site works, and I am re-writing how KSP-MA finds the info it needs.)

https://github.com/4o66/KSP-Mod-Admin/releases will get you my latest (unofficial) build of the 1.x series.

[HealeyV3]

2.0 is in closed beta, PM MacTee to get a copy. (Supports only CurseForge for now.)

1.4 is no longer being maintained actively by MacTee, I am keeping it alive on GitHub until 2.0 goes live.

I have release up to a version 1.4.0.17, but mod updating is currently broken in both versions. (Curse changed the way their site works, and I am re-writing how KSP-MA finds the info it needs.)

https://github.com/4o66/KSP-Mod-Admin/releases will get you my latest (unofficial) build of the 1.x series.

I'm dying for either that 2.0 Beta or a fix for Curse 1.4

[4o66]

Another day, another set of problems:

CurseForge is in DDOS protection mode right now, this means both 1.4 and 2.0 don't work.

So now I get to try and figure out how to code around the CloudFlare service. (I have some ideas, lets see if I can get it to work...)

[HealeyV3]

Another day, another set of problems:

CurseForge is in DDOS protection mode right now, this means both 1.4 and 2.0 don't work.

So now I get to try and figure out how to code around the CloudFlare service. (I have some ideas, lets see if I can get it to work...)

Could using multiple proxies get around this? I have access to a few hundred private proxies that I can give you access to if it'd help.

[4o66]

Could using multiple proxies get around this? I have access to a few hundred private proxies that I can give you access to if it'd help.

Nope, the way Cloudflare's DDOS protection works means proxies do nothing (but thanks for the offer!)

Basically, in a normal browser you get a simple page that requires javascript and cookies enabled. If javascript is running, you get redirected after a few seconds (the browser calculates the forward link from an obfuscated chunk of given javascript.) Having a good cookie from that page lets you bypass on future visits. Also, the page never finishes loading, so that programmed DDOS attacks spend time waiting for the page to finish, but it never does.

To program around it, you need to catch a "connection ended unexpectedly" error, then re-request the page via a stream reader, and parse the stream looking for a signature of the DDOS page. Once you have than, run the javascript and use the given cookie on all further transactions.

Unfortunately, I havn't quite figured out how to do all that quite yet (luckily there is at least one person who has, and I am trying to understand their code so I can port it.)

tl;dr It can be done, not sure how yet.

Good news is that it is back to normal, and I can confirm that 1.4.0.17 is in fact working for CurseForge. Get it here.

Since the Anti-DDOS period was relatively short, I will back burner that and try to get updates working in 2.0 (they work in 1.4 since there is no "update" option, just browse and click download again.)

[Gribbleshnibit8]

Really like this project, and this game needs a good mod manager. I'd like to help if possible, and have been looking through the code and playing with adding KerbalStuff support. I've gotten some parts of the mod data handled (not looking into downloads yet) but am having a few issues with parsing the page. I don't have my code edits to look at right now. I'll reply with a more detailed question later.

I do have some suggestions requests that I'm going to look into implementing myself.

1. Part viewer brought back in 2.0 and updated with a part editor.
   
2. MM config editor with some kind of part detection/autocomplete. Shouldn't be too hard to do, but certainly not an easy thing to pull off. Would be useful though.
   
3. MM viewer that shows what parts are being edited by MM configs, conflicts between them, etc.
   

I think I had some more, but I'll wait and see. I know I can do some of this myself, so I'll plug at it until things work.

[HealeyV3]

Nope, the way Cloudflare's DDOS protection works means proxies do nothing (but thanks for the offer!)

Basically, in a normal browser you get a simple page that requires javascript and cookies enabled. If javascript is running, you get redirected after a few seconds (the browser calculates the forward link from an obfuscated chunk of given javascript.) Having a good cookie from that page lets you bypass on future visits. Also, the page never finishes loading, so that programmed DDOS attacks spend time waiting for the page to finish, but it never does.

To program around it, you need to catch a "connection ended unexpectedly" error, then re-request the page via a stream reader, and parse the stream looking for a signature of the DDOS page. Once you have than, run the javascript and use the given cookie on all further transactions.

Unfortunately, I havn't quite figured out how to do all that quite yet (luckily there is at least one person who has, and I am trying to understand their code so I can port it.)

tl;dr It can be done, not sure how yet.

Good news is that it is back to normal, and I can confirm that 1.4.0.17 is in fact working for CurseForge. Get it here.

Since the Anti-DDOS period was relatively short, I will back burner that and try to get updates working in 2.0 (they work in 1.4 since there is no "update" option, just browse and click download again.)

Not to burst any bubbles, but seems like they must have continued Anti-DDOS, as using your Get it here link still results in : Mod download started ...Mod download failed!

[4o66]

Really like this project, and this game needs a good mod manager. I'd like to help if possible, and have been looking through the code and playing with adding KerbalStuff support. I've gotten some parts of the mod data handled (not looking into downloads yet) but am having a few issues with parsing the page. I don't have my code edits to look at right now. I'll reply with a more detailed question later.

I do have some suggestions requests that I'm going to look into implementing myself.

1. Part viewer brought back in 2.0 and updated with a part editor.
   
2. MM config editor with some kind of part detection/autocomplete. Shouldn't be too hard to do, but certainly not an easy thing to pull off. Would be useful though.
   
3. MM viewer that shows what parts are being edited by MM configs, conflicts between them, etc.
   

I think I had some more, but I'll wait and see. I know I can do some of this myself, so I'll plug at it until things work.

Which version are you working on, 1.4 or 2.0? (Both are on GitHub now, so you can fork and do pull requests.)

For parsing the page, take a look at the changes I amde to use HtmlAgilityPack, makes it a lot easier than doing string operations.

For KerbalStuff, make sure you query mod info via API, since that will be a lot easier to do than parsing the html page.

Not to burst any bubbles, but seems like they must have continued Anti-DDOS, as using your Get it here link still results in : Mod download started ...Mod download failed!

I assume 1.4.0.17 then? From my end it looks like DDOS is off, but I will see what I can do.

If you open a new incognito/privacy mode tab in you browser and go to a CurseForge link do you get a DDOS protection page?

Also, what url are you giving it? (I have been testing with TweakScale and DMagic's Orbital Science.)

Edited December 4, 2014 by 4o66

[Gribbleshnibit8]

I'm working from 2.0. Had forgotten about the API last night but after your reminder here got it and I think I've got kerbalstuff downloads working.

I did make a fork, and a new branch to work on it all in, as there were plenty of other changes I wanted to attempt. Running into an issue now though that I don't know how to re-upload a fork/branch etc. Using github on windows and the program gives me a "Permission denied" error I'm assuming because I'm not allowed to push a new branch to that repo. So... help please?

[4o66]

I'm working from 2.0. Had forgotten about the API last night but after your reminder here got it and I think I've got kerbalstuff downloads working.

I did make a fork, and a new branch to work on it all in, as there were plenty of other changes I wanted to attempt. Running into an issue now though that I don't know how to re-upload a fork/branch etc. Using github on windows and the program gives me a "Permission denied" error I'm assuming because I'm not allowed to push a new branch to that repo. So... help please?

I would recommend SourceTree instead of the native GitHub client.

Are you sure you cloned your fork, and not the upstream master?

The GitHub client is a little confusing, espcially for setting up the credentials. (Get SourceTree.)

If you are not familiar with git and github, go watch

right now. Then download SourceTree, since it lets you see graphical diffs and what is happening way better.

My workflow for starting work on an existing Git project that I don't have push access to:

Log in to github, and fork the project.

Using SourceTree, clone the project via URL.

If you feel the need, use a branch per issue. Merge the branch as each issue is resolved.

(Using branches makes it easy to discard you edits if you want to try a totally different method.)

Commit Often! Use notes EVERY time you do, even if the code is incomplete. This is you log so when you come back after a week or two of absence you can remember what the hell it was you were doing.

Once you are stable, push to your fork.

From the fork's page on github, generate a pull request.

Since you have already downloaded a copy and done work, after you clone the fork, just replace the files with the working copy you have. SourceTree will pick up the changes (seriously, love this app.)

I would also encourage you to look at my fork, since it uses (IMHO) a much better parsing method for the websites.

[Gribbleshnibit8]

That is a MUCH better program, thank you! Ignoring the fact you linked an hour long video, I actually learn software better by poking at it until I figure things out. Which I did, and I'm getting the hang of git slowly but surely.

I'm pretty sure I now have my local copy set up to pull from the master and your fork as needed. I added your fork as a remote, and was able to download and merge it into my local branch and it all ran after I got the HtmlAgility pack installed, so I guess I did it correctly. Won't know for sure until you or MacTee push an update whether or not I've got it set up properly to tell me about updates and whatnot.<br><br>I just ran another test with your update to curse stuff and it seems to work. Got my mechjeb updated, so it says. Added info to all my mods from KS and ran an update check on one of them. I don't know that any are out of date though, so I'm not sure if it's seeing and getting updates correctly.

It's all on my github page, which I don't know why I'm linking since you can just check it out through the forks, but whatever.

Edited December 4, 2014 by Gribbleshnibit8

[4o66]

That is a MUCH better program, thank you! Ignoring the fact you linked an hour long video, I actually learn software better by poking at it until I figure things out. Which I did, and I'm getting the hang of git slowly but surely.

I'm pretty sure I now have my local copy set up to pull from the master and your fork as needed. I added your fork as a remote, and was able to download and merge it into my local branch and it all ran after I got the HtmlAgility pack installed, so I guess I did it correctly. Won't know for sure until you or MacTee push an update whether or not I've got it set up properly to tell me about updates and whatnot.<br><br>I just ran another test with your update to curse stuff and it seems to work. Got my mechjeb updated, so it says. Added info to all my mods from KS and ran an update check on one of them. I don't know that any are out of date though, so I'm not sure if it's seeing and getting updates correctly.

It's all on my github page, which I don't know why I'm linking since you can just check it out through the forks, but whatever.

Looks awesome, please do a pull request against MacTee's project. I would like to see if it lets me merge in from upstream pulls. (If not, I will just merge you in manually.)

Works for downloading mods so far. (Only tested against one so far...)

Some comments/requests:

The reason for the pull request is that you get the credit for the code on GitHub.

The source code file KerbalStufHandler.cs is misspelled... (missing an "f")

Think I can get you to add version as one of the fields it saves to the config?

(Look at Config/KSPConfig.cs and Constants.cs)

In KerbalStufHandler.CheckForUpdates(), use modInfo.version as the determining factor, since no api support for Mod date info (yet).

A great tool for looking at the config files (and just about every human-readable file) is Notepad++ if you don't already have it.

Also: forgot the video was an hour long, but it is quite good. Helped me out since I was moving from a Centralized versioning mindset (SourceSafe/Subversion) and didn't quite get the whole push/pull/branching that Git does.

Edited December 4, 2014 by 4o66

[Gribbleshnibit8]

Did a pull request. I'm pretty sure that'll work, I tried it with updating on two different mods, and got info from over a dozen with no issues.

I've been looking at adding GitHub support next, and actually have it working, so long as all you want to do is download the latest file, and you provide a link directly to the releases. That's a bit too limiting to me, and I want to expand it. More in spoiler.

I looked for a decent Git API and the best I found was octokit, which should do just fine, though it's more for if you're trying to build an interface into git than just download releases. It's also .NET 4.5, so I don't know if the project was supposed to stay 4.0 or what but there's that too. It does provide the ability to get all releases, but everything I've been able to do with it has it returning a <TResult> value that just doesn't do anything. The whole API seems to be in async, and I have no clue how to work with that, so it's beyond me. You can add it through NuGet and I'm gonna create a new branch with the work I've done on it.

I also added some preliminary support for a Game Version field in mods, which can be gotten from KerbalStuff and Curse at least, and if more Git parsing were done maybe for Git, though I don't think everyone puts what game version in every update notes, but still useful for manually adding versions. The problem I ran into is that I have to edit the usModSelection.cs file and a bunch of other stuff gets changed at the same time. It also requires updates to the .lang files to support the new field and overall becomes somewhat complicated. And a field would need to be added to the Mod Info panel to let it be edited. Everything on the backside is done though.

[MacTee]

Hi 4o66 & Gribbleshnibit8,

thanks for your work on KSP MA 2.0!

I have merged your changes to the main branch.

@Gribbleshnibit8:

I would prefer to stay on .Net 4.0 to support WinXP too and be on the save side with mono.

[Gribbleshnibit8]

I figured, and my difficulties with the module I was trying to use made me not like it that much. The main changes I made to github downloading can be easily merged back without the 4.5 stuff, so no loss on that.

I think I've got GitHub support just about done now too. Don't have any checks to ensure that a repository actually contains any releases though, but it will handle naming the mod, author, and downloading the most recent file. Not sure how to check versions against github though. Extra work could be done to get the tags of the release file, but I'm not going to do that right now. Maybe in a day or so. It should be simple to get the release tag and date though, so that'll be good.

I have not done a pull yet, since I want to finish github first.

[Gribbleshnibit8]

So, I want to add support in for game version, but I don't know how to update/modify the language files to reflect the new rows in the list. Adding that will also require modifying the Edit Mod Info window, and it's already so well laid out I'm not sure what to change where. I also have some other general GUI changes I'd like to make that would just kind of clean things up. Totally unnecessary really, but (I think) it looks better. Obviously there's nothing that says you have to merge anything I do, but was just wondering if you would accept that. I can post screenshots of changes when made.

I've started looking into enabling the forum support stuff, which is going to take some doing, and I'm going to look into adding Bitbucket too, since several mods I like use it. Lots of new options!

[4o66]

I have not done a pull yet, since I want to finish github first.

Actually, with Git you want to pull now since you are in a branch. You will get any updated code that does not effect what portion you are working on.

This way when you are done with your branch, merging back will be less painful.

(Always pull just before a branch merge, makes the pull request back to upstream less confusing.)

[Gribbleshnibit8]

I don't think I did it exactly correctly, but so far as I can tell I'm up to date on the current version, though mine is now like, 5 or 6 commits ahead? Still learning, but figuring it out. The code is much less confusing

I rewrote a bit using HTMLAgility and omg that's so much easier. That'll make adding BitBucket pages much easier. I want to add Jenkins pages, but those are a lot less standardized, being hosted individually, not sure what to do about URL verification yet. I'm also trying to do a bit of code consolidation as well, moving some of these page parsing functions into more logical locations, mostly cleaning up after myself, but still.

I added in a control to make 64-bit launching optional, since it's less stable. Just an extra checkbox in the launcher that lets you select it, 32-bit by default. Otherwise, the biggest UI change I've made is restructuring the options panel to put the color selections in two columns so that there's more room for other things there.

Oh, and the github branch was just to test using an API for access, since it required me to set the project at .NET 4.5 and make a few other changes. That didn't work so that was really nothing more than the few edits to get github working anyway.

EDIT: 6/12/2014 11:38PM CST

Added a handler for Bitbucket, I'm on a roll! I can't think of any other sites besides people's personal sites to handle, and I might look into that by creating a handler that will just find all download links. Of course, no way to version that so not sure. Anyway, I had more to say and then accidentally refreshed the page and lost it, so I guess that's it for now. I'll look over all my changes when I'm more awake and probably do a pull request tomorrow.

Edited December 7, 2014 by Gribbleshnibit8

[MacTee]

So, I want to add support in for game version, but I don't know how to update/modify the language files to reflect the new rows in the list. Adding that will also require modifying the Edit Mod Info window, and it's already so well laid out I'm not sure what to change where. I also have some other general GUI changes I'd like to make that would just kind of clean things up. Totally unnecessary really, but (I think) it looks better. Obviously there's nothing that says you have to merge anything I do, but was just wondering if you would accept that. I can post screenshots of changes when made.

I've started looking into enabling the forum support stuff, which is going to take some doing, and I'm going to look into adding Bitbucket too, since several mods I like use it. Lots of new options!

Hi Gribbleshnibit8,

i would prefer to make GUI changes on "Core" Views by myself (for now).

I'm almoste done with the GameVersion integration. Expect a GitHup update soon

EDIT: See here

Some notes to add localization support to new Controls.

• The Control name MUST be unique through the complete project AND plugins projekts (otherwise last Language file entry wins).
  
• Add a new entry to all language files for the new Control like this:
  A)
  <Control Name="{UniqueControlName}" Value="{The text of the Control}">
  For notmal Controls like labels, buttons, etc.
  or
  
  <Control Name="{UniqueControlName}" Value="{The text of the Control}">
  <ChildControl Name="{UniqueControlName}_ToolTip" Value="{The text of the Control ToolTip}" />
  </Control>
  
• Adding Items to the ModInfo Panel is a little complicated
  You have to change the ID of the language file nodes accordingly to the changes of the IDs of the ModInfo Items.
  But till i have redone parts of that code i guess asking me for changes would be better
  EDIT: This commit shows you how to change the language files.
  

Added a handler for Bitbucket, I'm on a roll! I can't think of any other sites besides people's personal sites to handle, and I might look into that by creating a handler that will just find all download links. Of course, no way to version that so not sure. Anyway, I had more to say and then accidentally refreshed the page and lost it, so I guess that's it for now. I'll look over all my changes when I'm more awake and probably do a pull request tomorrow.

Nice that we have Bitbucket support now. Thanks! =)

When you do your pull request, please do a pull request for each feature like BitBucket support, GUI Changes for GameVersion, etc...

Edited December 7, 2014 by MacTee

[SVlad]

MacTee, why do you use that unusual internationalization system with XML and controls name? Usually lang file is key-value pairs in plain text file. That file is easy to edit with any tool. An XML can be edited with text editor too, but it's bit more difficult. And GUI components just refer to keys in that file. So one string can be used in several components instead of duplication it for each component. As column header and as field label, for example.

[MacTee]

Hi SVlad,

the problem is that i want sometimes a translation of items from a controls and ToolTips too.

Like the ModInfo entries.

And i love XML =)

[4o66]

Oh, and the github branch was just to test using an API for access, since it required me to set the project at .NET 4.5 and make a few other changes. That didn't work so that was really nothing more than the few edits to get github working anyway.

.

Found an issue with the GitHub integration, but not sure there is anything that can be done.

The Active Texture Management Mod has multiple binaries with each release, 2 for x64, and 2 for x86. Since all 4 are under the same tag/release, the current implementation will grab the first binary of the release.

Any way we can change this to ask user when multiple binary links are present (and store for future update checks?)

Thanks!

[Gribbleshnibit8]

Any way we can change this to ask user when multiple binary links are present (and store for future update checks?)

Missed that some would do that. It's possible yeah, it'll need a list dialog popup when multiple files are present, the same dialog should be usable for the forum handler too. I can throw something together for it unless MacTee has an idea for it already (or something done).

New fields would be a list of the available downloads (probably doesn't need to be stored), and the index of the selected download. Though, it doesn't always appear to be the same order, so storing some kind of name value might be necessary to select the correct one, or just offer every time, if order or naming is inconsistent.

I'm going to try to work on the forum handler over the next few days, there's a lot of oddities to handle, like inconsistently named ID properties and stuff. Frustrating but not impossible to work with.

To ensure I do this correctly, I have a couple questions about what I need to do for easy merging later. The way I understand it is that I should:

1. Pull the dev branch into my local master.
   
2. Branch my local into a branch for working on the forum handler
   
3. Make all my changes for forum handler in there, and push it to github
   
4. Make a pull request against the dev branch with that branch as my source.
   

Edited December 9, 2014 by Gribbleshnibit8

[4o66]

To ensure I do this correctly, I have a couple questions about what I need to do for easy merging later. The way I understand it is that I should:

1. Pull the dev branch into my local master.
   
2. Branch my local into a branch for working on the forum handler
   
3. Make all my changes for forum handler in there, and push it to github
   
4. Make a pull request against the dev branch with that branch as my source.
   

My recommendation:

1. Fork the Dev Branch to your github

2. Clone that repo to your computer.

3. In your repo, 1 issue = 1 branch.

4. When issue fixed, merge that branch

5. Push to origin/master

6. Pull request to upstream/dev.

7. Profit! (Wait, can't: licensed under CC BY-NC-SA)

Or if you are lazy like me and have a fork going that you don't want to scrap...

Skip step 1 and 2, add upsteam as a remote, and fetch & pull the dev branch in local.

[MacTee]

Missed that some would do that. It's possible yeah, it'll need a list dialog popup when multiple files are present, the same dialog should be usable for the forum handler too. I can throw something together for it unless MacTee has an idea for it already (or something done).

I have added a "SelectDownload" dialog which was intended for that purpose.

Feel free to edit it.

New fields would be a list of the available downloads (probably doesn't need to be stored), and the index of the selected download. Though, it doesn't always appear to be the same order, so storing some kind of name value might be necessary to select the correct one, or just offer every time, if order or naming is inconsistent.

When the index and naming changes, i think let the user select the download every time.

I'm going to try to work on the forum handler over the next few days, there's a lot of oddities to handle, like inconsistently named ID properties and stuff. Frustrating but not impossible to work with.

Don't miss to have a look at the old KSPForum handler =).