Calling All Mod/Plugin Developers, I have a proposal, I need your support

[katateochi]

Hi everyone; Moustachauve and I have been talking about a possible way to provide a website which provides a centralised place for all Mods/plugins. I want to put to you a proposal and see what you think. For this idea to work it will really need the support/approval of the mod developers, you guys are fundamental to making this work.

The approach we want to try is a distributed, cloud based and out-sources all the nasty problems like backups, security, scaling, redundancy to a third party.

The third party is GitHub and it would provide a lot of gain and one significant catch.

The catch is that mod developers would need to learn the source code management (SCM) system called 'git'. If you don't know what git is I can explain further, but all you'd need are the basics.

The idea is that each mod developer will have a (FREE) github account and will host their own mods, each mod in a separate "repository" (this is about as easy as sneezing). Every github repository(repo) gets its own wiki pages and bug/issue tracking system and each has a main page which shows a file browser for the repo and displays it's readme file (nicely formatted with markup).

The site which we would build would simply act to index these repositories and users requesting downloads would be redirected to the github download (transparently I hope). Our site would provide search, tags, comments all that kinda stuff and also enable users to submit bug reports directly to the mods issue tracking system.

The real gains however are in resilience, scaling and security. Github provide a corporation grade service and have automatic backups, redundancy etc to ensure their systems stay up and scale to huge numbers of users. This brings in a high level of resilience and takes all the responsibility for important aspects like security, bandwidth, growing storage requirements etc away from our site. It means that if our site goes down it will not effect mod developers in any way. It will not actually stop users from downloading/finding mods either as google will return the github pages. All the high load traffic goes onto GitHub rather than on our little site meaning it would be a very low maintenance/cost site to run (prob free).

There are other gains to github; collaboration being a major one, but I could waffle on for pages, I'll spare you.

There are still some unknowns to work out but I think this idea is very possible. I've tried to put this as briefly as possible without going into the details (I did, 2 pages in and I'd bored myself). I can explain the idea more as needed.

The main question which I'd like to hear your feelings on is; how do you Mod Developers feel about being forced to use git as the way to publish your mods? This is key, without your support this won't be a good idea!

[Corax]

how do you Mod Developers feel about being forced to use git as the way to publish your mods? This is key, without your support this won't be a good idea!

Being forced into something as the only way to do things is always a bad idea.

Several developers are already using github, some aren't, and likely some won't ever - to be honest, I'm still struggling with getting github working from within MonoDevelop, as long as I can't get that worked out, I'm not willing to go to the extra length of manually maintaining a git repository.

[nibula]

As user of these mods id like to download all the mods from comfort of this forum.

+ there is no way you could force anyone to use that because i could just put my files to dropbox and post link here

[Seret]

I don't think it's a case of being forced to anything is it? It's just that if you wanted to be featured on their site you'd have to use Git.

[Corax]

I don't think it's a case of being forced to anything is it? It's just that if you wanted to be featured on their site you'd have to use Git.

Granted I may have read more into the OP than what was actually intended. It is of course fine for anyone to create another mod repository with whatever backend they opt to use.

There are already quite a few external repositories. In that respect, yet another repository can only increase the possible choices for users and authors alike, which I think is generally a good thing; time will tell which one will emerge as "the" repository favoured by people looking for mods.

I'd say go for it, create that site, anounce it once it's done, and see if it is being accepted.

[EndlessWaves]

I know being offensive/unpleasant is fashionable in certain circles but some people are going to be put off by the name so it's not the best choice for a universal mod database.

Additionally github is specifically code hosting isn't it? Would they allow other mods like art assets and new music?

[comham]

I know being offensive/unpleasant is fashionable in certain circles but some people are going to be put off by the name so it's not the best choice for a universal mod database.

lol what

Anyway, some kind of unified mod system would be very nice. Beats the hell out of a cluttered forum.

[Dani-Sang]

I've got a 4man plugin/mod team and we are never ever going to post something outside KSP forums unless it is an official KSP website itself. We got difficulty enough making our mod work wonderfully and having one good downloading link is more then sufficient. Having more download links means more maintenance work time, and time that I could spend better on making the mod better.

I can tell you that my plugin-coder is not willing to learn yet something else how git or whatever it is called work. Forcing somebody to do it will only scare more potential modders away. I'm against in general of external websites for mod-database because of the above stated reason. Also eventually you will need to post on multiple websites because member-base is going to ask for it and of-course you want your modification to be popular.

Lets keep it with the forums, there is no use for external websites, all the ones that I know are either unsuccessful or a blatant failure and keep modders away from what they should do.. --> create content.

All the high load traffic goes onto GitHub rather than on our little site meaning it would be a very low maintenance/cost site to run (prob free).

There is no such thing as free in this world

There are other gains to github; collaboration being a major one, but I could waffle on for pages, I'll spare you.

I'm already collaborating with many other people. 3 within my own team, and several outside of it. I don't see how a website can improve collaboration between people, only thing you need is some will to do, but I could waffle on for pages, I'll spare you.

The main question which I'd like to hear your feelings on is; how do you Mod Developers feel about being forced to use git as the way to publish your mods? This is key, without your support this won't be a good idea!

Forcing others while asking for my support... since when did the world work this way?

EDIT:

If you want to create something for the community. Make a website like mediafire or dropbox where we can drop our mods for unlimited download. But I geuss you would have to pay for that eventually anyway :/

Edited August 27, 2012 by Dani-Sang

[togfox]

op: if you can index github then can you also index this forum? it seems you'll get more joy that way

[Seret]

I can tell you that my plugin-coder is not willing to learn yet something else how git or whatever it is called work. Forcing somebody to do it will only scare more potential modders away.

If your coder isn't using a version control system currently, then they're doing it wrong. Git is one of the biggest ones out there, learning to use it is a valuable, marketable real-world skill. Something to think about.

[Dani-Sang]

If your coder isn't using a version control system currently, then they're doing it wrong. Git is one of the biggest ones out there, learning to use it is a valuable, marketable real-world skill. Something to think about.

Still, for a simple game mod on a external website?

[comham]

KSP is not simple, some of the mods are far from simple, and the modding scene (as a whole) for this game is set to get very not-simple rather fast.

I'm not a modder, I can't even program. All I know is my experience from Arma 2 which has an uncountable number of mods, everything from graphical changes to 8000 new uniforms for people who care what water bottle the enemy is wearing, new sounds, new maps, extensive plugins... everything. Setting up your mods and joining a server would be a nightmare without the Six Updater. Browsing existing mods would be painful if they were all in one ugly forum.

The OP could have phrased it better, since the way he did it is provoking an emotional reponse. Imagine he said "I think we should adopt this [github] as a community standard because [reasons]" instead of "how about you're forced to use this"

[Dani-Sang]

KSP is not simple, some of the mods are far from simple, and the modding scene (as a whole) for this game is set to get very not-simple rather fast.

I'm not a modder, I can't even program. All I know is my experience from Arma 2 which has an uncountable number of mods, everything from graphical changes to 8000 new uniforms for people who care what water bottle the enemy is wearing, new sounds, new maps, extensive plugins... everything. Setting up your mods and joining a server would be a nightmare without the Six Updater. Browsing existing mods would be painful if they were all in one ugly forum.

The OP could have phrased it better, since the way he did it is provoking an emotional reponse. Imagine he said "I think we should adopt this [github] as a community standard because [reasons]" instead of "how about you're forced to use this"

KSP is a lot easier to mod then all those other games, but back to the point. I still think that github or any external website is not going to work unless it is extremely good. And I haven't seen one yet.

[Moustachauve]

op: if you can index github then can you also index this forum? it seems you'll get more joy that way

We have no way to index the forum. GitHub provide a nice API that give us the ressource needed to do it automaticly.

KSP is a lot easier to mod then all those other games, but back to the point. I still think that github or any external website is not going to work unless it is extremely good. And I haven't seen one yet.

We could possibly give an alternate choice to upload your plugin from our website that would put it automaticly on our github repo and for you and the other user, it will be the same as f they downloaded it from the forum, you can even copy the download link from our site and post it on the forum if you want, you won't have to upload it at many places

Edited August 27, 2012 by Moustachauve

[katateochi]

Hi everyone, thanks for input. I'd meant to reply sooner, life just got in the way.

I may have focused you all on the specific 'git' aspect too much. I used the term "forced" to put it as strongly as possible that the 'ideal' way would be for everyone to be all using the same thing and that IMHO that common thing should be github.

I knew peps who'd never heard of git would react with "oh thats an odd name", everyone does the first time! I'd also written like 2 pages to explain it and thought it would be way too much.

At its core, this is not really about using git vs anything else, this is about using old fashioned "server" thinking vs modern cloud computing.

So let me Rewind.

The main thing I'm trying to tackle is the idea of centralising the mod repo all in one place. We all want one place to find/upload mods, but if that one place goes down then everything stops, developers can't upload, users can't download. It breaks both workflow and playtime - major fail!

Putting all the data in one place (ie this forum) is a "single point of failure". It's also costly and implies a ton of admin work (but who cares/thinks about them right?). Almost however you do it, be that on this forum or on a dedicated web site you can't get away from putting all the data in one place, except with a cloud based service.

The gain of github is it is a major distributed service. It sits on a massive layer of virtual machines spread across many servers, many countries. It has ridiculous levels of redundancy meaning that if any bit of it falls over its not a problem, the system just self corrects. It would take a major internet outage to stop it.

The same is true for the Amazon Elastic Cloud (EC2) which could well be another option, but it will cost (although very little if we follow the distributed model).

Hosting on github would mean that if this forum or any indexing site goes down nothing has to stop. Developers can still publish and users can still download.

I've actually had this use case happen already. The forum was down(during the upgrade) but the mod I wanted was actually on github and I could still get the mod and carry on playing KSP regardless of forum issues. (can't remember the mod, but here's a space craft repo I found on github; https://github.com/jasonpl/kerbal-spacecraft - (with a nicer url than anything you get from this forum!))

btw dropbox is basically a simplified implementation of github. It has similar backend qualities (on a much smaller scale) and much much less front end power. githib is aimed at coding so comes with wiki pages and bug/issue tracking systems for each separate project. I know every dev has their own bug tracking system, I don't care if you use a pinata full of post-its, its about the community of users having a unified way of submitting bugs for all mods.

We could prob do this using dropbox but it would be very much more clunky. We've already seen dropbox held mods hit bandwidth limitations. Github also has bandwidth limits but the expectation is that people will be downloading large projects from each other so they are higher.

My primary goal is to put the mod hosting into some form of cloud type architecture which brings a level of resilience and dependability that servers can only dream off.

This could be done using (in order of preference) GitHub, Amazon EC2 or dropbox. Either way puts the repo into a cloud type architecture and gets away from having a single-point-of-failure, brings resilience and takes the stain away from admins. However github is the clear leader as it would be COMPLETELY FREE and is designed around programming projects.

Git also is the leader in terms of production grade programming workflow. It makes deploying/publishing your code very easy. 'git push ksp master' done. It's authenticated using SSH keys so very secure.

To summarise;

I think that the most effective, dependable way of providing mods to KSP users is to use a cloud type architecture. The best one available is githib but that implies learning/change for some. If git is just too much there are other options but they will be harder and less elegant.

-edit/addendum-

To answer the (so old man-y) "nothing's free, everything costs" mindset;

Github makes its money of the big guys, like Microsoft, Twitter, Facebook. They pay terrifying amounts to host private accounts. GitHub wants to promote open-source so it sucks on them and lets the open-source guys graze for free.

The other big player in the "give users free stuff" domain is Heroku - originally a Rails only host, it now hosts anything. You get one virtual machine for free (it's not even a VM its way cooler) and loads of free addons. That is MORE than enough to run a simple low to medium traffic site. You can pay to scale up or use additional addons but you only pay for what you use, metered to the second (like with EC2, which is what it sits on). Scaling is very granular, you only need to add what you specifically need, none of this "package" nonsense.

EC2 is almost free. You pay for exactly what you use to the second, I get monthly bills of around $0.05 for hosting some backups, I laugh.

-----

Another option is to allow mod devs with the "git or whatever its called" mindset (lol) to upload their zips (or whatever their called; archaic file structures) to the web site and have it create and manage repos automatically (cos the github API is just that easy). Why upload more than just the files that have changed is beyond me but ok.

-edit/addendum-

Edited August 27, 2012 by katateochi

[Dani-Sang]

I've looked up github and read about it.

I honestly still don't see how this is beneficial to modders.

Not after your long post or going to github itself. I can see it can be beneficial for large open sourceprojects where it is mostly about code. But I really wonder how you want to intergrade this along with models, art and whatever.

Also, a game isn't going to die because a forum is down for a few days, neither are the mods that you can find on it.

Edited August 27, 2012 by Dani-Sang

[Moustachauve]

Also, a game isn't going to die because a forum is down for a few days, neither are the mods that you can find on it.

The other day my friend really wanted to download a mod, but the forum was down. I can say that he was REALLY impatient about the forum being down. Maybe its nothing for you, but it may be really frustrating for others.

[Dani-Sang]

The other day my friend really wanted to download a mod, but the forum was down. I can say that he was REALLY impatient about the forum being down. Maybe its nothing for you, but it may be really frustrating for others.

If you are getting frustrated about the fact that you can't download a little mod of a game (while you can play the game) then you need a checkup....

[katateochi]

I've looked up github and read about it.

I honestly still don't see how this is beneficial to modders.

Not after your long post or going to github itself. I can see it can be beneficial for large open sourceprojects where it is mostly about code. But I really wonder how you want to intergrade this along with models, art and whatever.

Integration with static assets is not a problem, a git repo is just a folder. What you put in it is up to you, it works best with smaller files, using it for large media is wasteful but that is all.

Git works for the single developer up to the large corporation teams.

Just a question, how do you merge the work of two different devs working on the same project? How do you track which lines each one changed?

Also, a game isn't going to die because a forum is down for a few days, neither are the mods that you can find on it.

I see this as the classic developer focused on the product, not the end user.

[Capt'n Skunky]

Keep it civil or I'm going to lock this topic.

There's a reason every attempt to create some form of central mod repository has failed, they take more work and effort than just posting on the forums.

Trying to convince someone to change their workflow because you prefer a different method will never fly. Right now, using the forums to distribute your plugin is the best way to get exposure and contact with players and that's not likely to change for some time. Until Squad makes a statement on how they plan to support mods in the future, all this off-forum work might end up amounting to spinning your wheels.

Cheers!

Capt'n Skunky

KSP Community Manager

[Dani-Sang]

You are taking this way too serious katateochi. I'm just a simple modder doing this in my freetime, and telling me how to work better is kind of offensive, you are even comparing me how I should work and act like modding is some kind of job. Thats not the point of community modding and defiantly not for many. We just create a mod we like, we have to file it all in some kind of database or cloud systems, then you don't get the point of mods and why people create them.

[katateochi]

Keep it civil or I'm going to lock this topic.

Sorry' date=' too OTT, i'll be more chilled.

There's a reason every attempt to create some form of central mod repository has failed, they take more work and effort than just posting on the forums.

It is the very aspect of centralization that I want to tackle. Centralized, top-down always fails when it gets too big, I was hoping to thrash out the concept of cloud hosting but we've got rather side tracked by the git aspect.

Trying to convince someone to change their workflow because you prefer a different method will never fly. Right now' date=' using the forums to distribute your plugin is the best way to get exposure and contact with players and that's not likely to change for some time. Until Squad makes a statement on how they plan to support mods in the future, all this off-forum work might end up amounting to spinning your wheels.

[/quote']

I'd put it over too strongly. The perfect is for everyone to follow the same way but unless by some miracle everyone already used git that was never going to happen. I'd hoped people would see the cloud aspect as being so very valuable that git being a means to an end would not be an issue and people might consider it rather than out right rejecting it.

[katateochi]

You are taking this way too serious katateochi. I'm just a simple modder doing this in my freetime, and telling me how to work better is kind of offensive, you are even comparing me how I should work and act like modding is some kind of job. Thats not the point of community modding and defiantly not for many. We just create a mod we like, we have to file it all in some kind of database or cloud systems, then you don't get the point of mods and why people create them.

I'm sorry for making you feel like I'm tell you how. That was not my intent. I assumed you where a professional/production grade coding team by your "I've got a 4man plugin/mod team" line in your reply and thought that this would only be a major benefit for you.

You make it sound like I want you to file your mods in some bureaucratic system. You miss my aim entirely. The aim is to make the whole publishing to community and tracking of versions a complete no-brainier for you and totally dependable for the users.

Yes I'm taking this seriously, I take everything seriously, so add a pinch of salt, that's just me, sorry.

[katateochi]

http://kerbalspaceprogram.com/forum/showthread.php/18269-So-many-List-of-addons-mods-are-broken-due-to-the-switch-to-this-new-forum-%28con-%29

case in point

[Bluejayek]

http://xkcd.com/927/