Community Mod Repository and The Majiir Challenge

[Majiir]

Community Mod Repository

The goal of this project is to replace Kerbal Spaceport with a community-built, community-run alternative. Spaceport is in disrepair, and both it and its planned successor are effectively abandoned. By producing our own repository, we can ensure that it serves our interests as the producers and consumers of mod content.

There's been some discussion (e.g. in #kspmodders) on what a replacement should look like. I hope this thread will both consolidate those scattered discussions and attract more of the community at large.

The Majiir Challenge

One of the greatest obstacles to this project is the cost of server infrastructure. Weighing in at just 15MB, Kethane alone generated 650 gigabytes in download traffic in January. A popular mod repository would easily overwhelm any free or inexpensive hosting plans.

To eliminate that obstacle, I am offering free hosting for a community mod repository. My own server infrastructure is highly underutilized, and some back-of-the-envelope calculations indicate it's more than enough to handle this project. Specifically, I can provide virtual private servers on a baremetal hypervisor that I manage. This means I can support any needed server software and configuration so long as it doesn't incur additional licensing costs. (That means Linux, please.) The server is located in a Tier III datacenter (that's good), and it has a gigabit Internet link with plenty of bandwidth. I can also provide SSL certificates and (probably) domain names. In other words, I can take care of anything that really costs money.

Since we may see multiple projects in the works, I'm willing to provide service for any promising candidates until a clear winner emerges. Note that I have a preference for scalability, so a lightweight server will be more likely to win the challenge than a Wordpress reskin. (Super bonus points for using Node.)

There are no hard-and-fast rules at this point. My goal is to encourage development by removing the cost concern.

My Two Bits

As a mod developer, these are some things I'd like to see that would convince me to host Kethane on a repository:

• Detailed download statistics with an option to export raw data. This is my biggest sticking point.
  
• Effective search system. I don't care if it picks up every keyword, but if someone searches for Kethane, my mod ought to come up first.
  
• No moderation queue. At the very least, there should be some way to graduate out of moderation.
  
• Previous version support. I like to keep previous versions available for reference purposes, and some users like to hold off on updating.

But above all, I suggest to start simple. There's no point in getting wrapped up in a complicated categorization system before producing a first version. Let's talk core features, technologies and objectives.

Edited April 3, 2014 by Majiir

[blizzy78]

This is what I would like to see per mod. Some of these would be optional, of course.

- Screenshots.

- Videos.

- Changelog.

- License.

- Link to source code repository.

- Donate button (linking to where mod author decides.)

- Forum thread link.

- No commenting (use the forums!)

- Do we ever need rating?

[ferram4]

I'll chime in and agree with what Blizzy has suggested for each mod, especially the no commenting bit. Creating another place for support requests to appear is a pain, since for FAR I already need to check the FAR release thread, the Addon Support forum and r/KSP and r/KerbalAcademy, and then the Spaceport comment section if I remember (normally I don't, and normally comments don't appear there). Having the option to comment there will just frustrate everyone (users don't get timely support, modders have another place to look).

[rbray89]

One thing I wouldn't mind having is an easy way to manage source releases (from a modder perspective). Ie. I push to github, and hit generate release on the spaceport and it builds and packages it. I have sooo many projects on my plate ATM. One more can't hurt right?

[Greys]

Rating is actually a really valuable thing if harnessed correctly.

The biggest thing in my eyes is that we need a Crowd Sourced Content Moderation system, similar to how Reddit works. Users are given a quick system to convey how they feel about a listing and flag things that they feel are inappropriate. Ratings, views vs downloads, and how recently a mod was posted impact listing order after relevancy, so if you search for Kethane, Kethane is first, but if you search for Rockets, the listings will be ordered based on community response, with newer mods being preferred. A sufficient quantity of flags in a given period relative to the total number of views may cause a listing to be hidden, but all flags will be brought to the attention of a small number of official moderators to be handled.

Ideally the users should not be forced to have an account, but users logged into accounts could be treated as more valuable for weighting their opinions; and there could be a system where users define an instance of KSP, what version they're using with it; and what mods they're using with it; and the system will alert them when updates for those mods are posted. There could even be a system of sharing builds that provides an easy method of getting all of them.

Second to that, there should be a strong versioning awareness to the system that doesn't force developers into one scheme for their own versioning. Mainly it needs to be aware that this version of MechJeb is newer than that version. Side to that, the version of X mod needs to have it's own compatibility list that users are able to contradict; so say UbioZur's Welding mod 'supposed' works for 0.23, but last I used it it crashed everything constantly, I could go to the listing, look at that version, and click a thumbs down next to the Version Y Works With: KSP 0.23 item, and these results would be conveyed back to the users for them to make their own decision about it. Similarly I could thumbs up that it does work with 0.23 to give relevancy to the thumbs down count; and I could thumbs up version matches that maybe aren't addressed or supposedly don't work. In combination with the Voluntary Version Locking movement, mod makers should be able to specify that their mod will refuse to work with certain versions of KSP.

Also mandatory license selection for all submitted addons, with a list of common licenses selectable via a randomized order checkbox list, a license helper link; and the ability to upload a license file; No means that could result in a listing that does not contain a license.

Finally, as a feature to the modders, offsite hosting. Even if the new ~port is good, some people may still want to host the files themselves or exploint GitHub's release system; the ability for a listing to have a link instead of an uploaded file would just be nice.

[Superfluous J]

As a(n extremely light) modder, I'd like the ability to host my own stuff yet still have an entry on the modding site.

As a player, I'd like a way to be notified (email, for example) when a mod I'm using is updated, if I've so chosen to be notified.

[rbray89]

As a player, I'd like a way to be notified (email, for example) when a mod I'm using is updated, if I've so chosen to be notified.

Oooh... I like that.

[Justin Kerbice]

Good idea, in order to reduce hosting cost to its bare minimum, a torrent tracker might be the best, torrent can even be rated by the bittorrent client, community will be invited to contribute by seeding and whatever happen to the main server, sharing can still be up.

In addition to that, as Greys wrote, versionning should be mandatory + I add: DOCUMENTATION inside/with the files.

[Superfluous J]

Good idea, in order to reduce hosting cost to its bare minimum, a torrent tracker might be the best, torrent can even be rated by the bittorrent client, community will be invited to contribute by seeding and whatever happen to the main server, sharing can still be up.

Great idea! I'd personally torrent everything I use and leave them up all the time.

[Greys]

Good idea, in order to reduce hosting cost to its bare minimum, a torrent tracker might be the best

Majiir has 100TB monthly transfer to offer, and most mods don't compete with the popularity of Kethane, reducing the hosting cost isn't really necessary and involving a torrent client adds a barrier for entry that a significant number of users won't bother with. Not a bad idea but in this situation I don't think it'd be advantageous in this situation. Maybe as a secondary offering.

The big problem with torrents in this situation is that as mod makers we want to limit the recirculating of outdated versions, so there would need to be a way to kill the torrent swarm when that version is replaced; given that most torrent clients these days are designed to keep a torrent running even once the tracker has been cut off this may be impossible.

[DerekL1963]

Good idea, in order to reduce hosting cost to its bare minimum, a torrent tracker might be the best, torrent can even be rated by the bittorrent client, community will be invited to contribute by seeding and whatever happen to the main server, sharing can still be up.

As a user, no thank you. If I need additional software or plugins/add-ons to download mods, it'll likely be a no-go. KISS.

[blizzy78]

Same here, I'd just want to download it using my browser I'm already browsing the site with.

[Majiir]

- Screenshots.

- Videos.

- Changelog.

- License.

- Link to source code repository.

- Donate button (linking to where mod author decides.)

- Forum thread link.

In the spirit of simplicity, I'll distill this down to one feature: rich description field. A simple text formatter like Markdown can be extended to support video embeds.

- No commenting (use the forums!)

- Do we ever need rating?

I agree that commenting is a bad idea. I also think traditional rating systems are a bad idea, although I'll address this in more detail in response to Greys.

One thing I wouldn't mind having is an easy way to manage source releases (from a modder perspective). Ie. I push to github, and hit generate release on the spaceport and it builds and packages it.

Automated build is a huge feature, but something that's very easy and allows for the same end result is a simple submission API. If the site is well-built, this wouldn't even need to be a separate feature. This way, you can use an automated build system like Jenkins and have your built script automatically upload releases.

The biggest thing in my eyes is that we need a Crowd Sourced Content Moderation system, similar to how Reddit works.

I think I agree with this at a fundamental level, but what you've described is getting very complex, especially when you talk about logins, versions, flags, moderators, et cetera. A system that brings more responsibility to mind than Reddit is StackOverflow, but that works because reputation is earned as a sort of currency, and that must be spent in order to vote down. Another alternative is a "star" system, which is essentially upvote-only (and can seem less like a voting system if it's used to give users a "favorites" list). Crowd rating systems are hard to get right (even Reddit is still refining its algorithms) so I'm not sure this is a core feature.

Second to that, there should be a strong versioning awareness to the system that doesn't force developers into one scheme for their own versioning. Mainly it needs to be aware that this version of MechJeb is newer than that version.

There are a number of ways to do that, but it can get complicated if you try to be too automatic. If the system is to support multiple versions of a mod, is it not enough to have a "latest" (or "default") and then a chronological listing of all other versions? The fanciest you have to get is allowing the mod author to manually reorder the listing.

Side to that, the version of X mod needs to have it's own compatibility list that users are able to contradict; so say UbioZur's Welding mod 'supposed' works for 0.23, but last I used it it crashed everything constantly, I could go to the listing, look at that version, and click a thumbs down next to the Version Y Works With: KSP 0.23 item, and these results would be conveyed back to the users for them to make their own decision about it.

This is getting way too complex. It seems simple on the surface, but when you consider all the API endpoints, database routines and front-end scripts that would go into it, this becomes a huge feature item. It's not worth it. Either the mod will have been updated after the latest KSP release or it won't. If someone wants detailed compatibility information, they can go to a linked forum thread.

In combination with the Voluntary Version Locking movement, mod makers should be able to specify that their mod will refuse to work with certain versions of KSP.

Sure... in the general-purpose description field. KISS.

Also mandatory license selection for all submitted addons, with a list of common licenses selectable via a randomized order checkbox list, a license helper link; and the ability to upload a license file; No means that could result in a listing that does not contain a license.

This is making my head spin. KISS again. That's a whole lot of work for a problem that isn't fully articulated. This won't be operated by Squad, so it doesn't need to conform to Squad's community rules. I personally disagree with making license selection mandatory; so long as a sensible default policy is clearly communicated, the worst case is that mods are licensed more restrictively than intended (and this can always be amended).

Finally, as a feature to the modders, offsite hosting. Even if the new ~port is good, some people may still want to host the files themselves or exploint GitHub's release system; the ability for a listing to have a link instead of an uploaded file would just be nice.

As a(n extremely light) modder, I'd like the ability to host my own stuff yet still have an entry on the modding site.

I've said this about Spaceport, but the reason is because Spaceport doesn't have good analytics, and as an author I want to be able to track downloads. Off-site download is not completely unreasonable, but it does introduce complexity, so I'd be interested in justifications. There are advantages to requiring that files be hosted. (If a link is broken or leads to a slow download server, that could irritate users while reflecting poorly on the mod repository.)

As a player, I'd like a way to be notified (email, for example) when a mod I'm using is updated, if I've so chosen to be notified.

I like this idea. I don't think it's a core feature, but it's a good second-round addition. It could go very nicely with a "star" system since users would be disinclined to star mods they don't care to keep easy track of.

Good idea, in order to reduce hosting cost to its bare minimum, a torrent tracker might be the best, torrent can even be rated by the bittorrent client, community will be invited to contribute by seeding and whatever happen to the main server, sharing can still be up.

As others have mentioned, this is both detrimental to users and simply unnecessary. (Seriously, I have plenty of bandwidth. I host one of the most popular KSP mods, and my monthly bandwidth readout always rounds down to 0% utilization.)

In addition to that, as Greys wrote, versionning should be mandatory + I add: DOCUMENTATION inside/with the files.

Documentation is just something that quality mods do. You can't force quality.

[Superfluous J]

Very interesting stuff, here, Majir. I'll distill as well to those things I have some opinion on, which is amusingly one thing:

One feature: rich description field. A simple text formatter like Markdown can be extended to support video embeds.

One problem with a single description field for everything is that every mod will look different on their own page. The site will look disjointed and will confuse users. I feel the site should enforce SOME rules lest people (users, not modders) don't bother using it because it's ugly most of the time.

I like this idea. I don't think it's a core feature, but it's a good second-round addition. It could go very nicely with a "star" system since users would be disinclined to star mods they don't care to keep easy track of.

I see a problem with that: If I get updates for everything I give a star rating to, I'd never give something a low star rating. Unless by "star" system you mean "thumbs up" in which case I'm all for that.

Documentation is just something that quality mods do. You can't force quality.

...but you can force standards, and if a mod can't bother to conform to (reasonable) standards it's probably not worth my time as a user.

Don't let me make you think I'm down-talking the idea. This is just discussing the particulars

[regex]

I was about to say I'd give this a go (I'm a web dev by trade and we have an open-source MongoDB framework that would work perfectly for this) but then I remembered that I have a hard enough time working on my own mod after eight hours of programming.

This all sounds like a fantastic idea, though, and thanks for putting forward your resources, Majiir. Your post here echos my thoughts on most of the meat of this discussion. I agree, keep it simple.

[Majiir]

I see a problem with that: If I get updates for everything I give a star rating to, I'd never give something a low star rating. Unless by "star" system you mean "thumbs up" in which case I'm all for that.

Yes, that's what I mean. Think of Github's star system, or a browser's favorites list. Github displays a count of how many users have starred a repository, and each user can view his list of stars. It's a loose way of gauging popularity, and it tends to work as long as the action of starring also does something like adding the item to a quick reference list. (This way users don't just star everything they see.)

One problem with a single description field for everything is that every mod will look different on their own page. The site will look disjointed and will confuse users. I feel the site should enforce SOME rules lest people (users, not modders) don't bother using it because it's ugly most of the time.

...but you can force standards, and if a mod can't bother to conform to (reasonable) standards it's probably not worth my time as a user.

I mentioned to Greys in #kspmodders that NPM is a great example of a simplistic repository: https://www.npmjs.org/

NPM serves a very different purpose, but it does a good job at providing a consistent and informative experience without much complexity. I agree that it's possible to be too open-ended, but I also think it's important to carefully justify each additional piece of information that's made into a first-class feature. For KSP mods, it might be important to display author, version and release date, but a donate button might be too esoteric. These are issues to sift out keeping in mind that complexity can always be added later.

I was about to say I'd give this a go (I'm a web dev by trade and we have an open-source MongoDB framework that would work perfectly for this) but then I remembered that I have a hard enough time working on my own mod after eight hours of programming.

This is why I created this thread instead of working on code. I'd really like to see this happen, and while I think I have the skills to get the job done, I don't think I can make the time commitment. Plus, a lot of the feedback so far deviates from what I expected, so that's valuable knowledge.

MongoDB is certainly a good candidate technology for a site like this. Of course, that final determination will require more details on what exactly the site will do.

Edited March 1, 2014 by Majiir

[sumghai]

Very generous of you, Majiir - all of this sounds neat!

I have a fair amount of experience working in PHP / MySQL for web-based stuff similar to this, so if you guys need such expertise, let me know how I can help.

[DMagic]

Either the mod will have been updated after the latest KSP release or it won't. If someone wants detailed compatibility information, they can go to a linked forum thread.

It might be a good idea to have some kind of designated version indicator, not just the [version] text that some people add to the name of their mod.

As others have mentioned, this is both detrimental to users and simply unnecessary. (Seriously, I have plenty of bandwidth. I host one of the most popular KSP mods, and my monthly bandwidth readout always rounds down to 0% utilization.)

The one, and really only, thing that spaceport does well is that big red download button. There is no other software required, and no pop-up ridden download host to deal with. Having that simplicity in actually getting the files is a huge benefit.

[DerekL1963]

The one, and really only, thing that spaceport does well is that big red download button. There is no other software required, and no pop-up ridden download host to deal with. Having that simplicity in actually getting the files is a huge benefit.

This, +1, "Like", Thumbs Up, and assorted other noises indicating support and deep admiration.

The one thing the site must do well is make the download button obvious, consistent, and above-the-fold.

[Faark]

Allowing listings without download links that just refer to some external location (forum thread or even spaceport) might be a good idea as well, especially for the transition phase.

Being comprehensive is very important for users accepting such a database and sth like that might make it easier for mod creators or even allows entries to be created initially by someone else.

I'm also a little worried about the competition between multiple such projects. As mentioned, having a comprehensive listing is very important for such a project and both users and mod developers tend to be lazy. That means who ever launches first to get a head start will most likely win, regardless of its usability features (competing against spaceport with lots of modders behind you is one thing, but but an open competition sth else). Unless ofc there would be some rule that forces them to share content between each other...

[Rich]

I propose that we use an open source GitHub clone, such as GitLab, as a starting point. I suggest this based on my experiences using GitHub itself to develop and distribute my small contributions to the KSP community.

To demonstrate the partial viability of using a git backed system, I will address the OP's "Two bits" in terms of how GitHub fulfils these requirements (coupled with the vision to specialise and extend GitLab to support similar features).

1) Analytics - GitHub provides some analytics, but not nearly enough. However, I suspect these extra analytics could be added to GitLab.

2) Searching - The clone I am using as an example includes a search function. Is it any good? I have no idea, but again I am sure as a community we could improved it if needed.

3) Moderation - I feel a solution to this is independent of the platform developed.

4) Versioning - GitHub releases is amazing.

Potential issues that I see:

* The scalability of GitLab.

* Git is not suitable for every project, but coupled with a GitHub releases style system this would not be a problem.

* Everyone is not familiar with such rich project tools? Well, there would need to be some serious UI specialisation to make it suitable for a modding community.

Edited March 1, 2014 by Rich

[NeoMorph]

I propose that we use an open source GitHub clone, such as GitLab, as a starting point. I suggest this based on my experiences using GitHub itself to develop and distribute my small

2) Searching - The clone I am using as an example includes a search function. Is it any good? I have no idea, but again I am sure as a community we could improved it if needed.

3) Moderation - I feel a solution to this is independent of the platform developed.

4) Versioning - GitHub releases is amazing.

Just a few thoughts off the top of my head...

Searching... If there is one thing I absolutely HATE about both Spaceport and the KSP forums is the really crappy search engine. Trying to find a specific mod is hard enough without it sorting the results incorrectly and getting wierd stuff mixed in that has nothing to do with what you searched for. And what REALLY drives me silly is four letter minimum search query when we have a LOT of three letter mods (kOS for example) that you cannot search for. Want to search for SAS mods... HAHAHAHAH... dammit!

Moderation - Someone is going to need to weed out the trash/spam etc AND help make the mods go in the right subsections. Who is going to spend a LOT of time moderating the stuff (which is what has caused Spaceport to fail so dismally). It's something to think about.

Versioning - Here's an idea... I think we need to see one main thing shown next to the download button "[This mod is compatible up to V0.23]" so we know we are downloading a good version. The problem comes when a mod gets abandoned but is so good that it keeps working for a while afterwards... Maybe change it to "Confirmed working with V0.23". Also what to do with versions that are no longer working with the current version of KSP but still works fine in older versions? Some people may play older versions just because of their specific mod.

I also want to say thanks to Majiir for offering his server infrastructure.... I'm a big mod user and regularly have problems finding stuff that I am after. Having a mod base that will finally be organised will be a HUGE help. +10000000 rep.

[sharpspoonful]

After reading this thread I have a few requests and points ( and please bear with me, I'm on my phone) to make.

1. As a frame of reference I liked the direction Spaceport was headed in terms of how they wanted to organize mods by what they had; command and control, science, utility, etc. The problem was that most mods that have parts, cover most of the spectrum of part classifications. This, coupled with the obscene amount of craft files, poorly organized or documented secondary and tertiary mods caused the whole thing to collapse in a pile of mediocrity.

I propose an organizational hierarchy that is limited to name, author, a handful of predesignated tags (much like it already is organized in the mods library thread), and version. Most modders like me either already know some of that information or loosely know names, and could thus cut down on the search engine requirements.

2. This repository could go with out the aforementioned craft file downloads. They clutters server space and have few downloads if any. The forum already has a section for this and there is always red it for showing of and providing downloads. 'Nuff said.

3. I will add this to the discussion as food for thought. If this does take off and actually begin developing, I think that it would only be appropriate to ask SQUAD for some official support, and allow them to focus on the game itself rather than the mod repository. If we get support, there stands the possibility of having logins tied into our forum accounts and simplifying the voting, reputation, and bug support issue.

I would like to note that I am not a web dev, mod dev, and have little to no experience with networking outside of military infrastructure. I am simply a player with a lot of time spent downloading mods.

[Talisar]

2. This repository could go with out the aforementioned craft file downloads. They clutters server space and have few downloads if any. The forum already has a section for this and there is always red it for showing of and providing downloads. 'Nuff said.

I would agree with this 100%, or at the very least have some exclusionary capability included with search functions so people could easily filter out results that they have no interest in.

[Greys]

One of spaceport's top failings, among many top failings, is that craft files are in no way separated from things that are not craft files, this effectively exploits all of the other top failings to make the site useless unless you have a link to exactly what you want, and even then....

So long as our replacement for it does separate craft files, they shouldn't matter at all; and that is predominantly an issue with the search engine excluding things that don't have the desired category, how those categories are cut up is a discussion on to itself that we will probably have repeatedly, but craft files are not addons, they are not mods, they are not parts, they are not plugins, and they are not assets.

In one way you could consider this a key metric of if the search engine is working adequately, given that both addon and craft listings exist; if any craft files show up in the results when you don't want them to, the engine has failed.