ModStatistics 1.0.3 - Anonymous mod usage statistics - Now for public distribution!

[xEvilReeperx]

Because as it is designed now, the plugin sends the info each time KSP closes. Also no personal info is ever needed or collected.

That doesn't explain how you find the "common link"...?

[JewelShisen]

That doesn't explain how you find the "common link"...?

Does everybody have Mod X? If that is the only mod in common then there is the problem. If they have three mods in common then it is one of the three. It is a simple matter of just looking through and doing a compare.

[xEvilReeperx]

Does everybody have Mod X? If that is the only mod in common then there is the problem. If they have three mods in common then it is one of the three. It is a simple matter of just looking through and doing a compare.

Why would people who are having a problem show up in your reports at all? They're going to uninstall your mod and you have no way to track them when they're gone.

[JewelShisen]

Why would people who are having a problem show up in your reports at all? They're going to uninstall your mod and you have no way to track them when they're gone.

You assume that they will uninstall the mod the moment they find any issues at all and that simply isn't the case. Besides while this will still report usage even if you only have it once it is mostly made for tracking the usage by players that KEEP it installed. That was my point originally when I mentioned using this to see if my users also use FAR.

[codepoet]

At the moment we don't really know anything about what users have installed other than what gets downloaded (and that can be hard to know based on the varying quality of download hosting options.) We also don't know what errors users are having other than what gets reported on forums / github / similar. Something that automates the gathering and presentation of data in either of these areas would be a step forward, and I commend Majiir for taking one such step. I am looking forward to see what else others have to contribute in this area beyond just criticism.

[Majiir]

I can picture a twinkle in your eye as you write that. We both know the average user.

Uh? The average user probably won't care, if that's what you mean. There's nothing sinister here. Again I ask: Which collected data do you object to?

Download tracking can and does get you this information, if you have an accurate counter.

It turns out it doesn't. With restricted redistribution, download counts give a reasonable upper bound, but it says nothing about how many people actually use the mod. What percentage of players update within a week of release? How many times on average does a user download the mod? Is it the same userbase, or is there a lot of churn? Usage stats combined with download stats can answer a lot of these questions, but download stats alone don't say much. (I have raw server logs for a number of mods, including some I didn't author. They're nice to have, but not as useful as you'd think.)

Writing a plugin like this but saying that the most obvious, useful function that plugin could do is somehow not in its scope is silly and makes the real reason obvious.

Erm, what? It's a usage statistics mod. You're asking it to be an error reporting mod. You might as well ask me to make Kethane an autopilot. There is nothing special or hard about gathering a list of installed plugins; if you want, go ahead and make an error reporting tool. That's just not what this project is about.

My opinion is that you need to consider some way to make it useful to the user.

"Gosh, my game is crashing a lot. Why don't I check the stats site and see which of my mods has a high crash rate?" It's not very useful for a user to see his own data for a single session, but aggregate statistics (for a single user or the entire population) can provide insight. Have you not seen the hundreds of forum threads and Reddit posts titled "what are some popular mods I should download?" It would be nice to have some real data to answer that question. What are the most popular mods? Kethane and KAS? Interstellar and B9? MechJeb and KER? What mods cause loading time to increase? What mods do people tend to try for a minute and then immediately uninstall? What mods do people play with for hours on end? Are there mods that people leave running overnight? (Could be a bad thing!)

This plugin can be useful for a lot of people in a lot of different ways. There are a lot of ways in which it can be improved; I just coded it yesterday. You don't see the value? Fine, but please don't spam my thread just because you aren't satisfied by the many good answers already given.

[xEvilReeperx]

Sorry, I didn't mean to imply anything sinister. Text can make things sound different than what was meant. Getting users to read instructions can be challenging as I'm sure you know

Erm, what? It's a usage statistics mod. You're asking it to be an error reporting mod. You might as well ask me to make Kethane an autopilot.

Well, the actual meat of the mod is little more than a few handfuls of code. Loop through the various loaded assemblies, get their versions, and then use whatever you're using to serialize it and send it out. I figured since the error reporting would be so similar and related that it'd just make sense to roll it in there too rather than to make an entire second framework where most of that code is going to end up duplicated. "Error reporting" and "usage statistics" are closely related in my mind; that's typically what tools like this do. But you're right, it's your mod and I'm sorry

"Gosh, my game is crashing a lot. Why don't I check the stats site and see which of my mods has a high crash rate?"

That would be error reporting, apparently a separate and unrelated mod...? This is exactly the kind of thing I'm trying to talk you into adding, so I don't understand if this was sarcastic or what. I'll stay out of your thread if my feedback is unwanted

[Majiir]

That would be error reporting, apparently a separate and unrelated mod...? This is exactly the kind of thing I'm trying to talk you into adding, so I don't understand if this was sarcastic or what.

Crash reporting is relatively easy to fit into the stats regime, mainly because it's a necessary feature. (Stats are collected when the game closes, but if it crashes instead, we need to detect that and report on the session as far as it got.) The problem with error reporting in a more general sense is that it vastly complicates the privacy issue. I don't want to have the mod send any data that could potentially include personal information, and error logs certainly could.

[Faark]

Yes but error logs don't have to be submitted automatically. Just show the user the data and let him press "Send this stuff". At least from a legal pov this should be safe? Maybe with some flag to only

Loop through the various loaded assemblies, get their versions, [..] "Error reporting" and "usage statistics" are closely related in my mind; that's typically what tools like this do.

Most assemblies don't have a version info embedded, so you might have to do ugly stuff like generating a file hash and resolving that from your database. Also error reporting has a bunch of other difficulties, especially preventing spam of known reports or resolved issues. It would extend this project by a lot and would require constant effort to maintain the project and while it would be awesome, i can understand that Majiir doesn't want that.

[xEvilReeperx]

I wasn't thinking automatic reports, more just a way for a user to quickly and easily upload their depersonalized log and a short message describing the problem.

Re:

Most assemblies don't have a version info embedded, so you might have to do ugly stuff like generating a file hash and resolving that from your database.

I thought this

• List of plugins detected and their versions

indicated that version info was included already. That's why not having any way for the user to upload a log is so frustrating to me; I thought about 95% of the work was already done. You could copy-paste the entire project, add about 30 lines and for the cost of storing some logs on the server for a day have a very useful, widely desirable debugging utility for modders to distribute with all of their mods.

Anyway, won't cloud the thread any more

[Justin Kerbice]

I just think of a good use to this plug-in Majiir and others, as people talks about necro here, stats allow modders to know some people still have a real interest in their work and may push them to keep their mods alive and updated. As I guess anyone will be more please to spend time on something people actually use than on something on one even know about it.

But for such purpose, again, your plug-in needs to know exactly parts use, and how much they are used, not just a subdir of GameData dir is there. Which also allow modder to know which part to take care of in a big set (example: if one engine in KWRocketry or NovaPunch is very appreciated, and one is almost never use, if both have issues, priorities comes by themselves)

[Majiir]

ModStatistics 0.0.2 has been released. The download link in the first post has been updated.

This is another beta release. Changes were primarily related to the reporting format. The plugin now has a dependency which is packaged with the mod; I'm interested to see how this fares on various platforms.

Changes in this version:

• Reports are now sent as JSON instead of XML.
  
• Scene times are sent ordered by the GameScenes enumeration. This acts to hide the order in which the user entered scenes.
  
• The current time is now only sampled once during scene switches. This makes scene timers slightly more accurate.

[Majiir]

I thought this ("List of plugins detected and their versions") indicated that version info was included already. That's why not having any way for the user to upload a log is so frustrating to me; I thought about 95% of the work was already done. You could copy-paste the entire project, add about 30 lines and for the cost of storing some logs on the server for a day have a very useful, widely desirable debugging utility for modders to distribute with all of their mods.

It's true that version information is included, but getting versions is basically a one-liner (assuming the file version is correct, which it almost never is):

AssemblyLoader.loadedAssemblies.Skip(1).Select(a => a.name + " " + assembly.assembly.GetName().Version);

If someone needs a server to run an error reporting service, they're free to get in touch with me. I don't think error reporting is in the scope of this project. I'm not writing an error reporting mod. If there's some context in which reporting errors becomes easy or useful from a statistics point of view, I'll consider it.

It seems you're frustrated because this is "almost" what you're looking for, but I think you're off in that estimation. Besides, if that's really the case, then it shouldn't take anyone more than an hour or so to put together what you want. I'd head to the requests forum.

But for such purpose, again, your plug-in needs to know exactly parts use, and how much they are used, not just a subdir of GameData dir is there. Which also allow modder to know which part to take care of in a big set (example: if one engine in KWRocketry or NovaPunch is very appreciated, and one is almost never use, if both have issues, priorities comes by themselves)

I'm working on a configuration and plugins system that will allow modders to customize how ModStatistics reports certain things. I'll also be considering part usage, but that could very well become a plugin to ModStatistics rather than a full-fledged component of it, at least at first.

[Greys]

Majiir wants a post to segregate his posts so they don't molest eachother, therefore everybody go test the thing I made and tell me how broken it is http://forum.kerbalspaceprogram.com/threads/79175-Virgin-Kalactic-Responsive-Fuel-Burning-Generator-and-RequestResource%28%29-upgrade

[Majiir]

ModStatistics 0.0.3 has been released. The download link in the first post has been updated.

This is another beta release. Note that reports generated by prior versions are no longer accepted by the server. That said, the server can now properly parse and validate reports, so data collected with this version can be analyzed. I'll be working on a simple website to serve as proof of concept. The plugin still needs some work, but I'd like to see how the whole system works before I develop it much more.

Changes in this version:

• Crashes are now detected, and reports for the crashed session are sent at the next start. Crash reports send the same information as normal ones, but they indicate that a crash occurred.
  
• Several changes were made to the report format to make reports easier to parse, validate and process on the server.
  
• Fixed a minor discrepancy in how report end times are generated.
  
• Added assembly version to the user agent string.

[Greys]

I feel that with this new version my anonymously reported nonidentifiable usage statistics reports are more secure than ever before.

[codepoet]

I just updated to v0.0.3 and KSP hung on loading with module manager going bonkers on loading, logging:

NullReferenceException: Object reference not set to an instance of an object
  at UrlDir+.MoveNext () [0x00000] in <filename unknown>:0 

  at ModuleManager.ConfigManager.Update () [0x00000] in <filename unknown>:0 

repeatedly.

Could it be that ModuleManager is trying to parse the .cfg file for ModStatistics and getting into a mess?

Update:

Just so you know the attempt to load it worked. I have noticed in the past that an empty (or empty other than comments) .cfg file can cause module manager to get upset. Perhaps having added the id to the cfg file, it was then able to start the second time.

Edited June 4, 2014 by codepoet
added update

[blizzy78]

I remember having problems with that exact issue in my IRC plugin. It's probably best to rename the config file.

Edited June 4, 2014 by blizzy78

[Justin Kerbice]

I just updated to v0.0.3 and KSP hung on loading with module manager going bonkers on loading, logging:

NullReferenceException: Object reference not set to an instance of an object
  at UrlDir+.MoveNext () [0x00000] in <filename unknown>:0 

  at ModuleManager.ConfigManager.Update () [0x00000] in <filename unknown>:0 

repeatedly.

Could it be that ModuleManager is trying to parse the .cfg file for ModStatistics and getting into a mess?

Update:

Just so you know the attempt to load it worked. I have noticed in the past that an empty (or empty other than comments) .cfg file can cause module manager to get upset. Perhaps having added the id to the cfg file, it was then able to start the second time.

I have had the same* issue the first time I launch KSP (0.23), but the first exception comes from Active Texture Management mod., Second run was ok.

* except it is ModuleManager.ConfigManager.OnGUI ()

in my case.

Edited June 4, 2014 by Justin Kerbice

[Majiir]

Thanks for the (bug) report. I've sorted out a few issues with server parsing as well. Some of you probably have unsent reports stacked up, so those should be coming in soon. (The server was incorrectly rejecting them before.)

[codepoet]

I have had the same* issue the first time I launch KSP (0.23), but the first exception comes from Active Texture Management mod., Second run was ok.

* except it is ModuleManager.ConfigManager.OnGUI ()

in my case.

Surely the ultimate solution though is for a change to ModuleManager to make it handle .cfg files with no contents. (I had the problem myself with CLS trying to ship a blank .cfg file to remove config that had shipped with a previous version and was now obsolete.)

[Majiir]

Okay, some basic stats are up!

By plugin name: http://stats.majiir.net/

By plugin folder: http://stats.majiir.net/folders

Neither of these gives a very good picture, so I'm still thinking about how to best categorize plugins. This is mostly a test to make sure the data is structured well for querying. I'll probably have to change (again) how scene times are reported, but everything else seems fine.

I'll be adding various tables as I go, but let me know if there's anything in particular you'd like to see.

[EDIT] And here's crash info: http://stats.majiir.net/crashes

[EDIT 2] And here's average total loading time: http://stats.majiir.net/loading

So it turns out computing statistics is hard. Some of the more complex analysis will have to wait, especially because it's hard to work with such a small data set. I'm still not sure how to approach the mod categorization issue, so I'm still operating under the (flawed) assumption that plugins correspond to mods. The queries will work fine once that changes anyway.

Edited June 5, 2014 by Majiir

[JewelShisen]

Okay, some basic stats are up!

By plugin name: http://stats.majiir.net/

By plugin folder: http://stats.majiir.net/folders

Neither of these gives a very good picture, so I'm still thinking about how to best categorize plugins. This is mostly a test to make sure the data is structured well for querying. I'll probably have to change (again) how scene times are reported, but everything else seems fine.

I'll be adding various tables as I go, but let me know if there's anything in particular you'd like to see.

[EDIT] And here's crash info: http://stats.majiir.net/crashes

Awesome that so far I only have a single crash listed. Can I start spreading this among my user base yet and is there a way that mod authors can get access to see what our users are also using with our mods?

[Majiir]

Awesome that so far I only have a single crash listed. Can I start spreading this among my user base yet and is there a way that mod authors can get access to see what our users are also using with our mods?

I'd rather not put the current version into the wild. It has a few issues and limitations. I could possibly have the next version ready for a first run with the public (although I might code in an expiration date so it doesn't cause problems too far into the future).

Right now the only available information is what's on the site, but that will change daily. "This mod commonly goes with" is something that will be public. There might be stats available for author's viewing only, but I'm trying to keep all the stats as innocuous as possible, so there shouldn't be any need to lock certain information down.

[JewelShisen]

I'd rather not put the current version into the wild. It has a few issues and limitations. I could possibly have the next version ready for a first run with the public (although I might code in an expiration date so it doesn't cause problems too far into the future).

Right now the only available information is what's on the site, but that will change daily. "This mod commonly goes with" is something that will be public. There might be stats available for author's viewing only, but I'm trying to keep all the stats as innocuous as possible, so there shouldn't be any need to lock certain information down.

Awesome. Perhaps just let authors see the crash data as they are the only ones who really need it?