Lisias

GCMonitor logging discussion (split from Kolyphemus System)

Recommended Posts

15 minutes ago, linuxgurugamer said:

Right now it's running on a long term test to see if it is stable.

One thing that would make me very happy is a IPC call between GC Monitor and some external logging tool (something as kRPC). I'm pretty confident that cross-referencing KSP.log, Player.log and this log would help to diagnose some mysterious "Unplanned Instant Application Closures".

4 minutes ago, linuxgurugamer said:

I think I know what's going on with the rings

 What is the Instantiator?

It's the tool Kargantua's writer used to create the Kargantua's rings.
 

 

Share this post


Link to post
Share on other sites
57 minutes ago, Lisias said:
1 hour ago, linuxgurugamer said:

Right now it's running on a long term test to see if it is stable.

One thing that would make me very happy is a IPC call between GC Monitor and some external logging tool (something as kRPC). I'm pretty confident that cross-referencing KSP.log, Player.log and this log would help to diagnose some mysterious "Unplanned Instant Application Closures".

Ksp.log and player log contain the same info, one just has more than the other.

Would be very easy to have GC Monitor write to it's own log file, no need for any IPC calls

Share this post


Link to post
Share on other sites
Posted (edited)
9 hours ago, linuxgurugamer said:

Would be very easy to have GC Monitor write to it's own log file, no need for any IPC calls

It would add a lot more disk I/O to the process (imagine logging memory consumption each 1/4 of a second for hours). Writing to a pipe would delegate the burden to another process - what is more performatic nowadays as computers with lots of cores are affordable.

And also free the GcMonitor from having to implement logging strategies. It's up to the client to decide how it want to crunch the numbers. 


POSTEDIT:

Ksp.log and player log contain the same info, one just has more than the other.

Sorry, but this doesn't apply. I just fire up KSP, and didn't found any Memory and GC information on KSP.log neither Player.log. At least, until the Main Menu screen.

Edited by Lisias
new info

Share this post


Link to post
Share on other sites
1 hour ago, Lisias said:

It would add a lot more disk I/O to the process (imagine logging memory consumption each 1/4 of a second for hours). Writing to a pipe would delegate the burden to another process - what is more performatic nowadays as computers with lots of cores are affordable.

And also free the GcMonitor from having to implement logging strategies. It's up to the client to decide how it want to crunch the numbers. 

Umm, the disk IO would be minimal compared to everything else.  And it's a LOT simpler to add a bit of logging code than to implement IPC.

In fact, why would it need to be in a separate file?  Why not let Unity handle the io, and just let GC Monitor write to the standard Unity log file?  That would then eliminate the need to match up different log files, and it's very easy to pull out specific things from a log file using a filter such as grep.

1 hour ago, Lisias said:

It would add a lot more disk I/O to the process (imagine logging memory consumption each 1/4 of a second for hours). Writing to a pipe would delegate the burden to another process - what is more performatic nowadays as computers with lots of cores are affordable.

And also free the GcMonitor from having to implement logging strategies. It's up to the client to decide how it want to crunch the numbers. 


POSTEDIT:

Sorry, but this doesn't apply. I just fire up KSP, and didn't found any Memory and GC information on KSP.log neither Player.log. At least, until the Main Menu screen.

I mean, I can easily add the logging and have it write to the log file, AND I can have that start immediately

Share this post


Link to post
Share on other sites
Posted (edited)
21 minutes ago, linuxgurugamer said:

Umm, the disk IO would be minimal compared to everything else.  And it's a LOT simpler to add a bit of logging code than to implement IPC.

It should, but a IPC is even less. And save me from parsing the log after the event. Better. It would allow me to use some DevOps tools and monitor KSP on the fly

 

21 minutes ago, linuxgurugamer said:

In fact, why would it need to be in a separate file?  Why not let Unity handle the io, and just let GC Monitor write to the standard Unity log file?  That would then eliminate the need to match up different log files, and it's very easy to pull out specific things from a log file using a filter such as grep.

Well, you said yourself. :) Imagine the huge waste of processing by setting up a fswatch to monitor the file, then parsing it to extract the information I need, and then consuming it. ;) Now imagine this running 4 times per second.

Not to mention that I would need to write different tools for Windows.

A huge log file is convenient for post-mortem analysis. For runtime monitoring, not that much.

 

21 minutes ago, linuxgurugamer said:

I mean, I can easily add the logging and have it write to the log file, AND I can have that start immediately

At least on UNIX, I could setup a named pipe and get what I want the same way. But that should limit such activities for UNIX machines.

 

 

POST-EDIT: Uh... Dude... We hijacked the thread... :P Better move it somewhere else. :)

Edited by Lisias
typos. but you already knew it.

Share this post


Link to post
Share on other sites
18 minutes ago, Val said:

Thank you

1 hour ago, Lisias said:

It should, but a IPC is even less. And save me from parsing the log after the event. Better. It would allow me to use some DevOps tools and monitor KSP on the fly

 

Well, you said yourself. :) Imagine the huge waste of processing by setting up a fswatch to monitor the file, then parsing it to extract the information I need, and then consuming it. ;) Now imagine this running 4 times per second.

Not to mention that I would need to write different tools for Windows.

A huge log file is convenient for post-mortem analysis. For runtime monitoring, not that much.

 

At least on UNIX, I could setup a named pipe and get what I want the same way. But that should limit such activities for UNIX machines.

Your initial comment was about an ips to an external logging system.

I stil think it would be overkill.  The amount of IO at 4x a sec really isnt that much.  This is a case of keeping it simple to implement.  Log file (any, not just to the KSP/Unity log) is trivial.  Im not familiat with Windows monitors, but I wrote a mod a while ago called KSPLogger, which writes certain values to a file multiple times a sec.  This was for a streamer.  It works well, and no noticible impact on stream

Share this post


Link to post
Share on other sites
Posted (edited)
2 hours ago, linuxgurugamer said:

Your initial comment was about an ips to an external logging system.

It is what would solve "my" problem in the most convenient way for me. :)

 

2 hours ago, linuxgurugamer said:

I stil think it would be overkill.  The amount of IO at 4x a sec really isnt that much.  This is a case of keeping it simple to implement.  Log file (any, not just to the KSP/Unity log) is trivial.  Im not familiat with Windows monitors, but I wrote a mod a while ago called KSPLogger, which writes certain values to a file multiple times a sec.  This was for a streamer.  It works well, and no noticible impact on stream

Overkill? Well... Perhaps. The named pipe would solve the thing for me the same way.

Frankly? Thinking twice about the matter, I would be "happy" either way. And, of course, I didn't considered the developer side of the problem - where you are pretty right about.

Edited by Lisias
yeah. some bad grammars. =/

Share this post


Link to post
Share on other sites
Just now, Lisias said:

It is what solve "my" problem in the most convenient way for me. :)

 

Overkill? Well... Perhaps. The named pipe would solve the thing for me the same way.

Frankly? Thinking twice about the matter, I would be "happy" either way. And, of course, I didn't considered the developer side of the problem - where you are pretty right about.

Ok.  So I'll look into adding that.

I think I'll do it something like this:

Config file to enable/disable each of the following optional items

Each of the following items will be optional:

  1. Write all desired values to a file for instant monitoring.  This file will get overwritten every time
  2. Append values to a log file for long-term review.  This line will have a date/timestamp
  3. Write data to KSP log
  • Like 1

Share this post


Link to post
Share on other sites
1 hour ago, Lisias said:

It is what solve "my" problem in the most convenient way for me. :)

 

Overkill? Well... Perhaps. The named pipe would solve the thing for me the same way.

Frankly? Thinking twice about the matter, I would be "happy" either way. And, of course, I didn't considered the developer side of the problem - where you are pretty right about.

Are you aware that it apparently already does logging to the KSP log?  Not sure if it's useful.  I'm only looking at the source, not near my dev system right now

Share this post


Link to post
Share on other sites
21 hours ago, linuxgurugamer said:

Are you aware that it apparently already does logging to the KSP log?  Not sure if it's useful.  I'm only looking at the source, not near my dev system right now

Yeah, but that would make realtime monitoring cumbersome. I'm planning to "monitor" KSP the way I do my servers, DevOps style - not confuse it with DevWhoops style, a slightly different approach. :D 

Share this post


Link to post
Share on other sites

Will do in the morning! Thx!

Share this post


Link to post
Share on other sites

better yet, here is a full release:

 

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now