GCMonitor logging discussion (split from Kolyphemus System)

[Lisias]

15 minutes ago, linuxgurugamer said:

Right now it's running on a long term test to see if it is stable.

One thing that would make me very happy is a IPC call between GC Monitor and some external logging tool (something as kRPC). I'm pretty confident that cross-referencing KSP.log, Player.log and this log would help to diagnose some mysterious "Unplanned Instant Application Closures".

4 minutes ago, linuxgurugamer said:

I think I know what's going on with the rings

 What is the Instantiator?

It's the tool Kargantua's writer used to create the Kargantua's rings.
 

 

[linuxgurugamer]

57 minutes ago, Lisias said:

1 hour ago, linuxgurugamer said:

Right now it's running on a long term test to see if it is stable.

One thing that would make me very happy is a IPC call between GC Monitor and some external logging tool (something as kRPC). I'm pretty confident that cross-referencing KSP.log, Player.log and this log would help to diagnose some mysterious "Unplanned Instant Application Closures".

Ksp.log and player log contain the same info, one just has more than the other.

Would be very easy to have GC Monitor write to it's own log file, no need for any IPC calls

[Lisias]

9 hours ago, linuxgurugamer said:

Would be very easy to have GC Monitor write to it's own log file, no need for any IPC calls

It would add a lot more disk I/O to the process (imagine logging memory consumption each 1/4 of a second for hours). Writing to a pipe would delegate the burden to another process - what is more performatic nowadays as computers with lots of cores are affordable.

And also free the GcMonitor from having to implement logging strategies. It's up to the client to decide how it want to crunch the numbers. 

POSTEDIT:

Ksp.log and player log contain the same info, one just has more than the other.

Sorry, but this doesn't apply. I just fire up KSP, and didn't found any Memory and GC information on KSP.log neither Player.log. At least, until the Main Menu screen.

Edited June 13, 2018 by Lisias
new info

[linuxgurugamer]

1 hour ago, Lisias said:

It would add a lot more disk I/O to the process (imagine logging memory consumption each 1/4 of a second for hours). Writing to a pipe would delegate the burden to another process - what is more performatic nowadays as computers with lots of cores are affordable.

And also free the GcMonitor from having to implement logging strategies. It's up to the client to decide how it want to crunch the numbers. 

Umm, the disk IO would be minimal compared to everything else.  And it's a LOT simpler to add a bit of logging code than to implement IPC.

In fact, why would it need to be in a separate file?  Why not let Unity handle the io, and just let GC Monitor write to the standard Unity log file?  That would then eliminate the need to match up different log files, and it's very easy to pull out specific things from a log file using a filter such as grep.

1 hour ago, Lisias said:

It would add a lot more disk I/O to the process (imagine logging memory consumption each 1/4 of a second for hours). Writing to a pipe would delegate the burden to another process - what is more performatic nowadays as computers with lots of cores are affordable.

And also free the GcMonitor from having to implement logging strategies. It's up to the client to decide how it want to crunch the numbers. 

POSTEDIT:

Sorry, but this doesn't apply. I just fire up KSP, and didn't found any Memory and GC information on KSP.log neither Player.log. At least, until the Main Menu screen.

I mean, I can easily add the logging and have it write to the log file, AND I can have that start immediately

[Lisias]

21 minutes ago, linuxgurugamer said:

Umm, the disk IO would be minimal compared to everything else.  And it's a LOT simpler to add a bit of logging code than to implement IPC.

It should, but a IPC is even less. And save me from parsing the log after the event. Better. It would allow me to use some DevOps tools and monitor KSP on the fly

 

21 minutes ago, linuxgurugamer said:

In fact, why would it need to be in a separate file?  Why not let Unity handle the io, and just let GC Monitor write to the standard Unity log file?  That would then eliminate the need to match up different log files, and it's very easy to pull out specific things from a log file using a filter such as grep.

Well, you said yourself.  Imagine the huge waste of processing by setting up a fswatch to monitor the file, then parsing it to extract the information I need, and then consuming it.  Now imagine this running 4 times per second.

Not to mention that I would need to write different tools for Windows.

A huge log file is convenient for post-mortem analysis. For runtime monitoring, not that much.

 

21 minutes ago, linuxgurugamer said:

I mean, I can easily add the logging and have it write to the log file, AND I can have that start immediately

At least on UNIX, I could setup a named pipe and get what I want the same way. But that should limit such activities for UNIX machines.

 

 

POST-EDIT: Uh... Dude... We hijacked the thread...  Better move it somewhere else.

Edited June 13, 2018 by Lisias
typos. but you already knew it.

[Val]

Split from Kolyphemus System.

@linuxgurugamer

@Lisias

[linuxgurugamer]

18 minutes ago, Val said:

Split from Kolyphemus System.

@linuxgurugamer

@Lisias

Thank you

1 hour ago, Lisias said:

It should, but a IPC is even less. And save me from parsing the log after the event. Better. It would allow me to use some DevOps tools and monitor KSP on the fly

 

Well, you said yourself.  Imagine the huge waste of processing by setting up a fswatch to monitor the file, then parsing it to extract the information I need, and then consuming it.  Now imagine this running 4 times per second.

Not to mention that I would need to write different tools for Windows.

A huge log file is convenient for post-mortem analysis. For runtime monitoring, not that much.

 

At least on UNIX, I could setup a named pipe and get what I want the same way. But that should limit such activities for UNIX machines.

Your initial comment was about an ips to an external logging system.

I stil think it would be overkill.  The amount of IO at 4x a sec really isnt that much.  This is a case of keeping it simple to implement.  Log file (any, not just to the KSP/Unity log) is trivial.  Im not familiat with Windows monitors, but I wrote a mod a while ago called KSPLogger, which writes certain values to a file multiple times a sec.  This was for a streamer.  It works well, and no noticible impact on stream

[Lisias]

2 hours ago, linuxgurugamer said:

Your initial comment was about an ips to an external logging system.

It is what would solve "my" problem in the most convenient way for me.

 

2 hours ago, linuxgurugamer said:

I stil think it would be overkill.  The amount of IO at 4x a sec really isnt that much.  This is a case of keeping it simple to implement.  Log file (any, not just to the KSP/Unity log) is trivial.  Im not familiat with Windows monitors, but I wrote a mod a while ago called KSPLogger, which writes certain values to a file multiple times a sec.  This was for a streamer.  It works well, and no noticible impact on stream

Overkill? Well... Perhaps. The named pipe would solve the thing for me the same way.

Frankly? Thinking twice about the matter, I would be "happy" either way. And, of course, I didn't considered the developer side of the problem - where you are pretty right about.

Edited June 13, 2018 by Lisias
yeah. some bad grammars. =/

[linuxgurugamer]

Just now, Lisias said:

It is what solve "my" problem in the most convenient way for me.

 

Overkill? Well... Perhaps. The named pipe would solve the thing for me the same way.

Frankly? Thinking twice about the matter, I would be "happy" either way. And, of course, I didn't considered the developer side of the problem - where you are pretty right about.

Ok.  So I'll look into adding that.

I think I'll do it something like this:

Config file to enable/disable each of the following optional items

Each of the following items will be optional:

1. Write all desired values to a file for instant monitoring.  This file will get overwritten every time
2. Append values to a log file for long-term review.  This line will have a date/timestamp
3. Write data to KSP log

[linuxgurugamer]

1 hour ago, Lisias said:

It is what solve "my" problem in the most convenient way for me.

 

Overkill? Well... Perhaps. The named pipe would solve the thing for me the same way.

Frankly? Thinking twice about the matter, I would be "happy" either way. And, of course, I didn't considered the developer side of the problem - where you are pretty right about.

Are you aware that it apparently already does logging to the KSP log?  Not sure if it's useful.  I'm only looking at the source, not near my dev system right now

[Lisias]

21 hours ago, linuxgurugamer said:

Are you aware that it apparently already does logging to the KSP log?  Not sure if it's useful.  I'm only looking at the source, not near my dev system right now

Yeah, but that would make realtime monitoring cumbersome. I'm planning to "monitor" KSP the way I do my servers, DevOps style - not confuse it with DevWhoops style, a slightly different approach.  

[linuxgurugamer]

Take a look at this:

https://github.com/linuxgurugamer/MemGraph/releases/tag/1.1.0.3-logging

It's not done, but it is now logging to files.  Suggestion for improvements is welcome

[Lisias]

Will do in the morning! Thx!

[linuxgurugamer]

better yet, here is a full release: