[1.12.x] - Modular Kolonization System (MKS)

[RoverDude]

Oh I thought you said 50m not 50km   

 

Kontainers can no longer push to PL on their own.  Something with logistics capabilities are needed.  The new MPUs were added explicitly with push-only to help with remote miners

[Murdabenne]

2 hours ago, FirroSeranel said:

Because it's slow and requires lots and lots of clicking if you have multiple fuel tanks you want to transfer from/to. I can refuel a ship within maybe 30 seconds with GPO... or take about 10 minutes and hundreds of clicks, and wind up with sore hands (carpal tunnel and some other stuff) with stock. It is, at this point, a convenience mod, rather than a necessity, but frankly, many of the 1.2 changes are not very user-friendly in terms of sheer number of clicks required to do things. I logged issues in the beta for things like adding a global preference for Rigid Attachment, but it never made it. That's another feature I basically can't use. My hands just can't handle the hundreds of extra precise clicks needed during a vessel design process to turn it on for all parts.

For global, at least per-ship, in the VAB, I think EEX-redux does what you want with rigidity and autostrut

 

 

[DDay2021]

7 minutes ago, RoverDude said:

Kontainers can no longer push to PL on their own.  Something with logistics capabilities are needed.  The new MPUs were added explicitly with push-only to help with remote miners

Thanks.   This newfound understanding will bring a new challenge to upgrade the remote miner.   I appreciate the help and the wonderful mod.

[jd284]

17 hours ago, jd284 said:

@RoverDude, I found a problem with scavenging and background catch-up mechanics of converters.

Basically, ModuleLogisticsConsumer doesn't operate at all during catch-up so containers run dry and aren't filled from other nearby vessels until physics kicks in after catch-up. For instance, I can watch my kontainer get drained of its 20k MetallicOre in about 5 steps (which are 6 hour intervals right?), and then it stays empty for a few seconds. Then right after physics kicks in again, it fills up to about 12k MetallicOre. Meanwhile the other vessel's kontainer is full all that time until physics kicks in.

This seems to have gotten overlooked, so I wonder if I could get input from someone else if they have these same issues with local logistics during catch-up not working or if it's just me. And if you have these issues, do you also get the BaseConverter_GetDeltaTime exception spam in your log?

[Gilph]

3 hours ago, DDay2021 said:

It's 50km so local logistic won't work.  It seems the source of my issue is the lack of a logistics hub at the remote mining site.  Although I could swear I read that kontainers could push to PL,  I've only had 2 cups of coffee this morning and my memory could be off. 

In some cases I have been using the small MPL to push resources.  They are small and lighter than a logistics module if you only need to push drilled resources into PL.

[Gilph]

9 minutes ago, jd284 said:

This seems to have gotten overlooked, so I wonder if I could get input from someone else if they have these same issues with local logistics during catch-up not working or if it's just me. And if you have these issues, do you also get the BaseConverter_GetDeltaTime exception spam in your log?

As most can attest, I think I have run into my fair share of scavenging and catch up issues.  I don't have the log spam in my recent log, but I haven't been checking carefully.  Other than the scavenging issue using the wrong tanks issue I have in Github, I resolved my issues by always making sure that consumption rates never exceed my tank capacity for a 6 hour stretch.

I would need to understand your scenario a bit better. Since it looks like the storage is a large kontainer, does it get filled as much as it gets emptied in that 6 hour window?

[RoverDude]

21 minutes ago, jd284 said:

This seems to have gotten overlooked, so I wonder if I could get input from someone else if they have these same issues with local logistics during catch-up not working or if it's just me. And if you have these issues, do you also get the BaseConverter_GetDeltaTime exception spam in your log?

FixedUpdate does in fact get called - it's how all of the conversions work   If you're getting that spam then something is horribly broken...  Repro it with a USI and Stock only save, then if it persists, toss a save file on a github issue.

14 minutes ago, Gilph said:

In some cases I have been using the small MPL to push resources.  They are small and lighter than a logistics module if you only need to push drilled resources into PL.

I am making a new part specifically for this.

[Warezcrawler]

I am missing the old MKS-lite. Simpler resource production chains, simpler everything really.

So I've been working on simplifying it, and when good enough share it.

Currently I'm looking at simplifying the Kerbal types back to the stock 3 ones, while retaining the skill sets in a meaningful manner.

For that I have a question. I have tried to remove the many new types of kerbals added into the game by this mod, but currently it's like the old ones are enhanced, but the new ones still remain, somehow. 

@RoverDude What elements do I need to consider in order to simplify this aspect of MKS? Advise would be deeply appreciated.

This is what my last (failed) try looked like (MM):

Spoiler

//Extend skills of base characters of the game - avoid having 13 types
@EXPERIENCE_TRAIT[Pilot]:AFTER[MKS]:FOR[GTI_SimpleMKS]
{
    EFFECT
    {
        name = FundsBoost
    }
    EFFECT
    {
        name = ScienceBoost
    }
    EFFECT
    {
        name = RepBoost
    }
    EFFECT
    {
        name = LogisticsSkill
    }
    EFFECT
    {
        name = ExplorerSkill
    }
}

@EXPERIENCE_TRAIT[Engineer]:AFTER[MKS]:FOR[GTI_SimpleMKS]
{
    EFFECT
    {
        name = FundsBoost
    }
    EFFECT
    {
        name = ExConstructionSkill
    }
    EFFECT
    {
        name = ConstructionSkill
    }
}

@EXPERIENCE_TRAIT[Scientist]:AFTER[MKS]:FOR[GTI_SimpleMKS]
{
    EFFECT
    {
        name = BotanySkill
    }
    EFFECT
    {
        name = ScienceBoost
    }
    EFFECT
    {
        name = GeologySkill
    }
    EFFECT
    {
        name = FundsBoost
    }
    EFFECT
    {
        name = AgronomySkill
    }
    EFFECT
    {
        name = RepBoost
    }
    EFFECT
    {
        name = MedicalSkill
    }
}

//Remove MKS added Kerbal Types
!EXPERIENCE_TRAIT[Kolonist]:AFTER[MKS]:FOR[GTI_SimpleMKS]{}
!EXPERIENCE_TRAIT[Miner]:AFTER[MKS]:FOR[GTI_SimpleMKS]{}
!EXPERIENCE_TRAIT[Technician]:AFTER[MKS]:FOR[GTI_SimpleMKS]{}
!EXPERIENCE_TRAIT[Mechanic]:AFTER[MKS]:FOR[GTI_SimpleMKS]{}
!EXPERIENCE_TRAIT[Biologist]:AFTER[MKS]:FOR[GTI_SimpleMKS]{}
!EXPERIENCE_TRAIT[Geologist]:AFTER[MKS]:FOR[GTI_SimpleMKS]{}
!EXPERIENCE_TRAIT[Farmer]:AFTER[MKS]:FOR[GTI_SimpleMKS]{}
!EXPERIENCE_TRAIT[Medic]:AFTER[MKS]:FOR[GTI_SimpleMKS]{}
!EXPERIENCE_TRAIT[Quartermaster]:AFTER[MKS]:FOR[GTI_SimpleMKS]{}
!EXPERIENCE_TRAIT[Scout]:AFTER[MKS]:FOR[GTI_SimpleMKS]{}

 

[TauPhraim]

10 minutes ago, RoverDude said:

FixedUpdate does in fact get called - it's how all of the conversions work

We've seen cases when some of the callbacks were not called during catchup, at least not right when catchup begins (cf a recent-ish fix about recyclers and catchup). To me it is possible FixedUpdate would suffer from the same issue.

[DDay2021]

12 minutes ago, RoverDude said:

I am making a new part specifically for this.

I'm interested to see your creation. Earlier, I was thinking that a part which is similar to NecroBones probe cores in MRS and SpaceY, coupled with the logistics capability, would be perfect for this role. 

[DStaal]

5 minutes ago, DDay2021 said:

I'm interested to see your creation. Earlier, I was thinking that a part which is similar to NecroBones probe cores in MRS and SpaceY, coupled with the logistics capability, would be perfect for this role. 

My thought is what the part really is is a beacon: It designates the exact location of the ship, and where to hook up for resource transfer.  Then you can tell you automated crawler (or send your manned pickup rover) to beacon #XZY to pick up $K units of $Resource$.

[RoverDude]

That's pretty much my thought, @DStaal - hence free push, but to pick it up and deliver (pull) is a different kettle of fish.

Tho what I am making is bigger than just a probe core   

18 minutes ago, TauPhraim said:

We've seen cases when some of the callbacks were not called during catchup, at least not right when catchup begins (cf a recent-ish fix about recyclers and catchup). To me it is possible FixedUpdate would suffer from the same issue.

Those were cases where I had an intentional delay though (most of which I dropped).  FixedUpdate is part of a MonoBehavior.

[Deus Zed Machina]

5 hours ago, Gilph said:

Hi, been doing this very thing over the last few days.  Had about a 70 ton ferry I used between Kerbin and Duna that I have slowly been reducing over the past few days.  Can post some screenshots over the weekend if you are interested.

That sounds cool! Version 1.0 of what I'm calling Aristaeus is already nearing completion, but an example craft would certainly help me figure out whether or not I'm doing this right.

[TauPhraim]

3 minutes ago, RoverDude said:

Those were cases where I had an intentional delay though (most of which I dropped).  FixedUpdate is part of a MonoBehavior.

I don't mean things like a stored "lastCheck" variable, I mean PostProcess (to name it) was not called during the beginning of catchup. It's very possible that a similar thing could be happening: for example KSP instanciates converter modules first, they start doing their things, and logistics modules are instanciated next. But in the meantime converters have a few ticks running without logistics.

[RoverDude]

Just now, TauPhraim said:

I don't mean things like a stored "lastCheck" variable, I mean PostProcess (to name it) was not called during the beginning of catchup. It's very possible that a similar thing could be happening: for example KSP instanciates converter modules first, they start doing their things, and logistics modules are instanciated next. But in the meantime converters have a few ticks running without logistics.

What is happening is very likely an issue with LastCheck since that's the guard clause (PostProcess is called at the end of FixedUpdate from the converters).  Since we use FiixedUpdate, they will always be in lockstep (which is why FixedUpdate is used for precision).

In this case what it looks like is that after the first tick, since LastCheck is being set to universaltime, you're missing the remainder of your catchup chunks.  So for these, it makes sense to use an increment vs a straight set.

[DStaal]

18 minutes ago, RoverDude said:

That's pretty much my thought, @DStaal - hence free push, but to pick it up and deliver (pull) is a different kettle of fish.

Tho what I am making is bigger than just a probe core  

I'll admit, I was thinking if it's just a beacon it could look like a radial-mounted Sunflower light with a built-in KAS pipe.  But I suspected you were likely to have some sort of packaging and pickup designed into the part.

[TauPhraim]

22 minutes ago, RoverDude said:

What is happening is very likely an issue with LastCheck since that's the guard clause (PostProcess is called at the end of FixedUpdate from the converters).  Since we use FiixedUpdate, they will always be in lockstep (which is why FixedUpdate is used for precision).

In this case what it looks like is that after the first tick, since LastCheck is being set to universaltime, you're missing the remainder of your catchup chunks.  So for these, it makes sense to use an increment vs a straight set.

I'm not sure callbacks being as solid as that would explain that old recycler problem (or jd284's observations). This would rely on the game instanciating all of a vessel's modules first, before starting ticking them (not sure this is even possible in Unity). But I agree that logistics might also need something regarding lastCheck In USI-LS, you are able to detect catchup (ModuleLifeSupportSystem), maybe you could do something similar.

[jd284]

52 minutes ago, Gilph said:

As most can attest, I think I have run into my fair share of scavenging and catch up issues.  I don't have the log spam in my recent log, but I haven't been checking carefully.  Other than the scavenging issue using the wrong tanks issue I have in Github, I resolved my issues by always making sure that consumption rates never exceed my tank capacity for a 6 hour stretch.

I would need to understand your scenario a bit better. Since it looks like the storage is a large kontainer, does it get filled as much as it gets emptied in that 6 hour window?

The tank capacity on both vessels is good for about 24 hours of production. It empties during catch-up and doesn't get filled again until catch-up is completed and normal physics resumes.

53 minutes ago, RoverDude said:

FixedUpdate does in fact get called - it's how all of the conversions work   If you're getting that spam then something is horribly broken...  Repro it with a USI and Stock only save, then if it persists, toss a save file on a github issue.

The exception spam went away on its own for some reason, so disregard that.

I also added some more info to the print statement I have as first line of ModuleLogisticsConsumer.FixedUpdate, and I think I found the issue.

The FixedUpdate function is called with Planetarium.GetUniversalTime() already set to "now", not the time it's catching up. For instance, in my save the vessel in question has lastUT = 15879278 whereas now it's 16014157. So I'd expect it to step through this time interval (about 37.5 hours?) during catch-up. But the first call to catch-up is with Planetarium.GetUniversalTime()=16014156. So the logistics only get called every 5 seconds of "real time", which probably corresponds to days and days of catch-up time.

As an idea based on this I just removed the "lastCheck" guard clause so that logistics would run every tick. And lo and behold, the container didn't empty, logistics worked, and everything was fine. It's not nice for the framerate though, I suppose.

So is there a different time it should be checking instead of Planetarium.GetUniversalTime(), for it to work with catch-up where 5 seconds correspond to an eternity? Some other timestamp or a flag that allows logistics to run every tick during catch-up but only every 5 seconds during normal operation?

[RoverDude]

30 minutes ago, TauPhraim said:

I'm not sure callbacks being as solid as that would explain that old recycler problem (or jd284's observations). This would rely on the game instanciating all of a vessel's modules first, before starting ticking them (not sure this is even possible in Unity). But I agree that logistics might also need something regarding lastCheck In USI-LS, you are able to detect catchup (ModuleLifeSupportSystem), maybe you could do something similar.

Trust me on this one (source:  Wrote the stock resource system)

14 minutes ago, jd284 said:

The tank capacity on both vessels is good for about 24 hours of production. It empties during catch-up and doesn't get filled again until catch-up is completed and normal physics resumes.

The exception spam went away on its own for some reason, so disregard that.

I also added some more info to the print statement I have as first line of ModuleLogisticsConsumer.FixedUpdate, and I think I found the issue.

The FixedUpdate function is called with Planetarium.GetUniversalTime() already set to "now", not the time it's catching up. For instance, in my save the vessel in question has lastUT = 15879278 whereas now it's 16014157. So I'd expect it to step through this time interval (about 37.5 hours?) during catch-up. But the first call to catch-up is with Planetarium.GetUniversalTime()=16014156. So the logistics only get called every 5 seconds of "real time", which probably corresponds to days and days of catch-up time.

As an idea based on this I just removed the "lastCheck" guard clause so that logistics would run every tick. And lo and behold, the container didn't empty, logistics worked, and everything was fine. It's not nice for the framerate though, I suppose.

So is there a different time it should be checking instead of Planetarium.GetUniversalTime(), for it to work with catch-up where 5 seconds correspond to an eternity? Some other timestamp or a flag that allows logistics to run every tick during catch-up but only every 5 seconds during normal operation?

Already sorted this in my dev code

[jd284]

11 minutes ago, RoverDude said:

Trust me on this one (source:  Wrote the stock resource system)

Already sorted this in my dev code

Cool! Would you mind pushing the change to github so I can get this fixed without killing my framerate?

[RoverDude]

Sure, incoming shortly.  Will be in USITools - let me know if it works out

[jd284]

15 minutes ago, RoverDude said:

Sure, incoming shortly.  Will be in USITools - let me know if it works out

Yes it fixed it logistics not running during catch-up, but had the same bad framerate as when I was having it check every tick. I believe it's now running up lastCheck from 0 to the current time and performing logistics every tick along the way. And then after a few minutes my framerate got better again, I guess it reached 16014157 then.

So it's not quite fixed yet.

I think you should probably add something to set lastCheck after loading. Though just initializing it to Planetarium.GetUniversalTime() broke logistics again for me...

[RoverDude]

tbh it probably just needs to persist will poke at it

 

[jd284]

Well, I tried initializing lastChange to vessel.lastUT, but that way ModuleLogisticsConsumer was done catching up before the converters even started running, and when they did the container emptied with no logistics to fill it back up. So there's still some desynchronization between converters and logistics somehow.

At least in my case it was sufficient to slow it down to 1/20th of the catchup speed with the following:

            if (lastCheck < ResourceUtilities.FLOAT_TOLERANCE * 10)
                lastCheck = vessel.lastUT;
                        
            var planTime = Planetarium.GetUniversalTime ();
            if (Math.Abs(planTime - lastCheck) < LogisticsSetup.Instance.Config.LogisticsTime)
                return;                 

            lastCheck = Math.Min(lastCheck+ _maxDelta/20, planTime);

But that "/20" there is pretty arbitrary and probably only worked by accident.

But persisting the field should be about equivalent to setting it to vessel.lastUT so I don't think that on its own will help. There's still the problem with converters only starting to run some time after ModuleLogisticsConsumer gets done with its catch-up (at least without that "/20" in my case).

Another idea might be to use the actual converter's lastUpdateTime property if that is somehow accessible. Since if you get the delta time using that, you would definitely be synched to converters. At least for parts that have ModuleResourceConverter_USI...

[RoverDude]

@jd284 - try what I just checked in.

 

(Actually - hold off a second...)

 

(ok now try!)

Edited March 24, 2017 by RoverDude