Efficiency of ISRU has dropped?

[Alshain]

Do the drills shut off when the ore tank fills?  Is it possible the ore tank is filling faster than the converter removes it? (I'm just shooting in the dark here, doesn't look like he has much room for ore)

[monstah]

Guys, guys. This is just personal bickering by now. Some people like stock better, other like mod X or mod Y better, but all this caps lock going around and calling something crap or saying "I would say something personally nasty" is just detracting from the point which was someone asking for help. Now, the author of the mechanic is here, offering his help, take this somewhere else would ya?

[Snark]

2 hours ago, Alshain said:

Do the drills shut off when the ore tank fills?  Is it possible the ore tank is filling faster than the converter removes it? (I'm just shooting in the dark here, doesn't look like he has much room for ore)

Nope, they shut off while the tanks still have oodles of space left.  I'll try RoverDude's diagnostic suggestions as soon as I have a chance, and will post the results here.

[Plusck]

I too have had problems with my drill ships shutting off. However it is certainly not something that happens all the time, and I don't really know why. I don't use fuel cells (but I think I'm going to regret sending a miner/converter to the Jool system without one).

My ships' designs are all based on much the same core set-up. From memory, the last times they shut down while absent were: on Ike on a 7% ore resource, and on an asteroid. On Mun and Minmus, ore concentrations are lower and I don't recall them ever shutting off.

So if I combine that with the hints above, it may well be that my drills were filling the ore tanks too quickly. Drills will happily spin up to 100% efficiency while tanks are prohibited (useful for smaller asteroids so as not to waste asteroid mass) but could it be that this only works while the ship has focus?

 

As for the stock mining system, I have no complaints about it. The instant survey could maybe benefit from a slight time-based requirement (e.g. you have to click the survey button twice with at least one orbital interval) for the role-players amongst us, but since the result of the survey is so general anyway that doesn't bother me. Any other problems I've had with the stock system have been entirely my own doing: a great one is my habit of forgetting to put antennae on my survey craft. :/

Edited January 10, 2016 by Plusck

[RoverDude]

FYI - your asteroid drill mass loss was actually fixed in 1.0.5

What *MIGHT* be happening (and again, this is why I am asking a lot of specific questions) is that in the physics load your ship is 'hopping' just enough that it thinks it is flying, and shuts down the drill.  This is especially prevalent on uneven terrain/slopes.  Option B is intermediary storage based on the timewarp lerp - hence why I asked how it behaves as you go to max warp.  

[Plusck]

58 minutes ago, RoverDude said:

FYI - your asteroid drill mass loss was actually fixed in 1.0.5

Odd, because I tested this very recently. While the drills were at under 100% efficiency, KER was telling me that the combined mass of my ship (i.e. ship plus asteroid) was slowly dropping. Once they got to 100% efficiency, the mass stayed constant as the ore tanks filled. This is with build 1028.

I'm pretty sure I have pics somewhere because I was wanting to work out the exact rate of mass loss (i.e. to see if it was really 90% loss at 10% efficiency, or whatever). I could find them if that helps.

 

edit: uploaded a couple of screenshots here: http://imgur.com/a/MwPDZ

Over 6 seconds, mass dropped by 0.12t while at around 50% drill efficiency. Ore increased by 11 in this time. At 50% efficiency, therefore, half the mass is lost.

The asteroid was about 95% ore at this stage, the engineer was (I'm pretty sure) level 2, and using the wiki's info of 0.075 x concentration x multiplier, that gives 3.8 per second. Again this is coherent with slightly over 50% efficiency for the ore production rate, and the other 50% causing disappearing mass. Rounding errors are mostly due to the fact that the time is only measurable to the nearest second, and the KER info on mass has only 2 decimal figures.

Edited January 10, 2016 by Plusck

[davidpsummers]

23 hours ago, RoverDude said:

Looking at the OP - you're pretty much doing every single thing to cripple the stock resource system and guarantee failure.

You are not prospecting (tools are provided for this, ignore at your own risk).  The prospecting game is a mini-game unto itself.

Not sure what you mean about prospecting.  I did the orbital survey and when I land in a region of high concentration, I get an ore concentration that is acceptable.  

23 hours ago, RoverDude said:

You are not cooling your drills (this has a significant impact, take a peek at what your thermal efficiency is - I bet it's below 20%)

This is likely my real problem, since I'm not getting the fuel I should from the ore available.

23 hours ago, RoverDude said:

You are on an unmanned mission.  By the way, if you correct the first two, the latter is still a barrier but likely only about 50% of your problem.  It makes unmanned mining ops possible, but more in line with unmanned missions where you don't have the advantage of someone clearing jams, monitoring the hardware, etc. 

If your drill is shutting off between runs, there's likely a good reason - provide an example craft, and I can likely tell you why.  Hint:  If you're dropping half a dozen large drills and an ISRU on a vessel and trying to power it with a couple of solar panels and a 100-EC battery, you are going to have issues.

It doesn't shut down.  I did put in a picture on my original post.

[Snark]

18 hours ago, RoverDude said:

Ok - let's start.  Does the drill remain operational upon your return or is it shut down?

Sometimes it's shut down, sometimes it's still running.  I haven't been able to figure out any consistent correlation as to what conditions cause it to be one or the other-- seems to be random, as far as I can tell.

18 hours ago, RoverDude said:

 If shut down, what does it say in the status message?

(edit for clarity - what I mean by status message is if you right click on a shut down drill, there will be a message).

Whoa, there's a status message?  Huh, did not know that.  Been playing with these things the better part of a year and had no idea that it was there (and wouldn't have guessed that there'd be one).

Anyway, grrr.  Naturally now that I'm trying to reproduce the problem, I can't get it to do it.  I've just now tried switching away to another ship, warping a day or so, coming back... it's still running.  Tried various combinations of doing that when I have a full electric charge load, when it's low, when I've got lots of ore in the tanks, when I have little ore in the tanks... no dice.  Just tried like a dozen times, couldn't get it to do it.  So I'm guessing that in the set of saves I currently have, there's something about the situation that's not triggering the bug and that's why I can't reproduce it now.  All I know is, with this same ship on this same moon, I've been running into this problem again and again and again; just can't make it do it right now.  No idea why.

(Side note:  As long as we're on the topic of mining in absentia, I have a bugfix-or-feature-request, don't know which it is.  If I have a mining ship, and it's both extracting ore and running ISRU to convert it to fuel, and the ISRU can convert faster than the drills can mine, then if I go away for a long time, I'd expect to see full fuel tanks when I come back.  But that's not what happens.  Instead, the miner continues to mine ore in my absence until the ore tanks fill up, but none of that gets converted, and when I come back (even if it's a year later), the only progress that it has made is that if filled up the ore tanks without ISRU conversion thereof.  Would really be nice if that could be fixed.  As it currently stands, this bug makes it essentially impossible to have a mining ship that runs usefully in the background; it has to be babysat.  Given how long it can take to fill up a mining ship, that's a significant impact on player's experience of the game.)

18 hours ago, RoverDude said:

Do all drills shut down, or just some?

All of them.

18 hours ago, RoverDude said:

If you warp manually, will it work happily at max warp?

Yes.  As long as I'm controlling the ship, it never runs into any problem, including warping at max warp.  Nothing shuts down.

The only time I ever experience the mysterious shutdown is if I switch away to another ship somewhere, and then come back after some reasonable length of time has passed (i.e. not right away).  And it doesn't do it every time even in that case-- just sometimes.

Edited January 10, 2016 by Snark

[RoverDude]

2 hours ago, davidpsummers said:

Not sure what you mean about prospecting.  I did the orbital survey and when I land in a region of high concentration, I get an ore concentration that is acceptable.  

This is likely my real problem, since I'm not getting the fuel I should from the ore available.

It doesn't shut down.  I did put in a picture on my original post.

Prospecting - the orbital scanner simply tells you areas that are potentially worthy of looking into.  Those numbers represent an average for that biome.  If you want better rates you *REALLY* need to use the surface scanner and ground truth a biome or two, then use the NBS to find potential spots.  You are absolutely shooting yourself in the foot if you only depend on the orbital scan.

The shut down discussion was for another poster, not you.  

1 hour ago, Snark said:

The only time I ever experience the mysterious shutdown is if I switch away to another ship somewhere, and then come back after some reasonable length of time has passed (i.e. not right away).  And it doesn't do it every time even in that case-- just sometimes.

Because it is intermittent it is very likely a physics hop, especially since I saw you were on a slope.  A way to test this is to hop back and forth with a minimum time lapse.  Since you were able to operate at max warp (which exceeds the default lerp cap) I am inclined to think your issue is physics hop.

[Plusck]

Now I think about it I do recall coming back to my Ike miner (the one that shut down) and finding it about 8 feet up the air. It settled in a few seconds without any adverse effects (i.e. the game realised it shouldn't be up in the air and plonked it back down on the surface). If that was the cause (I honestly can't remember whether that was the time it had shut down), I take it there is nothing that can be done about it?

[RoverDude]

Not outside of modifying stock code, no.  

[Snark]

2 hours ago, RoverDude said:

Because it is intermittent it is very likely a physics hop, especially since I saw you were on a slope.  A way to test this is to hop back and forth with a minimum time lapse.  Since you were able to operate at max warp (which exceeds the default lerp cap) I am inclined to think your issue is physics hop.

Thanks for the tip, good to know-- will keep an eye out for it in the future.  (Also good to know about the status message.)

I get that the physics-hop problem is part of the engine, and may not be easy for Squad to fix.  Would love it if the 1.1 update to Unity 5 might provide better physics that works around the problem.

However, even if no physics-based solution is forthcoming:  it would be nice if the resource mining could be fixed so that it's not so sensitive to momentary flickers of ground contact.  For example, if the ship loses "landed" status, don't immediately turn everything off-- give it a grace period of a little while (even a few seconds), where it resumes mining if it re-establishes contact with the ground.

The problem is exacerbated by the (presumably unrelated) problem of mining + ISRU not working in concert when the ship's in the background.  As I mention in a previous post in this thread:  if I leave a ship with both drills and ISRU going, with ISRU capable of keeping up, I'd expect it to run continuously and fill up all fuel tanks in the background, but instead it just fills up the ore tanks and stops any further ore processing.  The presence of this counterintuitive behavior increases the player's perception of the land-hop bug's severity.  The player comes back to the ship, expecting to find drills running and all tanks full, and instead sees stopped drills and mostly-empty tanks, and assumes that the one is the result of the other when in reality they're unconnected.  It adds FUD to the situation.

So, I guess my main asks as a player are:

1. top priority:  fix the drill+ISRU bug
2. next priority:  do something to mitigate drill interruption due to physics hop

Anyway, thanks for taking the time to look at this!

[RoverDude]

1 hour ago, Snark said:

The problem is exacerbated by the (presumably unrelated) problem of mining + ISRU not working in concert when the ship's in the background.  As I mention in a previous post in this thread:  if I leave a ship with both drills and ISRU going, with ISRU capable of keeping up, I'd expect it to run continuously and fill up all fuel tanks in the background, but instead it just fills up the ore tanks and stops any further ore processing.  The presence of this counterintuitive behavior increases the player's perception of the land-hop bug's severity.  The player comes back to the ship, expecting to find drills running and all tanks full, and instead sees stopped drills and mostly-empty tanks, and assumes that the one is the result of the other when in reality they're unconnected.  It adds FUD to the situation.

As noted, that's a separate issue and likely has a just as simple answer.  Usual deal, start with the ship in question, watch how it behaves while focused as warp ramps up, look at the status messages.  

[davidpsummers]

Well, I can confirm that the problem was heat.  Radiators seemed to solve the problem.  It was working OK before, so I think it was a change in going from 1.0.4 to 1.0.5?

As to using orbital scans, how much does it very in a region?  I've landed maybe 1/2 dozen times in my current favorite spot and always gotten .003 to .005 ore/sec.  I might be willing to spend some time with the M-4435 Narrow Band Scanner, but I would rather just drop my a refueler a number of weeks early than have to rove around and look for a psot.

[RoverDude]

You may want to read the patch notes... there were significant ISRU changes in 1.0.5.

Orbital scans show the average off of limited data points.  As noted before, this is just to give you a rough idea of where to look further with an NBS, and an NBS can be done from orbit if so desired ( @Geschosskopf has a tutorial he linked above).  And every save is going to be different.

Bottom line, if you are bypassing the prospecting part, you're effectively just making a very rough guess.

[Geschosskopf]

10 hours ago, davidpsummers said:

As to using orbital scans, how much does it very in a region?  I've landed maybe 1/2 dozen times in my current favorite spot and always gotten .003 to .005 ore/sec.  I might be willing to spend some time with the M-4435 Narrow Band Scanner, but I would rather just drop my a refueler a number of weeks early than have to rove around and look for a psot.

0.003-0.005 ore/sec is not a very good rate at all.  You can get 10x better than that if you prospect and find good ore.  By "good", I'm talking ore concentrations around 8% but even 5% should give you several times the rate you're getting now.

If you're worried that prospecting requires virtually doing the Elcano Challenge, don't be.  This is because ore distribution on a planet is tied to its biome map.  Each biome has an average ore concentration that's essemtially constant for each patch of that biome no matter where it is on the planet.. So the key to prospecting is to learn the average ore concentrations of the biomes.  Once you know that, you can eliminate most of the planet and focus only on the better biomes.  And there, you're just looking for localized hotspots within the biome's overall avrage concentration that are 1) on reasonably flat terrain and 2) are conveniently located for your needs (usually meaning not too far off the equator, depending on the planet's gravity).  Thus, most rover trips are going to be fairly short, just getting the details of an area you already know is worth looking into in detail.  And if the planet has low gravity (like Minmus), you don't use a rover, you use a little lander with enough fuel to hop between several biomes.

However, you MUST have access to an in-game biome map to do all this.  There are 2 ways to do this.  In stock, you can toggle a biome map overlay in the map view via the ALT-F12 menu, which you might do already anyway when hunting science in the various biomes.  Or you can use the SCANsat mod to make your own interactive in-game biome map  (and terrain map, and average ore concentration map, too)..  Of these, SCANsat is by far the most convenient because it will show you average ore concentrations in all biomes once you spend a month or 2 mapping the planet from orbit.  This removes all the guesswork from prospecting.  You can immediately see which biomes are good, which are bad, where these are in relation to where you want to build your base (usually near the equator), and generally how flat the terrain probably is.  Thus, you can immediately create a short list of potential base locations based on ore, flatness, and general location, and only have to bother checking those places out.  And then you're just looking for the flattest landing zones and maybe localized hotspots.  No long rover trips.

With stock, you have more work to do because you get zero ore concentration info until you actually land on biomes.  So first you compare the survey overlay to the biome overlay.  The best-looking spots on the survey overlay are usually big enough to cover parts of several adjacent biomes, so you need to visit all those biomes to figure out what's really going on.  You might have to do this for a couple of regions, too.  That's where the bulk of the prospecting work comes in, and it may involve a long rover trip or 2, but only on high-gravity, airless planets.  If the planet has low gravity, use a biome-hopping lander.  If the planet has an atmosphere, fly an airplane.  Only then you can get into the final site selection, which is much less work.  In contrast, SCANsat lets you jump straight to final site selection.

6 hours ago, RoverDude said:

 @Geschosskopf has a tutorial he linked above). 

Thanks for the endorsment

The link is in my signature.

Edited January 11, 2016 by Geschosskopf

[RoverDude]

FYI @Geschosskopf thats not quite right RE biomes.

Without ground truthing, you get biome data on the NBS and it includes the biome average.  So a common practice is to have an NBS on your same orbital satellite that did the initial survey, and use it to narrow down the biomes.  Since the NBS also includes terrain data it's pretty easy to get a feel for what biomes are where, and to scout out areas worth more investigation before your first rover hits the ground.

(For the thread in general, this is why we don't consider it an insta-scan - it's a mini game unlike anything offered up by prior mods).

[Mikki]

... With at least a three star experienced engineer on the vessel and an average ore field abundance around 7% one large drill looks absolutly efficient to make a normal mining task.

1. fill the ore tanks
2. generate fuel
3. again fill the ore tanks
4. lift off, move along, whatever
5. done

A picture of an off-world stockparts ISRU lander, specs for Pol, Bop, Minmus, even Vall, Gilly, Dres, Ike... it can supply a large ship in lower orbit with ore/fuel:
(Pic taken after starting the drill, takes some time to achieve 100% efficiency.)


 

Edited January 11, 2016 by Mikki
typo:D

[monstah]

58 minutes ago, RoverDude said:

(For the thread in general, this is why we don't consider it an insta-scan - it's a mini game unlike anything offered up by prior mods).

You know, I never quite got how that system worked, and then I starded a career save, a new career save, and another one, and never got to ISRU again.

I still like the SCANSat method better for the first scan pass, but I get what you mean about the minigame.

[Geschosskopf]

1 hour ago, RoverDude said:

FYI @Geschosskopf thats not quite right RE biomes.

Without ground truthing, you get biome data on the NBS and it includes the biome average.  So a common practice is to have an NBS on your same orbital satellite that did the initial survey, and use it to narrow down the biomes.  Since the NBS also includes terrain data it's pretty easy to get a feel for what biomes are where, and to scout out areas worth more investigation before your first rover hits the ground.

I find this to be of extremely limited value, which is why I never mention it.  There are several major difficulties with using the NBS in orbit that drastically devalue it for me, to the point that I view it as merely an adjunct to the surface scanner and use SCANsat to fill the NBS's orbital role.  These difficulties are:

1. The NBS shows snapshots instead of video.  When moving over the ground at orbital speeds (except at very low-gravity worlds),  you have to update very frequently or you'll miss seeing the next patch of terrain.  This is bad because....
2. The NBS doesn't save any info so you have to jot down paper notes.  The faster you're moving, the less time you have for writing or even mousing over the NBS snapshot for something to write down to begin with.  And if you miss a spot, there's no real going back because you're so focused on the NBS display that you don't know where you are in any detail on the in-game map view.  Plus, you're in a non-synchronous orbit so it might be weeks before you fly over that spot again.
3. You can try taking a bunch of screenshots of clusters of NBS snapshots, but you can only see them when you exit the game.  And your cursor doesn't show up in the screenshots so you don't know where on the NBS display the cursor was when it showed the text in the display.  And even worse, you have no idea where this is on the planet because the game doesn't show you lat/lon on the map view when mousing over terrain.  So this is pretty much useless.
4. Because of the above, and because the NBS unlocks after all the other ore-related parts, and because the NBS competes for science points with other relatively high-end parts that you'll use way more often, I often go to Duna before I unlock the NBS and use SCANsat to fill its orbital role because SCANsat does a way better job.

EDIT (forgot to say):  Because of the problems of using the NBS in orbit, therefore, I only use it on rovers or low-flying, slow-flying aircraft and biome-hoppers.

My opinion of the NBS would change A LOT if it showed video and saved its data for future reference.  You know, building up an easily accessible, interative, in-game map that could be studied at leisure even from the space center while not flying a ship.  Even if it drew this as an overlay in the map view instead of a pop-up window like SCANsat, it would be great.

But even with that, stock still needs a much better way to have in-game access to the biome map, not just for ore but for science, too.  It's ridiculous that both of these major game functions are tied so heavily to biomes but you can only see a biome map by using ALT-F12 with all tis connotations of cheating.  And with the NBS, you need to see both ore and biomes simultaneously, or at least be able to toggle between them easily without moving your mouse from where it is on the planet's surface.  I certainly hope the interface improvements coming in 1.1 address this.

Edited January 11, 2016 by Geschosskopf

[RoverDude]

tbh I've had no issues using it in high orbits, but then that's probably just a matter of being used to it's mechanics (which I should be - given I wrote it ;))  the snapshot is a fairly large area, and the biomes end up being clearly marked on any given snapshot.  I usually take a snapshot of an interesting area, then look at it and evaluate it while I orbit over to the next spot.

[davidpsummers]

It sounds like a NBS scanner has to be on the mining lander rather than on a separate satellite?

[RoverDude]

No, works perfectly fine either way.  I personally put them on both.  Also - flags show up on the NBS which can be super handy when trying to zero in on a nice landing site and you have no handy terrain features to get your bearings off of.

[ShotgunNinja]

@Snark @RoverDude Hi guys. A bit of necro perhaps, sorry if that is the case.

I was thinking, if the ModuleResourceConverter and ModuleResourceHarvester are simulated 'post-facto' at vessel loading time, using the lastUpdateTime module data (correct me if I'm wrong), then of course it is possible that this is the reason of so many apparently random problems people encounter after off-rail time of ISRU vessels. Let me explain: you got a complex system (composed of multiple interacting parts: in this case sources, sinks, and storage) and the game engine perform a single update, at a single point in time (loading time). Then it is impossible for the system to be modelled accurately, or even approximately, if not only for the simplest cases, because there is only one integration at the end.

The solution is to mimic what physical integrator does to maintain stability: perform multiple simulation steps over time. Eventually, you can also compute all the steps still at vessel loading time, in succession. I hope this is clear and maybe even helpful. Cheers!