[KSP 1.11.1] LaserDist, a distance measuring laser part (for kOS, kRPC, etc)

[Dunbaratu]

Well I changed it on my github master (AGAIN, as far as I can remember.  Unless I'm hallucinating a memory, because I definitely *already* did this last night, despite there being no evidence of it to be found in the git logs).  I'll wait for tomorrow to see what happens when CKAN looks at it again.

[HebaruSan]

4 minutes ago, Steven Mading said:

I'll wait for tomorrow to see what happens when CKAN looks at it again.

It'll be the same, because the local value of VERSION (1.0.0) doesn't match the remote value (1.1.0). The ability to tweak the version info for already released versions only applies when VERSION is the same.

Edited October 2, 2018 by HebaruSan

[Dunbaratu]

1 minute ago, HebaruSan said:

It'll be the same, because the local value of VERSION (1.0.0) doesn't match the remote value (1.1.0). The ability to tweak the version info for already released versions only applies when VERSION is the same.

Then what the heck am I supposed to do from my end?  I can't lie by pretending LaserDist is now version 1.0.0 again in the version file.

[HebaruSan]

1 minute ago, Steven Mading said:

Then what the heck am I supposed to do from my end?  I can't lie by pretending LaserDist is now version 1.0.0 again in the version file.

One way or another you'll need to get a .version file with an accurate VERSION field into the download. Either a new release or replacing the asset of the current release would be the most obvious ways to accomplish that.

[Dunbaratu]

4 minutes ago, HebaruSan said:

 Either a new release or

But you JUST got done saying the fix won't work unless I claim it's version 1.0.0 in the file.  How does making a *new* release fix it then?  This is what I don't understand.  It seems fixing it requires a time machine.

Edited October 2, 2018 by Steven Mading

[HebaruSan]

4 minutes ago, Steven Mading said:

But you JUST got done saying the fix won't work unless I claim it's version 1.0.0 in the file.

No, I said they need to match. If you fix the download, then both would say 1.1.0, and they'd match.

[Dunbaratu]

54 minutes ago, HebaruSan said:

No, I said they need to match. If you fix the download, then both would say 1.1.0, and they'd match.

 

I think I can finally decode what you were trying so say, maybe, now that you started mentioning the word "download" file rather than obtusely saying "local value" or "the asset", which have a lot of other possible meanings.

Do you mean "make the ZIP download's .version file hosted on the release page match the master branch's .version file?"

When you used the words "local value" versus "remote value", did you really mean "value hosted in the LaserDist Github, coming from the ZIP file's LaserDist.version file" versus "value hosted on the LaserDist Github, but coming from the branch's (in my case, master) source code LaserDist.version file"?  If so, calling one of them "remote" and one of them "local" really wasn't clear that's what you meant.  They're both located in the same host site.

When you used the word "the asset" did you mean the download file attached to the release tag on github?  Yes, that's called an "asset", but pretty much any file is called an "asset" so it's not very clear.

If CKAN peeks inside the ZIP file, that was not something I knew.  I asked if it looks at the locally installed version file, specifically because I was worried about the version file inside the ZIP still saying 99.99.99, and you said no it doesn't look at that, so I sort of assumed it doesn't know about the contents of the download file.

Thus all this talk about the "local" version having to match the "remote" version in order to overwrite the old info for it sounded like you meant my repo versus what was in CKAN's database, not what was in two different parts of my repo.

Edited October 2, 2018 by Steven Mading

[HebaruSan]

5 minutes ago, Steven Mading said:

When you used the words "local value" versus "remote value", did you really mean "value hosted in the LaserDist Github, coming from the ZIP file's LaserDist.version file"

Yes.

5 minutes ago, Steven Mading said:

When you used the word "the asset" did you mean the download file attached to the release tag on github?

Yes.

5 minutes ago, Steven Mading said:

If CKAN peeks inside the ZIP file, that was not something I knew.  I asked if it looks at the locally installed version file, specifically because I was worried about the version file inside the ZIP still saying 99.99.99, and you said no it doesn't look at that, so I sort of assumed it doesn't know about the contents of the download file.

There are two programs here with different answers. First is the CKAN client app, the GUI that you install and run and see on your screen. That one doesn't look at .version files and doesn't know anything about them. It just downloads metadata and uses it.

Then separately in the background there's a bot that runs on a server to auto update the metadata that CKAN uses. That bot does peek inside ZIP files to check their .version files.

[Dunbaratu]

1 hour ago, HebaruSan said:

Then separately in the background there's a bot that runs on a server to auto update the metadata that CKAN uses. That bot does peek inside ZIP files to check their .version files.

I take it this all means the CKAN bot doesn't pay attention to the github tagging system?  As in, it does NOT do this: "We saw a new release for the LaserDist project appear in this run, let's look at it.  Hmm it goes with tag 'v1.1.0'.  I'll build my database information by asking github to show me what LaserDist.version used to look like at tag `v1.1.0`, which might not be what it looks like right now since several hours have passed since then."

(Thus my talk about needing a time machine to historically alter what was in an already created tag.  That was when I thought it worked like this.)

If it does not do this (pay attention to the github tagging system), that has an implication I need to pay attention to.  Namely this:

If the CKAN bot only looks at what the current newest state of the project's files are at the time it visits my repo, and all past versioning it tracks is built up from its own records of what it saw on previous runs, NOT from github's tag history, then this means if I make two releases in quick succession, fast enough that the CKAN bot didn't run in between them, then CKAN's database won't know the first of those two releases exists.  It will only see the second one.  This is relevant in cases where sometimes right after making a normal release, I follow it up with a backport release for the realism overhaul people who are stuck on older versions of KSP.  Often the backport release is easy enough to make that all I have to do is edit some version numbers and recompile against different reference DLLs.  If what I said in this paragraph is true, then that means if I want to follow up a normal release with a backport release, I need to hold off and wait a day before making that backport release.

Edited October 2, 2018 by Steven Mading

[HebaruSan]

5 hours ago, Steven Mading said:

If the CKAN bot only looks at what the current newest state of the project's files are at the time it visits my repo, and all past versioning it tracks is built up from its own records of what it saw on previous runs, NOT from github's tag history, then this means if I make two releases in quick succession, fast enough that the CKAN bot didn't run in between them, then CKAN's database won't know the first of those two releases exists.  It will only see the second one.  This is relevant in cases where sometimes right after making a normal release, I follow it up with a backport release for the realism overhaul people who are stuck on older versions of KSP.  Often the backport release is easy enough to make that all I have to do is edit some version numbers and recompile against different reference DLLs.  If what I said in this paragraph is true, then that means if I want to follow up a normal release with a backport release, I need to hold off and wait a day before making that backport release.

Yes, that's correct. Each pass of the bot looks at only the current latest release. Some mods have very long release histories, and trying to re-index every one on every pass would require much more latency, network bandwidth, and download cache space.

[Dunbaratu]

8 hours ago, HebaruSan said:

Yes, that's correct. Each pass of the bot looks at only the current latest release.

Actually, it sounds like CKAN's bot will not limit itself to the current latest release at all.  It sees the latest copy of the file in the master branch even when it's newer than the current release.

That means every release MUST be followed by a waiting period in which the master branch is not touched so the bot doesn't try to pair the future .version file (edited in preparation for a future release that's not  ready yet) with the current release.  If the .version file has already incremented the version by the time the bot sees the release, it will never make the association between them.

[HebaruSan]

32 minutes ago, Steven Mading said:

Actually, it sounds like CKAN's bot will not limit itself to the current latest release at all.  It sees the latest copy of the file in the master branch even when it's newer than the current release.

... only if that .version file is pointed to by the URL property of the one in the download and is for the same version as the current release, as indicated by its VERSION property, as per the KSP-AVC spec.

32 minutes ago, Steven Mading said:

That means every release MUST be followed by a waiting period in which the master branch is not touched so the bot doesn't try to pair the future .version file (edited in preparation for a future release that's not  ready yet) with the current release.

No, that's not correct. If the VERSION property is different, then no such "pairing" will occur. No waiting period is needed.

32 minutes ago, Steven Mading said:

If the .version file has already incremented the version by the time the bot sees the release, it will never make the association between them.

... yes? No? I'm not sure exactly what scenario you're describing here. If you're specifically referencing what happened with this mod, there is no way to alter the VERSION property after the fact other than fixing the download.

[Jacke]

I think a bit of the problem is having the .version file URL field point at the master branch.  After much thought, I think this practice, although very common, is wrong.

"URL": "https://raw.githubusercontent.com/Dunbaratu/LaserDist/master/LaserDist.version",

It should instead point to a per-release online copy.  That way the KSP version fields can be updated online for KSP-AVC and I think for future runs of the LaserDist .netkan as well.

@HebaruSan mentioned this very concept in another topic.

Quote

The KSP-AVC spec doesn't require the URL property to point to your master branch on GitHub. If your master branch is special to your work flow, you're free to pick any other location as long as you can store an up to date .version file there.

 

[Dunbaratu]

6 hours ago, HebaruSan said:

No, that's not correct. If the VERSION property is different, then no such "pairing" will occur. No waiting period is needed.

When you describe that failure to perform the pairing, you are describing exactly the problem I specifically mentioned, but then you prefaced that with the phrase "no that's not correct", so I clearly don't get what you're saying.  You just repeated back to me the same problem I mentioned, as far as I can see.  If I update the version file, make a release, then update the version file *again* to the next version up, and do all these things without letting the bot run between them, then the bot ends up seeing the newer version file *instead of* what it looked like when the release was made, and therefore it will never read the .version file that was correctly paired to the ZIP, because that occurred when it "wasn't looking" and it's already been updated to a higher version since then, which will now fail the pairing.  This is a consequence of it always reading from the current HEAD version of the .version file, instead of paying attention to the tag that github explicitly said goes with the release ZIP.

Quote

The KSP-AVC spec doesn't require the URL property to point to your master branch on GitHub. If your master branch is special to your work flow, you're free to pick any other location as long as you can store an up to date .version file there.

I think there might be a better way, and may start doing it - Instead of this statement that implies the fix is to put it in some OTHER branch other than the master branch, instead just tell people to go ahead and use the master branch, but tell them to put the specific TAG OF the master branch into the URL, which github lets you do, like so:

Points to the master branch version file, and will always give the newest HEAD version any time it changes:

https://raw.githubusercontent.com/Dunbaratu/LaserDist/master/LaserDist.version

 

Points to the master branch version file, but it's an older commit of it, from back when tag v1.1.0 was made, not the *current* master branch:

https://raw.githubusercontent.com/Dunbaratu/LaserDist/v1.1.0/LaserDist.version

 

Since making a tag is generally part of what happens when you make a release, it makes a lot more sense to tell people to point the URL at that tag, rather than point at some other branch other than the master.

This prevents the ability to update the version file for an already-made release by just updating the file in master (because CKAN's bot isn't looking at the most recent version in master anymore if you change the URL to this style that encodes the specific tag, if I understand all this correctly), but if doing that ad-hoc version file change on the spot often requires replacing the old ZIP with a new one *anyway* because CKAN refuses to take the change if the ZIP version differs from the repo version,  then I don't see losing that feature as a big loss.  I have to ensure I make the .version file inside the ZIP match the newly updated .version file in the repo *anyway* if I want to use this feature, so I may as well make the .version file inside the ZIP point it at a new URL too if I do that.  And at that point, I may as well just make a new release if that happens.  I'm already doing 80% of the work of a new release anyway.

 

It would mean the workflow of making a release would have an additional step added:

- In addition to editing the version number sections of the .version file, also edit the URL of the .version file to point it at the specific tag you will be giving to this release.

 

What I'm proposing would basically be me doing a bit of extra typing to manually force the CKAN bot to behave intuitively and treat releases like you'd expect it to - Github makes a git tag for each release - when you make a release it's allegedly a snapshot of what the branch looked like at that time, and the release's download file (ZIP) is supposed to be made from that snapshot.  So when CKAN's bot sees the new release, it knows to look at that specific tag that goes with it.  It sounds like I can't make it do this automatically, but instead I have to explicitly tell it each time I release what the tag is (type it in the URL field) even though Github already tells it which tag the release is associated with, and it *could* have found this automatically.

If this works, I may script it to help make it less likely to make an error (part of making a release would include automatically populating that URL field in the .version file with the tag name just before the tag is created.)

Edited October 3, 2018 by Steven Mading

[Dunbaratu]

It may sound like I'm coming across as hostile, and if it does I apologise.  It's not out of hostility, it's borne out of *frustration* at trying to understand what the heck is going on with this system and constantly being thrown a new contradictory bit of information every time I thought I had it. I don't think you're trying to contradict, but rather that the use of CKAN's alien terminology which is a wrapper around Github's alien terminology, which itself is a wrapper around git's alien terminology, makes clear speaking nearly impossible.  Terse sentences often mean the opposite of what it sounds like they mean when dealing with that sort of terminology.  It's a situation where being verbose really *helps* because it provides communications parity to say the same thing two different ways.

Also, terse communication makes the reader have to "fill in the blanks" with assumptions about all the things you *didn't* say, and if they're not already on the same page as you, those things they fill in the blanks with won't be what you thought they would be.

Edited October 3, 2018 by Steven Mading

[HebaruSan]

5 hours ago, Jacke said:

I think a bit of the problem is having the .version file URL field point at the master branch.  After much thought, I think this practice, although very common, is wrong.

"URL": "https://raw.githubusercontent.com/Dunbaratu/LaserDist/master/LaserDist.version",

It should instead point to a per-release online copy.  That way the KSP version fields can be updated online for KSP-AVC and I think for future runs of the LaserDist .netkan as well.

Wouldn't that defeat the purpose of KSP-AVC? If "URL" doesn't point to the latest .version file, then users wouldn't get notified of newly available versions.

[HebaruSan]

21 minutes ago, Steven Mading said:

Also, terse communication makes the reader have to "fill in the blanks" with assumptions about all the things you *didn't* say, and if they're not already on the same page as you, those things they fill in the blanks with won't be what you thought they would be.

I'm trying to avoid writing a complete technical manual for both KSP-AVC and CKAN here, partly because that belongs on a wiki rather than spamming some poor modder's forum thread, and partly because it just isn't necessary. There's nothing particularly complicated going on here.

1. Make a correct .version file (meaning, ensure that VERSION, KSP_VERSION, KSP_VERSION_MIN, and KSP_VERSION_MAX are accurate)
2. Put it in your ZIP file
3. Also put it online somewhere (master branch on GitHub is a common choice, but not the only option by any means)
4. Set the "URL" property to the location of the online copy (granted, this has to be done in step #1, but that's easy enough)
5. Then if a new KSP version comes out and your existing version turns out to be compatible with it, you can update the KSP_VERSION, KSP_VERSION_MIN, and/or KSP_VERSION_MAX properties in the online copy, and users who installed the original ZIP file will benefit from those changes, and they'll be reflected in CKAN.
6. If you decide to make a new release, start over from step #1, replacing the previous online .version file at step #3

That's it. The rest is just me trying to answer your questions without creating even more confusion. You're having difficulties because step #1 wasn't done.

[HebaruSan]

1 hour ago, Steven Mading said:

When you describe that failure to perform the pairing, you are describing exactly the problem I specifically mentioned, but then you prefaced that with the phrase "no that's not correct", so I clearly don't get what you're saying.  You just repeated back to me the same problem I mentioned, as far as I can see.

Maybe if you spell out why you think there's a problem? Such a setup would work well.

Quote

If I update the version file, make a release, then update the version file *again* to the next version up, and do all these things without letting the bot run between them,

... then the bot will index your download based on its internal .version file, which will work fine. It will ignore the online version file, but that's fine, since it doesn't apply to the version in the download.

Quote

Instead of this statement that implies the fix is to put it in some OTHER branch other than the master branch,

Actually what I had in mind was Cybutek's server at http://ksp-avc.cybutek.net/, where some modders store their "current" .version files.

Quote

Points to the master branch version file, but it's an older commit of it, from back when tag v1.1.0 was made, not the *current* master branch:

https://raw.githubusercontent.com/Dunbaratu/LaserDist/v1.1.0/LaserDist.version

As I told @Jacke, this would defeat the purpose of KSP-AVC. Your users would not be notified of available updates. That's the main purpose of the "URL" property. Checking for game version compatibility updates of the same version is a sideshow. I would strongly recommend against this approach.

Quote

This prevents the ability to update the version file for an already-made release by just updating the file in master (because CKAN's bot isn't looking at the most recent version in master anymore if you change the URL to this style that encodes the specific tag, if I understand all this correctly), but if doing that ad-hoc version file change on the spot often requires replacing the old ZIP with a new one *anyway* because CKAN refuses to take the change if the ZIP version differs from the repo version,

OK, stop. CKAN doesn't refuse anything. I thought we were talking only about the case where you want to make an update just to the online copy of a .version file and have it apply to people's already installed mods, but now it sounds like you're conflating it with just normal indexing of a new version. Please be clear about which scenarios you're discussing.

Edited October 3, 2018 by HebaruSan

[Jacke]

1 hour ago, HebaruSan said:

As I told @Jacke, this would defeat the purpose of KSP-AVC. Your users would not be notified of available updates. That's the main purpose of the "URL" property. Checking for game version compatibility updates of the same version is a sideshow. I would strongly recommend against this approach.

Then there's two things that KSP-AVC (and CKAN) needs to know.

1.  When the KSP_VERSION* of a particular VERSION of the mod changes, because a new KSP version has released and existing version(s) of the mod can support it.

2.  What the existing VERSION's of a mod exist, including a new release.

I can see #1 being solved by per-VERSION online copies of the .version file, pointed to by the URL field of release .zip .version file.  That online copy can have its KSP_VERSION* fields updated.

Given #1, that directly clashes with using that URL field pointing to a master .version file for the latest release, as you pointed out, @HebaruSan.

How does KSP-AVC and CKAN figure out when a new release of a mod comes out?

[Dunbaratu]

5 hours ago, HebaruSan said:

OK, stop. CKAN doesn't refuse anything.

The extra piece of information you supplied in this recent post was important to clearing things up.  Until now, it wasn't made clear that the CKAN bot uses the information from the downloaded ZIP file when it ignores the URL file. (That it uses that as a second backup place to get the information from, so ignoring the URL file doesn't make the attempt to get the release into CKAN's database fail.)

Edited October 3, 2018 by Steven Mading

[Dunbaratu]

Tested, and it seems to still work fine in KSP 1.5 without needing a recompile.

[funkheld]

Hi good afternoon.

How is laserdist with krpc?

Thank you.
greeting

[SaucerChrome]

On 2/14/2019 at 2:00 AM, funkheld said:

Hi good afternoon.

How is laserdist with krpc?

Thank you.
greeting

It works pretty well. Here's a (rather poor) video of a LiDAR scanner program I wrote to build pointclouds out of readings from laserdist parts.

[Dunbaratu]

Version 1.2.0 out (compiled for KSP 1.7.3).

This fixes an issue that cropped up in KSP 1.7.x where if you edited the part.cfg file (or used ModuleManager) to change the min/max values for bending the laser aim, LaserDist didn't use those values properly and kept using the default, ignoring your edit.

(This was because previous to KSP 1.7.x, it was sufficient to alter those values for the Editor (VAB) and that change would get inherited by the Flight scene too, but as of KSP 1.7.x, the Editor and Flight scene need two different API calls to apply the override to both of them.)

 

Edited July 23, 2019 by Steven Mading

[Menroka]

Is there any chance to get a 2d, or even 3d Lidar as well?