[1.4.1] KerbTrack v1.5: Head tracking using your webcam, TrackIR, Oculus Rift, joystick (2018-03-14)

[Monkthespy]

Hey, could you point me toward that info? I've been getting back in PS2 and would love to get TIR working with it.

I don't know where I got it, it was a long time ago.

[Legwan]

Great thing! Thank you for developing that.

Idea:

Instead of amplified head motion use the head position to get "looking through the window" effect i.e. something what you can see on this promo video.

This way instead of pretending to be a pilot and make a head movement which will be represented on screen BUT I still need to look straight at that screen to se the result. Instead with this idea idea in place the virtual pilot merges with myself, just is limited by the flat window (the screen). Then I can just lean to the side to see something there (act intuitively). Of course it would be still possible to move the whole window using mouse as usual.

Edited March 23, 2014 by Legwan
making more sense

[pizzaoverhead]

Great thing! Thank you for developing that.

Idea:

Instead of amplified head motion use the head position to get "looking through the window" effect i.e. something what you can see on this promo video.

This way instead of pretending to be a pilot and make a head movement which will be represented on screen BUT I still need to look straight at that screen to se the result. Instead with this idea idea in place the virtual pilot merges with myself, just is limited by the flat window (the screen). Then I can just lean to the side to see something there (act intuitively). Of course it would be still possible to move the whole window using mouse as usual.

That's what the plugin does, the only difference is the sensitivity setting. You'll be able to set it to match your head for "through the window" or magnified for when you want to be able to look over your kerbal's shoulder without moving your monitor. Though currently, the headtracking movement takes the place of the right-click mouse movement, so the window style movement wouldn't be as useful. I'll see if I can fix that.

I got some time to work on this over the weekend, and try it out with some actual gameplay for the first time, so I've updated the original post. It's revealed some usability issues that prevent it from being suitable for doing full missions with, but I hope to have those fixed for a first release soon.

[DaMichel]

Ha ha nice! It is about time someone comes up with this.

Cannot wait to try it together with RasterPropMonitor and do some flying.

[mushroomman]

Good god. Thanks for introducing this to me! It changes everything!

[comham]

TrackIR5 owner here humbly requesting joystick support for tir2joy compatibility. I'd love to get a sense of IVA perspective going on while I wait for the Oculus Rift to be released!

[Dirt_Merchant]

this is an absolute necessity for my Iva missions! I'm a longtime freetracker and I have been absolutely salivating over this mod. Please Please Please let me test it for you! I might have had a dream that I have freetrack and trackir api (tirviews) available, in addition to facetracknoir. Let me know what I can do to help you out in any way.

[pizzaoverhead]

I've cleaned this up enough for open testing. The alpha release of KerbTrack is now available here.

If you have any issues, crashes or suggestions, leave a comment, though it'll likely be next week before I'm able to work on this again.

[deadshot462]

Very nice, this will make it worth it for me to get a trackIR gear.

[pizzaoverhead]

Very nice, this will make it worth it for me to get a trackIR gear.

There is currently no TrackIR support in the current version of FreeTrack due to legal action by NaturalPoint. You can use this plugin with a regular webcam, but I hope to find a way to support TrackIR in future.

[Nuke]

the trackir is just a high speed monochrome camera with an fpga for fast video processing, turning a bunch of pixels into a few vectors. this keeps the usb bus traffic to a minimum. this was a good idea with single core machines. in a modern machine with many cores this becomes a non issue, and you have cpu time to do video processing while your game clunks along in another thread. so ultimately you just want a monochrome camera with a high frame rate, and usb 2.0 helps. i did a setup with an old security camera and a video capture card, and some ir filters. it worked great.

[ZodiaK]

I have a normal internal webcam in my laptop. when I go on IVA with KerbTrack enabled, It won't work. do I need to have facetrak?

[Nuke]

facetracknoir is some kind of magic, i think i played with it when it was still in alpha. you dont need to mod your webcam either. i dont think its as precise.

freetrack on the other hand takes some setup. you need something to track, leds (usually of the infrared variety) are often used as markers. this requires some basic electronics skill. basic "what resistance do i need to keep my leds from burning out" kinda stuff. these of course need to be arranged in one of several ways (freetrack lets you pick which and tweak the dimentions). then of course you need an ir pass filter (old floppy discs, vhs tape, film negatives, all work well), and some webcams need to be modded (removal of an ir block filter that interferes with the whole thing).

if you dont want to or cant mod your cam you can use colored leds, and block as many other colors as possible. one thing ive seen done is take a couple pairs of red/blue anaglyph glasses, and fold them over and tape it to your camera, then use amber leds, just make sure there aren't any lights behind you.

i got it to work fine with that old version of freetrack and my trackir 4. had to copy the FreeTrackClient.dll to the root ksp folder though. i think if you install the application the usual way, it does that for you, but i just launched freetrack off of a backup drive from another machine.

Edited March 28, 2014 by Nuke

[pizzaoverhead]

I have a normal internal webcam in my laptop. when I go on IVA with KerbTrack enabled, It won't work. do I need to have facetrak?

As Nuke said, FreeTrack uses IR LEDs. You need FaceTrackNoIR to use it with a regular webcam.

i got it to work fine with that old version of freetrack and my trackir 4. had to copy the FreeTrackClient.dll to the root ksp folder though. i think if you install the application the usual way, it does that for you, but i just launched freetrack off of a backup drive from another machine.

You've successfully tested this with TrackIR? Glad to hear that's working. I don't have a TrackIR to test with myself.

Edited March 28, 2014 by pizzaoverhead

[Nuke]

i dont think this works for the trackir5, because of data stream encryption on that device. im using a trackir4 and i hear this method also works well for all earlier versions of the trackir camera as well as the professional optitrack cameras.

for reference im using the 2.2.0.267 version (support for trackir was dropped in 2.2.0.279) of freetrack, and it requires optitrack sdk version 1.1.034.

Edited March 28, 2014 by Nuke

[pizzaoverhead]

i dont think this works for the trackir5, because of data stream encryption on that device. im using a trackir4 and i hear this method also works well for all earlier versions of the trackir camera as well as the professional optitrack cameras.

for reference im using the 2.2.0.267 version (support for trackir was dropped in 2.2.0.279) of freetrack, and it requires optitrack sdk version 1.1.034.

Thanks for the info Nuke, I'll add it to the OP.

[Rockyfelle185]

Deeply interesting

[Helix935]

so wait i saw the release but i'm confused what tracking software do i have to download since i just downloaded the alpha?

also i am using a laptop that has a webcam built in

[MK3424]

so wait i saw the release but i'm confused what tracking software do i have to download since i just downloaded the alpha?

also i am using a laptop that has a webcam built in

FaceTrackNoIR.

[Helix935]

FaceTrackNoIR.

ok downloaded and installed

Edit: ok i have it installed and the alpha too with the settings sliders appearing in game but nothing seems to happen as i turn on the FaceTrackNoIR. Does anyone know why?

Edited March 29, 2014 by Helix935

[Helix935]

ok i am willing to double post(even though it is frowned upon) to preserve this mod and receive help...

i still can't get it to work with FaceTrackNoIR installed

Edit: Ok i just realized that it is using the crap out of my CPU

Edited March 29, 2014 by Helix935

[Dirt_Merchant]

@Helix935: I'm very glad to see that you are enthusiastic about mods for ksp. More fans of mods will result in more support for mods if previous experience holds true. However, and don't take this personally, I've noticed you've quite the habit of clamoring for attention in other mod dev threads, and reading your signature further encourages me to implore you sir, to take a step back, breathe, and remember that mod devs are people too. People with limited time, resources, and patience for this sort of behavior. Given that you are not paying, or in any way compensating these guys and gals for their contribution to this game, please *slow your roll*. This advice is to help prevent your alienating everyone you come into contact with.

Regarding FaceTrackNoIr: Read the manual, and ensure that you're actually employing the correct outputs within the program. Further, test the face tracking within FTNIR. My experience with FTNIR was less than perfect as compared with my FreeTrack setup, however, this is expected given webcam resolutions and the cpu required with facetracking. Expect it to be less than perfectly precise, and use the smoothing features to help reduce jittering. Finally, re-read the OP carefully after ensuring that FTNIR is indeed functioning correctly. Report any updates you might have. Hope you find this helpful.

[Helix935]

@Helix935: I'm very glad to see that you are enthusiastic about mods for ksp. More fans of mods will result in more support for mods if previous experience holds true. However, and don't take this personally, I've noticed you've quite the habit of clamoring for attention in other mod dev threads, and reading your signature further encourages me to implore you sir, to take a step back, breathe, and remember that mod devs are people too. People with limited time, resources, and patience for this sort of behavior. Given that you are not paying, or in any way compensating these guys and gals for their contribution to this game, please *slow your roll*. This advice is to help prevent your alienating everyone you come into contact with.

hmm i am sad to say that you aren't the first person to tell me this and this is most likely due to the fact that i literally log on to this forum everyday and search through the forums looking for something to do (i am bored almost all the time) and have try to bump threads out of just being addicted to these mods...

so i apologize if i end up alienating or putting off anyone for my repeated clamors for attention

Edit: i should mention that i wasn't kidding about having ADD/ADHD as said in my Sig. so that may have affected my behavior

Regarding FaceTrackNoIr: Read the manual, and ensure that you're actually employing the correct outputs within the program. Further, test the face tracking within FTNIR. My experience with FTNIR was less than perfect as compared with my FreeTrack setup, however, this is expected given webcam resolutions and the cpu required with facetracking. Expect it to be less than perfectly precise, and use the smoothing features to help reduce jittering. Finally, re-read the OP carefully after ensuring that FTNIR is indeed functioning correctly. Report any updates you might have. Hope you find this helpful.

i have already tested the face tracking and it does work (although it would sometimes confuse my eye brows for my eyes) and although i should mention that sometimes when i test it in KSP, it would actually move but only rarely(like only twice since i started using this) and only front an back in one-movement jerks so i will continue testing this as i also test out my own mod

[Space_Kraken]

This plugin caused me massive ingame lag for some reason and tracknoir made it even worse. I love the idea, and I will come back at a more stable release.

[pizzaoverhead]

ok i am willing to double post(even though it is frowned upon) to preserve this mod and receive help...

i still can't get it to work with FaceTrackNoIR installed

Edit: Ok i just realized that it is using the crap out of my CPU

Picking through ~30 pictures a second looking for a face is very expensive for the CPU, so you need a spare core to run it on. With KSP, this means you should have a quad-core computer to run face tracking. You can improve this by reducing the resolution and frame rate your camera is running at. If you can't do that, FreeTrack uses less CPU power as it only has to track LEDs, so that setup would be more appropriate.

i have already tested the face tracking and it does work (although it would sometimes confuse my eye brows for my eyes) and although i should mention that sometimes when i test it in KSP, it would actually move but only rarely(like only twice since i started using this) and only front an back in one-movement jerks so i will continue testing this as i also test out my own mod

Make sure you are in a well-lit room and that the camera is centred on you at an appropriate distance. It can take a few seconds to find you, but it should work fine once it does. You can press the Home key to re-centre the tracking if it thinks you're looking sideways when you're not.

This plugin caused me massive ingame lag for some reason and tracknoir made it even worse. I love the idea, and I will come back at a more stable release.

If you've been having crashes, please post your logs here. Without FaceTrackNoIR running, this plugin shouldn't do anything to affect game speed. What other mods are you running? As above, FaceTrackNoIR is very CPU-heavy. FreeTrack with LEDs or a TrackIR 4 would be a better solution for you in this case.