[WIP] Unity Editor Plugin

[VR_Dev]

6 hours ago, kcs123 said:

Some slopes are still slippery for bot, but it walk much, much better than before Great progress.

Thanks man. Nothing is tuned right now, so I'm sure with a little love it will be much better. I was gonna send the project out to you and ZodiusInfuser this morning but I remembered I have some file Paths hard coded I'm gonna need you guys to change. So I want that to be as easy as possible. I dont think its a lack of friction between the foot and the ground, as much as the foot bouncing off the ground.

Like I explained in the IR next thread, the Servo Speed PID is responsible for slowing the foot down as it gets too the ground. If it hits the ground too hard it explodes, slightly to hard and it bounces back into the air, and if I turn the speed way down, the whole performance suffers. This is why I'm looking forward to trying to build suspension parts. So that the mech leg can hit the ground with some force, and not explode. This would also help increase ground contact. I'm going to draw the ground triangle a different color when all three legs have contact as opposed to just two. This should help debug maintaining ground contact throughout the translation forward.

6 hours ago, ZodiusInfuser said:

That is one impressive terrain adaptation algorithm you've got there! Am I right in thinking you check the height of the leg relative to the ground on each update rather than just on contact? I as because if you were to translate this to IRL getting that height value won't be possible so another approach to terrain adaptation would be required, which is something I'm currently looking in to.

Good luck tackling steering, especially if you want to combine translation/strafing with turning. Not impossible, but to do it well is likely to require a lot of maths and may not fit in with your current circular striding approach.

Thanks! I'm really happy with it. I do measure the ground every frame, but it could be done just on contact if need be without much change in performance. If it had to be on contact, I would just set the active balance target once contact was made, instead of throughout the whole stride.

I have high hopes for turning. My initial thought is i will rotate a transform in unity that will be the desired vector. Then, if the difference in vectors is less then say 15 degrees, the bot turns the way I have now. Anything higher than that the bot will stop, rotate, then begin walking. Still long way to go on that and I'm out of town all weekend so prob no progress anytime soon.

I've also thought about how easy it would be to make a few changes, and make this run stand alone in ksp. My method could be used for development of projects, then when they are done just a few changes will allow them to be run stand alone in ksp.

 I always thought one of these could get rigged onto an IRL foot.

Edited February 20, 2018 by VR_Dev

[ZodiusInfuser]

13 hours ago, VR_Dev said:

 I always thought one of these could get rigged onto an IRL foot.

I'm familiar with them, they're not very accurate and have a minimum range of at least 30cm, not to mention they don't report the distance of the point directly in front of them (it's more like a cone). You may get better results with one of these: https://www.sparkfun.com/products/242 but they again are not super accurate and also are affected by different materials. So in general you cannot rely on measuring the height of the ground IRL.

P.S I'll reply to the rest tonight.

[Dr_Goddard]

If you aimed two IR lasers downward from each foot, angled with a convergence, an IR detector aimed at the convergence would have a varying strength of return with the most powerful return being when the beams converged on the walking surface. I'm working with a client who uses a very similar principle to get measurements with an accuracy of a few mm. The components are cheap and easy to get.

Good work on the terrain adaptation. I'd love to get a peek at your code. 
I can trade you my gait generation and steering solutions. 

[Colonel Cbplayer]

If you need better foot pads a good way to do it is to use small landing gear wheels or rover wheel parts with brakes on and max suspension and dampening. Removing symmetry in the editor stops auto struts from strutting the different legs together. You can either do this by placing each part manually without symmetry or use a mod like editor extensions and LeftShift+LeftAlt+LeftClick apart to get rid of symmetry. Tweakscaling a group of small wheels for each leg in different directions should do the job.

[VR_Dev]

On 3/8/2018 at 7:24 PM, Dr_Goddard said:

On 3/8/2018 at 7:24 PM, Dr_Goddard said:

If you aimed two IR lasers downward from each foot, angled with a convergence, an IR detector aimed at the convergence would have a varying strength of return with the most powerful return being when the beams converged on the walking surface. I'm working with a client who uses a very similar principle to get measurements with an accuracy of a few mm. The components are cheap and easy to get.

Good work on the terrain adaptation. I'd love to get a peek at your code. 
I can trade you my gait generation and steering solutions. 

Thanks for the hot tip. Haven't even started on the real thing yet so I'll have to hold onto that. The terrain adaptation is actually super easy, gait generation is the tough part for me, so I'd be down to trade secrets. I sent you a pm about the project files.

11 hours ago, Colonel Cbplayer said:

If you need better foot pads a good way to do it is to use small landing gear wheels or rover wheel parts with brakes on and max suspension and dampening. Removing symmetry in the editor stops auto struts from strutting the different legs together. You can either do this by placing each part manually without symmetry or use a mod like editor extensions and LeftShift+LeftAlt+LeftClick apart to get rid of symmetry. Tweakscaling a group of small wheels for each leg in different directions should do the job.

Thanks dude, the friction is surprisingly ok, but I've def had that problem in the past. I've played with adding wheel modules to parts to try and increase their friction but I've never been really able to make it work. 

I haven't been working on this project lately because I am in the process of moving to a new apartment. What free time I can manage has been going to this guy lately. 

[VR_Dev]

Holy sht I just saw the 1.4 changelog. It claims Unity was updated to 2017. That means I should be able to use memory mapped files in ksp without my third party process. Also makes in game VR much more attainable.

If they updated the .net version this could be huge.

Edit:

Got super excited about this but turns out I was wrong. I knew that would be a huge change but figured all the old mods would still compile in a newer .net version

 

Edit again:

Confused cause this code tells me ksp is targeting 4.0, but mmap files don't work still.

RegistryKey installed_versions = Registry.LocalMachine.OpenSubKey(@"SOFTWARE\Microsoft\NET Framework Setup\NDP");
string[] version_names = installed_versions.GetSubKeyNames();
//version names start with 'v', eg, 'v3.5' which needs to be trimmed off before conversion
double Framework = Convert.ToDouble(version_names[version_names.Length - 1].Remove(0, 1), CultureInfo.InvariantCulture);
int SP = Convert.ToInt32(installed_versions.OpenSubKey(version_names[version_names.Length - 1]).GetValue("SP", 0));

Edited March 13, 2018 by VR_Dev

[VR_Dev]

I don't really have time to write out all the details right now but I got memory files to work between .net 3.4 and 4. While I thought my process trick was pretty clever, this is insanely faster. Here is a render texture captured in ksp, displayed in the unity editor at 20-30 fps. If I limit the texture display frame rate the ksp frame rate improves, but the Unity render texture frame rate drops, which is fine for now.

Essentially I have to rebuild the whole system based around the new memory bridge. But I'm going to try and make it a releasable platform for people to use.

This was the main problem in my project. Now that I have this figured out I'm gonna build some crazy stuff.

 

 

[Colonel Cbplayer]

I got a mecha I really want to test your controller with. Its based on the player model from Toribash, a martial arts physics game where players control the character by muscle moments. Since I already record everything in this game at slow motion anyway I wonder if its possible to replicate a similar control system where I can move the joints themselves on a model instead of using that mouse point and click menu
 and the actual robotic joints themselves 
If you are interested in the mech both are in my kerbalX and here is the armored one https://kerbalx.com/Colonel_Cbplayer/Toribash-Doll 

[fatcargo]

Concerning Memory Mapped Files : i'm interested in using this "development system", but to me, this isn't  well enough explained.

Is it a plugin ?

Or is this a piece of source code inserted into normal plugin code ?

Does it need debug version of Unity3D player ?

Where is a download link to a project/plugin for testing/development ?

Are there any plans to document this ?

Is KSP 1.4.x supported yet ?

 

Edited March 26, 2018 by fatcargo

[VR_Dev]

On 3/15/2018 at 6:31 PM, Colonel Cbplayer said:

I got a mecha I really want to test your controller with. Its based on the player model from Toribash, a martial arts physics game where players control the character by muscle moments. Since I already record everything in this game at slow motion anyway I wonder if its possible to replicate a similar control system where I can move the joints themselves on a model instead of using that mouse point and click menu
 and the actual robotic joints themselves 
if you are interested in the mech both are in my kerbalX and here is the armored one https://kerbalx.com/Colonel_Cbplayer/Toribash-Doll 

Damn dude that thing is slick. Really solid IR skeleton which will work well fully rigged. Also like some of the clipping joints you created.  As I'll explain below I've taken a step back form the robotics to rework the bridge, but when I get back at it I'll give yours a test.

On 3/26/2018 at 1:11 PM, fatcargo said:

Concerning Memory Mapped Files : i'm interested in using this "development system", but to me, this isn't  well enough explained.

Is it a plugin ?

Or is this a piece of source code inserted into normal plugin code ?

Does it need debug version of Unity3D player ?

Where is a download link to a project/plugin for testing/development ?

Are there any plans to document this ?

Is KSP 1.4.x supported yet ?

 

Yeah dude I hear ya. Everything in this thread right now has the robotics and memory files intertwined. It is also using my old custom process hack. Now that I can get memory files to work in ksp, I am building the whole thing from scratch as two separate projects. The independent memory mapped bridge, and the robotics on top of that. 

What I am in the process of building are two dlls, one server and one client, that will allow anyone to use this system. I am working hard to make it as easy as possible for people to use by requiring no up front initialization. If I recall correctly in kRPC, you have to declare the variables you want to use at start, which was kind of annoying. Using my method all you have to do is call ReadValue(string Key), and WriteValue(string key, float/vector3 value), and you will be able to access the value from the editor, or a stand alone application. I also plan to build the camera stream feature into the bridge.

So it's a giant step back but a huge step forward. Once I get the Memory Mapped plugin working, I will return to the robotics. As for the other questions, it just needs the standard unity editor now, but like I said, it will be supported in stand alone applications just like kRPC, no download now but soon, it the new system will be open and documented, and it should be 1.4 compatible there is nothing limiting it to a certain a ksp version.

On a completely separate note looks like someone beat me to native KSP VR. 

https://www.reddit.com/r/KerbalSpaceProgram/comments/87m019/you_cant_check_the_staging_with_vr/

Edited March 30, 2018 by VR_Dev

[fatcargo]

Thanks for the effort and a kind response ! I had kind of rushed my post and forgot to add personally most crucial question of all : will this bridge let me write code and re-test it without restarting KSP ?

If the above works as i assumed, then the workflow would go like this :

1. run unity editor, load unity-side of bridge

2.run ksp, load ksp-side of bridge as plugin (in my case as PartModule)

3. connect bridge, unity sends sources into KSP plugin code to be executed. code in KSP runs

4. run Visual Studio (or other C# IDE), open project

5. edit sources, change code as needed (squash bugs, add new features etc.. )

6. unity detects file changes in project and sends them over bridge into KSP to run again

This whole thing would let me write code in IDE that is aware of KSP's API (added references to DLLs) and test it near-instantly in KSP.

So... how much did i assumed wrong ? If this works, it would be next to best possible option of actually HAVING sources for KSP itself as unity project.

Edited March 30, 2018 by fatcargo

[VR_Dev]

Still hard at work at this. Didn't have much to show on the bridge rework but I've finally optimized it to be as fast as possible, and have begun adding features into the platform.

First off is the autopilot from my f-22 drone race. (https://www.youtube.com/watch?v=HEbhV4tjA8E&t=27s). You can input raw steering data or have it match a target. My joystick has never worked in KSP but now I am able to use it through unity. This video is raw control, but soon you will be able to position the second target vessel, and have the real one match its rotation. 

 

 

GenerateCamera() will automatically place a camera in the ksp scene and a render texture in the unity scene. You can also move the camera in unity and it will move in ksp. Render textures can now be sent at 60fps as well. Still need to remove the UI layers from the created camera however. This will be necessary to control the bots in VR, and still see whats happening in the game.

 

 

As shown in the examples above I can also send the meshes over the bridge to unity. This will be helpful for a visual reference on top of the line skeletons. I have to figure out a way to get the manipulated IR components to look correct however. My next big project is writing a component that will automatically map any IR arm, and automatically calculate it's IK. This will most likely not be easy, but I believe it is doable, and would be the most useful feature to other people. I plan on rebuilding the hexapod with this system, but am also trying to build this. I want a mech that can walk, as well as have VTOL short to medium range capabilities. Also seems like a good stepping stone towards an upright, bidepal mech.

 

 

 

Edited June 21, 2018 by VR_Dev

[VR_Dev]

I can now auto generate a skeleton on any IK structure based on this simple naming convention. The limb has to be its own group, which includes "ik" somewhere in the title. The base (1st) joint needs to include "base", and the last joint needs to include "wrist". Anything you want to skip, and any extentrons need to include "skip". Feel free to rig up your mechs and send them my way for testing.

These skeletons are auto generated, and the servos grouped by their pivot axis. The sections with the same color will move the leg along the same axis. I have also taken rotation into account, so servos that are 180 of their group will be adjusted.This is all the groundwork needed to attempt autogenerating the IK.

[Hooligan Labs]

Hello @VR_Dev, I have to say something because this work is very interesting. I was looking at the KSP forums for about the first time in a... year? I found your YouTube videos with tech like Leap Motion very interesting since I work with ARcore and the like. I have KSP to thank for this, because I learned OOP and Unity by doing an airship mod years ago.  

Also, your progress with the walking robot looks very impressive. I have a number of projects I started to create a walking robot like this in Unity, but have not come close to the level of that you've been able to achieve by hooking it up to KSP with IK like this.

Are you still working on this project? I see you invited others to try building their own robots.

[Stone Blue]

Hi @VR_Dev, I dont know if you ever saw @fatcargo 's post a little while back,

But I was wondering the same about the workflow, and if you could confirm or correct his assumption?

I'm mostly interested in whether you could indeed use this to "refresh/view" mesh and texture changes in "real-time"?
If so, I would imagine this could be very helpful to not only plugin developers, but part mod devs as well... vOv
I dont suppose it would include live .cfg changes, as I assume KSP would still need to reload the database for those?

Edited September 18, 2018 by Stone Blue

[Hooligan Labs]

That is a good point, Stone Blue. Having to restart KSP when deving is time consuming.

Maybe VR_Dev has a wizardry for this, but I doubt you can change what KSP compiles after you start the run. What makes the plugins and mods work is so cool that Squad hired the guy who invented it. Feedback into the KSP runtime may be limited. Probably could throttle engines, turn the ship, etc. It would need a mod like TweakScale to go deep into the system and expose its variables.

I could see it working in the other direction -> you could have a Unity environment where you run tests based on the output from an active KSP game. Like a "hardware in the loop" test.

In the meantime, there are mods out there that skip the start screen and do little things to improve dev workflow. The best would be if Squad released a Unity sandbox that was just like KSP, but that is too much to ask.  

[Stone Blue]

10 hours ago, Hooligan Labs said:

Having to restart KSP when deving is time consuming.

Yup... As a dabbler in part modding, just being able to view "live", near real-time changes to meshes/colliders, textures and cfgs would save HUGE amounts of dev time for a part maker.

Even if VR_Dev's plugin can help with just meshes, and/or textures, but not cfgs, that would still be worth it. Even with the recent addition of being able to reload databases in the Alt-F12 menu (which does NOT always seem to work), without having to completely restart KSP, it still takes huge amounts of time waiting while everything reloads, at least for peeps like me, who are on potato computers.

10 hours ago, Hooligan Labs said:

In the meantime, there are mods out there that skip the start screen and do little things to improve dev workflow.

Out of curiosity, of which mods does the Great Hooligan speak of?

I've been using Malah's QuickStart, and the one that skips going to the spaceport between scene changes... Thats helped quite a lot.

Edited September 19, 2018 by Stone Blue

[Hooligan Labs]

I hate to say it, but a big reason why I stopped modding KSP was because it took less time to write a whole game prototype from scratch in Unity than to make a simple KSP plugin. 

I did a quick search and this is the dev tool I was referring too. Maybe it still works!

https://kerbal.curseforge.com/projects/magic-smoke-industries-devhelper-v0-4?gameCategorySlug=ksp-mods&projectID=221289

[Stone Blue]

3 hours ago, Hooligan Labs said:

I did a quick search and this is the dev tool I was referring too. Maybe it still works!

https://kerbal.curseforge.com/projects/magic-smoke-industries-devhelper-v0-4?gameCategorySlug=ksp-mods&projectID=221289

Interesting... Sirkut made a re-appearance on the KSP dev scene a few months ago... i spent alot of time chatting with him, and I dont ever recall mention of an MSI dev tool...
I'll have to ask, if I chat with him again.

 

3 hours ago, Hooligan Labs said:

I hate to say it, but a big reason why I stopped modding KSP was because it took less time to write a whole game prototype from scratch in Unity than to make a simple KSP plugin. 

I wonder whether the consensus among "old-time" KSP mod devs think KSP modding has gotten any better, or worse... vOv lol

[VR_Dev]

hey @Stone Blue and @Hooligan Labs, I am still working on this, but it has been sidelined for my new project. I'll respond to all your questions later today.

With the end goal being automated flying, something like this.

 

Edited September 21, 2018 by VR_Dev

[Enceos]

Jawdropping! 

[Superdave42]

OK....   I was led here by a link from Trossen Robotics website.  I just purchased and built a PhantomX Mark III Hexapod..    I see this here now and am totally amazed and intrigued.    What the heck is this KSP ??  can I use it for robotics?    is this something extremely difficult to use ?    I'm a mechanical design engineer,   just learning coding very slowly,   can barely stumble around in Arduino so far.   Bought the robot kit to help me learn more coding.

Sorry if this post is wacky lol.    

Dave

[ZodiusInfuser]

Talk about jumping in at the deep end! Some of the algorithms involved in getting hexapods walking can be very complex, so I hope they provide you with a code base to get you started!

If you really do not know what KSP is, then the tl:dr is, it's a physics-based space simulation game. One of it's big features is the ability to support mods, so mods like Infernal Robotics add moving parts to the game, making it great for re-creating unmanned space missions, but also for testing our mechanical concepts without forking out all the money on parts first. Having built my own hexapod IRL I am very aware of how expensive that can get!

@VR_Dev 's project attempts to use KSP and Infernal Robotics as both the user interface for creating contraptions and the physics engine for simulating them, but with the control being by a completely separate application. I'm sure he can elaborate on any specifics you may have.

[Hooligan Labs]

Yes, KSP is unique in that it simulates interplanetary rocket construction and travel at real time or faster. The physics are surprisingly accurate too. There is a good reason why NASA officially supports this game.

KSP is made with a game engine called Unity, which is like a 3D modeling program fully integrated with Visual Studio. A lot of physics is available free out of the box. This makes it easy to have code manipulate objects realistically.

Tying KSP and robotics together makes sense because our community is mega nerdy and totally supports it.

[Superdave42]

Right on !  Thanks for the explanation.    I'm a total gerd as well (geek/nerd)  I'm going to check this out.  very cool.  Thank you