I'll update with a picture when I get home, but after I realized my original design wasnt working (and before I posted), I created a litmus test by sticking the joint directly against a fuel tank of a lander that I know functions with a small structural arm attached on the other end, just to test out exactly what you are describing. I received the same result after I unlocked the joint.
I'm pretty sure the joint I'm using is this after looking at the parts pictures while I'm not at my computer. I'm assuming it's a powered joint. It's the first one in the tech tree I've unlocked from IR, the robotron and pivotron are the only ones available to me for now.
I also have access to the G-00 joint: https://wiki.kerbalspaceprogram.com/wiki/G-00_Hinge
The G-00 joint works correctly when placed in the same location, but I'd prefer to use IR joints, especially in future projects