It's not very clear from your post which Lagrange point you are trying to simulate. I think you're trying to simulate L4/L5, which should be possible (well... they won't be automatically stable like the real ones would, you'd have to get the trajectory absolutely perfect for it to work - but you probably know that), but it should be stated somewhere in your post. "Just outside SOI" is pretty general and might get interpreted like you are trying to simulate L1 or L2, which will, of course, never work in a 2-body system. As for the reason why it does work with the Mun but doesn't work with Kerbin... No idea. It most definitely should.