It was the engine with the column on the top. I noticed everytime i put the launch clamps to low on the engine, the whole ship exploded right when the physics kicked in. Putting them near the top of the engine solver that issue.
This is what i don't like that much. First they get rid of the Launch tower and the VAB and the SPH and not even the second KSC is there to remind us of the good old days
It works perfect. No wobbling at all. Edit: i discovered a little bug. It seems like if you attach things too low on the engine the whole thing wobbles itself to death on the pad.