Chewy

I always include the short ladder sections in my inventories for this reason. It's hard enough to let one kerbal hover long enough to detach/attach a part, and much much harder for two or more... Just have an engineer attach five small ladder sections or 2 or 3 retractable ladders before the other four go on eva. It works as long as you use [ or ] to switch vessels and you don't need to time warp.