I'm not so sure about that, since the Hubble velocity is a property of the space-time and not of the object within, so I'd guess you'd move in comoving coordinates, which aren't affected by the expansion. Also the expansion of the universe doesn't affect gravitational bound objects (much), such like galaxies. The far greater source of differences in velocity would be plain gravity (e.g. Andromeda rushes with 300 km/s in our direction).