How to queue up two houses in beginning efficiently?

I know you can queue up two houses using the SHIFT key, but when I do that in the first part of the game, my two villagers move from the first house to the second house build. I want to avoid having all three villagers moving to build on the same house, i.e. two vills on one house and one on the second. Thanks!