Does spam clicking a vil to build something actually make them walk slower?

The enemies are pounding your palisade walls and to "hurry" up a villager you spam click it to build the house that's behind the attacked wall. Does doing that actually make the villager get to the house slower? That seemed to be the result in the scenario editor tests but I want to confirm that.