How to avoid losing villagers to scouts?

I hover between 1400-1500 elo and at this elo people are starting to be super active with their scout. What I find now is that almost every single game I'll lose a walling vill, boar luring vill, first vil to gold or whatever to the enemy scout. Either it'll be dark age pre-loom, or as soon as they hit feudal. I played a quick play recently and got matched with a 1700. He killed 3 of my damn villagers with just his scout.

My micro skills are pretty terrible so being able to quick-wall out a scout is pretty much a non-starter for me. So as far as I can tell that means I have to either have 2 villagers doing everything to avoid being picked off, or I have to play super conservative and only make tiny walls. It's kind of been tilting me recently because it just feels so random whether they find me or not. So any tips on dealing with or avoiding this would be useful.