Strange Idea: Infantry, that can gather resources, like villagers

It's been quite a while, since I've played Age of Empires in any capacity, but last night, I had a strange dream involving it.

In it, there was a Teutonic Knight mining stone next to some villagers. Much the same animation, with the pickaxe and everything. And what was probably more horrifying, was how it was moving at the same speed as the villagers.

Now, it's probably not feasible to mod into the game, or anything...but what would you folks think of it? If infantry units could be used to gather resources, in a similar way that villagers do? Most likely they'd be nowhere near as efficient--maybe they couldn't carry as many, maybe their gather speed would be notably smaller, and they wouldn't be able to build/repair buildings.

But, do you think the idea of infantry (not cavalry, navy, nor siege) units working the economy could be something...that could be worked with? How do you think it would affect strategies? What civs would benefit the most from them?

Feedback is always appreciated!