Do archers allow for faster castle age time than other units?

I have heard in a Youtube tutorial that building archers in feudal age has an advantage over building scouts or men-at-arms, because they cost no food, so you can get to castle age faster.

Is that really the case? Aren't all types of resources equivalent in a sense, because you can always just put more villagers to gather the resources you need?

Maybe you just need more villagers to continuously build scouts than you would need to build archers.