Is building trash units during early castle age considered bad?

Hi, I’m kind of a new player to the ranked ladder of aoe2, currently around 1.1k elo. So I’m still learning the fundamentals of the game and as such, still watching and learning from guides and high elo gameplay.

One thing I’ve noticed is that at least in really high elo 1v1 games, especially among pro players, players never build trash units during the early castle age, and usually never build trash units until imperial age or late castle age. Instead, players are consistently building only gold units. Either producing only crossbows, or only knights, or even both knights and crossbows (and depending on civ, camels too).

From what I’ve seen atleast, during early castle age, players don’t build skirms if the opponent is amassing crossbows, and never build pikemen if they are massing knights. High elo players would rather go for full gold-unit armies instead of building counter-trash units to the opponent’s gold unit.

I was wondering, would the reason be that building counter trash units to enemy gold units is considered sub-optimal during early castle age? And if so, why? When should I consider building a trash unit to counter the enemy main gold unit?

Thanks!