How to deal with players who only make hard counters to your units?

I see pro players very rarely go for counter units in the mid game: they rarely go for pikes against knights for example.

A lot of the people I play use exclusively trash counter units and although I can switch into something else, this hurts me more than them because they don't have to commit very hard into for example skirms to make my crossbow army useless.

I also often don't want to invest in units that are bad for my civilization (say Frank crossbows or Briton knights) just because my opponent made a few pikes or skirms, but if I don't switch I end up losing fights and lots of expensive units.

Advice? Feel like counter units and trash dominate noob games for this reason but not sure what I'm doing wrong.