I always lose before Castle Age, help!

To try and keep this as short as possible; I am trying online games for the first time (well, ranked anyway. Tried unranked and that was a BAD idea), and I pretty much lose around the time I reach Castle Age every time. I always get assaulted by unending waves of Feudal Age units, cannot produce enough counter units fast enough and my economy goes down the drain. But what confuses me is that my score is often around my opponents, or even higher, and I advance to Feudal around the same time as my opponent. I am clearly doing something wrong, but I have no idea what it is.

For background; I have done the online practice levels and passed them all, and know all the unit type matchups. But I just cannot get enough economy (despite keeping villager and town centre idle time to an absolute minimum) to afford a military by the time my opponent has enough of an army to kill all my villagers.

I am looking for general tips or anything really that can help.