Mid-Late game problems

I usually have a good start with even scores till castle age. However there I notice that the opponent starts getting a higher score and even though I research, have a lot of villagers, wall up and keep a respectable army, I end up losing in one way or another. I need tips to up my late game where I tend to fall behind.