To what point am I supposed to commit so Feudal/Castle Army ?

For me the question is more important for the transition from Feudal to Castle, because in Castle you still have some reasonable ways in defending yourself, whereas Feudal army really doesn't stand a chance against Castle army.

Anyway, let's say that I am in Feudal Age, reasonably walled so my enemy can't really surprise attack me, and I see that he has some army, how am I supposed to react ? Do just a little bit of army and barely resist his attack, so that I can click castle faster than him, or invest a lot and crush him in Feudal ? To me it seems that the first one to castle will be the one to win on the long run, so I always try to have a really strong eco while getting my army killed, but I might be wrong. What do you think about that ? The same applies in Castle age, where sometimes basically the first one to get a treb out will have a huge advantage, so I was wondering how you guys make your decisions.