How do you defend against a rush? Knight/Archers/scouts?

I play with my cousins and we want to beat 4 Extreme AIs before we move on to playing against people online - I've seen people like TheViper/SOTL wall up like crazy but is that the only way? The AI brings in 20+ archers and Skirms (and all of us are attacked at once) and even if we train enough military to match that, it smashes its way through and before you know it, it's rams and mangonels and done. You have to resign!

Really want to understand, and may be get some tips on how to defend?

Hopefully this will help some other new players too!