When should I start worrying about building an army?

I'm practicing against the AI before going into the multiplayer, and I find myself always falling behind militarily. I'd be too focused on getting my economy up and running, by the time it's satisfactory, I would've already lost my allies, and the enemies grouped up against me. I'll be sending my troops one by one from the barracks straight to their deaths. I bumped down the difficulty from hard to moderate and I still find myself outnumbered. At what age should I start building an army? And is the AI too difficult or should I expect the same challenge when playing against other players?