Age 3-4 Japanese Shrine rush question

Hey everyone. I've been playing the game for a few years and I like to think I understand the game to a decent extent.

The issue I'm having is the sheer amount of time it takes for me to collect enough resources to age 3. I usually play the Japanese, never online. I find myself in age 2 around three or four minutes using the shrine boom strat with a small twist I added myself. Once I'm in age 2 I don't really know how to proceed in terms of balancing military production and economy. I'm always the first to get to age 2, but almost always one of the last to get to age 3.

I can't figure out if I just have a bad habit of dumping too many resources into my age 2 unit and market upgrades, or if my eco is slumping due to aging up with only 6 villagers.

Could anyone give me some insight into how to play play age 3-4 as Japanese with a shrine rush?