What is the point of early aggression?

I learned the scout rush build order, and can execute it with minimal idle time. I send my five scouts only to find they have mostly walled and have spears. I manage to get in there base and pick off 3 vils. I retreat my scouts and come back to a fully walled base. I shifted my economy to go to castle so I could get siege, but they got up before me and had 3 this when I hit castle. What is the point of being aggressive if defending is way cheaper and way easier, and how do I counter a full wall with a scout rush bo?