What's the argument for going for feudal play when walling up early and going for FC almost always results in better units and better economy?

Just played a game(around 1100 elo-golden pit) in which my opponent went for scouts, I was going to do the same thing (I was berbers and he was lithuanians btw) but unfortunately I fucked up my dark age, couldn't find my boar in time and for some reason my boar stopped following my vil after hitting it. so anyway after being late in the scout war, I decided that going for scouts this late would have no ponit so I just walled up and added farms and he couldn't do anything about it, I was really surprised how well it paid off. he tried to pressure by building a tower but my walling was so good that all he denied was 100 food left in my berries. I advanced to castle age a lot earlier and was able to field many knights and camels, and I added a second TC while pressuring and booming behind it. all the added farms paid off greatly, (the map was gold pit btw) my second TC was right near the gold too. I fielded a lot of knights and camels, I supported them with spears and monks, and despite my opponent having around the same amonut of knights and a monk I won the big fight and after that it was gg.

So my point is, why should I bother with scouts in the first place? Your opponent can just wall up and you are left alone with your dick in your hand having wasted about 500 food into units that won't be able to compete in castle age. Tower pressure doesn't really pay off either because you can only build 1 tower and if you want to make more you need to put vils on stone which will further put your castle age time at a later time.

Sorry for the wall of text but I'm really curious as to why I shouldn't just wall up and go for FC every time.