Sportsmanship and laming

The other day I was playing a game Cumans vs Huns. He went scouts. I went with a rather memey approach and built spears and made forward seige and rams. After destroying stable and barracks he started to stone wall me out but I managed to break in at one spot near his gold. He had just gotten to castle age and I was still probably a minute til I clicked up. So I used my 2 vils to house wall in his gold he was mining. Needless to say he was upset and said I ruined the game. But personally I feel like if you're a 14xx and you allow someone to house wall in the gold you were mining in castle age, that's a tiny bit your fault. Earnestly, was I a jerk for that?