How to deal with an opponent intent on creating strong defensive structures?

Someone I play against a lot tends to build a ton of castles, bombards and walls, making it extremely difficult to win.

They almost always choose the Koreans or the Byzantines.

Best CIV to counter them?