Question about conservation of energy

Alright right out of the gate I wanna say I’m 19 and it’s been 2 years since I did grade 12 physics.

I was under the impression that it will take the same amount of energy to do something whether it’s fast or slow. So if you brake harder should it not produce the same amount of wear as if you did it slowly. You’re stopping the same mass, just over a longer period. Can someone explain why it wears brakes faster. Am I not factoring something in?

I know this is probably a pleb question for a bunch of smart people but I’m genuinely curious so please no hate.