Has anyone phased sneakers out of their casual wardrobe?

Quick disclaimer, I'm not referring to sneakers that you own for a specific activity. I have hiking sneakers and running sneakers that get used pretty exclusively for those sports. I'm referring to sneakers you might wear "out" casually.

As much as I love the look of a converse or a vans the lack of comfort and their tendency to wear out quickly has me considering phasing them out of my casual wardrobe entirely. Has anyone else here done this? What do you wear with a casual outfit where you might do a fair bit of walking? I was thinking a good pair of ankle boots but I'm afraid they might not be too comfortable either. Have you done this and just gone back to sneakers eventually?

EDIT: by ankle boots I meant boots with a shorter shaft height that stops at the ankle, not heeled boots, sorry everyone 😭 not looking to walk 10,000 steps in heels