I’ve never had a problem.
Got my first rubber handled BK back in 2014 and used it extensively for years without issue.
Back then the rubber had small bits of what I assume was some type of red fiber mixed in and occasionally some of the rubber would flake off with use but nothing that even remotely affected the integrity of the handle.
The current rubber has some type of white fibers in it and seems more refined and doesn’t suffer from what I considered a non issue to begin with.
Rubber is my favorite handle material for the knives but not so much for the axes… I find it too grippy and abrasive for any extended periods of chopping.
I use to have a Combat Axe, Hammer Combat Axe and a Stealth Axe all in rubber… sold two, gifted the other and switched to the smooth Maple for the axe handles.