Machinability is likely the biggest factor, it just takes *massively* longer to remove the coarse scratch patterns with the high carbide steels. It isn't hardness as much as it is carbide content and distributions. Note there isn't uniform agreement on polishing as with most effects. People have complained about D2 in that regard for a long time but there are makers who can produce a very high quality finish. I tried it once on D2 just to see how difficult it was, forget about it without power tools. There are other issues like actual uniformity of the steel, some people have complained for example that ATS-34 was/is "dirty" in this regard which prevents a high finish. Really low end steels can often take a very high finish so it definately isn't a sign of quality.
-Cliff