When I first started looking at mills, I though I could get away with the HF mini mill. Then I decided, why not go a little bigger to the LMS benchtop mill. Then I figured, why stop there? I'll look at a Grizzly G0705... but then I wanted a square column, so I looked at the G0704, then the G0465, then the PM-20V all the way up to the PM-45M and about another half dozen "benchtop" mills besides that.
After it was all said and done, however, I decided to just suck it up and get the biggest and best that I could afford, so I bought a used 3 phase knee mill. Ultimately, I actually paid just a couple hundred more than the G0704 would have cost, and I now have a mill that I can do just about anything I want to with.
Now, I will say that not all mills are created equal, and I may have gotten lucky on mine, but if you know what to look for and keep a sharp eye, you should be able to find a used knee mill that will put out every bit as good of a part as the smaller, imported benchtops, with room and power to spare.
And as far as size goes, I really don't think that a 3 phase knee mill really takes all that much more room than a G0704 (or similar) mill would in the first place. Sure, it's a little wider, longer and taller, but not what I'd consider outrageously so.
Just my $.02. YMMV.