You're not gonna find a Solstice, Z3, or s2000 anywhere near your price range. And if you do, you probably aren't gonna want the headaches that come with them if you do.
Even high milage s2k's are starting at $10k. Nicer examples run in the $15k range.
Z3's and Solstices are gonna be around $5k at the bottom end of the spectrum for a driver, so if you can double your budget, you might be able to find one, but be prepared to add in some cash for things that have been neglected. Clean, well maintained cars will be closer to $7-10k. Of the two, the Z3 would be the better choice.
A Miata might be a viable option, but you'll most likely be looking in the $3k range for a lesser quality one. They hold their value quite well - I sold my 95 for $6k about 7 years, and that's still what a clean one sells for. The nice thing about them is, mechanically, they are pretty bulletproof. So even a clapped out one will most likely run just fine (or can be made to cheaply). At that lower price ranges, look for worn suspension, neglected interiors and soft tops, and accident damage - or just general neglect. There is also a huge aftermarket for them, as they are still very popular in road racing.
I'd stay away from the Fiero. They were plagued with issues, and expensive to maintain. You can probably find one in your price range, but keeping an old one running well is expensive and frustrating. Just don't do it.
Might want to look into a Toyota MR2 as well. The first gens (84-89) might be had in your price range. They were also fairly reliable if I remember correctly. Handling was a tad funky, being mid engined, but not bad (just different if you've never driven a mid engined car), and much better than the Fiero by far. If I had a choice though, I'd shoot for a Miata over an MR2.
Now, if it doesn't have to be a drop-top, the options open up a little bit....
Old Honda CRX's are a hell of a fun car and drives like a go-kart. They are front-wheel drive, with only 100-120hp depending on the trim, but also only wight 2000lbs. I autocrossed and did track days in one for years when I was younger, and it's still a popular car in those circles. I'd say look for an Si version, but they only came in a stick. They are also getting a bit hard to find in general because so many turned into tuner cars in the late 90's and early 2000's, or the quarter panels rusted out.
First gen RX-7's might also be had fairly cheap. But they are a rotary engine, and require a specific set of skills (or cash) to work on.
Most of what you'll find in your budget are gonna be 80's or 90's, Japanese compacts, and not proper sportscars. That's not a bad thing, as this was a great time for compact imports. Most are dead reliable, and quite a few are extremely rewarding to hustle down a twisty road, even if not fast by today's standards. Just remember, 30 year old cars come with 30 year old car problems. Likewise, $2000 cars come with $2000 car problems.
Some to look for:
Honda Prelude, Civic (2 door was quite lively to drive),or CRX (see above), early Toyota Celica or Supra (81-86, not the Mk-IV), or maybe an Acura Integra. If you're looking for 'cool', none of these are gonna scream it, but for $2000, it's about what you're gonna get unfortunately. I will say, I've driven them all at some point, and they all were quite decent drivers for their time.
Your price point is just too low for a true, bonafide toy... unless of course, you eliminate 2 wheels.
