A good forge can be built for the cost of the insulation materials (wool and satanite) plus about $100. While 2" of insulation is better if the forge will run for hours on end, 1" will suffice if you are just a hobbyist.
A simple small forge can be built for around $50 total with some scrounging skills.
The only items you need to be of a specific type and quality are the insulation (Ins-wool/Kao-wool/mineral wool or UK equivalent) and the refractory (satanite or UK equivalent like Plistex). The rest can be any sort of stuff.
The shell can be anything that is not going to melt. A tin waste bin, old fire extinguisher, old oxygen tank, old propane tank, old air tank, metal 5 gallon paint pail, etc. It doesn't have to be heavy or pretty. I saw one forge made with the shell made from rolled up chicken wire. Scrap yards have hundreds of items that will work.
Small forges can be made with a large coffee can, 1 Sq.Ft of insulation wool, a pound of satanite, and a good plumbers torch.
If you have a good supply of soft firebricks (like from the lining of a burned out pottery kiln), you can make a rectangular forge, or angle the sides of the bricks and line a round forge. (The Atlas forges have K-26 soft firebrick lining)
While a fancy burner is great, a simple venturi or blown burner can be cobbled together from the stuff you find in a couple friends garages. Junk yards are again a great place to find stuff to use. Any pipe fittings from 3/4" to 1.5" will likely work. Small burners can be made with 1/2" pipe.
Black iron pipe is the type pipe you want ( or stainless) for the burner tube, but the rest can be galvanized.
Blowers can be many things if you are on a budget. Old larger size hair dryers, blowers from furnaces, small electric leaf blowers, old shop vacs, surplus blowers from ebay, etc. The manifold ( piping from the blower) and air gate valve can be PVC. You need steel once it gets to the elbow before the gas insert point.
This is a list of the basics for a decent basic forge ( off the top of my head):
14" to 16" shell of 7"-8" pipe (or anything about that size).
2 feet of 24" wide Kao-wool/Ins-wool (or 4 feet for a 2" thick liner).
10# satanite ( or equivalent) - get 20# if you can afford the extra $20, it is useful for many things.
A simple blower fan of some sort that delivers a good flow of air ( at least 50CFM)
12" of 2" pipe
2" elbow
2X1" bell reducer
2" gate valve (or 1")
12" of 1" black iron ( or stainless) pipe
a short nipple to fit the gate valve
1" tee
1X1/4" bushing ( or 1X1/2")
1/4" or 1/2"gas valve and nipple to screw into the Tee
Gas line fitting for the gas valve (welding hose fitting)
propane regulator that goes from 1-30PSI
12 foot propane hose
20# or larger size propane tank
A few firebricks ( hard or soft) will do for the front and back walls. You can use the cut-off pieces of ins-wool and some satanite to make covers with ports for the ends and hold them in place with the bricks. Some chaps just use the uncoated wool on the ends.
That sounds like a big list, but I bet you could find half of it in a few weekends of scrounging friends junk bins, visiting some flea markets, and going to the hardware store. You can drop by a plumbing and heating shop and often get free small pieces of pipe, maybe be allowed to have old pipe fittings from their scrap bin, and scrounge gas valves from old water heaters.