raptor racing, rabidchimp, and 935motorsports have the largest selection of MK2 parts. I'm looking to pick up a MK2 any day now and will be adding more parts to our site.
Good first modifications:
Intake: Rabidchimp sells a nice mandrel bent metal intake pipe to replace that restrictive plastic/rubber OEM system. They also have an open filter to get rid of the restrictive factory airbox
Exhaust: Replace the cat with a high-flow or test pipe as it has probably clogged from years of use. A Pacesetter header for the 7mge can be made to fit, or you can buy a modified version from rabidchimp. 935motorsports makes a nice cat-back to round out the exhaust.
Springs: Eibachs lower the car nicely and still ride well. Dobson (sp?) make very nice springs but take a while to get in the US. For what its worth, the camber is not adjustable on the MK2 so you may want to get Cusco front camber plates (often in stock at raptorracing -- otherwise a 6-week wait from Japan). The rear suspension has to be modified to adjust camber -- so don't slam the car or look at celicasupra.com for details on making it adjustable.
Bushings: 935motorsports carries some nice bushing kits to help take out the slop from 20+ year old rubber bushings
Brakes: Jim King did a big brake kit a while back, but it may be difficult to find. Precision brakes has a couple options available too (see my group buy *plug*). Rotors, pads, and stainless lines are readily available...
Transmission: The clutch and flywheel from a 7m-ge engine are the same, so you have a number of lightweight flywheels and strong clutches available. Short-shifters for the MK3 also fit, so you can either do the SOGI "cut shifter" modification, or buy a nice C's or Jim King short-shifter (the Jim King shifter is very very nice and is my personal choice)
Wheels: MK2's have an unusual offset. It's difficult to find 4X114.3 wheels with +10 offset, but look around or hook up with a group buy on the celicasupra forum. Corolla GTS guys will be your allies as they are in the same boat
Engine: If you want a little more power, you can drop-in a 6M-GE relatively easily. It uses the same block and head so your factory sensors will plug right in. You will need to use the oilpan and oilpump from a 5M-GE and have to do something with the crank pulley (its been 2 years since I did a 6m swap so I forget
) If you want more power, you can drop in a 7m-gte. The oilpan and pump will need to come from a 5M-ge, and you'll need the sensors, ECU, and wiring harness from the 7m-gte. You'll also want to do some custom plumbing for intercooler and exhaust and will have to do some wiring to make your gauges work. See Celicasupra.com for more accurate/detailed info on these swaps
That's all I can think of at the moment, good luck