Hi Nathan, the image quality between the two cameras is roughly similar, so you'd be buying the 40D on features and build. Do have another look at our 40D verdict page where we compare it against the D80.
http://www.cameralabs.com/reviews/Canon ... dict.shtml
If you want or need features like 6.5fps, Live View, superior build etc, then the 40D is a great camera. But as Thomas says, the D80 is half the price, so you could buy it with money to spare which could alternatively go on lenses or other accessories.
And while I admire your purist approach of going for a 50mm lens alone, you may find it a bit restrictive as your only lens - remember this will be acting like 80mm on your 40D, so you're going to miss out on wide angle until you can afford a seocnd lens. Be great for your portrait work though...
I'd also urge you to try the 50mm f1.8 before buying. The optical quality is good, but it's a slow and noisy focuser. The f1.4 model is MUCH classier with quick and quiet USM focusing, full-time manual focusing and of course a slightly brighter aperture too.
PS - if you end up buying from Ritz and they'll give you the price online, don't forget you can click through to their store from our Affiliate page
to help support this site!