I've long been tempted by the E-620 but noise remain the main issue for me. Retail presence of lenses and XD/Compact Flash support to a lesser degree.
There's also the issue of prime lenses which, I find, there's no good option with Olympus. But that could change in the future:
Part of me is also waiting to see how good their upcoming m4/3 DSLR-type camera will ne (not the compact E-P1, though I am curious about that too) and what will happen with the legacy of Zuiko lenses Olympus has built. I don't expect them to dump everything, but will Olympus introduce new lenses like Panasonic did specifically with the G1?
I know there's always something new just around the corner, but with DSLRs, I feel it's important to carefully and patiently shop around since you're buying into a system you're likely going to stick with for years to come. It's almost like choosing between Betamax and VHS in the old days.