although presently a Nikon shooter I will chime in because I used to own the E510 and the E520 and I do have a considerable appreciation for Olympus.
Fact: Olympus is worse than the competition at high ISOs. Perhaps not by much and, certainly, less than it used to be, but worse nonetheless, as the competition - especially Nikon, progresses as well.
High ISO is not the priority it seems to be, unless you DO shoots a lot in dim light. For example, shooting concerts, or in sport arenas in conjunction with f2.8 lenses, etc...
You really have to see what kind of shots you like to take, or you tend to take most of the time. For instance, I very rarely shoot at high ISOs because a) I don't have the need to, and b) I really do not like high ISO noise, even with cameras capable of mitigating it.
If you really NEED high ISO-capability, then get a full-frame D700, because it's the best at it. However - having owned a D700 after succumbing to the high-ISO 'hype' I found that I was a victim of what I would call the 'high ISO Debate". Victim, because I only bought it for its high-ISO performance, not because I actually needed it. And with the D700, came a bunch of issues: you might be in the full-frame 'league', but you also find yourself with a camera that is big, heavy and, mated with corresponding high-grade lenses, even MORE heavy and bigger, and really visible. I felt quite self-conscious of taking my camera and associated lenses with me, whereas I do enjoy the anonymity of being able to carry my camera around and 'catching' the moment as it happens.
And, because I mated the D700 with equally pro lenses, suddenly I was carrying a full bag of lenses, as pro lenses tend to be limited in their focal range. For a pro, this is not necessarily an issue, but it certainly was for me. In short, photography with a high-ISO capable camera and f2.8 lenses, became an (expensive!) chore and lost its 'click-clack Kodak' appeal for me. In other words, this is what you get if you chase the high-ISO game.
I find that I took very, very few high-ISO shots that I do like. These tend to be more 'arty' types of photos because of the high-ISO grain, no matter how good the camera and how good it is at suppressing high ISO noise. It's always there from 1600 ISO, with any camera. The colours suffer too. I might take some rare high-ISO shots, but it's mainly because I need to capture a specific - and for me, rare - moment. And in that case, Oly or Nikon or Canon would do just as well, because it's a 'utility' shot, not one taken for its composition or image quality.
So, if you find yourself taking mainly shots during daytime, or at worst, in museums, the latest generation Olys are great. I maintain that no other camera does colours like Oly does - warm, with the most beautiful blues. Pictures just look nice with Olympus. And now that they improved their TruPic, allowing for a weaker anti-aliasing filter, they're sharper than what they used to be with the E520/420 series. In addition, Olympus lenses are fantastic, even the kit lenses. And they tend to be smaller than Nikkor/Canon, and, unless you go for the very top-grade Zuikos, cheaper than Nikkor and Canon lenses.
Without ranting on too much, my main point is that one should not get too much caught in the high-ISO game, unless you actually need that feature. There's no clean high-ISO, no matter what brand of camera you use; there might be less or more ISO noise, but it's there. I haven't seen many 'wow' pictures taken at high ISO. If high-ISO performance is indeed critical to you, then you need to consider another brand such as Nikon or Canon because the physical limitation of 4/3 sensor will probably always be a step behind larger ones. As Olympus improves theirs, so does the competition.
But if you are interested in beautiful pictures in a portable package, then Olympus has much more to say in that regard...