I just upgraded to a D90 from a D40 myself, and I can tell you I was sorry to let go of my good old D40; as some others have already written, it's just a great camera that can take stunning looking pictures right out of the box, it's compact, it's light and it's easy to use. And I couldn't care less about the pixel count, I never missed a single extra MP, that "humble" 6.1 MP sensor still delivers the goods, period (except those interested in heaving cropping, of course).
My advise is that if you're happy with it, just keep it!
The only reason I upgraded is because I really wanted more flexibility and control, I missed direct access to ISO, WB, quality, burst mode, etc. I found a great deal for a brand new D90 and decided to go for it.
If you decide to upgrade though, I wouldn't recommend you to go for a D3x00 or D5x00, is just not worth the effort or the money IMHO, the natural step forward from the D40 is an "enthusiast" model like the D90 or D7000. You'll see what I mean when you look through that glorious, bright pentaprism viewfinder and start playing with all those extra control buttons.