Biggest thing to consider here may be what kind of lenses you're going to be running with. Of note, the D40 doesn't have an in-camera motor so you would be limited to AF-S lenses if you want auto-focus or having to learn how to manual focus/meter. Personally, I think it's a great exercise that'll help you're understanding of exposure and composition a great deal, but it's a decision that you'll have to make yourself as it'll significantly influence what kind of lenses you can buy.
I started with the D40 and used it for a couple of years quite happily before I upgraded. It's super compact and quite proficient at what it does. Just need to consider that it doesn't have a motor.