I'd also look at the transfer chain. The D40's front drive line is constantly in motion and you should hear things even when not in 4WD mode, although applying power through the system will make noises louder!
Might be time to put it on a hoist and move things around while inspecting it underneath.