Wow, that's a lot worse than I thought it was. It's hard to tell from the photo, but apparently there is a layer of dust on top of a white crust. I assume that the white crust is calcium deposits from the moisture in the air. Removing the dust and crust was a lot harder than I thought. I ended up taking the fan out of the housing and using q-tips and screwdrivers to break the dust and crust free. I vacuumed up the junk as I went.
After I thought I was all done I hit it with the air spray again and all sorts of more dust blew out of the fan assembly.
After it was all cleaned up I hooked it back up to power and it all sounded good. After putting the fan back in the ceiling and the low rumble that we used to have in there was gone.