You can still move down from 0, but you won't take damage unless you move from positive to 0. Some strange things could potentially happen - for instance, if you have a transport rune to 5 rooms up in the sky, fall 3 rooms down, cast a transport spell back to the castle, go through the rune, fall again, repeat as desired, then fall all the way to the ground you'll take damage as if you'd fallen however many number of rooms down... But if you just walk outside the castle you won't take any damage and your damage buffer is reset.
Pretty sure castles don't have the world map's code, but I don't see why they couldn't use gravity at some point. Guess we'll talk about this at the next Immortal meeting.