Figuring out file and directory permissions on a server is hard.

You have to balance security with functionality. What are the most restrictive permissions you can set and still have everything work?