Ok, I think I have it figured out. I had the module mod-evasive enable. Disabling this seems to have fixed it. At least I've gone over 100 clicks...
I have a Ubuntu 10.04 install with ISPConfig3. I can browse a given url for awhile but eventually it will throw a 403 forbidden error. If I wait...
Separate names with a comma.