It only takes 1 kiddy script hacker using a shit load of proxies to bring your server down to a slow crawl.
Once the hacker community finds out you don't have any brute force protection you are going to be a prime target.
The one day you awake to a $1,000.00 + bandwidth bill.
Just put something like
http://www.stopthathacker.com on the site and sleep easier.
Hugs,
Danielle