There is a few sites that give you an idea if you've linked to bad neighbourhoods, but generally they tag a lot of stuff from adult as negative, and also deem a bunch of blog systems as 'maybe bad' from spam, so they really arent much help (
http://www.bad-neighborhood.com/text-link-tool.htm) I wouldn't rely on them too much.
The only way to be sure is to keep an eye on what you link to. Check your links, check for redirect or dead links, possible exploits etc. You can use software like Xenu Linksleuth to spider your own sites and check for these, not exactly what it was made for, but great for doing it none-the-less.
Xenu will also give you a great list of orphaned files or pages on your site as well, another handy thing it does while spidering.
I guess bottom line is, police you're site, before they do.