Welp, I think I figured out why I dropped rankings. According to Google Webmaster Tools, I've got errors on my site crawls:
HTTP errors 0
Not found 2
URLs not followed 8
URLs restricted by robots.txt 0
URLs timed out 1
Unreachable URLs 5
So i've fixed the problems listed so if this works then I know what causes this problem
