Coder wanted

[mce]

I need a simple script that scrapes Google for search results and verifies those pages (ie., checks each result if it fits my parameters)

The ones that pass are put in a .txt file I can download

My budget is < $200

[just a punk]

There are many free Google scraper scripts in the Net. The problem is that you (the script) will be sent to recaptcha after a few dozen requests...

You will end up emailing the coder who made that script with messages like: "Ahh! There is a bug! Your script stopped working within a hour! Fix it! Do something! The sky is falling!"

Anyways, good luck with that )

[k0nr4d]

Originally Posted by CyberSEO

There are many free Google scraper scripts in the Net. The problem is that you (the script) will be sent to recaptcha after a few dozen requests...

You will end up emailing the coder who made that script with messages like: "Ahh! There is a bug! Your script stopped working within a hour! Fix it! Do something! The sky is falling!"

Anyways, good luck with that )

https://developers.google.com/custom-search/v1/overview

[Paul&John]

Originally Posted by k0nr4d

https://developers.google.com/custom-search/v1/overview

Custom Search JSON API provides 100 search queries per day for free. If you need more, you may sign up for billing in the API Console. Additional requests cost $5 per 1000 queries, up to 10k queries per day.

Not sure if the budget will be enough for this as well

[just a punk]

Originally Posted by k0nr4d

https://developers.google.com/custom-search/v1/overview

He said his budget is less than $200.

[Klen]

Originally Posted by CyberSEO

There are many free Google scraper scripts in the Net. The problem is that you (the script) will be sent to recaptcha after a few dozen requests...

You will end up emailing the coder who made that script with messages like: "Ahh! There is a bug! Your script stopped working within a hour! Fix it! Do something! The sky is falling!"

Anyways, good luck with that )

Yep, those fucks wants to limit almost everything of what they offer.

[k0nr4d]

Originally Posted by CyberSEO

He said his budget is less than $200.

Yeah, was just pointing out that there is another way to do it then scraping.

[ghjghj]

Originally Posted by mce

a simple script

Originally Posted by mce

that scrapes Google

[~Evilin~]

Originally Posted by CyberSEO

You will end up emailing the coder who made that script with messages like: "Ahh! There is a bug! Your script stopped working within a hour! Fix it! Do something! The sky is falling!"

[EddyTheDog]

Originally Posted by k0nr4d

Yeah, was just pointing out that there is another way to do it then scraping.

I think you could scrape the results from a CSE - I have never hit a recapture on those and I have been playing with a lot recently - Also, you get $300 credit when you open an account with Google - That would buy you a few searches with the JSON api - 60,000 in fact...

That would give you a lot of power over the results as well - You can perform all sorts of magic with it.....

Edit: Drop me a PM with more detail about what you want and if I think I can do it I will make you an offer - I could chuck in $50 worth of JSON api credit so you would get 10K searches - They would be highly customized searches so would work out better value than scraping standard Google results and without the recapture issue...

I could feed them into a DB and give you a way of pulling them in any format you wanted......

[Klen]

Originally Posted by EddyTheDog

I think you could scrape the results from a CSE - I have never hit a recapture on those and I have been playing with a lot recently - Also, you get $300 credit when you open an account with Google - That would buy you a few searches with the JSON api - 60,000 in fact...

That would give you a lot of power over the results as well - You can perform all sorts of magic with it.....

Edit: Drop me a PM with more detail about what you want and if I think I can do it I will make you an offer - I could chuck in $50 worth of JSON api credit so you would get 10K searches - They would be highly customized searches so would work out better value than scraping standard Google results and without the recapture issue...

I could feed them into a DB and give you a way of pulling them in any format you wanted......

Technically, with proxy/browser mix it might be possible to override limit issue.

[EddyTheDog]

Originally Posted by Klen

Technically, with proxy/browser mix it might be possible to override limit issue.

Yeah - If he wants all in for <$200 though he would be pushing his luck...

I'm not sure what he needs - It might even be better to go for a 15 trial on SEMRush and grab as much data that he can - It depends on the info he is trying to pull.....

Edit: If he is looking for data about site positions in SERPS then he would have to use a scrape - The API results are different to normal search results - SEMRush would be good for that as well - I have a link for the 15 day trial somewhere, you don't normally get it...

[CaptainHowdy]

[ANAL PASTE]

Lots of free scripts out there. If you know a bit of php use one of those.