Need an email scraper

[Wolfy]

I need to contact a shitload of people - (read: CONTACT, not spam) - who all happen to be members of a site that lists their email addresses and phone numbers. Is there a program that will search the site and collect the email addresses for me?

[The Duck]

so you are going to write personal messages to each and every single one of them? If not, its spam

[TeenCat]

if its your site you have access to the database and thats easy, if its not your site, you are going to spam no matter how you call it

[Wolfy]

I'm not here for your opinion. I'm here to find a solution to my need.

If anyone knows of a program, feel free to email me - wolfyman at gmail.

^[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}$

[Antonio]

use httrack to download the site (grab html only) then run an email extractor on the html files, I'm sure that there are better ways, google email extractor and see what you come up with

[Wolfy]

Originally Posted by Killswitch

^[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}$

Is that a search query? Where would I use that?

Google: "site:example.com ^[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}$" returned nothing, what am I missing?

Originally Posted by Antonio

use httrack to download the site (grab html only) then run an email extractor on the html files, I'm sure that there are better ways, google email extractor and see what you come up with

That's a start, thanks

[applebee]

LOL. That's a regular expression. Why don't you just hire some cheap coder on Rent-a-Coder and have him write you a script which does it? Probably cost like $50 max.

[fris]

do you have access to the sql db? if so it would be only a few lines of code

Originally Posted by Wolfy

Is that a search query? Where would I use that?

Google: "site:example.com ^[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}$" returned nothing, what am I missing?



That's a start, thanks

Regex to match email address.

[Wolfy]

Killswitch, it doesn't appear to work using it in google - how would I use that "regex"?

Your search - site:gfy.com ^[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}$ - did not match any documents.

Originally Posted by Wolfy

Killswitch, it doesn't appear to work using it in google - how would I use that "regex"?

http://www.php.net/preg_match_all

[Libertine]

Originally Posted by Wolfy

Is that a search query? Where would I use that?

Google: "site:example.com ^[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}$" returned nothing, what am I missing?

It's a regular expression. Used to identify patterns - in this case, email addresses.

[Wolfy]

I've retrieved all the email addresses on the site, thanks to a cool mofo that I'll happily give credit to if he wants it.

Originally Posted by Libertine

It's a regular expression. Used to identify patterns - in this case, email addresses.

Ok, I get that part.

For the sake of knowledge only, since my mission has been accomplished - how do I apply that regex expression to a task like this? I don't have control of the site, so any php pages I would build would be outside of the domain and would not have access to any databases.

What am I missing?

[raven1083]

i am sure you can buy it from internet classifieds.

[papill0n]

if what your doing is not spam then I don't know what is

[GrouchyAdmin]

I still prefer this regex.

[SmokeyTheBear]

Originally Posted by RageCash-Ben

if what your doing is not spam then I don't know what is

i would have to see the site , but if you put your email on a public site , arent you basiclly asking for unsolicited mail ?

Kinda like putting your # on a bathroom stall then saying every pervert who contacts you is "spamming"

the only exception to the rule i would think would be if its implied on the site the mail is to be used for a specific purpose, or its a whois info that isn't posted by the user

[Ozarkz]

I need a poop scraper.

[Angry Jew Cat - Banned for Life]

were you able to sort the first site out?

Originally Posted by Wolfy

I've retrieved all the email addresses on the site, thanks to a cool mofo that I'll happily give credit to if he wants it.



Ok, I get that part.

For the sake of knowledge only, since my mission has been accomplished - how do I apply that regex expression to a task like this? I don't have control of the site, so any php pages I would build would be outside of the domain and would not have access to any databases.

What am I missing?

First you use php to grab the source code of the page, then use the regex to browse that code to strip out the email addresses.

[Libertine]

Originally Posted by Killswitch

First you use php to grab the source code of the page, then use the regex to browse that code to strip out the email addresses.

To elaborate on what he said, what you'd normallly do is something along the lines of setting up a script that takes an url as input, downloads whatever is at that url (usually the homepage), strips out all links using a regexp and saves those, strips out all email addresses using another regexp and saves those too. Then, it uses the links found to determine new pages to repeat the process with - only those on the same domain if you're just getting all the email addresses on that site, or all if you just want to keep finding new email addresses on new sites forever.

Personally, I'd go for another language than php for this, but really, it can be done in pretty much any programming language.

Set a bot like that loose on a big directory, and you'll eventually build up a list of millions of email addresses. Of course, others do the same thing as well, so the email addresses won't exactly be fresh.

Keep in mind that site owners might have email harvester traps, which generate a list of random invalid email addresses and generate dynamic links to themselves as well, ensuring that if your harvester bot isn't protected from them, it will keep getting new invalid email addresses from them forever.

[Angry Jew Cat - Banned for Life]

Perl allows you to be one lazy ass coder. Jump on CPAN and install the Net::Scan::Extract module

use Net::Scan::Extract qw( :all );

my @emails = Extract_Email($source);
print "$_\n" for @emails;

fetch the source code of the page in question, then scan it using that code. done and done.