FHG producer please read. - GoFuckYourself.com

qw12er · 12-26-2007, 07:57 PM

I'm trying to make a data warehouse with all the FHGs out there and I'm having an hard time building a system to gather all those infos.

Some sponsors have csv export witch are good for one time update but requires manual interaction for each update.

Then some sponsors have the RSS feed which contain too less galleries to import (On most sponsors)

I though of building a spyder but it's a lot of work and some FHG are protected againt that kind of bot.

So am I missing an easy way to gather all those FHGs ? What would be the best way to do so ?

Mutt · 12-26-2007, 09:00 PM

the URL in your sig - did you let the domain expire?

qw12er · 12-27-2007, 08:35 AM

The more I check different sponsors the more I see new way of exporting data...

Isn't there a standard for XML and CSV exportation ! Barnak

just a punk · 12-27-2007, 08:44 AM

Quote:

Originally Posted by qw12er

I'm trying to make a data warehouse with all the FHGs out there and I'm having an hard time building a system to gather all those infos.

Some sponsors have csv export witch are good for one time update but requires manual interaction for each update.

Hint: use your own software to gather and process the data

qw12er · 12-27-2007, 09:12 AM

Quote:

Originally Posted by cyberxxx

Hint: use your own software to gather and process the data

Well I do ... I programmed my own piece of software but what I though would cover 80% covers 20% of the market and I have to do manually the 80% ...

Two cie would have 3 ways of doing things in this industrie

12-26-2007, 07:57 PM	#1
qw12er Confirmed User Join Date: Apr 2004 Location: Montreal Posts: 799	FHG producer please read. I'm trying to make a data warehouse with all the FHGs out there and I'm having an hard time building a system to gather all those infos. Some sponsors have csv export witch are good for one time update but requires manual interaction for each update. Then some sponsors have the RSS feed which contain too less galleries to import (On most sponsors) I though of building a spyder but it's a lot of work and some FHG are protected againt that kind of bot. So am I missing an easy way to gather all those FHGs ? What would be the best way to do so ? __________________ I have nothing to advertise ... yet.

12-26-2007, 09:00 PM	#2
Mutt Too lazy to set a custom title Industry Role: Join Date: Sep 2002 Posts: 34,431	the URL in your sig - did you let the domain expire? __________________ I moved my sites to Vacares Hosting. I've saved money, my hair is thicker, lost some weight too! Thanks Sly!

12-27-2007, 08:35 AM	#3
qw12er Confirmed User Join Date: Apr 2004 Location: Montreal Posts: 799	The more I check different sponsors the more I see new way of exporting data... Isn't there a standard for XML and CSV exportation ! Barnak __________________ I have nothing to advertise ... yet.