PHP gurus - Need your help - GoFuckYourself.com

qw12er · 01-11-2008, 06:10 AM

I'm trying to code a robot and some how the remote servers finds out it's not a real user and redirect my robot. So I made three files to catch the diff in the header I'm sending.

the first one is a server scirpt which displays $_SERVER values.

Code:

<?
	print("User : $_POST[user]<br>Passw : $_POST[pass]<br><br>");
	
	foreach($_SERVER as $key => $value){
		print("<b>$key</b> = $value<br>");
	}
?>

The second one is a Form I use to send info via a real post :

Code:

<body>
<form id="form1" name="form1" method="post" action="Server.php">
  <label>
  <input type="text" name="user" id="user" />
  </label>
  <input type="password" name="pass" id="pass" />
  <input type="submit" name="button" id="button" value="Submit" />
</form>
</body>

which returns :

Code:

HTTP_HOST = myfhg.com
HTTP_USER_AGENT = Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
HTTP_ACCEPT = text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
HTTP_ACCEPT_LANGUAGE = en-us,en;q=0.5
HTTP_ACCEPT_ENCODING = gzip,deflate
HTTP_ACCEPT_CHARSET = ISO-8859-1,utf-8;q=0.7,*;q=0.7
HTTP_KEEP_ALIVE = 300
HTTP_CONNECTION = keep-alive
HTTP_REFERER = http://myfhg.com/Lab/Form.php
HTTP_COOKIE = __utma=234929437.1319266469.1195355793.1200011167.1200017816.198; __utmz=234929437.1197320899.89.2.utmccn=(organic)|utmcsr=google|utmctr=myfhg.com|utmcmd=organic; __utmc=234929437
HTTP_CACHE_CONTROL = max-age=0
CONTENT_TYPE = application/x-www-form-urlencoded
CONTENT_LENGTH = 37
PATH = /sbin:/usr/sbin:/bin:/usr/bin
SERVER_SIGNATURE =
SERVER_SOFTWARE = Apache
SERVER_NAME = myfhg.com
SERVER_ADDR = 72.55.179.76
SERVER_PORT = 80
REMOTE_ADDR = 70.80.131.22
DOCUMENT_ROOT = /home/www
SERVER_ADMIN = root@localhost
SCRIPT_FILENAME = /home/www/myfhg.com/Lab/Server.php
REMOTE_PORT = 55452
GATEWAY_INTERFACE = CGI/1.1
SERVER_PROTOCOL = HTTP/1.1
REQUEST_METHOD = POST
QUERY_STRING =
REQUEST_URI = /Lab/Server.php
SCRIPT_NAME = /Lab/Server.php
PHP_SELF = /Lab/Server.php
REQUEST_TIME = 1200020756

qw12er · 01-11-2008, 06:15 AM

The last script is a bot script that's supposed to simulate the same post as the second script :

Code:

		$sock = fsockopen($URL, 80, $errno, $errstr);
	    fputs($sock, "POST $script HTTP/1.1\r\n"); 
	    fputs($sock, "Host: $URL\r\n");
	    fputs($sock, "Content-type: application/x-www-form-urlencoded\r\n");  
	    fputs($sock, "Content-length: " . strlen($data) . "\r\n"); 
fputs($sock, "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11\r\n"); 		
	    fputs($sock, "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5\r\n"); 
	    fputs($sock, "\r\n"); 
	    fputs($sock, "$data\r\n"); 
	    fputs($sock, "\r\n"); 
	
		while (!feof($sock)){
			$result .= fgets($sock, 128); 
		}

print($result);
	    
	    fclose($sock);

which returns :

Code:

HTTP_HOST = www.myfhg.com
CONTENT_TYPE = application/x-www-form-urlencoded
CONTENT_LENGTH = 25
HTTP_USER_AGENT = Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
HTTP_ACCEPT = text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
PATH = /sbin:/usr/sbin:/bin:/usr/bin
SERVER_SIGNATURE =
SERVER_SOFTWARE = Apache
SERVER_NAME = www.myfhg.com
SERVER_ADDR = 72.55.179.76
SERVER_PORT = 80
REMOTE_ADDR = 72.55.179.76
DOCUMENT_ROOT = /home/www
SERVER_ADMIN = root@localhost
SCRIPT_FILENAME = /home/www/www.myfhg.com/Lab/Server.php
REMOTE_PORT = 41244
GATEWAY_INTERFACE = CGI/1.1
SERVER_PROTOCOL = HTTP/1.1
REQUEST_METHOD = POST
QUERY_STRING =
REQUEST_URI = /Lab/Server.php
SCRIPT_NAME = /Lab/Server.php
PHP_SELF = /Lab/Server.php
REQUEST_TIME = 1200056719

WHY ARE THE RESPONSE DIFFERENT ?
How does the remote server spot my robot ? and how can I fix this ?

margarita · 01-11-2008, 06:20 AM

Try to sniff IE's request headers on that url and use the same request as IE

k0nr4d · 01-11-2008, 06:47 AM

You're using no subdomain on the first attempt and www.domain.com on the bot attempt. Other then that, and the fact you are missing a couple of headers on your bot post, You successfully posted as Mozilla so it's working fine.

qw12er · 01-11-2008, 03:29 PM

Quote:

Originally Posted by margarita

Try to sniff IE's request headers on that url and use the same request as IE

actually this is a good idea ! thanks

d-null · 01-11-2008, 04:04 PM

let us know how that works out

qw12er · 01-11-2008, 09:12 PM

my guess is that the response from the server tries to create cookies when I use my bot. My guess is that my server refuses to create those cookies.

How can I test if the my server accept those cookies ? or work around this problem ? (maybe by simulating interaction with cookies ?)

Zoose · 01-11-2008, 09:49 PM

If you use CURL you can accept cookies, however I've run in to the same problem with my robot/spider ( trying to spider FEMJOY hosted galleries ) even when using CURL so good luck.. you'll probably need it unfortunately.

http://us3.php.net/manual/en/ref.curl.php#75525 - there's a little function that will help if it is indeed cookies that are the issue. Please post back here if you happen to figure it out!

GrouchyAdmin · 01-11-2008, 09:49 PM

Just break down and use cURL, like you should be. Set a cookie file, init it, and forget all of your problems with your happy cURL wrapper.

woj · 01-11-2008, 09:54 PM

like GrouchyAdmin said, don't use fsockopen, curl will work 20x better...

01-11-2008, 06:20 AM	#3
margarita Confirmed User Join Date: Jun 2003 Posts: 917	Try to sniff IE's request headers on that url and use the same request as IE __________________ Check out my favourite kinky sponsor and must-have tool Nifty Stats

01-11-2008, 06:47 AM	#4
k0nr4d Confirmed User Industry Role: Join Date: Aug 2006 Location: Poland Posts: 9,228	You're using no subdomain on the first attempt and www.domain.com on the bot attempt. Other then that, and the fact you are missing a couple of headers on your bot post, You successfully posted as Mozilla so it's working fine. __________________ Mechanical Bunny Media Mechbunny Tube Script \| Mechbunny Webcam Aggregator Script \| Custom Web Development

01-11-2008, 04:04 PM	#6
d-null . . . Industry Role: Join Date: Apr 2007 Location: NY Posts: 13,724	let us know how that works out __________________ __________________ Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad! Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs Check out the #1 WordPress SEO Plugin: CyberSEO Suite

01-11-2008, 09:49 PM	#9
GrouchyAdmin Now choke yourself! Industry Role: Join Date: Apr 2006 Posts: 12,085	Just break down and use cURL, like you should be. Set a cookie file, init it, and forget all of your problems with your happy cURL wrapper. __________________ AIM: GrouchyGfy

01-11-2008, 09:54 PM	#10
woj <&(©¿©)&> Industry Role: Join Date: Jul 2002 Location: Chicago Posts: 47,882	like GrouchyAdmin said, don't use fsockopen, curl will work 20x better... __________________ Custom Software Development, email: woj#at#wojfun#.#com to discuss details or skype: wojl2000 or gchat: wojfun or telegram: wojl2000 Affiliate program tools: Hosted Galleries Manager Banner Manager Video Manager Wordpress Affiliate Plugin Pic/Movie of the Day Fansign Generator Zip Manager

01-11-2008, 09:12 PM	#7
qw12er Confirmed User Join Date: Apr 2004 Location: Montreal Posts: 799	my guess is that the response from the server tries to create cookies when I use my bot. My guess is that my server refuses to create those cookies. How can I test if the my server accept those cookies ? or work around this problem ? (maybe by simulating interaction with cookies ?) __________________ I have nothing to advertise ... yet.

01-11-2008, 09:49 PM	#8
Zoose Confirmed User Join Date: Aug 2006 Posts: 268	If you use CURL you can accept cookies, however I've run in to the same problem with my robot/spider ( trying to spider FEMJOY hosted galleries ) even when using CURL so good luck.. you'll probably need it unfortunately. http://us3.php.net/manual/en/ref.curl.php#75525 - there's a little function that will help if it is indeed cookies that are the issue. Please post back here if you happen to figure it out!