Shell wget/sed script help needed - GoFuckYourself.com

acctman · 01-28-2011, 02:16 AM

I need help using sed to parse html codiing, this is what i'm trying to do...

1. wget to http://site.com/xap/wp7?p=1
2. view the html extract all the ProductName's from in between title="Free Shipping ProductName"> ... ex: title="Free Shipping HD7-Case001"> , HD7-Case001 is extracted.
3. processes up to page 50

Code:

#! /bin/sh 

for ((  i = 1 ;  i <= 50;  i++  ))
do
	wget -q -O- "http://site.com/xap/wp7?p=$i" | 
	sed ... Need the parsing part

done < "products.txt"

fris · 01-28-2011, 08:06 AM

your answer has been answered

http://stackoverflow.com/questions/4...g-wget-and-sed

fris · 01-28-2011, 08:07 AM

Code:

[chris@jumbo ~]$ cat test.html
<a href="http://www.domain.com" title="Free Shipping HD7-Case001">
link</a>
<a href="http://www.domain.com" title="Free Shipping HD2-Case001">link</a>
<a href="http://www.domain.com" title="Free Shipping HD3-Case001">link</a>
<a href="http://www.domain.com" title="Free Shipping HD7-Case009">link</a>
<a href="http://www.domain.com" title="Free Shipping HD7-Case002">link</a>

Code:

[chris@jumbo ~]$ cat test.html | tr '"' '\n' | grep "^Free Shipping " | cut -d ' ' -f 3
HD7-Case001
HD2-Case001
HD3-Case001
HD7-Case009
HD7-Case002

acctman · 01-28-2011, 08:51 AM

Quote:

Originally Posted by fris

Code:

[chris@jumbo ~]$ cat test.html
<a href="http://www.domain.com" title="Free Shipping HD7-Case001">
link</a>
<a href="http://www.domain.com" title="Free Shipping HD2-Case001">link</a>
<a href="http://www.domain.com" title="Free Shipping HD3-Case001">link</a>
<a href="http://www.domain.com" title="Free Shipping HD7-Case009">link</a>
<a href="http://www.domain.com" title="Free Shipping HD7-Case002">link</a>

Code:

[chris@jumbo ~]$ cat test.html | tr '"' '\n' | grep "^Free Shipping " | cut -d ' ' -f 3
HD7-Case001
HD2-Case001
HD3-Case001
HD7-Case009
HD7-Case002

thanks... everything worked out this morning i had an extra character that I missed typed.

01-28-2011, 02:16 AM	#1
acctman Confirmed User Join Date: Oct 2003 Location: Atlanta Posts: 2,840	Shell wget/sed script help needed I need help using sed to parse html codiing, this is what i'm trying to do... 1. wget to http://site.com/xap/wp7?p=1 2. view the html extract all the ProductName's from in between title="Free Shipping ProductName"> ... ex: title="Free Shipping HD7-Case001"> , HD7-Case001 is extracted. 3. processes up to page 50 Code: #! /bin/sh for (( i = 1 ; i <= 50; i++ )) do wget -q -O- "http://site.com/xap/wp7?p=$i" \| sed ... Need the parsing part done < "products.txt"

01-28-2011, 08:06 AM	#2
fris Too lazy to set a custom title Industry Role: Join Date: Aug 2002 Posts: 55,359	your answer has been answered http://stackoverflow.com/questions/4...g-wget-and-sed __________________ Since 1999: 69 Adult Industry awards for Best Hosting Company and professional excellence. WP Stuff