Best way to do this? python/ruby/perl/sed/awk/php/etc?
 

I have a list of links.

example

i only want to get the search engine links being

best way to go about this?

i was using sed, but its printing the 2nd h3

any input would be great ;)

have you tried using xpath?

i don't know exactly what you're doing but in this case //a[position()<4] will bring the search engines in this case, but i'm sure xpath could handle whatever you wanted to do

i guess u just want everything after the first h3 tag and probably has dynamic number of links, but it was just an example of using xpath

I would first extract all the "a href" tags with regex, xpath, or this: http://simplehtmldom.sourceforge.net/

Then detect which urls contain search engine keywords or domains.

Something like this (Typed out fast, did not test):

Zoxxa, sorry, but this makes completely no sense.

If you know all search engine links ($search_engines array), why do you search them?
It's like "I *know* that lighbulb and toy car is inside this box, but I will check it anyway".

Also, what will be if you don't have a link in $search_engines that exists in test.txt ?

And, why you are firing up PHP, performing DOM/regexp processing, while it can be done with single sed command?

if you wanted to use xpath u could use //a[following-sibling::h3[1]]
but kazymjir's method is probably what you are looking for

To you.

Because from his example list he doesn't. He had links with paypal / paxum as well. I suppose he could just select all links between <h3>search engine links</h3> and <h3>payment links</h3> with regex or with xpath, but my code would work with all links he grabs from any section anywhere. If he is only concerned with that block, then regex that part out or xpath.

Like I said,he could use regex to grab the block, or just keep a simple list of search engines he wants to extract.

I code with php, not sed, so obviously my help would be provided with php. I didn't see your post so chill the fuck out allstar.

Fast and dirty -- should be in strict
in a foreach loop

Zoxxa, notice, that Fris provided an example input. example input. It doesn't have to be search engines, it can be anything.
He wants to get the content between given <h3>s. He may not know what content is between them, so your code is in this case useless.
Search engines are only example, as Fris said. There can be totally random links.

Zoxxa, you should chill out. I didn't have on mind attacking you, I just express my opinion.

I'd use the scrapy python framework

http://scrapy.org/

I apologize, I misread his post where it says "i only want to get the search engine links being".

I thought he was being literal and actually meant only urls being search engines, I did not read the text between the h3 tag or his last sed example which shows what he wants.

i dont want whats in the h3 tags, i just want to get the links after the h3 tags, but only those links, not the <h3> block of links after those.

:thumbsup

ugly, but it'll work... and less memory intensive than splitting the file:

foirgot to mention i wanna specify the links based on the h3 tag so <h3>payment links</h3> would grab those links under that h3 element ;)

cut and paste into a file editor, replace all <a href=" with \t<a href="
cut and paste into excel, select column, done

ya thats what i would end up doing ;)

quick mod:

this way it will even get the links if they are spread over multiple <h3>payment links</h3> blocks.

Maybe something along this line:

I guess if multiple h3 blocks continue this will work:

where is the value for the link block stored to grab the certain block?

assign it to a variable, ex. $link_block

fwiw, I prefer using xpath too like alcstrategy mentioned, but probably overkill depending on this need.

cause the input is coming from file_get_contents

that displays the 3rd link block everytime

It displays the first block for me, but all I had to go on was your sample links.txt code above.

ya see the file is full of h3 sections, i wanna specify which one and it will get those links, so its not just 1 h3, u-bobs works for this

its chrome bookmarks, so each folder has a h3 heading for the folder name, just wanna get those links for the h3 folder name.

Ah ok. If you're still interested in a php solution, maybe this?

Output usage:

you could have done it in excel by now or registered a website that parses chrome book marks with php preg_match

is it just chrome bookmarks? I'll make a damn site to stop reading this :)

I think it's more than just needing a quick solution, and not having one yet. It's a way to learn from others code, and to think of different approaches. Maybe one will suddenly be more elegant or preferable or faster or show an approach that you might not have chosen. Maybe a much better regex is presented. That's usually how I view these code threads. :2 cents:

great posts guys ;)

This is untested and you already have a lot of good examples but I believe you have more options to on the command line this way.

#!/usr/bin/perl

die "Usage is $0 <start> <stop> <filename>\n" unless $ARGV[2];

$start = shift;
$stop = shift;
$file = shift;

open(FILE, "$file") or die "Could not open $file $!\n;
while(<FILE>){
chomp;
$sp = 1 if $_ =~ /$start/;
die if ($_ =~ /$stop/;
next if $_ =~ /<h2>/;
if($sp == 1){
print "$_\n";
}
}