Search Engine script
 

Hello,
I am looking for a search enigne script, like goole .. with the feature to check linksback and modifying the quality score for keyword result ..
e.g. if site has 10 linkbacks rise the position of search result ...

Any idea ?

Thankyou
Jimba

indexing how many sites ?

approx 1000 sites

would like to know a good one ... 500k sites should be indexed

1000 Sites: Nutch + Hadoop
500k sites: get a small server farm then Nutch + Hadoop

It's worth noting that Nutch is not trivial to deploy, so if you modified your requirements there are plenty of far simpler script solutions to simple search.

explain better please ...
I've no idea of what is that, and what is used for ... :helpme:helpme

Thankyou

Unless you use Sphider http://www.sphider.eu or a similar script then implementing a search engine is non trivial.

What you want to do can't be achieved by Sphider because the indexing, weighting and ranking of sites based upon keywords and inbound links is a resource intensive task.

Sphider does not support ranking based on inbound links, however Nutch will as will some other open source search platforms.

Implementing such a thing is non trivial. You could develop your own solution and run it's crawler on Amazon EC2 instances to save having to invest in hardware, however you still need somewhere to house your database.

Here's something I posted on the subject two years ago in response to a similar question.

https://gfy.com/showpost.php?p=18075524&postcount=8

It's a bit out of date now, I'd lean toward Nutch these days.

Adult king you fuckin' rock!!!
 

Adult king you fuckin' rock!!! :playboy

Do you know PHP and/or a language you can dev with on a server (Perl, C++)? It's pretty simple to get a crawler happening if you're apt with programming. Once you've got an "indexer" happening, the weighting and all that jazz is the fun part :)

The search engine isn't really the hard part to program, more the crawler if you ask me. Depends on what you're crawling and how you're displaying it too.