Search Engine script

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • jimbaman
    Confirmed User
    • Jun 2003
    • 267
    #1

    Search Engine script

    Hello,
    I am looking for a search enigne script, like goole .. with the feature to check linksback and modifying the quality score for keyword result ..
    e.g. if site has 10 linkbacks rise the position of search result ...

    Any idea ?

    Thankyou
    Jimba
    Skype & Email ME -
    Tags: None
    • AdultKing
      Raise Your Weapon
      • Jun 2003
      • 15601
      #2
      indexing how many sites ?

        Comment

        • jimbaman
          Confirmed User
          • Jun 2003
          • 267
          #3
          Originally posted by AdultKing
          indexing how many sites ?
          approx 1000 sites
          Skype & Email ME -

            Comment

            • riofromrio
              Registered User
              • Dec 2011
              • 22
              #4
              Originally posted by AdultKing
              indexing how many sites ?
              would like to know a good one ... 500k sites should be indexed

                Comment

                • AdultKing
                  Raise Your Weapon
                  • Jun 2003
                  • 15601
                  #5
                  1000 Sites: Nutch + Hadoop
                  500k sites: get a small server farm then Nutch + Hadoop

                    Comment

                    • AdultKing
                      Raise Your Weapon
                      • Jun 2003
                      • 15601
                      #6
                      It's worth noting that Nutch is not trivial to deploy, so if you modified your requirements there are plenty of far simpler script solutions to simple search.

                        Comment

                        • jimbaman
                          Confirmed User
                          • Jun 2003
                          • 267
                          #7
                          Originally posted by AdultKing
                          1000 Sites: Nutch + Hadoop
                          500k sites: get a small server farm then Nutch + Hadoop

                          explain better please ...
                          I've no idea of what is that, and what is used for ...

                          Thankyou
                          Skype & Email ME -

                            Comment

                            • AdultKing
                              Raise Your Weapon
                              • Jun 2003
                              • 15601
                              #8
                              Unless you use Sphider http://www.sphider.eu or a similar script then implementing a search engine is non trivial.

                              What you want to do can't be achieved by Sphider because the indexing, weighting and ranking of sites based upon keywords and inbound links is a resource intensive task.

                              Sphider does not support ranking based on inbound links, however Nutch will as will some other open source search platforms.

                              Implementing such a thing is non trivial. You could develop your own solution and run it's crawler on Amazon EC2 instances to save having to invest in hardware, however you still need somewhere to house your database.

                                Comment

                                • AdultKing
                                  Raise Your Weapon
                                  • Jun 2003
                                  • 15601
                                  #9
                                  Here's something I posted on the subject two years ago in response to a similar question.

                                  http://gfy.com/showpost.php?p=18075524&postcount=8

                                  It's a bit out of date now, I'd lean toward Nutch these days.

                                    Comment

                                    • Bogdan
                                      So Fucking Banned
                                      • Dec 2012
                                      • 384
                                      #10
                                      Adult king you fuckin' rock!!!

                                      Adult king you fuckin' rock!!!

                                        Comment

                                        • Porndex
                                          Registered User
                                          • Oct 2012
                                          • 28
                                          #11
                                          Do you know PHP and/or a language you can dev with on a server (Perl, C++)? It's pretty simple to get a crawler happening if you're apt with programming. Once you've got an "indexer" happening, the weighting and all that jazz is the fun part

                                          The search engine isn't really the hard part to program, more the crawler if you ask me. Depends on what you're crawling and how you're displaying it too.
                                          Last edited by Porndex; 01-24-2013, 04:45 AM.
                                          Porndex

                                            Comment

                                            Previous template Next
                                            Working...