Solr PHP help. - GoFuckYourself.com

xmonkeys · 07-04-2011, 12:46 PM

Hey Guys I am trying to integrate solr into my tgpx system, which runs on php, but I am having some issues with it.
does anyone have any experience in this? Or could anyone help or offer their services?

Thanks.

Presnus · 07-05-2011, 04:35 AM

contact me on ICQ

Babaganoosh · 07-05-2011, 02:38 PM

I have some experience with solr and quite a bit with Sphinx. I have to ask, why do you need it if you're just using tgpx? Is the DB really huge or something?

xmonkeys · 07-05-2011, 07:54 PM

ya the db has close to 300k thumbs...and growing...am i setting it up wrong? is 300k not that big? :-) i m new and learning!

Babaganoosh · 07-05-2011, 08:32 PM

I can't see any reason to use something like that for a db of that size. I have one with 1.4 million records and a simple PHP script, a fulltext index and a well-configured my.cnf file make it nice and fast.

I'm running a custom script but it still stores data in mysql.

I guess I should ask how many searches you're anticipating per day before I try to talk you out of a search daemon.

xmonkeys · 07-06-2011, 08:34 AM

well i am just starting, so I dont know how many searches I am anticipating...
so ur saying my db isnt that huge and my search should be done rather quickly and without real delay, and without solr?
So here is my question, the search results take approx 4 to 5secs to load, you think it could be due to configuration?
you wanna maybe take a look and see y it would be loading slow? or maybe in ur eyes it is loading fast. check out the search at porntubehunters dot com lemme know if the search is rather fast or slow. I find that it takes a bit of a while to load.
Thanks again babaganoosh :-)

Babaganoosh · 07-06-2011, 05:04 PM

Yeah, that looks a little on the slow side. Are you on a shared server?

xmonkeys · 07-06-2011, 06:00 PM

Babaganoosh, I am on a shared server.
But, was in the process of moving it to a vps, in order to install solr, but that is where I got stuck the solr was giving me problems.

Do you think that transferring to a vps will make that much of a difference?

xmonkeys · 07-06-2011, 06:08 PM

hey Babaganoosh, someone suggested I try sphinxsearch.com, have you ever heard of it? what do u think of it?

Babaganoosh · 07-06-2011, 09:06 PM

In my first post I mentioned Sphinx. It works well if you have the hardware to throw at it. Your queries shouldn't be that slow even on a shared server. The host's mysql server is probably overloaded. See how it performs on a decent VPS. You'll almost definitely need to tweak your my.cnf but a db of that size shouldn't be beating you up too badly.

I don't know how tgpx works either. I don't know its db structure or anything so it might be a script issue too. For the kind of searching you're trying to do a fulltext index should exist on whatever column holds the descriptions and then use a fulltext query. If your search script is doing something like "SELECT * FROM tablename WHERE description LIKE %SEARCHTERM%" then that's a massive problem.

xmonkeys · 07-07-2011, 07:24 AM

Hey Babaganoosh, ya i just scrolled up and re-read ur first post you did mention sphinx.
Im gonna try and play with some settings today and tomorrow. i'll keep you posted on my progress. Waiting for the transfer to go through from my shared server to the vps.

thanks again for all your help and patience. Much appreciated.

xmonkeys · 07-07-2011, 03:00 PM

By the way Babaganoosh, we are doing a ""SELECT * FROM tablename WHERE description LIKE %SEARCHTERM%"" what do u suggest I do to change it? what other solutions r there? :-S

Babaganoosh · 07-07-2011, 06:01 PM

You're definitely going to want to do a fulltext query instead of a LIKE. Like queries are painfully slow for things like this.

Check this page for info on fulltext searching.
http://dev.mysql.com/doc/refman/5.1/...xt-search.html

xmonkeys · 07-07-2011, 06:51 PM

Thanks again babaganoosh. Reading it as we speak.
will keep u posted on progress! ur an amazing help. you have any site u want me to hard link to once my site is complete? will gladly hardlink it.

xmonkeys · 07-10-2011, 08:01 PM

Babaganoosh, thanks for all the help, it worked out perfectly with the fulltext query. The link u sent was amazing. Thank you so much for the help.
And again, if there is anyway I can help in return let me know, would gladly help.

thanks again.

Babaganoosh · 07-15-2011, 06:19 PM

That search is still really slow. I've tried it a few times to see if it improved. If you've got it implemented already then something is definitely wrong. You may need to tweak your my.cnf or even the PHP code for the search. Leave some contact info (email or ICQ) and I will see what I can do to help you out.

xmonkeys · 07-18-2011, 10:05 PM

hey babaganoosh, wasnt on for a bit. Ya you are right the search did get slow again, it was working fine for 2 days, then for some reason became extremely slow again.
you can reach me at icq: 601142993 or email [email protected]
Thanks again for all your patience and help.

xmonkeys · 07-19-2011, 07:41 PM

lemme know when ur on.
like check the site now, it seems much faster. it does it everytime, i dunno y.
some days it s fast, some days it s not!

07-04-2011, 12:46 PM	#1
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	Solr PHP help. Hey Guys I am trying to integrate solr into my tgpx system, which runs on php, but I am having some issues with it. does anyone have any experience in this? Or could anyone help or offer their services? Thanks.

07-05-2011, 04:35 AM	#2
Presnus Confirmed User Industry Role: Join Date: May 2011 Posts: 1,087	contact me on ICQ

07-05-2011, 02:38 PM	#3
Babaganoosh ♥♥♥ Likes Hugs ♥♥♥ Industry Role: Join Date: Nov 2001 Location: /home Posts: 15,841	I have some experience with solr and quite a bit with Sphinx. I have to ask, why do you need it if you're just using tgpx? Is the DB really huge or something?

07-05-2011, 07:54 PM	#4
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	ya the db has close to 300k thumbs...and growing...am i setting it up wrong? is 300k not that big? :-) i m new and learning!

07-05-2011, 08:32 PM	#5
Babaganoosh ♥♥♥ Likes Hugs ♥♥♥ Industry Role: Join Date: Nov 2001 Location: /home Posts: 15,841	I can't see any reason to use something like that for a db of that size. I have one with 1.4 million records and a simple PHP script, a fulltext index and a well-configured my.cnf file make it nice and fast. I'm running a custom script but it still stores data in mysql. I guess I should ask how many searches you're anticipating per day before I try to talk you out of a search daemon.

07-06-2011, 08:34 AM	#6
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	well i am just starting, so I dont know how many searches I am anticipating... so ur saying my db isnt that huge and my search should be done rather quickly and without real delay, and without solr? So here is my question, the search results take approx 4 to 5secs to load, you think it could be due to configuration? you wanna maybe take a look and see y it would be loading slow? or maybe in ur eyes it is loading fast. check out the search at porntubehunters dot com lemme know if the search is rather fast or slow. I find that it takes a bit of a while to load. Thanks again babaganoosh :-)

07-06-2011, 05:04 PM	#7
Babaganoosh ♥♥♥ Likes Hugs ♥♥♥ Industry Role: Join Date: Nov 2001 Location: /home Posts: 15,841	Yeah, that looks a little on the slow side. Are you on a shared server?

07-06-2011, 06:00 PM	#8
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	Babaganoosh, I am on a shared server. But, was in the process of moving it to a vps, in order to install solr, but that is where I got stuck the solr was giving me problems. Do you think that transferring to a vps will make that much of a difference?

07-06-2011, 06:08 PM	#9
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	hey Babaganoosh, someone suggested I try sphinxsearch.com, have you ever heard of it? what do u think of it?

07-06-2011, 09:06 PM	#10
Babaganoosh ♥♥♥ Likes Hugs ♥♥♥ Industry Role: Join Date: Nov 2001 Location: /home Posts: 15,841	In my first post I mentioned Sphinx. It works well if you have the hardware to throw at it. Your queries shouldn't be that slow even on a shared server. The host's mysql server is probably overloaded. See how it performs on a decent VPS. You'll almost definitely need to tweak your my.cnf but a db of that size shouldn't be beating you up too badly. I don't know how tgpx works either. I don't know its db structure or anything so it might be a script issue too. For the kind of searching you're trying to do a fulltext index should exist on whatever column holds the descriptions and then use a fulltext query. If your search script is doing something like "SELECT * FROM tablename WHERE description LIKE %SEARCHTERM%" then that's a massive problem.

07-07-2011, 07:24 AM	#11
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	Hey Babaganoosh, ya i just scrolled up and re-read ur first post you did mention sphinx. Im gonna try and play with some settings today and tomorrow. i'll keep you posted on my progress. Waiting for the transfer to go through from my shared server to the vps. thanks again for all your help and patience. Much appreciated.

07-07-2011, 03:00 PM	#12
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	By the way Babaganoosh, we are doing a ""SELECT * FROM tablename WHERE description LIKE %SEARCHTERM%"" what do u suggest I do to change it? what other solutions r there? :-S

07-07-2011, 06:01 PM	#13
Babaganoosh ♥♥♥ Likes Hugs ♥♥♥ Industry Role: Join Date: Nov 2001 Location: /home Posts: 15,841	You're definitely going to want to do a fulltext query instead of a LIKE. Like queries are painfully slow for things like this. Check this page for info on fulltext searching. http://dev.mysql.com/doc/refman/5.1/...xt-search.html

07-07-2011, 06:51 PM	#14
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	Thanks again babaganoosh. Reading it as we speak. will keep u posted on progress! ur an amazing help. you have any site u want me to hard link to once my site is complete? will gladly hardlink it.

07-10-2011, 08:01 PM	#15
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	Babaganoosh, thanks for all the help, it worked out perfectly with the fulltext query. The link u sent was amazing. Thank you so much for the help. And again, if there is anyway I can help in return let me know, would gladly help. thanks again.

07-15-2011, 06:19 PM	#16
Babaganoosh ♥♥♥ Likes Hugs ♥♥♥ Industry Role: Join Date: Nov 2001 Location: /home Posts: 15,841	That search is still really slow. I've tried it a few times to see if it improved. If you've got it implemented already then something is definitely wrong. You may need to tweak your my.cnf or even the PHP code for the search. Leave some contact info (email or ICQ) and I will see what I can do to help you out.

07-18-2011, 10:05 PM	#17
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	hey babaganoosh, wasnt on for a bit. Ya you are right the search did get slow again, it was working fine for 2 days, then for some reason became extremely slow again. you can reach me at icq: 601142993 or email [email protected] Thanks again for all your patience and help.

07-19-2011, 07:41 PM	#18
xmonkeys Confirmed User Industry Role: Join Date: Feb 2011 Posts: 121	lemme know when ur on. like check the site now, it seems much faster. it does it everytime, i dunno y. some days it s fast, some days it s not!