|
deny from 180.76.15.0/24
but I think that iptables or ufw firewalls are a better way to go than .htaccess Baidu is velly sneeky ... Code:
root@(none):~# dig ANY baidu.com |
Quote:
full path to THAT log not the domain config log. Also pipe into the grep $ tac <path/to/file/error.log> | egrep -i 'this|or|that' |less |
Thanks a lot for the answers
Unfortunately I see myself forced to postpone this thing because of 247host, which right now has suspended the account of my second server. The one where I only keep testing sites or that make a few visits 10-200 range about, have it suspended for excessive resource consumption... :disgust They are about 40 sites, but practically do nothing... It is the second time in a week that compel me to buy a more expensive service. They do not warn, do not give warnings, do not say anything, immediately suspend the account and send you an email with the link to buy a more expensive service... Along with the account also die all mails... including those of work... :mad: I would have changed short service, but I was hoping to be able to do everything quietly, and instead not...:mad: They're really shit. Anyway, leaving aside the anger... I thought I would try Linode, so as not to have them all on Digitalocean, a small server but fully updated, with 16:04 or 17:04, PHP7 etc etc etc... Particular indications or differences between Linode and Digitalocean? P.S. As I am happy, I will spend the next week moving sites again... always because of these crappy services... :( |
I use both.
I have used Linode for 9 or 10 years. I think they are comparable. I am going to try Leaseweb when I have time -- their pricing for small VPS is very good -- have to see if there are any issues ... I think you have PHP code issues if you are exceeding your resources. error_log is a PHP error log name? you might get some of the script errors in you use the php cli output at the terminal Code:
cd path/ |
Linode does not accept my credit card...:(
So maybe Digitalocean for the moment, if I can get some mail, since my mails were in the account closed...:angrysoap Today is really a shitty day :( |
So, I moved my sites with the mails here Freehostia and set the mails.
Now I receive all the mails of the world, except those with the password of the new droplet of Digitalocean:angrysoap What do you think about vultr? |
In the end I managed to install a new droplet LAMP 16.04 in Digitalocean
The mails do not work yet, but I managed to access the new droplet via SSH key authentication and changed the password (with some difficulty, because as usual in the guides of Digitalocean always missing a piece) I installed pretty much all the lamp server fast enough (with all the times I threw away and restarted in 14.04, I have now learned) I'm starting to upload my sites in a bit. This time however I really need the mail, 2 on two different domains. How to proceed? Install all a mail server on my droplet? Postfix? Dovecot? Roundcube? Are there any other alternatives? I just need to be able to configure them in my Thunderbird and receive and send mail. And maybe have a place to see them online if I'm not at home pc... And maybe if possible recover the backup of cpanel... |
Maybe Zoho?
|
You need access to your DNS records to change the MX entry.
You can enter a spf text record sending the email (MX) to some other location. |
So, I configured my mails with Zoho, with DNS, SPF etc, for the moment I am not very relaxed because I notice some strange behavior in the configuration phase through Digitalocean and I received some error messages, such as non-existent domain or relay disabled, I tried to send me mail through the same mail or to them, and not all the message arrived...
I hope it's DNS propagation issues, we wait a couple of days and see if everything starts to work... Just in case, are there any other similar free services? In the next few days I will to restore my sites, from 247host, after 2 days they gave me a backup that weighs 4.7 GB in which however missing about 40 sites... they are really crap... |
I have a problem, the MySQL server goes away...
Warning: PDOStatement:: Execute (): MySQL server has gone away in/var/www/html/xxxhashtag.com/sitemap_generator.php on line 88 I am reloading the sites from old backups, but I need to sync the sitemaps. I have a script, which reads the new data in the database, adds links to the Sitemap, and updates the database by entering the value "Insitemap = 1", the next start reads only the data with value "Insitemap = 0". Normally everything works, but now I have to sync everything in the Sitemap, about 600.000 lines... The script will stay there quite a bit, then it returns errors of this type. I have increased the limits on mysql.cnf and php.ini, but still does not work, what else can I do? |
So,
For the emails I solved with Yandex, it works... Zoho kept doing strange things... For MySQL, I tried different configurations, several restarts etc, but I did not have great results. I doubled the performance of the server and I installed a local lamp where I recreated the sitemaps that I then uploaded to the server. For the moment I solved that. I still can not run the files of the cronjob manually (the server remains stationary on a white page for hours), I created the cronjob with curl, let's see if it succeeds in completing the processing of the file or at least indicates some error... I have already found a problem with version 5.7 of MySQL, to him are unsympathetic to the GROUP BY, for now I solved disabling "Sql_mode = only_full_group_by", I have too many queries to modify to do it now... in the future I update the sites. |
my best thought it to run that script you are having all those issues with in a php cli
Code:
$ cd /path/to/script/PHP Code:
Code:
$ php scriptname.php |
It shows me the PHP code of the file... as did the cronjob on 14.04...
Weird, weird, weird... Looking online the first (and perhaps only) solution you find is: You forgot to enable "Short_open_tag" on php.Ini. But it's the first thing I've activated.. |
So,
To begin with I found this Code:
2017-09-26T19:51:42.376416Z 678542 [ERROR] /usr/sbin/mysqld: Table './bigbreasthardpics/tgp_search' is marked as crashed and last (automatic?) repair failedThen I have decomposed my cronjob, good or bad almost all files seem to work, albeit slowly But a particular file remains stationary for hours, is a file that synchronizes data between these sites: adulthashtag.com, xxxhashtag.com, porn-tags.com, Exchange each day the new rows added to the database and add each one that is missing. The swap file is a .txt and weighs about 50kb and usually contains 1500-2000 new rows. Each file is on its own site and is uploaded to others with a function like this: Code:
$file = "Http://www.adulthashtag.com/new_query_search.txt";I have these in my syslog, UFW should be the firewall, but I did not understand if Is he just doing his job or if he's blocking me??? Code:
Sep 26 06:31:34 ubuntu-1gb-nyc3-01 kernel: [36766.523428] [UFW BLOCK] IN=eth0 OUT= MAC=36:1a:36:97:ff:ba:84:b5:9c:f9:08:30:08:00 SRC=210.146.241.198 DST=104.236.230.48 LEN=52 TOS=0x00 PREC=0x00 TTL=115 ID=17950 DF PROTO=TCP SPT=58001 DPT=8118 WINDOW=8192 RES=0x00 SYN URGP=0 |
check those IPs;
$ whois ip the 104.xxx is you server? https://lists.torproject.org/piperma...ch/004159.html https://lists.torproject.org/piperma...ch/004160.html possibly ... the former IP user? |
Yes, and I usually use Tor, but not so often to fill the logs.
I thought... could it be yandex with emails? |
:angrysoap After the problem with the table Bigbreast I have taken a look at all the databases
I'm finding a lot of MySQL import errors from the backup of 247host. Missing auto increment, primary key, default value, null value, etc etc etc. Practically almost all the imported tables have problems... The backup of 247host is really a crap... Maybe it is the fault of MySQL problems if I find errors in the firewall, I do not receive news of the conjob, I can not start files etc etc etc??? |
So, after correcting an infinity of problems on MySQL caused by importing, something seems to start working
Also conjobs and the problem of cross-site synchronization was also due to the missing of an auto increment field. Now the idea is to leave the server free to run for a few days, and see what happens... Especially see if this chart normalizes. http://porn-update.com/temp/Schermat...2022-26-54.png Now it scares me... but also in the other server in the first days consumed many more resources than its normal use, and he did not have all these problems on the MySQL tables. |
These are not server problems they are software problems.
|
In the end I did just that, I installed a local lamp and recreated all the possible tables via script.
The imported ones were not recoverable, in many tables were missing the auto increment and primary fields, and the auto increment fields were filled with empty fields, missing numbers and zeros, etc etc etc., really a mess... Recreating the tables locally and importing mine, everything seems much more correct. Today it has also arrived the first report by mail of the cronjob of xxxhashtag.com. But something maybe still does not work... The cronjob took 8800 seconds...:eek7 usually this cronjob takes 2... Now I have to figure out if there's still something wrong or if he's just dealt with a heavy sync with other sites. Unfortunately the backups I had were not up to date and Xxxhashtag works with many sites and many databases. Usually it checks the data of the last 24 hours, but having restored the databases, probably many data have been added in the last 24 hours... We will see in the next few days... |
Looking through the tables I found a lot of duplicate indexes that I did not create...
Code:
CREATE TABLE IF NOT EXISTS `xxxhashtag_search` (Now I check all the other tables looking for duplicate indexes... (I will take about 3 days, as I am happy... :Oh crap) |
And now...
Everything seems to begin to work... and begin to come to mind questions like: Can I make some backups? Would only the databases (all files I can restore from the databases), would weekly, but the procedure should not weigh too much on the server (some MySQL databases now weigh 60-70 MB of data and the server is small). Maybe it would also be nice if the databases were sent somewhere, like my pc, or Yandex disk, or a gigamail, or something like that... just to be sure that if the server goes on fire them are somewhere else...:upsidedow but anyway all this should not abuse server resources too much. Then, what will be the fastest way to move a site to another server? (Always unmanaged) I was a little bit the desire to Linode or maybe vultr (although I asked, but I did not understand if Vultr accepts adult) For quite a while my sites will remain here, after all the effort I made, however I wonder, if I wanted to move a site I have to reload one by one all the databases via phpMyAdmin and via FTP all files? Exist a faster, more practical and more secure way? |
SSH
Warning Will Robinson: Use mysql as root user so you can lock the tables make a directory for your mysql backups cd to that directory you make Code:
#!use root to locktablesis made^^ use scp or rsync to move the backup to other locations You will have to create the database user and grant permissions as needed CREATE [DATABASE NAME]; then read this https://stackoverflow.com/questions/...-line-in-mysql |
So...
I have taken a look and tried to understand everything. To restore I understood, because I had to use the command line to load those damn corrupted databases, otherwise via phpMyAdmin I could not import them. For backup I have a few questions: Can I create a single file to use in crontab? Eg. .sh executable... (I do not know what they are called or what they are, but I happened to make someone on my PC...) to create a single file with all the commands in order to backup all the databases and use it in a cronjob? And launch this file once a week or once a month via crontab? It depends on how many resources it consumes by launching all backups together. I did not know SCP, but I like it... So much, I have a little raspberry PI attached to my router that spends the day making backups of my data between my PCs, the mobile phone, the tablet and some cloud services. So if I can take SCP backups of the databases on my raspberry, he save them anywhere... The backup files in order to be "taken" with SCP must be in home/user/? Also if ssh logs in as root? ------------------------------------------------------------------ Other little thing out of topic but it came a minute ago... Doing Code:
sudo apt-get updateCode:
Processing triggers for libapache2-mod-php5.6 (5.6.31-6+ubuntu14.04.1+deb.sury.org+1) ...I installed PHP 5.5 along with the lamp, then PHP 5.6 later, but I don't remember ever asking for FPM... Better to use FPM on the 14.04? Is fpm now obsolete? If I enable it I have to remake hp.ini configuration, re-enable opcache, etc etc etc.? |
I found this script, almost perfect for what I want to do
Code:
#!/bin/bashCode:
#elimino vecchi backupCode:
# DO NOT BACKUP these databasesCode:
#FILE="$MBD/$db.$HOST.$NOW.gz"Code:
#comprimo tuttoCrontab? Only this code? Code:
0 6 * * 4 /var/backup/mysql_backupI hope I didn't make any big errors... I added, edited, deleted lines, but I practically have no idea what the language with which the file is written... looks like PHP, it seems to work again... it's the best I can say... it would be nice if you would you warn me if I made some horrendous error :helpme One strange thing I saw is that in the compressed file I find the folder structure /var/backup/mysql, while I was expecting only /mysql, not a big problem, but strange... Now I try to bring everything on my raspberry via SPC, hopefully everything works. The next and last step will be to understand when to start all to not create problems to the server |
rm -f is a bad idea if you do not need it
plain rm is fine Make a copy like I told you manually then test your script manually have a plan b try adding at the bottom of your script Code:
$ echo "`date` backup done"Wed Oct 4 21:28:16 EDT 2017 backup done add this line to your cron and check what happened in the morning :) Code:
0 6 * * 4 (cd /var/backup/mysql_backup/; ./backup_script_name.sh) | mail -s "subject backup done" [email protected] |
Thank you
I tried everything, and it works!!! :thumbsup Even the mail arrives... without telling him anything... (probably because of the link cronjob and postfix, now comes anything has an output) The output that arrives is the result of zip, but it is OK, at least tells me if it did something and if it worked. Only strange thing, 16.04 complains a bit about this: Code:
mysqldump: [Warning] Using a password on the command line interface can be insecure.Rather than the date, can I have the time spent by the script? To understand how long it used the server. In php I usually put a time() at the beginning of the script and one at the end and calculating the difference, but here I do not know how to do...:( |
Code:
$ barry@paragon-DS-7:~$ echo `date +%s`; barry@paragon-DS-7:~$ echo `date +%s`; sleep 3; echo `date +%s`; 1507240274 1507240277 in seconds since epoch (just subtract the values (reversed)) If you don't have a MAIL_TO= at the top of your crontab you have to state it (or the right email address) in the cron itself. The password warning is for security. This is not done over the internet so it is a root cron? Well if you can't trust root locally on your server --- reformat fast! Quote:
|
OK, almost...
Excuse my stupidity, but this programming language sounds strange to me. So, is that it? Code:
#At the beginning of the script, this:Sorry if it seems trivial and stupid, but from the script I can hardly understand how to do even the dumbest things. Compared to my world in PHP I miss "$", ";", I do not understand why the variables are all written in uppercase and how the lines end... I think I understand that everything still works in cascade and that without $ I define a variable, while with the $ I read and use it, but of everything else I'm not sure... Just to understand, what programming language is this? Quote:
|
no in bash.sh the ; at the end of a statement is not needed
var=something (declaration) like JavaScript $var beneath (a declared variable) like echo $var the caps are what i did they could be in lowercase too -- but bash .sh is case sensitive in a terminal $ dothis; sothat; is this; && dosomethindgood | (<pipe>) to the next statement |
So,
I did that... Code:
#!/bin/bashCode:
/var/backup/mysql_backup: line 15: +%s: command not found |
STARTTIME=(`date +%s`)
try like this and the time will be in epoch seconds ENDTIME=(`date +%s`) |
It works, but the result is kinda odd...
Code:
Elapsed_time: 1507671766-1507671705I tried to put quotes, parentheses etc etc, but he does not want to do it... can we do this last thing too? It takes more time for this little thing than to configure all the server... :) |
Maybe in $()
TOTTIME=$($ENDTIME-$STARTTIME) Do the math -- the sum is in seconds :) barry@paragon-DS-7:~$ bc <<< 1507671766-1507671705 61 seconds |
Says:
Code:
/var/backup/mysql_backup: line 93: 1507756208-1507756139: command not foundCode:
TOTTIME=`expr $ENDTIME - $STARTTIME`Now I'm worried about those odd quotes... In PHP when I find those quotes it means that there was a copy paste error from the HTML and nothing works anymore. So I have the habit of removing them as soon as I see them and replace them with a normal apex... in sh instead it seems to be fundamental... I surely have removed someone thinking they were a error...:error I shouldn't have done any damage, because everything seems to work, but maybe I'm going to look for the original script and I see if there was someone...:upsidedow P.S. It's strange how we can install an entire server, and then the simplest things make us crazy...:) |
these are called backticks
bc is a terminal calculator program apt install bc man bc Quote:
Quote:
Like our @array =(<FILENAME>); |
There's one last thing that scares me a lot
Load http://porn-update.com/temp/Schermat...2023-13-39.png Much of that red is due to the phase of moving sites and all the importing error of those damned databases. Also the other server at the beginning had very red, then slowly it is normalized. This is taking a little more... But what sounds strange to me is that going to see the detail of the server, you do not understand why there is all that red. http://porn-update.com/temp/Schermat...2023-15-54.png The CPU rarely arrives at 90%, the memory is a bit chubby but it works, disk there is plenty, errors or special problems there are none... The sites are running well, fast, without interruptions, or visible slowdowns... Cpu sometimes says "stolen" even if it is working maybe at 70%, and already this is odd. But it is the usual load to give more worries, sometimes even 4-5, I also saw 7 in the days of cronjob (they are still synchronizing many data due to the lack of cronjob in the other servers) What does it actually indicate load? And how much do I have to worry? On a scale it goes from "quiet, goes all right" to "shit the server is going to explode, run away all before it's late, shit we'll die all :eyecrazy", where am I? |
Your problem is your PHP script and the MySQL daemon (server). Software for your application; or,
If you look at the times of the peak usage and grep those times in the server access logs you may find that bing is indexing too many pages too fast -- you can place a directive in the robots.txt User-agent: bingbot Crawl-delay:$v 5 10 see https://www.siteground.com/kb/how_to..._eng ine_bot/ https://www.bing.com/webmaster/help/...ntrol-55a30302 Slow bing down -- don't Disallow Bing they bring good converting traffic and the sell their PSaaS or indexed database to Yahoo and other search engines. You may find Baidu is indexing too many pages too fast -- block them at your firewall I have had luck that way Porn is illegal in China and you won't sell legit Chinese buyers either. # Free IP2Location Firewall List by Search Engine # Source: Whitelist Robots by Search Engine | IP2Location Code:
whois -h v4.whois.cymru.com " -c -p 183.131.32.0/20"https://github.com/arineng/nicinfo that will give you full RDAP/whois information Third way is just $ whois <ip address> If you are generating many dynamic pages search engines may be causing this problem Scrapers and *bad bots* may be the issue too. This is what server logs are for to search for problems and find patterns. A firewall is the way to go -- just do not answer -- drop the packet. |
But Holy cow :angrysoap
I was away 2 days and the server was invaded by bots, just like you said... http://porn-update.com/temp/Schermat...2000-34-49.png Code:
51.255.65.66 - - [16/Oct/2017:22:25:31 +0000] "GET /27 HTTP/1.1" 302 3634 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"Thanks, just in time |
| All times are GMT -7. The time now is 02:51 PM. |
Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc