GoFuckYourself.com - Adult Webmaster Forum

GoFuckYourself.com - Adult Webmaster Forum (https://gfy.com/index.php)
-   Fucking Around & Business Discussion (https://gfy.com/forumdisplay.php?f=26)
-   -   Tech Any idea why this cron job isnt doing its job? (https://gfy.com/showthread.php?t=1381137)

Publisher Bucks 01-13-2025 03:44 PM

Any idea why this cron job isnt doing its job?
 
It *was* working but stopped about a week ago, nothing has changed on the hosting account to my knowledge :Oh crap

Quote:

<?php

$directories = [
'/path/to/directory1',
'/path/to/directory2',
'/path/to/directory3'
'etc...'
'etc...'
];

$threshold = 30 * 24 * 60 * 60; // 30 days

foreach ($directories as $directory) {
if (is_dir($directory)) {
$dir = opendir($directory);

while (($file = readdir($dir)) !== false) {
if (pathinfo($file, PATHINFO_EXTENSION) === 'jpg') {
$filePath = $directory . DIRECTORY_SEPARATOR . $file;

if (file_exists($filePath) && time() - filemtime($filePath) > $threshold) {
unlink($filePath);
echo "Deleted: $filePath\n";
}
}
}

closedir($dir);
} else {
echo "Directory does not exist: $directory\n";
}
}

echo "Cleanup complete.\n";
?>

sarettah 01-13-2025 07:07 PM

not enough info. could be permissions, could be the cron setup, etc.

what does it show if you run it from the browser?

Pipecrew 01-13-2025 07:12 PM

Your missing commas in the directories array. Try this

<?php

$directories = [
'/path/to/directory1',
'/path/to/directory2',
'/path/to/directory3',
// Add more directories as needed
];

$threshold = 30 * 24 * 60 * 60; // 30 days in seconds

foreach ($directories as $directory) {
if (is_dir($directory)) {
$dir = opendir($directory);

if ($dir) {
while (($file = readdir($dir)) !== false) {
// Check if the file is a JPG
if (pathinfo($file, PATHINFO_EXTENSION) === 'jpg') {
$filePath = $directory . DIRECTORY_SEPARATOR . $file;

// Check if the file exists and is older than the threshold
if (file_exists($filePath) && time() - filemtime($filePath) > $threshold) {
if (unlink($filePath)) {
echo "Deleted: $filePath\n";
} else {
echo "Failed to delete: $filePath\n";
}
}
}
}

closedir($dir);
} else {
echo "Failed to open directory: $directory\n";
}
} else {
echo "Directory does not exist: $directory\n";
}
}

echo "Cleanup complete.\n";
?>

Publisher Bucks 01-13-2025 07:18 PM

Quote:

Originally Posted by sarettah (Post 23337888)
not enough info. could be permissions, could be the cron setup, etc.

what does it show if you run it from the browser?

It says its completed the cleanup, but images that are 45 days old are still present.

I have a ticket into the host to see if they've changed anything, no reply yet though :/

Publisher Bucks 01-13-2025 07:18 PM

Quote:

Originally Posted by Pipecrew (Post 23337889)
Your missing commas in the directories array. Try this

That was a typo on my part posting and editing the code lol

WiredGuy 01-13-2025 07:23 PM

Is there an output from your crontab? Usually it updates to mail on completion.
WG

Publisher Bucks 01-13-2025 07:56 PM

Well I just heard back, apparently the host disabled cron because it was using up 'excessive resources' running daily and deleting a few hundred images every day from multiple directories after they were created :1orglaugh

Currently emailing to get them to turn it back on LOL

Mystery solved, I do appreciate the assistance on this.

As a side note, is there a way I can make this quite simple code, use less resources?

WiredGuy 01-13-2025 08:27 PM

Quote:

Originally Posted by Publisher Bucks (Post 23337898)
Well I just heard back, apparently the host disabled cron because it was using up 'excessive resources' running daily and deleting a few hundred images every day from multiple directories after they were created :1orglaugh

Currently emailing to get them to turn it back on LOL

Mystery solved, I do appreciate the assistance on this.

As a side note, is there a way I can make this quite simple code, use less resources?

Run it in smaller batches. Keep track of a counter, after processing X images, halt the loop and then run it again every hour instead of daily.
WG

Killswitch 01-13-2025 11:00 PM

bruh you just let your host go on your servers you pay for and disable things without your permission?

YIKES.

mechanicvirus 01-14-2025 01:02 PM

Quote:

Originally Posted by Killswitch (Post 23337942)
bruh you just let your host go on your servers you pay for and disable things without your permission?

YIKES.

Please dont kink shame

Killswitch 01-14-2025 01:36 PM

Quote:

Originally Posted by mechanicvirus (Post 23338080)
Please dont kink shame

my bad b

mechanicvirus 01-14-2025 02:31 PM

Quote:

Originally Posted by Killswitch (Post 23338089)
my bad b

Now that I think about it, you are 100% right... lol killing a cronjob because 15 people clicked an image and it "overloaded" a server.

What a world we live in.

Killswitch 01-14-2025 03:08 PM

Quote:

Originally Posted by mechanicvirus (Post 23338109)
Now that I think about it, you are 100% right... lol killing a cronjob because 15 people clicked an image and it "overloaded" a server.

What a world we live in.

$65/mo managed vps with half a shared cpu and 256mb of ram. Those 15 clicks were a too much resource usage.

cerulean 01-14-2025 05:33 PM

If I were you, I would avoid doing this in PHP (and I'm primarily a PHP developer.) PHP is certainly capable of this, but loading a programming language runtime just to run existing system commands does complicate the process.

If you're on Linux, you can use the find command and your host should be well-versed in how to do this anyway:

https://stackoverflow.com/a/69374901

If you're on Windows, similar things exist:

https://stackoverflow.com/a/51069

Either way, you should make backups of these files before deleting them, and you should capture the output of whatever you run, and send a notification on success. That way you know when it runs, and whether it's successful.

Killswitch 01-14-2025 06:47 PM

Quote:

Originally Posted by cerulean (Post 23338171)
If I were you, I would avoid doing this in PHP (and I'm primarily a PHP developer.) PHP is certainly capable of this, but loading a programming language runtime just to run existing system commands does complicate the process.

If you're on Linux, you can use the find command and your host should be well-versed in how to do this anyway:

https://stackoverflow.com/a/69374901

If you're on Windows, similar things exist:

https://stackoverflow.com/a/51069

Either way, you should make backups of these files before deleting them, and you should capture the output of whatever you run, and send a notification on success. That way you know when it runs, and whether it's successful.

For the record, the overhead of executing PHP to run file commands like unlink while is overhead, it's negligible and you shouldn't complicate things for yourself because someone else's preferences.

Publisher Bucks 01-14-2025 07:05 PM

Quote:

Originally Posted by WiredGuy (Post 23337901)
Run it in smaller batches. Keep track of a counter, after processing X images, halt the loop and then run it again every hour instead of daily.
WG

Thanks Charles, LTNS, think we last chatted in Tempe at the Phoenix Forum with Aly Drummond!

Publisher Bucks 01-14-2025 07:07 PM

Wow, I’m so glad that people are getting offended on my behalf about a host shutting down a cron job on a $2 a month shared hosting plan that is only used for image generation… Classic GFY (and a good example of why society as a whole is failing).

cerulean 01-14-2025 07:54 PM

Quote:

Originally Posted by Killswitch (Post 23338188)
For the record, the overhead of executing PHP to run file commands like unlink while is overhead, it's negligible and you shouldn't complicate things for yourself because someone else's preferences.

This isn't the first time this has come up for me personally, and I stand by what I've said. The OP is using readdir, unlink, and pathinfo, and while you are focusing specifically on unlink, all three in tandem across multiple files can compound memory and resource issues. Without benchmarking it, I couldn't definitively say so, but my experience tells me that using find would be a lot more performant and would answer OPs request.

I don't know if the host is being honest, and I am not sure of the underlying system allocation, but there is truth to what I'm saying, and it's not necessarily negligible. PHP handles its own memory optimization and garbage collection, and there's not much you can do to make these more performant. It's already a very performant language. That doesn't mean it can handle large file operations with ease. It's not the right tool for the right job, in this case, in my opinion.

I was looking for some of the old threads and expert opinions I researched when I was attempting to optimize a phar executable for code obfuscation purposes, but I think most of those old forums are now defunct. My bookmarks were dead, but these SO posts shed some light on how file operations can be resource hogs in PHP, if you're interested in reading them. The bit on unlink not being asynchronous is very interesting.

https://stackoverflow.com/q/6627952

https://stackoverflow.com/q/37633680

https://stackoverflow.com/questions/...nous-functions

We can differ on this and have our own opinions, but I do stand by mine.

Quote:

Originally Posted by Publisher Bucks (Post 23338193)
Wow, I’m so glad that people are getting offended on my behalf about a host shutting down a cron job on a $2 a month shared hosting plan that is only used for image generation… Classic GFY (and a good example of why society as a whole is failing).

I would not mind knowing the name of a managed shared hosting platform that offers $2/mo plans with cronjob support. That sounds like a diamond in the rough.

natkejs 01-18-2025 06:33 PM

Quote:

Originally Posted by Killswitch (Post 23338188)
For the record, the overhead of executing PHP to run file commands like unlink while is overhead, it's negligible and you shouldn't complicate things for yourself because someone else's preferences.

It's not about the overhead of running unlink, it's about the directory listing which is exponentially faster using find.

Since the PHP script does nothing that a simple find command can't do, the cron should simply run the find command not a PHP script.

Something like:

find /path/1/ /path/2/ /path/3 -type f -mtime +30 -exec rm -f {} \;

Killswitch 01-18-2025 09:56 PM

Quote:

Originally Posted by natkejs (Post 23339146)
It's not about the overhead of running unlink, it's about the directory listing which is exponentially faster using find.

Since the PHP script does nothing that a simple find command can't do, the cron should simply run the find command not a PHP script.

Something like:

find /path/1/ /path/2/ /path/3 -type f -mtime +30 -exec rm -f {} \;

You're assuming this person is comfortable and familiar with the command line and the find command.

My assumptions are they know how to write PHP but that's about it.

And if you're optimizing your cronjobs there's a root issue not being solved.

natkejs 01-18-2025 10:16 PM

Quote:

Originally Posted by Killswitch (Post 23339174)
You're assuming this person is comfortable and familiar with the command line and the find command.

My assumptions are they know how to write PHP but that's about it.

And if you're optimizing your cronjobs there's a root issue not being solved.

Well yes, since OP is running on $2 shared hosting I'm assuming it's not managed, and if you can setup a cron for a PHP script you can also setup a cron running a find command.

If your task is to remove images older than 30 days, I don't see too much issues doing it this way.

It would be faster having the images and dates indexed to a ram db, but on $2 shared hosting I imagine your options are limited. Depending on the sizes of the directories using find like this should be just fine, and much much faster than relying on PHP.

natkejs 01-18-2025 11:05 PM

Ok I get it, he should pay for more resources and not worry about optimization of a cron job. But optimizing is so much fun, and cron jobs can require a gazillion % more CPU than necessary when not done right.

I know from experience having run more than a few WP installs on the same server. Shit ain't fun when wp-cron pops off on 1k domains at once.


All times are GMT -7. The time now is 01:30 PM.

Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc