Backup solutions for big sites
 

With more and more tubes popping up with terrabytes of data. How do these guys manage the backups?

RAID isn't an option, since drive failures/ drive copies always can go wrong.

Offsite backups are pain in the ass to move 10+TB every day at like 10 mbits...

Incremental backups could help, but copying the whole system from time to time is needed and runs in the two digits or more TBs.

How do the big hosting firms do those needed backups of clients with LOADS of needed disc space?

Anyway, discuss...

:thumbsup

Actually that's why I do RAID on all my boxes PLUS secondary back ups.

RAID should save you if your main hard drive fails. So your site is not down when you are doing a hardware swap out. Then you have a secondary system in place for your daily/weekly/monthly.

As for myself personally, I also have back ups in my office of websites and their content. So should all else fail, I could FedEx to host/datcenter/office and have them put it on their server at the office.

I'd think most of the bigger sites use a CDN which has all their content on multiple servers worldwide. For all those to fail seems very unlikely. That leaves a much smaller backup - db, etc.

I guess this is one of the only options. What puzzles me how you can make a backup of a 20 TB site for example?

Just copying the files from A to B to do the backup will take a longgggggggggg time.

Cloud hosting would be a solotion, if the cloud is really robust.

Small sites you can always just make a tar/zip file and send it to another hoster/server. But doing this with 10TB+, madness!

Great backup plan you got there!

But doing RAID and daily/weekly, if you are sporting HUGE amounts of data?

How does that work?

Leaves me puzzled...

RAID with 20 drives would be doable, but backing that up :-(

Imagine how long it takes to upload 20TB and populate the CDN :Oh crap

Well, it sounds like you have more data then I do in that regard. But I do video heavy sites myself personally. So I feel your pain. Not to the 20TB degree, but I can relate.

Thanks on the kudos. Yeah, I try and make sure myself, and clients, are never going to be down. It costs more to do things this way with the RAID ARRAY. But it does make sure that if a hard drive fails, I do not need to worry about ordering a HD. I can do a simple swap, and tada. Client remains online while getting failed drive replaced.

Mirrored SAN with hourly, daily and monthly snapshots.

rsync + storage box in ovh.com, thats all :)

2 online boxes with 2x750gb
50gb at one host
50gb at another host
1 online backup service on one of those boxes
1 backup server with backup service at home (keepit)
sending a usb memory to a friend every now and then
gmail
dvd and hard drive in safe
hidden 2.5 wireless NAS

Damn :pimp:1orglaugh:1orglaugh:1orglaugh

Was looking into that too, they have super cheap 24 drive setups, but filling that thing over even a 1000 mbit line would take some time :-)

RAID is great for one..Also, remote backups are highly recommended. Make sure you are keeping your backups not only on your main server, but on a remote server as well!

OVH is insane! 36x 2000GB SAS array, 10 gigabit uncapped unmetered incomming, for only 1K$/month. How do they make money on that? Lol!

With something like rsync you would only be moving new data or data that's changed, not everything all the time. So there ya go.

Also i do offsite backups for some of my clients, so if anyone in here need a secure / specialized offsite backup solution at a reasonable price, hit me up and i'll see how i can help. sales [at] scriptworkz.com