[linux] backup and storage

[Donald J. Ankney]

Tar is still more useful to me as the smbtar variant can archive from  
smb shares on my remaining Windows Servers (which hopefully will be  
retired this summer).

Tar also has several flags that can be useful (-N date (after the  
last successful backup); -i for smbtar incremental). I do a little  
bit of Perl to create a log and process the last successful backup  
and keep in them in 1 week incremental sets.

Eventually, I'm planning on using some rsync magic as part of the  
formula (rsync to a backup server that can be put into production in  
case of failure, archive past weeks to a remote server as compressed  
tarballs), but I haven't gotten there yet. First I'm working on  
getting rid of the Windows boxes so at least I'm all Unix, even if  
it's mixed (OpenBSD, Linux, Mac OS X) across my servers.

-- Don

On Feb 14, 2007, at 3:57 PM, Justin S. Williams wrote:

> Donald,
>
> Detailed at the following web page are instructions about using  
> rsync to
> create an incremental backup that gives you a full snapshot for any
> given day.  Basically it works by using "cp -al" to create a second  
> set
> of hard links to an initial full snapshot (which doesn't duplicate the
> data on disk, only the links to the inodes), and then using rsync to
> update the new copy.  The end result is that you can have what appears
> on the filesystem to be complete full snapshots for each day while  
> only
> using the disk space of an incremental snapshot.
>
> http://www.mikerubel.org/computers/rsync_snapshots/
>
> I've used this technique with great success and I find it far more
> useful than using tar.  However, I'm curious if you (or anyone else
> here) have looked into this and have found using tar across ssh to be
> more useful/functional, or if there are factors I've not considered in
> my backup scheme.
>
> Thanks,
>
> --
> Justin Williams
>
> -----Original Message-----
> From: linux-bounces at mailman1.u.washington.edu
> [mailto:linux-bounces at mailman1.u.washington.edu] On Behalf Of  
> Donald J.
> Ankney
> Sent: Wednesday, February 14, 2007 2:43 PM
> To: Linux/Unix Users Group at the UW
> Subject: Re: [linux] backup and storage
>
> It doesn't take a lot of processing power to set up a software raid  
> and
> sync the files. You can use pretty much anything in the P4 class.
> 512 megs of ram is plenty.
>
> Since the University has a site license for RHEL, why not just use the
> current version? If you're using this for a personal project, you can
> use CentOS, which originated as a free (as in beer as well as
> speech) alternative to RHEL. Personally, I just use tar piped  
> across ssh
> (rsync doesn't work for me as I'm interested in multiple backups  
> with so
> that I can restore to a specific date).
>
> On Feb 14, 2007, at 2:28 PM, Helen Petropoulos wrote:
>
>> Hi Alan,
>>
>> That sounds like just the advice that I was looking for, but I was
>> wondering, when you say barebones system, what's the minimum  
>> processor
>
>> that I should look at?  What's the minimum version number of RHLE  
>> that
>
>> I should use?
>>
>> I've worked on Unix systems, but I'm new to Linux and I'm not sure
>> where to look for this kind of information.
>>
>> Thanks,
>>
>> Helen
>>
>> On Feb 14, 2007, at 1:37 PM, Alan Gibson wrote:
>>
>>> the best solution depends somewhat on what you are trying to  
>>> achieve.
>>> if you want to archive stuff, dvd may be best. if you just want
>>> redundant backups, i would set up a NAS.
>>>
>>> take a cheap barebones system, add a 4 port sata card and 4 250GB
>>> drives. use mdadm to turn the drives into a 1TB raid. then set up an
>>> hourly cron job that backs the whole thing up with rsync. the great
>>> thing about rsync is that it only transfers what has actually
>>> changed.
>>>
>>> this solution would be cheap, easier to maintain than tape or dvd,
>>> not to mention it would be a whole lot faster. if you need more
>>> storage, just put in another sata card and some drives and add them
>>> to the raid (you might want to pay a little extra for a big power
>>> supply).
>>>
>>> backing up multiple systems would be trivial; just give the backup
>>> server an account on every system you want to back up and let it go
>>> out and fetch the files.
>>>
>>> now if your data is irreplaceable, you should really have multiple
>>> copies, at least one being off site.  there is always a chance that
>>> the building will burn down. in this case i would say either add
>>> weekly dvd backups and take them home with you, or sign up for  
>>> one of
>
>>> the new remote backup services that are coming online.
>>>
>>> alan
>>>
>>> On 2/14/07, Helen Petropoulos <elenip at u.washington.edu> wrote:
>>>> Hi,
>>>>
>>>> I have a DELL PowerEdge 700 running linux and I was wondering what
>>>> kind of backup system I should obtain for it.  I would like at  
>>>> least
>
>>>> 1 T of storage.  Is there something that someone out there
>>>> recommends?  Should I get an external jukebox?  I would also  
>>>> like to
>
>>>> be able to back up other machines onto this one.
>>>>
>>>> Thanks in advance,
>>>>
>>>> Helen - Dept of Radiology
>>>>
>>>>
>>
>