[Babase] Question about file upload
Karl O. Pinc
kop at meme.com
Mon Mar 23 11:45:44 EDT 2009
On 03/23/2009 10:30:19 AM, Ryan Hardy wrote:
> On Mar 18, 2009, at 9:43 PM, Karl O. Pinc wrote:
>
>> The problem with the backups, as done at Duke, is that they
>> rely on dumping the databases into the filesystem. One file
>> per database. So, the total (compressed) backup of the
>> entire cluster, per your remark below, is limited
>> to 2TB.
>
> Ahh, I see. That is not "standard", even for our systems. It is
> certainly not ideal.
I thought it had to do with Biology's use of rsync, or maybe
rdiff. Rsync does a nice hardlinked "incremental" backup,
but this does not work for databases, which are not designed
to be backed up straight from the filesystem. So instead
the PG databases are dumped to the filesystem, which takes
care of all the transaction related issues and ensures
you get a snapshot with no broken relations. Then you backup
the filesystem image. If you do that with rdiff (?) you
can keep a "complete" set of "incrementals" analogous to
the rsync backup. That was what Hunter explained to me
was going on with the Biology databases.
It's not at all a bad system so long as you've got
the disk space to dump the db cluster. And that
space could be anywhere and re-used. Once you backup
the db dump you no longer need it. That way you can keep
many days of database backups without using up gobs
of space on your backup server.
Of course there is the 2TB limit, but until it becomes
a problem, it's not a problem.
Karl <kop at meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein
More information about the Babase
mailing list