[Babase] Question about file upload
Jeanne Altmann
altj at Princeton.EDU
Mon Mar 23 15:03:41 EDT 2009
Let's get quotes on both, and I'll check into funding options;
Thanks,
jeanne
-----Original Message-----
From: babase-bounces at eeblistserv.princeton.edu
[mailto:babase-bounces at eeblistserv.princeton.edu] On Behalf Of Karl O.
Pinc
Sent: Monday, March 23, 2009 2:43 PM
To: Ryan Hardy
Cc: The Baboon Database Project
Subject: Re: [Babase] Question about file upload
Jeanne,
We're looking at maybe $500 for option 2 and maybe
$175 for option 1. Option 2 gives us more space and makes the system
faster for big queries. Option 1 puts us back where we were before the
drive went out.
Ryan can get quotes on either or both options.
What would you like him to do?
Ryan, see below.
On 03/23/2009 11:44:33 AM, Ryan Hardy wrote:
> On Mar 18, 2009, at 10:33 PM, Karl O. Pinc wrote:
>
>> We do want to retain that setup. It's what's saved us now.
>
> I concur; I just wasn't sure that any new space would need that level
> of redundancy.
>
>> If we could it'd be good to do raid 0 over raid 1, (raid 1+0) so we
>> can take advantage of all the spindles.
>> I'm not heavy into md so I don't know if this is feasible without a
>> full restore.
>
> I'll have to look into it.
FYI, here's the current layout:
10K RPM LVD320 SCSI drives
sda -\/->md0 ---> /boot (300MB)
sdb -/\->md1 ---> Vol0 (68GB) ---> LV00 / (16GB)
\--> LV01 /var/lib/pgsql (the db) (50GB)
\-> LV01 swap (1.6G)
10K RPM LVD160 SCSI drive(s)
sdc --->md2 ---> /Vol1 (68GB) ---> LV20 /disk (sdd)-/
(sdd is dead and gone)
>
> You're correct (it was something I noticed myself). I believe two of
> the drives (the two forming the pair that hosts the OS) are 320's, but
> the two that belong to the pair in which the failure occurred are
> 160's.
>
>> I would think twice what we've got would be plenty.
>> I don't know. It depends on the money and what's coming down the
>> pike in terms of data requirements.
>> (This mention of 15GB came out of the blue.)
>
> Ok. Here are a few ideas. Let me know which you would be happy with,
> Karl, and I will get quotes:
Let's wait for Jeanne's response to the above.
(Or if Susan chimes in.)
> There are, of course, further options. If you want me to put
> together a quote to implement the raid 1+0 idea, I can do that too.
> I'm not too familiar with the application/server usage in general, so
> I'm not sure what sort of performance requirements various pieces
> have.
It's a database, so when it's running right it's cpu bound but when
we've big queries we go IO bound.
I'm thinking the right thing to do is to get 2 new fast drives and raid
1 them. Then we want to take the existing /var/lib/pgsql storage stripe
(raid0) it using the new drives and the currently allocated storage
This would put the db on 4 spindles so we could get up to 320Mbps
sustained data transfer rates off the drives when hitting the db. At
least if we get the stripe size and readahead configured right. Mostly
when we're slow it's a single query that's the problem so we'd want to
focus on single query performance rather than multi-user performance.
(Which is the harder option.) I don't know all the specs but my guess is
that at the moment we're probably getting half of 320Mbps sustained data
transfer at best. 4 spindles are better than 2 in this regard.
If we can get 15Krpm drives instead of 10K drives that'd be good too.
(I don't know if that's another quote in the making.) The key stat is
the sustained data transfer rate. We want to bump this up as high as
possible.
Whatdayathink? You're the one who's going to have
to setup and maintain this so I'll go with your choice.
I think Hunter's idea is that /disk did not need to be fast, it being
mostly scratch space and database backup into the fs for backup
elsewhere. But it seems to me that it's probably more cost effective to
just make it part of an otherwise fast raid1 array.
I can imagine that the whole backup strategy can play into how much disk
we need and how fast it should be.
Let me know how you think this plays out.
I thought that it might be better to use lvm to do the striping rather
than md, for flexibility. But looking at the docs it seems that you can
only specifying the striping when creating an lv in the first place so
no matter what we do we're probably looking at a restore. Maybe you
still want to stripe with lvm so we can easily mess with the size but I
leave that up to you.
Karl <kop at meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein
_______________________________________________
Babase mailing list
Babase at www.eco.princeton.edu
http://www.eco.princeton.edu/mailman/listinfo/babase
More information about the Babase
mailing list