[Babase] Question about file upload

Karl O. Pinc kop at meme.com
Mon Mar 23 14:43:01 EDT 2009


Jeanne,

We're looking at maybe $500 for option 2 and maybe
$175 for option 1.  Option 2 gives us more space and
makes the system faster for big queries.  Option 1
puts us back where we were before the drive went out.

Ryan can get quotes on either or both options.
What would you like him to do?

Ryan, see below.

On 03/23/2009 11:44:33 AM, Ryan Hardy wrote:
> On Mar 18, 2009, at 10:33 PM, Karl O. Pinc wrote:
> 
>> We do want to retain that setup.  It's what's saved us now.
> 
> I concur; I just wasn't sure that any new space would need that level  
> of redundancy.
> 
>> If we could it'd be good to do raid 0 over raid 1,
>> (raid 1+0) so we can take advantage of all the spindles.
>> I'm not heavy into md so I don't know if this is feasible
>> without a full restore.
> 
> I'll have to look into it.

FYI, here's the current layout:

10K RPM LVD320 SCSI drives
sda -\/->md0 ---> /boot (300MB)
sdb -/\->md1 ---> Vol0 (68GB) ---> LV00 / (16GB)
                               \--> LV01 /var/lib/pgsql (the db) (50GB)
                                \-> LV01 swap (1.6G)

10K RPM LVD160 SCSI drive(s)
sdc --->md2 ---> /Vol1 (68GB) ---> LV20 /disk
(sdd)-/

(sdd is dead and gone)


> 
> You're correct (it was something I noticed myself).  I believe two of  
> the drives (the two forming the pair that hosts the OS) are 320's,  
> but the two that belong to the pair in which the failure occurred are  
> 160's.
> 
>> I would think twice what we've got would be plenty.
>> I don't know.  It depends on the money and what's
>> coming down the pike in terms of data requirements.
>> (This mention of 15GB came out of the blue.)
> 
> Ok.  Here are a few ideas.  Let me know which you would be happy  
> with, Karl, and I will get quotes:

Let's wait for Jeanne's response to the above.
(Or if Susan chimes in.)


> There are, of course, further options.  If you want me to put  
> together a quote to implement the raid 1+0 idea, I can do that too.   
> I'm not too familiar with the application/server usage in general, so  
> I'm not sure what sort of performance requirements various pieces  
> have.

It's a database, so when it's running right it's cpu bound but
when we've big queries we go IO bound.

I'm thinking the right thing to do is to get 2 new fast drives
and raid 1 them.  Then we want to take the existing /var/lib/pgsql
storage stripe (raid0) it using the new drives and the currently
allocated storage  This would put the db on 4 spindles so we could
get up to 320Mbps sustained data transfer rates off the drives
when hitting the db.  At least if we get the stripe size
and readahead configured right.  Mostly when we're slow
it's a single query that's the problem so we'd want to focus
on single query performance rather than multi-user
performance.  (Which is the harder option.)
I don't know all the specs but my guess
is that at the moment we're probably getting half of 320Mbps
sustained data transfer at
best.  4 spindles are better than 2 in this regard.

If we can get 15Krpm drives instead of 10K drives
that'd be good too.  (I don't know if that's another quote
in the making.)  The key stat is the sustained data transfer
rate.  We want to bump this up as high as possible.
Whatdayathink?   You're the one who's going to have
to setup and maintain this so I'll go with your choice.

I think Hunter's idea is that /disk did not need to be fast,
it being mostly scratch space and database backup into the fs
for backup elsewhere.  But it seems to me that it's probably
more cost effective to just make it part of an otherwise
fast raid1 array.

I can imagine that the whole backup strategy can play
into how much disk we need and how fast it should be.
Let me know how you think this plays out.

I thought that it might be better to use lvm to do the striping
rather than md, for flexibility.  But looking at the
docs it seems that you can only specifying the striping
when creating an lv in the first place so no matter what
we do we're probably looking at a restore.  Maybe you still
want to stripe with lvm so we can easily mess with the size but I
leave that up to you.

Karl <kop at meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein



More information about the Babase mailing list