[Babase] Re: Papio memory usage
Karl O. Pinc
kop at meme.com
Thu May 10 17:55:25 EDT 2007
More RAM would be great. How can you go wrong?
But to keep from crashing we're going to have to
work out what uses how much shared memory, and
how much VM might get used.
It might be eaiser to be sure we've got it right
with less RAM in the box, and then get more RAM.
I've not figured it out, so I'm going to do so as
I write here. But I'd like to talk with Hunter
about this for a sanity check and more ideas.
There's a maximum of 1GB shared memory, total
for all processes. (After tweaking config in
/etc/sysctl.conf.) No getting around this.
Shared memory must be backed by swap, AFAICT.
Pg loves shared memory. I want it to have lots,
and I can configure how much it'll take.
Pg uses both OS disk buffers and shared memory for buffering,
but more shared memory is better. Call this the Pg buffer.
After coming out of Pg the data is buffered in libpq,
linked to PHP and Apache. I think this happens on a
per-SQL statement basis. So, roughly, we need double the
memory for every SQL result. Call this the libpq buffer.
Then, the PPA PHP application also buffers the query result.
Now we need 3 times the memory for every query result.
Really, more because the buffering includes padding
for column alignment. Call this the PPA buffering.
The Pg buffer we want in RAM, at least while it's
being filled. The other buffers we don't care about
so much. They can be on swap. The disk is faster
than the network, and much faster than the client's
browser. So long as Pg does all it's work in RAM,
meaning it has the entire db in RAM including indexes,
and has enough RAM for the Pg buffer it needs for
all running queries, I don't think we care so
much about swap. Except for the query Marie was
just running it seems that CPU is the bottleneck.
Where is the RAM going? In particular where
is the shared RAM going? No matter how much
RAM we buy we've only 1G shared RAM. We see both the Pg
backend and the Apache childeren crash due to
lack of swap.
The Apache children seem to be the culprit. I've seen
one of them suck up 25% of the RAM. That'd be 500MB.
The wierd part about that is that PHP should be limited
to 128MB of RAM. Which points to the libpq buffer.
I've also seen the child keep that memory, seemingly
forever. Perhaps this is after the user/client
has decided they goofed, having gotten too many
rows back, and has shut down their browser or
some such. All that RAM can surely be in swap
until the TCP connection gets around to eventually
timeing out.
So, let's just guess that we could have 15 Apache
children going, plus the Pg backend, and round it
out and configure 15G of swap.
And buy more RAM.
That should give us plenty of swap plus have
enough RAM floating about that Pg can have 1G
of shared memory and there's plenty of OS fs
buffers as well.
The only thing I've otherwise got to worry about is
that Apache might snatch the shared memory instead
of Pg. But we'll deal with that should it happen.
I'll probably want to set Apache's max_requests_per_child
config to something like 40 or 50. That should
minimize memory leaks in PHP. If I cut down the
maximum number of children to 20 that should ensure
that we never have so many running that we run out of
swap.
I'm pretty sure PHP is leaky.
Because CPU _is_ the bottleneck should we be considering
a new server rather than nickle and dimeing fiddling with
RAM? (Although to ensure that CPU continues to be the
bottlneck I need to make sure the indexes are right.)
Hunter?
On 05/10/2007 01:37:50 PM, Susan Alberts wrote:
> Thanks Hunter,
>
> I am very happy to buy more RAM. Karl's concern is that this is not
> our problem and won't solve it. He'd like to talk to you tomorrow
> (Friday). What is a good time?
> Susan
>
> On May 10, 2007, at 11:50 AM, Hunter Matthews wrote:
>
>> I'll increase swap tomorrow if you really want me to, but may I
>> please
>> make the case that
>> a) hardware is relatively cheap
>> b) science is hard
>> c) programmers and grad students time is relatively expensive?
>>
>> papio has _at_least_ two ram slots free, and a ram upgrade to 4GB
>> would
>> cost only $199 at crucial.com
>>
>> (papio is a 2850 model dell poweredge, for reference)
>>
>>
>>
>> On Wed, 2007-05-09 at 20:39 +0000, Karl O. Pinc wrote:
>>> Hi Hunter,
>>>
>>> I read the units wrong. Papio has almost 2G of swap.
>>> Still, that's what it's running out of and I know shared
>>> memory has something to do with it.
>>>
>>> Postgresql has crashed again since I reduced the
>>> shared memory requirements. httpd is using
>>> lots of memory, and I'm not suprised. It would not
>>> hurt for the memory used by httpd to swap out....
>>>
>>> I'll look at restricting httpd's RAM usage in the meantime.
>>>
>>>
>>> Karl <kop at meme.com>
>>> Free Software: "You don't pay back, you pay forward."
>>> -- Robert A. Heinlein
>>>
>> --Hunter Matthews Unix / Network
>> Administrator
>> Office: BioScience 145/244 Duke Univ. Biology
>> Department
>> Key: F0F88438 / FFB5 34C0 B350 99A4 BB02 9779 A5DB 8B09 F0F8 8438
>> Never take candy from strangers. Especially on the internet.
>>
>> _______________________________________________
>> Babase mailing list
>> Babase at www.eco.princeton.edu
>> http://www.eco.princeton.edu/mailman/listinfo/babase
>
> -----------------------------------------------
> Susan Alberts, Dept. Biology, Duke University, Durham NC 27708. Phone
> 919-660-7272, Fax 919-660-7293. alberts at duke.edu
>
>
>
> _______________________________________________
> Babase mailing list
> Babase at www.eco.princeton.edu
> http://www.eco.princeton.edu/mailman/listinfo/babase
>
Karl <kop at meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein
More information about the Babase
mailing list