[Babase] A comprehensive Hunter to-do list

Karl O. Pinc babase@www.eco.princeton.edu
Tue, 9 Nov 2004 16:20:40 -0600


This is what I'm waiting for from Hunter.  I'm starting
to get to where I'm waiting for the first two.

1) More disk space for my user id.

2) Tuning of the database server.  It needs to use a good
chunk of RAM to run fast, and without this the conversion takes
too long to run.  This should also make regular use of the database
faster.  And the database needs to log errors detected by the
triggers.

3) Automated access to the posgresql log files.  For permanent
logging of data rejection or other babase data problems.
Babase parts of these logs (the only part I care about)
to be visible to babase users via https.  Must be kinda
real-time so users can go back and see error messages
they just got but wern't paying attention to.

4) Access to the apache log files, both automated and real-time.
For permanent logging of babase php program errors and
records of program
execution, just like the postgresql logs.
(Again, I'm only interested in the babase portion
of the apache logs.)
This allows us to backtrack and see who did
what when to the database, and whether they used
the program with a bug in it that eats the database
to do it.  (Note that this is not a complete changelog
as people can man 'manual' changes directly to the
database.)
Real-time access is required as this is the best way to
debug php, to get php error messages out of the log.

5) Some unix account, that does not belong to a user,
(like babase_admin) we can use to run
cron jobs (against the postgresql server).  Users
come and go (and are deleted) and do un-project-related things and
I want to keep the babase stuff clearly marked.

6) Some way for the above user to send mail to us
when it finds problems.  (User does not need to receive
mail.)

7) We'll be keeping lots of files as permanent records.
These will be uploaded by the users to the web server.
I had thought we'd be keeping these files in our
shared group directory, but so long as the web server
has the same backup facilities as the shared directory
there's no reason not to keep these files on the web
server.  (Along with the logs, above.)  I'm looking
to Hunter for some comment one way or the other.
(The files are what has been loaded into the database,
to get an idea of disk size.  No reason why we can't
compress them.)  As the whole project is visible
on the web to somebody (with the right passwords)
it occurs to me that we likely won't be needing the
shared data space once we're done with the conversion.
(Until the next go-round, whatever that is.)

8) And a question: how far away from the servers are the
backups?  If the whole building falls down do we
lose everything.

If you'd like to talk with me about any of this
I've got a land-line at Princeton:
609 258 6797

It's been difficult to come up with a list like this,
for instance it took me a long time to find that
users on login.biology.duke.edu cannot send mail
(item 6), but I think this is comprehensive.

Karl <kop@meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein