[Babase] Conversion speed & plan

Karl O. Pinc babase@www.eco.princeton.edu
Mon, 11 Oct 2004 23:40:09 -0500


I've sped up the interpolation algorithm and the conversion
now runs in (at a big guess) 36 hours.  I'm going to let
it run overnight and then kill it so it does not slow the
system down while I work.  The next step is speeding up
the repstats/cycstats algorithm, which should knock
another 6 hours off the conversion time.  I can also
look at tuning the database and that might significatly
speed things up, or might not.  I'll see about
finishing with this tomorrow which should leave plenty
of time to get done with the old/new database comparisons
by the end of the week.

Randomizing the census rows for
comparison should be easy, an hour or two, but will take a while to
run so we probably won't see results on this until
the end of next week.  We won't want to overlap running this
with running the conversion program as they'll both
be slow (alghough we can if we want to.)

Karl <kop@meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein