[Babase] Auto updating of PREGS.Resume, new conversion files,
new errors
Susan Alberts
alberts at duke.edu
Thu Sep 21 14:53:10 EDT 2006
Karl, thanks for this update. I am pleased to hear that this is done.
Yay indeed. Does this mean that we will do the "big" conversion
starting tomorrow? Leah, Tabby, are you ready for that? Should we
discuss this by phone? We had agreed that we would proceed with this
unless the number of conversion errors seemed very large. This is
subjective, and kind of up to you guys. Also, let us know if we need
to have a call about the points Karl raises below.
Susan
On Sep 21, 2006, at 7:30 AM, Karl O. Pinc wrote:
> Hi,
>
> It appears I have all the validation working, and all
> the automatic updating working but for cycstats
> and repstats. (As previously discussed.)
>
> Yay! It turns out there was quite a bit of
> trigger code throughout the sexual cycle tables that had not
> been exercized and had bugs.
>
> In getting all the validation and automatic updating
> working it proved easiest to have PREGS.Resume
> automatcially become the resuming Tdate. See the
> PREGS docs for more info.
>
> There's a new set of conversion errors with all the
> validation turned on, but only loading the sexual
> cycle, interaction, and point sample data as
> has been done lately. The new files are:
>
> papio.biology.duke.edu:/biology/groups/babase/errors/loadscript.pregs
> papio.biology.duke.edu:/biology/groups/babase/errors/errors.pregs
>
> The latter file does not yet exist. It will come
> into existance in an hour or so when the conversion
> finishes.
>
> (Leah and Tabby, please delete whatever files
> you are not using in
> papio.biology.duke.edu:/biology/groups/babase/errors/.
> Some of the files in there are large and
> while I don't know that there's a problem,
> I don't want to run out of space at an awkward
> moment.)
>
> Tabby can look at the sexual cycle errors and see
> what they are about. As usual, there's a lot of
> "cascading" going on. See below for some analysis.
>
> As Tabby and I discussed, there are 3 new files that
> supply input to the conversion. They contain
> SQL statements (mostly INSERTS).
>
> The first is:
> papio.biology.duke.edu:/biology/groups/babase/real_gaps
>
> This contains the contents of the CYCGAPS table, as we
> would like it to look after the conversion. This
> (and the other files) was (were)
> initially generated programatically, but there's
> since been manual changes and we may want to make further
> changes. (For instance, one change was made as a result
> of one of Catherine's findings where there was a CYCPOINTS
> data point one day before observation resumed, so the
> data was changed so that the observation resumed one day
> earlier for that individual.)
>
> The second and third are:
> papio.biology.duke.edu:/biology/groups/babase/fake_gaps
> papio.biology.duke.edu:/biology/groups/babase/undo_fake_gaps
>
> The Babase rules won't allow a pregnancy to have
> a resume cycle unless there's a BIOGRAPH.Pid
> value that references the PREGS row. And they
> won't allow cycling after a pregnancy
> without a resume unless there's a gap.
> And of course a PREGS must have a Conceive
> on CYCPOINTS. So, in order to load PREGS
> the conversion program first loads CYCPOINTS.
> This allows PREGS to have a Conceive.
> Then it temporarly creates a ("fake") gap on each birthday,
> allowing pregs with no Resume's to have subsqeuent
> CYCPOINTS. Then it loads PREGS, but without any Resume values
> because there are no births yet.
> Then it puts the PREGS pids in BIOGRAPH.Pid to
> get a birth. Then it puts the PREGS.Resumes
> in. And finally it removes all the temporary
> gaps it created just so it can get the PREGS
> in.
>
> The fake_gaps file contains SQL INSERT statements
> that create all the "temporary" gaps. The
> undo_fake_gaps file contains the SQL to remove
> the "temporary" gaps.
>
> (The SQL in the undo_fake_gaps file also has statements
> that check to see that the gap to be removed exists,
> and if not it produces a warning. This is a double-check
> to make sure we don't think we're removing a temporary gap
> and then fail to remove it. Leah might be interested
> in the SQL techniques in the 2 "fake" files.)
>
> So, if pregancies are moved about then the fake_gaps
> and undo_fake_gaps files might need adjusting.
>
> There is already an issue with about 20 of the
> fake gaps. They are for stillbirths. The date
> of the stillbirth is the female's next Tdate.
> Babase does not care if you want to do this,
> I don't think (somebody might want to check
> the docs), but the conversion program does care
> because the fake gap is on the "birthdate" and
> that means there's a CYCPOINTS row (the Tdate)
> that occurs during a gap in observation. This
> is a no-no. The conversion produces an error
> that says "Cannot have CYCPOINTS when there's
> no observation". The solution is probably to
> adjust the date of the fake gap to be (say
> a week) before the (still)birthdate where
> there's a problem. I leave that to Tabby
> if that's really what should be done.
>
> Here's the mothers that I took the time to
> look at in detail that have this problem
> with stillbirths. (There are some others
> with the above error message
> but I didn't take the time to check them.
> Tabby should check them an confirm that
> what I think is going on is really happening.):
>
> NZI 1993-03-31
> SIS 1988-12-18
> LUN 1995-10-18
> OCH 1997-03-29
> VIN 1997-10-29
> VIN 1997-12-14
> VIN 1998-01-16
>
> Another issue with the "fake" gaps is that the State of the gap
> must not conflict with the neighboring CYCPOINTS rows.
> I've code in the fake_gaps file to compute the common case
> but there's also some cases where I've edited the fake_gaps file
> and altered a particular data value. Changes
> to CYCPOINTS might require changing the State of the fake gap.
>
> Karl <kop at meme.com>
> Free Software: "You don't pay back, you pay forward."
> -- Robert A. Heinlein
>
> _______________________________________________
> Babase mailing list
> Babase at www.eco.princeton.edu
> http://www.eco.princeton.edu/mailman/listinfo/babase
-----------------------------------------------
Susan Alberts, Dept. Biology, Duke University, Durham NC 27708. Phone
919-660-7272, Fax 919-660-7293. alberts at duke.edu
More information about the Babase
mailing list