[Babase] how to - 07a update in Babase 2.0

Karl O. Pinc kop at meme.com
Tue Dec 18 15:38:27 EST 2007


On 12/18/2007 12:43:01 PM, kfenn wrote:
> Hi Karl,
> 
> I'm in the final stages of culling data from our binders for the  
> 2007a demography update. I will be doing the update for the first  
> time in Babase 2.0 and will be making everything up as I go along  
> since there is no set protocol yet. Below I've outlined everything we  
> did in FoxPro system, the order in which we did it, and the  
> programs/process we used.
> 
> I'm assuming the order of input has to be maintained, unless you tell  
> me otherwise. Now I need a new method for all the different inputs in  
> Babase 2.0. I've outline the input methods I think I should use, and  
> I'll need your feedback on whether I've got it right or not, or in  
> some cases you'll have to fill in the blanks for me. A phone call on  
> all this in probably needed, but I thought composing this would give  
> us a good starting point.
> 
> Reproductive Cycles data
> Prepare cycles sub-tables ( Ex:Vcyc06a) (Bb 2.0 - prepare excel  
> sheets and export into csv or tab delimited format for upload)

Tab delimited would be more "universal" as the upload program takes
tab delimited.  (As opposed to PPA import which does it all.)

No need to enter sequence numbers, cycle ids or any of that.
Those are all computed by Babase 2.0.

> Manually enter puberty dates into BIOGRAPH table (Bb 2.0 - use UPDATE  
> command for each record in biograph?)

Update statements:  Good: written record  Bad: more typing
PPA browse: Good: less typing  Bad: no record, perhaps hard to find row

> Validate cycles sub-table (do Valcycles) (Bb 2.0 - cannot validate  
> before upload, but UPLOAD program will give error messages?)

Yes.  Any updating, whether with PPA browse, SQL, or the Babase Upload
will give error messages and won't change the db unless the data is
good.

> Upload cycles sub-table to master CYCLES table (do Upcen) (Bb 2.0 use  
> UPLOAD program to append to mtd_cycles view?)

Yes.

Note that Babase 2.0 decides on it's own which dates belong to which
cycle.  Uploading will go faster if the cycle dates for any given
female are entered in datewise order.  (I think that's the fast way.)

In other words, you could upload into the CYCPOINTS_CYCLES view,
just the Sname, Date, and Code columns, if that was any easier.
Or just put Mdates, Ddates, and Tdates into MTD_CYCLES, but have
only a single Mdate, Ddate, or Tdate in each row, etc.  The system
should glom them together into cycles.

Regardless of how the data goes in, after the first update it's
probably worth checking (again!, for the last time?) that the
system has (re)glommed the dates into cycles like it ought.

> 
> Pregs data
> Once cycles are loaded, cids are available for each pregnancy so
> Fill in cids for all Z dates on summary sheets (Bb 2.0 - I think the  
> cids are now the tcpids and dcpids in the mtd_cycles view)

Yes.
The Z dates are the Dcpids, because Z dates are Ddates.  And the Tdates
are the resume dates.  Resume dates should be automatically assigned
by Babase 2.0.  Leave them blank (NULL).  But there's a catch.
(Cue Twilight-Zone music.)

If a pregnancy has a resume date (because enough time has passed)
then you'll need to enter a (fake) gap in observation, and then remove
it after entering the birth in BIOGRAPH.  The way to (kinda)
avoid this is to enter 6 months of cycles, then pregs, then births,
and then go on to the next six months.  That way not every pregnancy
will have a resume.

If you don't have a (fake) gap then you can't enter the pregnancy
because the system will complain that a female is having sexual
cycles while pregnant, while PREGS.Resume is NULL.  And you can't
fill in the Resume because there's not been a birth, no new BIOGRAPH
row for the offspring.  And while you can put in a new BIOGRAPH
row if you like you can't associate it with a pregnancy when there's
no PREGS row.  (!)  At least this is how I recall the rules go.
Feel free to poke it with a stick and see if I'm right about the
rules and or if there's a less troublesome way to proceed.
I may have made recommendations/notes as to how to enter data
in the manual.  I forget.

No need to put in a fake gap for every pregnancy, the system
will complain about pregnancies that need them.  I don't have
a good feel for how often the problem will appear.

Of course, if there are no subsequent sexual cycles then there's no
problem, no need for a fake gap.  This is why entering 6 months
at a time makes life easier, many pregnancies will be ongoing and
there will be no subsequent cycles.  If you were to do 3 months,
or 1 month, or 1 week, at a time then the problem would, uh,
increasingly decrease.

If data entry is awful we can always change the rules.
Let's see how it works out before we do that.

> Prep pregs sub-tables leaving Resume field blank (Ex:Vp06a) (Bb 2.0 -  
> prep excel table and export to csv or tab delimited)
> Append sub-table to master PREGS table (Bb 2.0 - use UPLOAD program  
> to append to pregs table?)

There are few enough new pregs that I would probably just use PPA to
input the new pregs rows, or write SQL INSERT statements and cut and
paste.  You can use spreadsheets/export/upload if you like too.

> 
> Biograph data
> Prep sub-table with ‘new’ entries births, immigrant M, aborts  
> (Ex:Vbr06a) (Bb 2.0 - prep excel table and export to csv or tab  
> delimited)
> Append sub-table to master BIOGRAPH table (Bb 2.0 - use UPLOAD  
> program to append to biograph table)
> Manually enter Death data to BIOGRAPH fields: statdate, Dcause,  
> status (Bb 2.0 - use UPDATE command for each record in biograph?)

Again, I'd ether edit the tables directly in PPA or write SQL.
Probably the former, maybe with a printout of a query when done
for review/recordkeeping.  YMMV.

At this point take out the fake gaps that were entered for
pregnancies.

> 
> Census data

There is an upcen program in Babase 2.0, because there's no table
that looks like the census sheet.  (Do data in excel, export
tab delimited.)  The db validates as usual; no
data is changed if there are any errors.

> Enter census (5 primary groups) data to sub-tables (Vn0106, Vn 0206,  
> Vn0306, Vn0406, Vn0506, Vn0606)
> Enter demog notes for the group to subtable (Ex:Vd06a)
> Enter Other Group census into a single subtable (Ex:Gd06a)

I don't think I'm familiar with the Other group census.

> Validate census for 5 primary groups (Bb 2.0 - UPLOAD program should  
> catch errors and report them?)
> Upload census for 5 primary groups (Bb 2.0 - prep in excel, export as  
> tab delimited, use CENSUS program to append to the census table
> Validate demog info for 5 primary groups
> Update demog info for 5 primary groups

Again, no separate validation step.  Use UPLOAD into the
DEMOG_CENSUS view.  (You could also use CENSUS_DEMOG, but
I think DEMOG_CENSUS makes more sense.)

> Validate & update Other Group info
> 
> Cycstats, Repstats, Members
> Rebuild Cycstats and Repstats manually (Do Makerep)(Bb 2.0 - not sure  
> of new procedure, but I think its in documentation...Help!)

rebuild_all_cycstats
https://papio.biology.duke.edu/babase_system_html/re13.html
rebuild_all_repstats
https://papio.biology.duke.edu/babase_system_html/re17.html

These need to be run a lot. Changing many things (statdate,
etc., etc.) changes the results.  It's always safe to re-run
them after any update of the db.

> Members may be automatically rebuild or require manual (do  
> Buildmembers) (Bb 2.0 - Help again!)

Members automatically builds, always, but can be manually
rebuilt with rebuild_all_members.

> 
> Biograph: Matured dates for males
> Manually enter matured dates for natal & immigrant males to BIOGRAPH  
> (Bb 2.0 - use UPDATE command for each record)
> 
> Biograph: Dispersal dates
> Manually enter dispersal dates for natal & immigrant males to  
> BIOGRAPH (Bb 2.0 - use UPDATE command for each record)

Karl <kop at meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein



More information about the Babase mailing list