[Babase] Conversion of interaction and point sample data, also rank validation question

Karl O. Pinc kop at meme.com
Sat Aug 26 20:48:22 EDT 2006


Hi,

I've finished with the data validation on the
social interaction part of the database.
This leaves the sexual cycle stuff.  And it
now occurs to me that it also leaves the ranks
table.  (See following email.)

This is mostly for Leah, but everyone will want to
read through this, especially the last part.

I've checked all the interaction
and point sample data by running the
data through the conversion.  This means
there's now data errors to go through.  There's
almost certainly still bugs in the code, so let me
know when you find one.

I've divided the errors into two parts, neighbors,
female point samples, and allmiscs and everything else.
(In two parts because I was able to not "start over"
and reconvert interactions when I had to fix bugs
later on in the process.)

This time, I've made a directory where I've put the files:
papio:/biology/groups/babase/errors/

The everything else errors are:
errors.interact
And the corresponding data file is:
loadscript.interact

There are about 1,700 errors, some of them 'cascading' --
resulting from previous errors.

The neighbors, female point samples, allmiscs related errors are in:
errors.neighbors
And the corresponding data file is:
loadscript.neighbors

There are about 3,000 of these, I'd say at least half
due to previous errors.

I believe the reason there's so many problems with
the neighbors may be that at various times various codes have
had different meanings.  For example, at some point in time
it was perfectly ok to have 2 different neighbors
be the same individual.  Later on, the same codes
were used but each neighbor was expected to be
unique.  A neighbor code in Babase (NCODES)
determines whether or not the neighbor must be
unique.  You _could_ use the same code for unique
and non-unique data, but then neither would get
checked for uniqueness.  That'd be silly.
The thing to do is to use a different code, and if
it's necessary to combine the two codes during
analysis we can have another column on the NCODES
table that says the two codes "go together".
We've done this already on the ACTS table with
respect to agonisims, the AS and AO agonisims are
all "A" type agonisims.

(The female point samples in particular seem to
have a lot of errors where one of the neighbors
is the focal of the point sample.)

All in all not bad for half a million interactions
and 30,000 point samples (10 times that many
individual points and 400,000 neighbors.)

If it matters, the result of the conversion is
in babase_test.  Note that there is no sexual
cycle related data in this database.

Karl <kop at meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein



More information about the Babase mailing list