[Babase] Morris (MOR) and interpolation rules in members table.
Karl O. Pinc
kop at meme.com
Tue Aug 18 18:05:41 EDT 2009
Hi,
There are no special interpolation rules for males. I do
recall some talk about it but I thought we decided against
making the end of the month be special in that way.
As far as females and juveniles go, there's few problems
because the individuals get marked dead and this stops
interpolation. The other "backstop" is that interpolation
does not happen until "later data", either censuses noting
the individual present or noting them absent, is entered.
This keeps interpolation from making assumptions about
what happens after "the end of time", the last census data
for an individual that's still living. Dead individuals
can be interpolated after the last entered census
based on the date of death. (More on this below.)
There are 2 ways to stop males from interpolating.
You could use a non-interpolating census code
("manual" or some other) to record the last date
you want the individual in the group. At this point
you are manually doing what the interpolation does.
The other way to go is to ensure that there's
always a subsequent absence recorded, even
if the individual leaves the group at the very
end of the month. (Vis Niki's point below.)
I recall quite a bit of discussion
about this some time ago. There can be a problem
for females and juveniles as well, even if they
do wind up marked dead. If say, a female is
censused present the last day of the month
but is absent thereafter, the female's group membership
should be in the unknown group if the death date
is long enough after the last time she
was censused present. For example, she
might be sick, and censused present on the
31st of the month and then absent on the
first. Perhaps the 4th is assigned as
the death date. If the absence on the first
is not recorded then interpolation will place
the individual in the group on the first, even
though she was censused absent.
I'm against Susan's proposal for "male interpolation"
rules for a number of reasons, but there's
no point in going over this right now. (Ultimately, I don't think
it can be reliably implemented, but that's something
we can determine if we want to talk further and work
out exactly what it is that we'd want implemented.) There was so much
discussion regarding the import of continuing to enter
absences for the next month after an individual disappears
that it should not be hard to find in the archives.
The problem is more apparent with males, but
can crop up with females and juveniles as well
depending on when the censuses are taken and when
the death date is assigned. I thought we had something
about this recorded in the data entry manual....
On 08/18/2009 02:32:49 PM, Niki Learn wrote:
> Karl,
>
> We have a babase puzzle for you.
>
> Jeanne, Susan, and I have been conversing over the male Morris. It
> all began when they were puzzled by his having a bstatus of 9 (which
> has impacts on Jordi's tooth abrasion data) since they were confident
> that he was of reasonably well-known age for an immigrant male.
> While
> investigating the bstatus issue we found some other irregularities.
> A
> couple are related to mistakes in oddball census information but one
> involves the interpolation rules for the members table. Please see
> the excerpts from the email string below and let me know what the
> current situation is on interpolating member presence for males
> (i.e.,
> is the 14-day rule in place for them even in study groups where
> censuses have been going on and the male is not marked as present or
> was this altered as Susan thought). I guess we'll move forward from
> there.
>
> Thanks!
> Niki
>
> -----Original Message-----
> From: Susan Alberts [mailto:alberts at duke.edu]
> Sent: Tuesday, August 18, 2009 12:59 PM
> To: Niki Learn
> Cc: 'Lacey Maryott'; 'Jeanne Altmann'
> Subject: Re: Morris (MOR)
>
>
> Niki wrote:
> >> While we're on ranks, Jeanne asked about Morris' rank during his
> >> short tenure with Dotty's group. His ALM ranks are below:
> >> 17098 ¦ MOR ¦ 1997-10-01 ¦ 1.20 ¦ ALM ¦ 7 (out of 20
> >> males)
> >> 17120 ¦ MOR ¦ 1997-11-01 ¦ 1.20 ¦ ALM ¦ 9 (out of 22
> >> males)
> >> 17142 ¦ MOR ¦ 1997-12-01 ¦ 1.20 ¦ ALM ¦ 9 (out of 24
> >> males)
> >> 17166 ¦ MOR ¦ 1998-01-01 ¦ 1.20 ¦ ALM ¦ 9 (out of 24
> >> males)
>
> >> Morris shows up in Dotty's on the 9 Oct 1997 census and leaves
> again
> >> somewhere between the 17th and 21st. The 9 Oct demog note reads:
> >> 1356, There is a new yellow adult male in Dotty's group
> today. He
> >> seems to be very well habituated to people and he seems to me as
> one
> >> of the Olkenya group males but I'm not quite positive about it.
> He
> >> was observed consorting with Asha today.
> >>
> >> 11 Oct 1997:
> >> The new adult male who was first found in the group on 9
> Oct 97 is
> >> still with them today. Roy and Piston were observed displacing
> him
> >> several times.
> >>
> >> 16 Oct 1997:
> >> The new adult male is still with the group. Confirmed by
> RSM/SNS
> >> that he is one of the Olkenya males. We have named him Morris.
> >>
> >> 20 Oct 1991:
> >> 0946 Morris not seen with the group today.
> >>
> >> Curiously, he was ABSENT all of November 1997 BUT HAS A RANK FOR
> >> THAT MONTH...
>
> Susan wrote:
> > I just checked this. In Members, he is interpolated as being
> present
> > in Dotty's, 1.2, from the 19th Nov onwards. This more or less
> forces
> > us to rank him, as noted above. I presume there are no Census
> records
> > that contravene the interpolated Members data fro 19th nov onwards?
> > Or, are there contradictory records?
> >
> > Niki wrote:
> > There is contradictory evidence for November. He is actually
> marked
>
> > absent on every census day in November but whoever was entering the
>
> > data then did not enter it since he had already been gone from the
> > group at the end of October. Census dates for November range from
> 1
>
> > Nov to 29 Nov (including 19 Nov, which was a census day) and the
> > first census date in December (where Morris is again present) is 3
> > Dec. Thus by the interpolation rules (pasted below) if he had been
>
> > marked absent in babase for November as the field data indicate, he
>
> > would not have been interpolated to be present with the group again
>
> > until 1 Dec and thus should not have been ranked in November (but
> > should have been ranked, according to interpolation rules, in
> > January since he did not appear on the January census despite being
>
> > present on the last census date in December - also the first
> January
>
> > census wasn't until the 9th so even if he had been marked absent on
>
> > that day, the interpolation rules would still have marked him
> > present for part of January).
>
> >From the babase documentation:
> > The premise of interpolation is that an individual is assumed to be
>
> > in the group where observed for a period of 14 days to either side
> > of the observation unless there's indication otherwise. To this
> end,
>
> > interpolation keeps an individual in the group where a census
> > locates him for a time period that is the shorter of:
> >
> > 1. Half of the time interval between the individual's next
> > (or prior) census that finds the individual in any group.
> > 2. Half of the time interval between the next (or prior)
> > recorded absence from the group in which the individual was
> > censused. Absences from other groups are ignored.
> > 3. The 14 day Interpolation Limit. Given no other
> > information, an individual is considered to remain (or have been)
> in
>
> > the group where observed for 14 days following (or preceding) the
> > date of observation.
>
> Niki wrote:
> > So this raises a few questions for me as the enterer of the census
> > data. First, I was also inclined not to enter absences like
> Morris'
>
> > November absences where the individual was absent the entire month
> > after already having been marked absent for at least the end of the
>
> > prior month. This is in part because I didn't realize there was
> any
>
> > interpolation going on and didn't know that not marking the
> > individuals absent might (depending on circumstances) affect the
> > interpolations. Also, usually the team does not mark an individual
>
> > absent for an entire month unless he or she had been present at the
>
> > end of the previous month (and then I do enter the zeroes for that
> > month) - they only occasionally do what they did in Morris' case.
> > So this raises two questions for me:
> >
> > 1) Should zeroes such as Morris' always be entered or should it be
> > done on a case-by-case basis depending on where the individual was
> > next/how the interpolation rules will be applied? It seems for
> > example that it would only really have an effect in cases where the
>
> > individuals move in and out of the group and happen to change
> groups
>
> > between the last census day of one month and the first of the next
> > as Morris did at the juncture between November and December
> (because
>
> > if they leave or show up in the middle of a month, due to the way
> > uploads work, they get marked absent for all census dates that they
>
> > were not present but they are usually not marked absent in the
> month
>
> > before if they appear at the beginning of a month or in the month
> > after if they leave after the last census date since the team
> > usually has, understandably, not included them on the census sheets
>
> > in those other months).
>
> Susan wrote:
> I think for females and juveniles, it is generally safe to not enter
> the zeros. If they disappear, they are pretty much dead. For males,
> unless we know they are dead,I think we need to enter the zeros when
> they appear in the census. I did think that Karl had rewritten the
> interpolation program so that males did not appear in a group via
> interpolation where they had not been censused -- isn't that right
> Jeanne? So the problem with Morris interpolated in Nov should not
> have
>
> happened. I'm confused. Jeanne, do we need to talk to Karl?
>
> Niki wrote:
> > 2) Especially given this 14-day rule it seems that it might be
> > prudent to go ahead and make sure there is a zero entered on either
>
> > end of an individual appearing in or disappearing from a group,
> even
>
> > if this means adding a month of zeroes that has not been marked on
> > the census sheets (since a lack of presence on the census sheets
> > implies absence during censuses). For example, in the case of
> > Morris above where he was interpolated as being present with
> Dotty's
>
> > beginning on Nov 19 - he was clearly not with the group on the 19th
>
> > nor on several other census days after. Does this make sense? Of
> > course, if we change how we enter the data now, the next question
> > is, do I have to go back and alter past data entry to match...?
>
> Susan wrote:
> Again, I think for females and juvs, this is generally not needed,
> because they don't come back. For adult males, it may be a risk but I
>
> thought we had fixed that risk, so that a male can't be interpolated
> into a group for which census records exist and he was not there and
> there are no demog notes.
>
> Niki wrote (in case this helps make sense of the comments about
> January interpolation above)
> >> He is marked on the census sheets as absent from Linda's
> throughout
> >> November and December (but not in January 1998) and these were not
> >> entered into babase.
> >>
> >> Morris is then marked as present in Dotty's on all 11 census dates
> >> in December 1997, which is consistent with the babase record.
> There
> >> are no accompanying demog notes.
> >>
> >> However, he again fails to appear in the babase census in January
> >> 1998, DESPITE HAVING A RANK FOR THAT MONTH.
>
> Susan wrote:
> > Right, but again, he is interpolated as having been there for a
> while
> > at the beginning of the month (see Members). That forces us to rank
>
> > him.
> >
> > Niki wrote:
> > Got it (now).
>
> >> All 1997 census entries for Morris (none appear from 1998):
> >> cenid ¦ date ¦ sname ¦ grp ¦ status ¦ cen
> >> −−−−−−−−+−−−−−−−−−−−−+−−
> >> −−−−−+−−−−−−+−−−−−−−−+−−−
> >> −
> >> 1635082 ¦ 1997-10-01 ¦ MOR ¦ 1.20 ¦ A ¦ t
> >> 1635083 ¦ 1997-10-03 ¦ MOR ¦ 1.20 ¦ A ¦ t
> >> 1635084 ¦ 1997-10-06 ¦ MOR ¦ 1.20 ¦ A ¦ t
> >> 1635085 ¦ 1997-10-09 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1635086 ¦ 1997-10-11 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1635087 ¦ 1997-10-14 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1635088 ¦ 1997-10-16 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1491260 ¦ 1997-10-20 ¦ MOR ¦ 9.00 ¦ M ¦ f
> >> 1635089 ¦ 1997-10-20 ¦ MOR ¦ 1.20 ¦ A ¦ t
> >> 1635090 ¦ 1997-10-23 ¦ MOR ¦ 1.20 ¦ A ¦ t
> >> 1635091 ¦ 1997-10-25 ¦ MOR ¦ 1.20 ¦ A ¦ t
> >> 1635092 ¦ 1997-10-26 ¦ MOR ¦ 1.20 ¦ A ¦ t
> >> 1635093 ¦ 1997-10-27 ¦ MOR ¦ 1.20 ¦ A ¦ t
> >> 1639088 ¦ 1997-12-03 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1639089 ¦ 1997-12-05 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1639090 ¦ 1997-12-08 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1639091 ¦ 1997-12-10 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1639092 ¦ 1997-12-12 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1639093 ¦ 1997-12-13 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1639094 ¦ 1997-12-15 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1639095 ¦ 1997-12-17 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1639096 ¦ 1997-12-19 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1639097 ¦ 1997-12-21 ¦ MOR ¦ 1.20 ¦ C ¦ t
> >> 1639098 ¦ 1997-12-29 ¦ MOR ¦ 1.20 ¦ C ¦ t
>
>
>
> >> -----Original Message-----
> >> From: Susan Alberts [mailto:alberts at duke.edu]
> >> Sent: Monday, August 17, 2009 12:26 PM
> >> To: Niki Learn; Lacey Maryott
> >> Cc: Jeanne Altmann
> >> Subject: Morris (MOR)
> >>
> >> Hi Niki and Lacey,
> >>
> >> Regarding male MOR (Morris):
> >>
> >> He has a current bstatus in biograph of 9 ("do not use"). He was
> in
> >> the study population for 10 years, entering DOtty's Group in 1997,
>
> >> and
> >> then gone for a while, then Linda's Group in 2001, where he stayed
> >> until he disappeared.
> >>
> >> Jeanne and I think that, based on the information we found, he
> should
> >> have a bstatus of 2, not 9. He entered Dotty's as an adult (which
> >> means he was at least 7.5 - 8 years), he was named at that time,
> his
> >> birthdate estimates him to be 10 years old at that time (but with
> >> bstatus of 9). He came from Olkenya group but we don't have
> specific
> >> information about him there, other than that he was there. I
> remember
> >> him throuighout his time in Linda's group (2001- 2007) as an older
> >> male. This also suggests that an estimate of 1987 is reasonable.
> >>
> >> We'd like you to check all the records we checked (Dotty's in
> 1997,
> >> Olkenya group before and around then, and most importantly any
> >> records
> >> of age estimates for him) and see if you find anything to
> contradict
> >> our idea that a bstatus of 2 makes sense for him.
> >>
> >> Also, he is currently in members as having been in Nyayo's from 11
> -
> >> 26 July 2001, a few weeks before he permanently immigrated into
> >> Linda's. This looks wrong to Jeanne and I based on what we found
> --
>
> >> we
> >> are not convinced that he should ever have appeared as a member of
> >> Nyayo's, and certainly not starting 11 July. Can you take a look
> at
> >> the records and see what you think might be going on?
> >>
> >> Thanks, let us know if you have any qeustions.
> >>
> >> Susan
> >>
> >> PS Lacey is out today and perhaps a few more days, so Niki if you
> can
> >> fit this in feel free, but if there is a natural division of which
> of
> >> these you versus Lacey should do, feel free to wait until she is
> >> back.
> >>
> -------------------------------------------------------------------
> >> Susan Alberts, Department of Biology, Duke University, Box 90338,
> >> Durham NC 27708 USA, Phone 919-660-7272, FAX 919-660-7293,
> alberts at duke.edu
>
>
>
>
>
> _______________________________________________
> Babase mailing list
> Babase at www.eco.princeton.edu
> http://www.eco.princeton.edu/mailman/listinfo/babase
>
>
Karl <kop at meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein
More information about the Babase
mailing list