The Social Group Residency Rules

Sometimes, an individual’s group membership in MEMBERS shows them temporarily disappearing from a group. This might mean that the individual is preparing to leave the group, but there are alternative explanations that do not involve such a dispersal. Even when physically absent from a group, an individual may remain socially present there. After the system uses interpolation to estimate an individual's physical presence each day of their life (each row in MEMBERS), the system may analyze those data to estimate the individual’s social presence, or residency, on each of those days. These residency analyses "smooth out” temporary absences and provide an objective method for identifying when an individual has dispersed from a group.

Note

In this context, “disperse” is used differently than it is in the DISPERSEDATES table, where a “dispersal” is strictly defined as the date when a male leaves his maternal group. In this section, a “dispersal” is any case where an individual of any sex leaves a known group for a significant period of time.

Caution

Unlike interpolation, residency (and supergroup) information is not automatically updated by the system. The data managers are expected to run one or more of the data analysis procedures to update residency information whenever changes to Babase data might affect residency.

In general, an individual becomes resident in a group on a particular date if interpolation places them in the group more than it places them in any other group in a period beginning that date and extending through the next 28 days. They will remain resident in the group until the end of the last consecutive 29-day (“today” + 28 days) window that meets this criterion. The 29-day interval was chosen because interpolation can place an individual in a group for up to 14 days on either side of the date on which the individual was censused present in the group. 29 days is the longest period during which an individual can be interpolated into a group with a single census.

The residency rules are asymmetric in that there is one set of rules for establishing residency and another for maintaining residency. It is not always possible to determine if an individual is resident on any given day without knowing about the determination of the individual's residency on surrounding days.

Some specifics of the group residency rules differ depending on the density of available census data for the individual. When the MEMBERS.LowFrequency is TRUE, the process used to determine an individual’s residency for that day may be based on a slightly different set of rules. Briefly: if an individual is a resident of a group, is not seen for an extended period, and is still in that group the next time both the individual and the group are seen, then the individual is presumed to have remained resident in the group throughout the time that they were not seen.

Residency is never assigned in groups 9.0 ("Unknown") and 10.0 ("Alone"), as these two GROUPS.Gid values explicitly do not represent actual social groups.

Which Group?

For brevity in this chapter, group identity is discussed as though it is absolute: a social group of individuals has a particular name or identity, and that group’s identity never changes. That presumption is often incorrect. Groups may gradually divide into subgroups, and multiple smaller groups may gradually assemble into a single larger group. This section describes how the residency rules determine the group in which an individual is assigned residency, especially during those periods of group fission or fusion when an individual’s presence in one group might be evidence for residency in another.

During fission and fusion periods, residency is determined with respect to the parent group(s). After fission/fusion completes — on the GROUPS.Permanent date of the daughter group(s) — individuals become resident in the daughter group(s). Consequently, all residency rules are with respect to the parent group(s) during periods of fission or fusion and otherwise are with respect to the current group[206]. There is one exception.

Shortly after a fission ends, some individuals may continue to float between the daughter groups. While the fission is still recent, these visits should still be treated as being in the same group. For the first 28 days after the fission ends, being in any daughter group is evidence for residency in both[207]. Still, during this period the system can only assign residency in one of the daughter groups. The group of residency is whichever group is visited first in this period.

To implement these rules in a way that is agnostic to the presence/absence of a group fission or fusion, the system uses the MEMBERS.Supergroup and Delayed_Supergroup columns — never the Grp — when determining presence in a group. On two given dates X and Y (where X < Y but X + 28 > Y), an individual is considered to be in the same group if any of the following conditions are met:

If... Then...
Supergroup on X = Supergroup on Y The Grp on these two dates are identical, or they're two non-Permanent subgroups of the same group and this is during their fission
Supergroup on X = Delayed_Supergroup on Y One of the above, or X is in a parent group during a fission or fusion and Y is in a daughter group after the fission/fusion
Delayed_Supergroup on X = Delayed_Supergroup on Y One of the above, or both dates are ≤ 28 days after a fission and each of these dates are in different daughter groups of that fission.

Note

This system is not especially robust to individuals switching between parent groups during a fusion. Special consideration is needed when adding those data to CENSUS, as discussed above.

When an individual is determined to be resident on a particular date and the individual is present in that group, the MEMBERS.GrpOfResidency is the individual’s Supergroup for that Date. If the individual was not present in the group on that date, the GrpOfResidency is the Supergroup of the first subsequent row in which the individual is present in the group.

Example 4.4. Determining the GrpOfResidency when absent

An individual is resident in Hook’s group, which is in the process of dividing into two new groups: Linda's and Weaver's. One week before the fission ends, the individual is present in (his MEMBERS.Grp is) Hook's on Monday, Linda's on Tuesday, unknown on Wednesday, and Weaver's on Thursday. Because this is before the fission ended, residency can only be obtained in the parent group. Therefore, the Supergroup (and the GrpOfResidency) on Monday, Tuesday, and Thursday is Hook's. On Wednesday, the supergroup is unknown, so the system looks to the Supergroup on the first subsequent day in Hook's/Linda's/Weaver's to get the GrpOfResidency. In this case, that would be Thursday, in Weaver's. The Supergroup on Thursday is Hook's, so the GrpOfResidency for Wednesday is Hook's.


A special case occurs when an individual retains residency on a date that they are not present in the group, but the Supergroup of the next “present” row is a group that was not yet permanent on the date of the individual’s absence. This happens at the end of fissions and fusions when the “not present” day is before the parent group(s) ceased to exist, and the next “present” day is after the daughter group(s) became Permanent. In this situation, the system uses the Delayed_Supergroup — not the Supergroup — of the first subsequent MEMBERS row in which the individual is present in the group.

Example 4.5. Resident in a nonexistent group

The same individual from the previous example is still in Hook’s group, the fission of which is just about to end. On Friday, the last day of the fission — when he's still a resident in Hook's group — he is absent. On Saturday — the next day that he’s present — he will be in Linda’s group, which at that point will be Permanent. Usually, when determining the GrpOfResidency for an absent day like Friday, the system would look at the Supergroup for that next "present" day ("Linda's") and use it as Friday's GrpOfResidency. But Linda's group wasn't Permanent on Friday, so the individual can't be resident there that day. Instead, the system uses Saturday's Delayed_Supergroup: Hook’s.


In another special case, an individual is not present in the group on a date and is not seen in that group again, but retains residency there because 1) the date is on or shortly before the individual's Statdate and 2) the individual's Status is a Residency_Special_Case. (See Statdate is (also) special for more details.) When this occurs, the system cannot look forward to the next "present" date and must instead look back. The GrpOfResidency is that of the most recent date that the individual was present in the group. In the rare case that the GrpOfResidency ceases to exist during this "special case" period, the new GrpOfResidency is whichever daughter group was most recently visited (in the individual's MEMBERS.Grp). If no such group is found, the new GrpOfResidency will be whichever daughter group is numerically first[208]. It is an error if the system still cannot determine a GrpOfResidency[209].

The 15/29 test

In principal, an individual is resident in a group for a given period of time if they are present in that group more than they are absent from it. To assess an individual’s presence/absence, the system often uses what will hereafter be referred to as the “15/29 test”.

For a given date, the system investigates the 29-day “window” that begins on that date and ends after the subsequent 28 days. If the individual is present in a group for at least 15 of those 29 days, the individual has “passed” the test. The individual will likely — but not necessarily — be assigned residency in the group. How the system uses this information and whether the individual actually is a resident on the given date are context-dependent, explained later.

While performing this test, the system also counts the number of distinct dates in the window in which the individual was censused in any group, including both “present” and “absent” censuses. When this number is 3 or fewer, the pertinent MEMBERS.LowFrequency is TRUE. Otherwise, it’s FALSE.

Figure 4.20. An example 15/29 test

If we could we would display here a diagram showing an example of the 15/29 test and how it is calculated.

When there are fewer than 28 days after the given date in MEMBERS, the individual must still be present for at least 15 of those days to pass the test, and there must still be > 3 census dates to have a LowFrequency of FALSE.

To be clear: the 15/29 test alone does not determine whether or not an individual is resident in a group on a given date. The test is an important part of how residency is determined, but there are other factors that also affect that determination.

The Residency Algorithm

Residency for each individual is calculated via day-by-day iteration through the individual’s life history in MEMBERS. The system begins on the individual’s BIOGRAPH.Entrydate and continues chronologically onward through the individual's Statdate. While iterating through each date, the system tracks whether or not the individual is in an ongoing bout of residency and deploys a concomitant series of tests. When the individual is not in an ongoing bout, the system will try to obtain residency. When there is an ongoing bout, the system will try to retain residency.

When considering the 29-day window that begins on a specific date, the process of obtaining residency asks if the individual was resident in the group beginning on day 1. In contrast, the process of retaining residency asks if the individual was still resident on day 29. That is, obtaining residency is about finding the beginning of a residency bout, while retaining residency is about finding the end. These rules are discussed in greater depth below.

After the system finishes trying to obtain or retain residency for an individual on a date, the MEMBERS.Residency, LowFrequency, and GrpOfResidency for the row with that Date are populated.

When a continuous period of residency ends, the system also adds a row to the RESIDENCIES table for the entire period, or "bout". The Start_Date and Finish_Date are populated as expected. The Start_Status and Finish_Status are populated as discussed below.

As mentioned earlier, rows in MEMBERS from days before the Entrydate or after the Statdate are not analyzed for residency. When a residency analysis is performed, the Residency column for those rows is set to X. Thus, it can safely be assumed that when an individual has any MEMBERS rows with a NULL Residency, that individual's residency information is in need of an update.

Obtaining Residency

When the system iterates over a date on which the individual's residency has not yet been determined, the system will try to obtain residency for the individual on this date. If the individual is present in a real group — not 9.0 or 10.0 — on that date, the system performs a 15/29 test to determine if residency is obtained in that group. The putative end of the residency — the last day of the 29-day window in which the individual was present in the group, hereafter referred to as the putative last date — will likely be extended onward as the system iterates through subsequent dates and tries to retain residency.

While obtaining residency, it is not sufficient to merely be present in the group for 15 of the 29 days; the individual must also be present in the group on the first day of the 29-day window. Otherwise, individuals could become resident in a group up to 14 days before they are first present there.

If the individual passes the 15/29 test, their MEMBERS row for day 1 is updated to indicate that they’re resident: the Residency is set to R, the LowFrequency is populated according to the 15/29 test’s result, and the GrpOfResidency is that row’s Supergroup.

If the individual doesn’t pass the 15/29 test, or the individual is in group 10.0 on the date and didn’t attempt to obtain residency there, the row is updated to indicate that they’re not resident: the Residency is set to N and the LowFrequency and GrpOfResidency are NULL.

If the individual is in group 9.0 on this date and therefore can’t attempt to obtain residency, the row’s Residency is set to U and the LowFrequency and GrpOfResidency are NULL.

When residency is first obtained, the system categorizes how the residency period began. Later, when the residency ends, that information will be inserted into the RESIDENCIES.Start_Status column for the residency period. See RESIDENCIES.Start_Status for more information about the possible values.

Entrydate is Special

In some cases, an individual's residency can be inferred from the way that the individual entered the study population. At birth, for example, an infant does not need to "earn" residency by being present for the requisite number of days; the infant simply is intuitively a resident of the group from the beginning. When attempting to obtain residency on the individual's Entrydate, the system allows the possibility that the decision of whether or not to obtain residency might depend on the individual's Entrytype.

When the individual's Entrytype is indicated in the ENTRYTYPES table as a Residency_Special_Case (when the Residency_Special_Case is TRUE), the individual automatically obtains residency in the group in which they were present on the Entrydate. They do not need to pass the 15/29 test; their residency in the group is obtained solely because of their Entrytype. The only exception to this: if the individual is in group 9.0 or 10.0 on their Entrydate, then they will not obtain residency[210].

Individuals with any other Entrytype are not automatically assigned residency on their Entrydate. They may still become resident on this date, but they must pass the 15/29 test to obtain it, as usual.

When determining LowFrequency, the system has more special treatment for individuals who were born into the study population. To be clear: this is only for individuals whose Entrytype is B, not for all Entrytypes that are Residency_Special_Cases. It is presumed that if the group was being watched frequently enough to know the individual's date of birth then it must not have occurred during a low frequency period. Because of this, in the MEMBERS rows for those individuals' Entrydate and subsequent 28 days, any days where the individual is a resident and part of the bout that began on the Entrydate have their LowFrequency automatically set to FALSE.

Note

These special provisions may seem unnecessary. Most individuals whose Entrytype is a Residency_Special_Case would likely obtain residency there anyway. For most individuals whose Entrytype is B, the 15/29 test likely would have determined that the individual's LowFrequency was FALSE in those early days anyway. Also, in rare occasions an infrequently visited group might happen to be visited on a newborn individual's birth date, in which case a TRUE LowFrequency would arguably be more appropriate. So why have these special cases at all?

The primary aim is to accommodate individuals who didn't remain under continuous observation for very long. Short-lived infants, for example, may live fewer than 15 days and couldn't pass the 15/29 test, but they should nevertheless be considered residents during their brief lives. Similarly, if these infants do not live long enough to be censused more than 3 days, their brief lives would incorrectly appear to have occurred in a period of low frequency.

It remains true that an infrequently visited group could be visited on a newborn individual's birth date, and that it would be more accurate for that individual's LowFrequency to be TRUE. However, it is expected that short-lived individuals in regularly observed groups will be much more common than observed births in infrequently observed groups, so these provisions have been created to preserve data integrity for the former, admittedly at the expense of the latter.

Regardless of the individual's Entrytype, the 15/29 test is always performed on the Entrydate. The test might not be used to obtain residency, but it is still needed to determine the bout's putative last date, and unless the Entrytype is B it is also needed to calculate the Entrydate's LowFrequency.

Obtaining Residency with Low Frequency

The process of obtaining residency is the same, regardless of the density of available data.

Retaining Residency

Once resident in a group, an individual remains resident until the end of the last consecutive 29-day window that passes the 15/29 test. The image below illustrates that there will inevitably be a number of days at the end of a residency period — at least 14 — that will not pass the 15/29 test.

Figure 4.21. 29-day windows at the end of a residency period

If we could we would display here a diagram illustrating how the last several 29-day windows of a residency period will not pass the 15/29 test.

Because of this, an individual's residency on any given date is not determined simply by whether or not they passed the 15/29 test. Once residency has been obtained, an alternative approach is needed to identify when a period of residency ends.

When the system iterates over a date on which the individual is already resident — when day 1 of the 29-day window is ≤ the putative last date of the current residency bout — the system will try to retain the individual’s residency. While the process of obtaining residency is about asking if the individual is resident on day 1, the process of retaining residency begins already knowing that to be true. Instead, the system attempts to extend the putative last date by asking if the individual is still resident on day 29.

Note

Even though retaining residency focuses on day 29 of a window, MEMBERS is only updated for day 1. The remaining rows in the window are updated in subsequent iterations when they become the “new” day 1.

Days 2-28 are neither ignored nor skipped; they are included in any 15/29 tests, they have already had a turn at being day 29 in earlier windows, and they will have a turn being day 1 in later windows.

While the individual continues to be present in the group with no absences from the resident group and no appearances in other groups, retaining residency is simple. In each consecutive window, the putative last date is updated to day 29 and the MEMBERS row for day 1 is updated to show that the individual was resident: Residency is R, LowFrequency is populated based on the density of data in that window, and GrpOfResidency is the row’s Supergroup. The system then iterates onward to the next 29-day window.

When the individual is not in the resident group on day 29, the system checks if the window passes the 15/29 test. If yes, the individual is away from the group but has not (yet) been away long enough to lose their residency. Lacking evidence to extend the putative last date, it is not changed. Regardless, the individual is still resident on day 1: the MEMBERS row with that Date is updated to indicate that the individual is resident: Residency is R, LowFrequency is populated based on the density of data in the window, and GrpOfResidency is populated as expected. The system then iterates onward to the next 29-day window.

When the individual is not in the resident group on day 29 and the window fails the 15/29 test, the individual has apparently been away long enough to lose their residency. First, the system considers the density of available data in the most-recent window that did pass the test: if LowFrequency for that window is TRUE, there may be an alternative explanation for the individual’s absence, as discussed below in Retaining Residency with Low Frequency. If that alternative is ruled out, or if the most-recent "passing" window’s LowFrequency is FALSE, the individual has certainly been away long enough to lose their residency. The putative last date becomes confirmed — no longer putative — and can no longer be updated. When this first occurs, the confirmed last date will necessarily be a date within the window and later than day 1. Therefore, the individual is still resident on day 1: the MEMBERS row with that Date is updated to indicate that the individual is resident: Residency is R, LowFrequency is populated with the value used in the most-recent window that did pass the test, and GrpOfResidency is populated as expected. The system then iterates onward to the next 29-day window.

Once the residency’s last date is confirmed (not putative), the system won’t attempt to extend that date anymore and the process of retaining residency becomes a simple game of waiting for day 1 to become the last date. While day 1 is before or equal to the confirmed last date, the individual is certainly resident on day 1 and the related MEMBERS row is updated accordingly: Residency is R, LowFrequency is that of the most-recent window that did pass the 15/29 test, and GrpOfResidency is populated as expected. The system then iterates onward to the next 29-day window, unless day 1 is the confirmed last date. In that case, the residency bout is over and a corresponding row is added to the RESIDENCIES table. In the next 29-day window, there will be no residency to retain so the system will revert to attempting to obtain residency.

Before adding a new row to the RESIDENCIES table, the system categorizes how the residency ended. This information is recorded in the row's Finish_Status. See RESIDENCIES.Finish_Status for more information about possible values.

Note

While retaining residency, the system asks only whether the individual was in their resident group. When not in the resident group, the system does not pay attention to which group was visited. If an individual visits a group while resident elsewhere, the days of these visit(s) cannot count towards obtaining residency in the visited group later. The individual must lose residency before its visits to another group can count towards obtaining residency there.

Retaining Residency with Low Frequency

When an individual is not regularly or frequently censused, there may be periods of time when the individual is not seen at all and is interpolated into the unknown group (9.0). They will appear (in MEMBERS) to have left the group when (in reality) they have not. Because of this, when LowFrequency is TRUE an individual’s residency bouts might be extended across periods in which their whereabouts are unknown.

When a resident of an infrequently censused group stops being seen and is interpolated into the unknown group (grp 9.0) long enough to fail a 15/29 test, their residency in that group may not actually end. The residency will extend through the “unknown” period in either of two situations. First, the individual retains residency if they are still in that group the next Date they are seen, provided that the individual was never censused “absent” from that group in the intervening time period. Second, the individual retains residency if the next Date that they are seen is the individual's Statdate and their Status is a Residency_Special_Case, as discussed below.

For every MEMBERS row in this "unknown" period, the Residency will be R, LowFrequency will be TRUE, and GrpOfResidency will be that of the last window that passed the 15/29 test. If that group ceases to exist during this unknown period, the GrpOfResidency is whichever of that group's daughter groups is first visited by the individual after this "unknown" period ends..

Outside of the two aforementioned situations, the residency does not receive any special treatment and the residency bout ends as normal: on the last day the individual was seen (interpolated present) in the nonstudy group.

Caution

If a low-frequency group undergoes a fission or fusion during one of these "unknown" periods, the individual's presence in a parent group before the "unknown" period and in a daughter group afterward can only be recognized as the same group if the fission/fusion was in progress at the beginning or end (or both) of the "unknown" period. That is, 1) the last day before the "unknown" period or the first day after it must be during the part of a fission/fusion when the parent group(s) has not yet ceased to exist, or 2) the first day after the "unknown" period must be during the 28 days after the parent group(s) has ceased to exist, when the Delayed_Supergroup is the parent group. Outside of these circumstances, the individual is interpreted as being in two separate and entirely unrelated groups, and residency is not extended across the "unknown" period.

Statdate is (also) special

Sometimes, special treatment is needed when retaining residency and the last day of the window is the individual's Statdate. As the system approaches the end of available data for an individual, retaining residency may become disproportionately difficult. If the individual happens to be away from their grp of residency on their last day in MEMBERS, that single non-"present" day will cause them to prematurely lose their residency. In some cases that may be appropriate, but in other cases it does not reflect the demographic reality.

For example, if an individual is sick or injured near the end of their life and unable to keep up with the rest of their group, they eventually might fall so far behind that they are censused "alone" in their final few days. This would cause them to lose residency after their last present day in the group, apparently having dispersed from the group shortly before dying. But no, this hypothetical individual didn't socially leave their group; they just happened to be physically away from it at the time of their death. Their residency should end on their Statdate, not the last date present in the group.

In contrast, the end of an individual's data might occur because they dispersed from a study group to some unknown (never observed) other group. They were present in their group one day, seen alone the next, then never seen again. In that case, their residency will have ended because of a dispersal, so it would be quite appropriate for the individual to lose their residency as normal: on the last day they were present in the resident group.

In the above examples, the two different residency assignments made at the end of the individuals' MEMBERS data could be described with the same CENSUS data. The only difference between them: the manner in which they left the study population, i.e. their BIOGRAPH.Status. Thus, when attempting to retain residency in the days leading up to and including the individual's Statdate, the system allows some variation in the rules, depending on the individual's Status.

When the individual's Status is indicated in the STATUSES table as a Residency_Special_Case (when the Residency_Special_Case is TRUE), the MEMBERS row representing the individual's Statdate is treated as a sort of "wild card" in any 15/29 tests performed to retain residency. That is, when attempting to retain residency and counting the number of days that the individual was present in a given group in a window, the Statdate will always count as "present" in that given group. Thus, an individual who was only briefly away will remain resident through their Statdate, but an individual who was away long enough to fail a 15/29 test can still lose their residency shortly before their Statdate.

When an individual is resident in a group with LowFrequency, this special case works just a little differently and in one specific circumstance. Suppose that an individual was last seen present and resident in such a group, then was not seen for an extended period of time until at last they were found dead[211]. It is unclear if the individual was still a resident of the group when they died, or if they had dispersed to another group during that period of nonobservation. The only datum available to address this is the length of time between the individual's last Date in the resident group and the individual's Statdate. When the Statdate is more than 90 days after the last Date in the group of residency, it has been too long to make an inference about the individual's residency and the special case does not apply. When the Statdate is 90 or fewer days after that last Date in the group, the system infers that the individual had not left their resident group, and the individual is eligible to extend their residency through their Statdate. As in other cases where low frequency residency is extended through "unknown" periods, the individual's whereabouts must truly be unknown during those final 90 or fewer days; all MEMBERS rows in that time — including that of the Statdate — must have a Grp of 9.0, and the individual must have no "absent" CENSUS rows in the group of residency.

The "wild card" status for the Statdate risks allowing a narrow opportunity for an individual to pass a 15/29 test that the individual should have lost. If the individual is not in their resident group on their Statdate, and the 28 preceding days are evenly divided between the resident group and anywhere else (14 days in the group, 14 days in any other groups), then the individual has been away too long, and loses their residency. The "wild card" status in a residency special case does not extend the residency through the Statdate.

Individuals with any other Status are not forbidden from being resident through their Statdate. They simply must remain physically present in the group to retain their residency, as usual.



[206] From this discussion, it's tempting to conclude that residency can never be obtained/retained in a group before it becomes Permanent, but that would be an overgeneralization. Individuals can become resident in a non-permanent group if there is no parent group to be resident in. That is, individuals can be residents of a group before its Permanent date if the group has no From_group and does not exist as another group's To_group in the GROUPS table.

[207] Note that this is only an issue after a fission. After a fusion, there are not multiple daughter groups to switch between.

[208] I.e. if group 1 divided into groups 2 and 3, the system would choose group 2 because it comes before 3 when ordered numerically.

[209] Which would only happen if the group ceased to exist and has no daughter group(s).

[210] In practice, individuals with those Entrytypes probably shouldn’t be in either of those groups on their Entrydate anyway, but there are no rules that explicitly forbid it.

[211] For individuals who have been fitted with a radio collar this is not unusual.


Page generated: 2024-03-06T15:02:28-05:00.