[Babase] Update trouble - Demog notes cannot have no note (no
nulls allowed)?
Karl O. Pinc
kop at meme.com
Tue Sep 7 13:02:10 EDT 2010
On 09/07/2010 11:22:14 AM, Niki Learn wrote:
> Yes, most of the M records in babase seem to be unnecessary and there
> are at
> least a small number that conflict with demography notes that are not
> yet in
> babase. I expect I will delete most of these Ms when I do the demog
> note
> backfill entries. There do, however, seem to be a small number of
> "real"
> Manual entries where, for example, we don't have an actual census
> datum but
> we know that an animal was in another group prior to arriving in a
> study
> group and so we manually place the animal in that other group prior
> to
> their
> arrival in a censused group (such as when a previously unknown animal
> is
> seen coming from another group or an easily recognizable but unnamed
> animal
> known to be in a peripheral group shows up in a study group). The M
> then
> provides a sense that the datum is a result of interpretation of
> circumstantial evidence rather than being an actual census point.
You probably want an 'N' in these cases, because 'N' does not
interpolate whereas 'M' does.
>
> Thanks for allowing comment to be null!
I exist to make your life complicated. Oh no wait, I mean
to serve. :)
>
> Niki
>
> -----Original Message-----
> From: babase-bounces at eeblistserv.Princeton.EDU
> [mailto:babase-bounces at eeblistserv.Princeton.EDU] On Behalf Of Karl
> O.
> Pinc
> Sent: Tuesday, September 07, 2010 11:35 AM
> To: The Baboon Database Project
> Subject: Re: [Babase] Update trouble - Demog notes cannot have no
> note
> (no
> nulls allowed)?
>
> Hi,
>
> It's taken a while to get through my email backlog.
>
> It has also taken me a while to work through the implications
> of having a NULL DEMOG.Comment. My analysis is below and
> is probably worth reading.
>
> In the end I came to the conclusion that everyone has made
> the right decision and a NULL DEMOG.Comment is the right
> way to go. The change has been made to the database
> and the documentation will be updated very shortly.
>
> ----------------<snip>-------------
>
> Allowing the demography note to be null is not the right way
> to go. Instead of doing that we should remove the DEMOG
> table entirely. But I don't think that's the right approach
> either.
>
> I'm not sure exactly what the problem is that you're having,
> or more precisely what exactly it is that you want to accomplish,
> so let me list the various issues that I see. I'm going to think
> out loud as I go.
>
> There are two possible sides to the problem. A data entry problem
> and a record keeping problem. Record keeping is paramount,
> although of course the data must go in, so let me start with
> record keeping. Once we know what we want to do with that
> we can have further discussion regarding data entry.
>
> There are a couple of possibilities. First, you have no regular
> census
> but do have a demography note, with a useless remark. This is
> exactly
> the sort of case where you would use the 'M' (manual) CENSUS.Status
> value -- creating a manual census entry. The 'M' Status was used
> prior to 2001, I think by Daphine in an attempt to force
> interpolation
> to do the right thing when we discovered that our early stabs
> at interpolation were not working. However the Status has not
> been used since so, if it's really necessary to record that the
> entry is the result of a demography note it could be used to mean
> 'census entry created as a result of a demography note'. (In any
> case I've a feeling that all those pre-2001 "M" statuses don't belong
> and should be removed. I could be wrong in this.)
>
> This brings us to the heart of the issue, whether it's necessary
> to record that the source of the CENSUS row is a demography note.
> If it's not necessary to note that the source
> of the CENSUS-row-that-does-not-appear-on-the-census-sheet is
> a demography note then 'M' works perfectly well. Alternately,
> again, if there's no other reason for manually fussing with CENSUS
> then the 'M' rows must be there because of demography notes.
> If there are multiple sources of census-like data, demography notes,
> something else, etc., that would cause you to make changes to CENSUS
> then we could either create separate codes for each, which we would
> want to do if it was important to query on this information, or
> we could have a comment column that indicates the information's
> source. In the latter case, that's pretty much exactly what
> the DEMOG.Comment column is -- and you'd enter in that column
> something like "A demography note records that so-and-so
> is in such-and-such group". Exactly the comment which
> started this thread, the comment that you want to get rid of!
>
> :-)
>
> (FWIW, I've a dim recollection of having had a discussion
> like this one before.)
>
> The second possibility is that there is already a CENSUS row
> for the individual and there is a demography note placing
> the individual in the group. In this case I think you do
> want to keep track of the existence of the demography note
> so that if the record of the census gets changed or otherwise
> goes away you need to keep a row on CENSUS because the
> demography note exists. Humm.... The right way to do
> this is to allow NULL demography notes.
>
> On 09/03/2010 04:35:55 PM, Niki Learn wrote:
> > Yes, mostly it is with other group censuses. There may also be
> > occasional
> > instances where a baboon is marked present using a demography note
> > during an
> > incomplete census.
> >
> > Thanks - Karl, please fix at your earliest convenience.
> >
> > Niki
> >
> > -----Original Message-----
> > From: babase-bounces at eeblistserv.Princeton.EDU
> > [mailto:babase-bounces at eeblistserv.Princeton.EDU] On Behalf Of
> Susan
> > Alberts
> > Sent: Friday, September 03, 2010 5:31 PM
> > To: The Baboon Database Project
> > Subject: Re: [Babase] Update trouble - Demog notes cannot have no
> > note
> > (no
> > nulls allowed)?
> >
> > I don't completely understand what situations produce a demog note
> > that has a group and an ID number but no explanation-I presume that
>
> > this arises with other groups censuses? And sometimes with regular
> > group censuses?
> >
> > If I'm understanding correctly then I agree with Jeanne; go with 1.
> >
> > Susan
> > On Sep 3, 2010, at 5:06 PM, Niki Learn wrote:
> >
> > > Jeanne thinks 1 is best. (2 is clunky and 3 is bad)
> > >
> > > If that's okay by Susan as well, Karl please remove the not-null
> > > constraint on the comment column in demog. Then the upload can
> > > commence.
> > >
> > > Thanks!
> > > Niki
> > >
> > > From: babase-bounces at eeblistserv.Princeton.EDU
> > [mailto:babase-bounces at eeblistserv.Princeton.EDU
> > > ] On Behalf Of Niki Learn
> > > Sent: Friday, September 03, 2010 4:26 PM
> > > To: 'The Baboon Database Project'
> > > Subject: [Babase] Update trouble - Demog notes cannot have no
> note
>
> > > (no nulls allowed)?
> > >
> > > I am working on the upload in test and have just come to the
> > > demography notes portion. Demography notes also include census
> > > entries and other groups notes entries on nonstudy groups, where
> > > often the "note" would just consist of "so-and-so is present in
> > such-
> > > and-such group".
> > >
> > > At the joint lab meeting we talked extensively about some
> revisions
> >
> > > to how we enter demography notes and what actually counts as a
> > > demography note. One of the things we discussed was that these
> > > notes should be as un-wordy as possible and that the ones that
> just
> >
> > > say things like "so-and-so is present in such-and-such group",
> > which
> >
> > > is already captured in the census record the demography note
> > > creates, that we didn't have to have any words because they were
> > > redundant. So I implemented that for this upload and, just on
> the
>
> > > first group, got this error 9 times:
> > >
> > > CAUTION -- This error may not be real; prior row(s) of the
> uploaded
> >
> > > file were rejected:
> > > Line 17: ERROR: null value in column "comment" violates not-null
> > > constraint
> > >
> > > So, the note can't be blank. I see three possible options
> > >
> > > 1) Remove the rule saying that the comment column (the
> actual
>
> > > note) cannot be null. (This seems simplest.)
> > > 2) Go back to "so-and-so is present in such-and-such group".
>
> > > (Perhaps preferred for some reason?)
> > > 3) Or I could enter those other groups entries the way we
> > enter
> >
> > > census data - it just might have a lot of "N" entries in the file
> I
> >
> > > upload (but babase reads "N" as "don't create a record for this
> > > animal on this date" so it wouldn't create extra records or
> > anything
> >
> > > like that). The downside there being that if there were actual
> > > demography notes to go with any of them I would need a census
> > record
> >
> > > and a demography note but it comes out the same in babase. Also,
>
> > > entering them as demog notes tells you the reference source
> (i.e.,
>
> > > is there an actual census sheet in the other groups binder or is
> it
> >
> > > an other groups note within one of the study group binders and
> > which
> >
> > > study group is it with).
> > >
> > > Thoughts?
> > >
> > > Thanks,
> > > Niki
> > >
> > >
> > > _______________________________________________
> > > Babase mailing list
> > > Babase at www.eco.princeton.edu
> > > http://www.eco.princeton.edu/mailman/listinfo/babase
> >
> > ------------------------------------------------------------
> > Susan Alberts, Professor of Biology, Duke University, Box 90338,
> > Durham NC 27708. Phone 919-660-7272, FAX 919-660-7293
> >
> >
> >
> >
> >
> > _______________________________________________
> > Babase mailing list
> > Babase at www.eco.princeton.edu
> > http://www.eco.princeton.edu/mailman/listinfo/babase
> >
> > _______________________________________________
> > Babase mailing list
> > Babase at www.eco.princeton.edu
> > http://www.eco.princeton.edu/mailman/listinfo/babase
> >
> >
>
>
>
>
> Karl <kop at meme.com>
> Free Software: "You don't pay back, you pay forward."
> -- Robert A. Heinlein
>
>
> _______________________________________________
> Babase mailing list
> Babase at www.eco.princeton.edu
> http://www.eco.princeton.edu/mailman/listinfo/babase
>
> _______________________________________________
> Babase mailing list
> Babase at www.eco.princeton.edu
> http://www.eco.princeton.edu/mailman/listinfo/babase
>
>
Karl <kop at meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein
More information about the Babase
mailing list