[Babase] Update trouble - Demog notes cannot have no note (no nulls allowed)?

Karl O. Pinc kop at meme.com
Tue Sep 7 13:02:10 EDT 2010


On 09/07/2010 11:22:14 AM, Niki Learn wrote:
> Yes, most of the M records in babase seem to be unnecessary and there
> are at
> least a small number that conflict with demography notes that are not
> yet in
> babase.  I expect I will delete most of these Ms when I do the demog
> note
> backfill entries.  There do, however, seem to be a small number of
> "real"
> Manual entries where, for example, we don't have an actual census
> datum but
> we know that an animal was in another group prior to arriving in a
> study
> group and so we manually place the animal in that other group prior 
> to
> their
> arrival in a censused group (such as when a previously unknown animal
> is
> seen coming from another group or an easily recognizable but unnamed
> animal
> known to be in a peripheral group shows up in a study group).  The M
> then
> provides a sense that the datum is a result of interpretation of
> circumstantial evidence rather than being an actual census point.

You probably want an 'N' in these cases, because 'N' does not
interpolate whereas 'M' does.

> 
> Thanks for allowing comment to be null!

I exist to make your life complicated.  Oh no wait, I mean 
to serve.  :)

> 
> Niki
> 
> -----Original Message-----
> From: babase-bounces at eeblistserv.Princeton.EDU
> [mailto:babase-bounces at eeblistserv.Princeton.EDU] On Behalf Of Karl 
> O.
> Pinc
> Sent: Tuesday, September 07, 2010 11:35 AM
> To: The Baboon Database Project
> Subject: Re: [Babase] Update trouble - Demog notes cannot have no 
> note
> (no
> nulls allowed)?
> 
> Hi,
> 
> It's taken a while to get through my email backlog.
> 
> It has also taken me a while to work through the implications
> of having a NULL DEMOG.Comment.  My analysis is below and
> is probably worth reading.
> 
> In the end I came to the conclusion that everyone has made
> the right decision and a NULL DEMOG.Comment is the right
> way to go.  The change has been made to the database
> and the documentation will be updated very shortly.
> 
> ----------------<snip>-------------
> 
> Allowing the demography note to be null is not the right way
> to go.  Instead of doing that we should remove the DEMOG
> table entirely.  But I don't think that's the right approach
> either.
> 
> I'm not sure exactly what the problem is that you're having,
> or more precisely what exactly it is that you want to accomplish,
> so let me list the various issues that I see.  I'm going to think
> out loud as I go.
> 
> There are two possible sides to the problem.  A data entry problem
> and a record keeping problem.  Record keeping is paramount,
> although of course the data must go in, so let me start with
> record keeping.  Once we know what we want to do with that
> we can have further discussion regarding data entry.
> 
> There are a couple of possibilities.  First, you have no regular
> census
> but do have a demography note, with a useless remark.  This is 
> exactly
> the sort of case where you would use the 'M' (manual) CENSUS.Status
> value -- creating a manual census entry.  The 'M' Status was used
> prior to 2001, I think by Daphine in an attempt to force 
> interpolation
> to do the right thing when we discovered that our early stabs
> at interpolation were not working.  However the Status has not
> been used since so, if it's really necessary to record that the
> entry is the result of a demography note it could be used to mean
> 'census entry created as a result of a demography note'.  (In any
> case I've a feeling that all those pre-2001 "M" statuses don't belong
> and should be removed.  I could be wrong in this.)
> 
> This brings us to the heart of the issue, whether it's necessary
> to record that the source of the CENSUS row is a demography note.
> If it's not necessary to note that the source
> of the CENSUS-row-that-does-not-appear-on-the-census-sheet is 
> a demography note then 'M' works perfectly well. Alternately,
> again, if there's no other reason for manually fussing with CENSUS
> then the 'M' rows must be there because of demography notes.
> If there are multiple sources of census-like data, demography notes,
> something else, etc., that would cause you to make changes to CENSUS
> then we could either create separate codes for each, which we would
> want to do if it was important to query on this information, or
> we could have a comment column that indicates the information's
> source.  In the latter case, that's pretty much exactly what
> the DEMOG.Comment column is -- and you'd enter in that column
> something like "A demography note records that so-and-so
> is in such-and-such group".  Exactly the comment which
> started this thread, the comment that you want to get rid of!
> 
> :-)
> 
> (FWIW, I've a dim recollection of having had a discussion
> like this one before.)
> 
> The second possibility is that there is already a CENSUS row
> for the individual and there is a demography note placing
> the individual in the group.  In this case I think you do
> want to keep track of the existence of the demography note
> so that if the record of the census gets changed or otherwise
> goes away you need to keep a row on CENSUS because the
> demography note exists.  Humm....  The right way to do
> this is to allow NULL demography notes.
> 
> On 09/03/2010 04:35:55 PM, Niki Learn wrote:
> > Yes, mostly it is with other group censuses.  There may also be
> > occasional
> > instances where a baboon is marked present using a demography note
> > during an
> > incomplete census.  
> > 
> > Thanks - Karl, please fix at your earliest convenience.
> > 
> > Niki
> > 
> > -----Original Message-----
> > From: babase-bounces at eeblistserv.Princeton.EDU
> > [mailto:babase-bounces at eeblistserv.Princeton.EDU] On Behalf Of 
> Susan
> > Alberts
> > Sent: Friday, September 03, 2010 5:31 PM
> > To: The Baboon Database Project
> > Subject: Re: [Babase] Update trouble - Demog notes cannot have no 
> > note
> > (no
> > nulls allowed)?
> > 
> > I don't completely understand what situations produce a demog note  
> > that has a group and an ID number but no explanation-I presume that 
> 
> > this arises with other groups censuses? And sometimes with regular  
> > group censuses?
> > 
> > If I'm understanding correctly then I agree with Jeanne; go with 1.
> > 
> > Susan
> > On Sep 3, 2010, at 5:06 PM, Niki Learn wrote:
> > 
> > > Jeanne thinks 1 is best.  (2 is clunky and 3 is bad)
> > >
> > > If that's okay by Susan as well, Karl please remove the not-null  
> > > constraint on the comment column in demog.  Then the upload can  
> > > commence.
> > >
> > > Thanks!
> > > Niki
> > >
> > > From: babase-bounces at eeblistserv.Princeton.EDU
> > [mailto:babase-bounces at eeblistserv.Princeton.EDU 
> > > ] On Behalf Of Niki Learn
> > > Sent: Friday, September 03, 2010 4:26 PM
> > > To: 'The Baboon Database Project'
> > > Subject: [Babase] Update trouble - Demog notes cannot have no 
> note
>  
> > > (no nulls allowed)?
> > >
> > > I am working on the upload in test and have just come to the  
> > > demography notes portion.  Demography notes also include census  
> > > entries and other groups notes entries on nonstudy groups, where  
> > > often the "note" would just consist of "so-and-so is present in
> > such- 
> > > and-such group".
> > >
> > > At the joint lab meeting we talked extensively about some
> revisions 
> > 
> > > to how we enter demography notes and what actually counts as a  
> > > demography note.  One of the things we discussed was that these  
> > > notes should be as un-wordy as possible and that the ones that
> just 
> > 
> > > say things like "so-and-so is present in such-and-such group", 
> > which
> >  
> > > is already captured in the census record the demography note  
> > > creates, that we didn't have to have any words because they were  
> > > redundant.  So I implemented that for this upload and, just on 
> the
>  
> > > first group, got this error 9 times:
> > >
> > > CAUTION -- This error may not be real; prior row(s) of the
> uploaded 
> > 
> > > file were rejected:
> > > Line 17: ERROR: null value in column "comment" violates not-null  
> > > constraint
> > >
> > > So, the note can't be blank.  I see three possible options
> > >
> > > 1)      Remove the rule saying that the comment column (the 
> actual
>  
> > > note) cannot be null.  (This seems simplest.)
> > > 2)      Go back to "so-and-so is present in such-and-such group". 
>  
> > > (Perhaps preferred  for some reason?)
> > > 3)      Or I could enter those other groups entries the way we 
> > enter
> >  
> > > census data - it just might have a lot of "N" entries in the file
> I 
> > 
> > > upload (but babase reads "N" as "don't create a record for this  
> > > animal on this date" so it wouldn't create extra records or 
> > anything
> >  
> > > like that).  The downside there being that if there were actual  
> > > demography notes to go with any of them I would need a census 
> > record
> >  
> > > and a demography note but it comes out the same in babase.  Also, 
> 
> > > entering them as demog notes tells you the reference source 
> (i.e.,
>  
> > > is there an actual census sheet in the other groups binder or is
> it 
> > 
> > > an other groups note within one of the study group binders and 
> > which
> >  
> > > study group is it with).
> > >
> > > Thoughts?
> > >
> > > Thanks,
> > > Niki
> > >
> > >
> > > _______________________________________________
> > > Babase mailing list
> > > Babase at www.eco.princeton.edu
> > > http://www.eco.princeton.edu/mailman/listinfo/babase
> > 
> > ------------------------------------------------------------
> > Susan Alberts, Professor of Biology, Duke University, Box 90338,  
> > Durham NC 27708. Phone 919-660-7272, FAX 919-660-7293
> > 
> > 
> > 
> > 
> > 
> > _______________________________________________
> > Babase mailing list
> > Babase at www.eco.princeton.edu
> > http://www.eco.princeton.edu/mailman/listinfo/babase
> > 
> > _______________________________________________
> > Babase mailing list
> > Babase at www.eco.princeton.edu
> > http://www.eco.princeton.edu/mailman/listinfo/babase
> > 
> > 
> 
> 
> 
> 
> Karl <kop at meme.com>
> Free Software:  "You don't pay back, you pay forward."
>                  -- Robert A. Heinlein
> 
> 
> _______________________________________________
> Babase mailing list
> Babase at www.eco.princeton.edu
> http://www.eco.princeton.edu/mailman/listinfo/babase
> 
> _______________________________________________
> Babase mailing list
> Babase at www.eco.princeton.edu
> http://www.eco.princeton.edu/mailman/listinfo/babase
> 
> 




Karl <kop at meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein




More information about the Babase mailing list