[Babase] Dupe Cenids in DEMOG

Catherine Markham babase@www.eco.princeton.edu
Thu, 25 Aug 2005 08:46:52 -0400


> It's supposed to mean exactly that, that it will catch the error
> no matter how it's entered into the database.  This particular
> error I'm confident it will always catch.  (Some others are
> trickier -- mostly when you can get the same error state by
> different paths of change in the data, particularly when
> multiple tables are involved.)  The conversion does not do
> any error checking.  It just inserts the data into the database
> in an order that does not violate any rules, biograph first, etc.
> I could have skipped all the rule checking as the conversion
> loads the data, but then we wouldn't be checking either the data or
> my rules.  The rules are built into the database, so we can use
> any program or none to alter the data and be assured the data is good.
> 
> The only tricky part is that there's more than one way to get the
> data into the same state.  First change this table then change that,
> or vice versa, for example.  As the conversion is loading every thing
> the same way the rules are not fully exercised and no doubt there will
> still be bugs that allow bad data.  But it's better than what we
> had before.

Sorry, Karl - I do remember you explaining that to me before.  But two 
follow-up questions for you...

1.  How do you get the system to run that Babase-wide proofs of rules 
and such?  Is it something that happens on an automated schedule?

2.  Regardless of the details for the system error checking, I don't see 
why we still wouldn't want red flags going up with the validation 
program.  Ultimately, the database can be kept clean either way but I'd 
much rather be warned about a mistake at the point of data entry as 
opposed to resolving the issue later on.

Maybe the best bet is just for me to ask you about this on a conference 
call at some point.  It is a piece of the big picture that I don't quite 
get yet, but certainly not something worth holding us up on right now.

Thanks for the quick feedback,
Catherine