[Babase] Extra quotes in data -- Was: Re: min_max file revised

Karl O. Pinc kop at meme.com
Wed Mar 18 10:06:55 EDT 2009


On 03/18/2009 07:32:08 AM, kfenn wrote:
> Karl O. Pinc wrote:
>> Hi Tabby,
>> 
>> attached are 2 files.
>> 
>> foo.txt is the same error messages I just sent to the list,
>> plus additional error messages I got when I moved
>> "multiple observers" into the comments.  (I generate
>> an error message when there's already a comment
>> and the observers are added to the end of the comment.)
>> 
>> I don't know if you care about these new
>> "errors" or not.
>> 
> I'll look.  I don't know if I care.  I hope I don't.
> 
>> The second file is the result of all the magic
>> manipulations.
>> 
>> Those extra quotes that excel sometimes puts around
>> the data are making the comments look ugly.  (Excel
>> didn't do that once upon a time but Microsoft keeps
>> making changes to make it harder to get data
>> out of Microsoft products.)  It might be time to
>> figure out what's going on and if there's some
>> way to keep excel from adding quotes.  (My guess is
>> no.)  I'm thinking that some of your demography
>> notes or some other uploaded data may also
>> be getting extra quotes added that we don't want.
>> In any case we should figure out what's going on
>> so we can do something if we need to.
>> 
> I know for sure there are quotes in the demog notes.  At times, I  
> think I've bothered to delete them manually, but I'm pretty sure I  
> gave up when I had to put in 1200 demog notes from incomplete census  
> data one update.  I don't know where they come from or how to get rid  
> of them.  Any suggestions?


It's Excel putting the extra quotes in when you tell it
to export as text, so one way to attack the problem is
to get help with Excel and keep it from doing that.
There might possibly be some way to get it to revert
to it's old behavior.  There must be some Excel support
on-campus, or some expert forum on the net or something.
I've no doubt that _something_ can be done to solve
the problem on the Microsoft side.  If nothing else
I'm sure that purchasing Microsoft Visual Basic and
programming some extension or something would do it.
But that sort of approach sounds both excessive and
fragile.

Maybe Lacey has a clue?

Or, stop using Excel.  Lordy, you could even switch
to notepad.  For a spreadsheet option, OpenOffice calc works with
Excel files and may have a more sane export.  (Or
not, as they need to stay "compatible" with Microsoft.)
(You'd set to load/save in Microsoft format by default
using the dialog shown here:
http://wiki.services.openoffice.org/wiki/Documentation/OOo3_User_Guides/Getting_Started/Choosing_options_for_loading_and_saving_documents
)

Alternately, I should modify the upload program so
that it can check for this Microsoft-ism and
remove the problem.  We need to do something to have
control over the data.  Modifying the upload
program mostly solves the problem, but does not
help in other areas like the quick data fixes
I'm doing now so it'd be nice to tackle the problem
closer to it's source, if possible.



> 
>> If you _do_ get the quotes out of the weather data
>> I need to know when you send another
>> file because the program that does
>> all the messing about is assuming that when
>> there's multiple observers there's quotes.
>> 
>> Note that the "messing about" that produces
>> the observers_moved.txt file is a one-time
>> program and does not have anything to do with
>> the regular upload process.  The regular upload
>> program should take as input a file with all
>> the insanity removed.
>> 
> New files won't have that insanity.  I'll just stamp it with my own  
> brand of yet-unknown insanity... to be discovered by some poor sucker  
> 10 years from now.

Karl <kop at meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein



More information about the Babase mailing list