[sugar] Database Activity

[Simos Xenitellis]

The current Python installations use ascii as the default encoding.
The new Python (Python 3?) will switch the default to UTF-8.
What to do to test if the problem is this?

Go to /etc/python2.5/site.py
and find the function setencoding(). Change the value from "ascii" to 
"utf-8".
Try again your test program and report back if it was fixed.

The issue with having the default encoding as ascii is that you have to 
know to convert all text input to utf-8 by hand; a buggy situation.

Simos Xenitellis
http://blogs.gnome.org/simos/

7150 wrote:
> This will be my last post on this topic until I learn a bit more.
>
> It appears that pysqlite has some data integrity issues.
>
> Interesting link to pysqlite "bug" discussion: 
> http://osdir.com/ml/python.db.pysqlite.user/2006-04/msg00020.html
>
> ---
>
> "I have built sqlite databases containing utf-8 encoded text.  When I
> access them using pysqlite, utf-8 codepoints appear to be converted to
> latin-1."
>
> It does this.
>
> How to stop it?
>
> An answer on the list was 
> (http://osdir.com/ml/python.db.pysqlite.user/2006-04/msg00022.html):
>
>  > This should be:
>  >
>  >      req.write(elem.encode('utf-8'))
>  >
>  > write() expects a str object, so what happened
>  > in your code was an implicit conversion of your
>  > unicode object to a str, doing the encoding using
>  > the default system encoding, here 'ascii'.
>
> The req.write stuff is Greek to me, but I'll see what I can find out 
> about it.
>
> Then from the same post:
>
> "So, upon reflection, my situation boils down to this:
>
> (1) A utf-8 encoded string in an sqlite database is queried using
> pysqlite which returns a latin-1 string (a
> gratuitous/silent/unspecified/unrequested conversion).
>
> . . .
>
> pysqlite QUESTIONS:
>
> Question 1:  Why does pysqlite convert the utf-8 sting to latin-1 in the
> *query process* when my sqlite settings are for utf-8?
> i.e., sqlite.h macro: #define SQLITE_UTF8 1
>
> Question 2:  Does the pysqlite user (programmer) have any control over
> this query/conversion?  (I see nothing in the Python Database API, or
> the pysqlite/sqlite documentation, about either automatic or specified
> conversions.  Also, this conversion does not occur using the sqlite3
> client.)
>
> Question 3:  Is this a pysqlite bug or a feature, and why?"
>
> The group had no further answer for the guy.
>
> ---
>
> Someone else has noted thet the SQLite3 client does not transform the 
> data.  If I learn how to query my UTF-8 database on the XO, I'll let you 
> know. But, it's not an XO problem.
>
> ---
>
> http://www.litenverden.org
> _______________________________________________
> Sugar mailing list
> Sugar at lists.laptop.org
> http://lists.laptop.org/listinfo/sugar
>