DjangoBot: #django-dev

Speaking about database backends ... I just posted a proposal to django-dev that the PostgreSQL backends stop paying attention to time zones, so we can properly support time zones. (Sounds counterintuitive, I know.) Thoughts from anyone?

korpios: you mean passing it datetimes with tzinfo fields, and avoid calling "SET TIMEZONE"?

lsoto: The idea is that the time zone shouldn't be handled by Postgres (or any backend), since it only supports fixed offsets. A new Field (already written and working in our code) could support it by taking an argument of either a tzinfo instance, or a lookup to a string (e.g., "America/Chicago"), and always yield timezone-aware datetime instances

also, postgres's timezone handling is pretty damned broken, as it blows up the moment you change settings.TIME_ZONE or database backends

if we just use a plain TIMESTAMP (aka TIMESTAMP WITHOUT TIME ZONE) and stop using SET TIME ZONE on connections, the mentioned Field could handle everything cleanly

korpios: but, if you give psycopg a datetime instance with a tzinfo field, isn't it saved properly (as a timestamp with timezone)?

lsoto: it saves a fixed offset, not the actual zone

oh, crap

so you're left wondering, e.g., am I talking about Arizona, or DST Central (in America)?

jacobkm, lsoto: re ctypes on jython, yes it's very possible. JNA is modeled on ctypes, but apparently PyPy has a portable solution they developed (to work with their java backend, other backends) that we plan to look at. so that will be one of the things that we will be looking at the EuroPython sprints

I have a DateTimeZoneField that you can pass in, e.g., time_zone='city__time_zone', and it'll look that relation, and use that string to set its own tzinfo

or you can just have the relation on the same model if the zone will always be changing (e.g., tracking a road trip)

my experience (drawn from oracle) is that it's better not to use the TIMESTAMP WITH TIMEZONE; at least at some point, cx_oracle didn't properly support it; and worse, such columns could not participate in indexes

jbaker_: right ... if we don't use WITH TIME ZONE we can let a timezone-aware Field assume that the backend data is always in UTC

right, better to normalize the timestamps to UTC, and manage for the user in some interesting way

that only can be done at the app level anyway

I've got the "interesting way" down ... I just need the time zone references removed from postgres to make it sane (as it is, I need to throw an error if TIME_ZONE isn't UTC and the backend is postgres)

that's definitely the right approach. similar to normalizing your unicode encoding to UTF-8 in the database - or comparable. (can't remember if sql server wants to do utf-16 or utf-8)

jbaker_: cool, so I will implement such UTC normalization on the postgresql_zxjdbc backend

sounds like a plan

jbaker_: "Generally, SQL Server stores Unicode in the UCS-2 encoding scheme"

this is getting a bit off-topic, but i think .Net has done the same ucs-2 -> utf-16 switch seen in java. for some reason, i developed some background in this ;), see jython. so it's likely that sqlserver at some version is doing the same, especially since the aspect of surrogates can be often be ignored by db level code

jbaker_: well this is where I got that: http://msdn.microsoft.com/en-us/library/bb33096...

I'll say no more on the sub

empty: i wholeheartedly concur, just concluding that they at least support utf-16 where it's perhaps most important, in their xml support; and let you access CLR functions when you do care about surrogates

yup

hey guys, what do you think of #6398?

jezdez: OK idea, but as a new feature it'll probably have to be post-1.0

not really :P

marking it post-1.0

looks like #7190 is breaking the test suite under mysql

(I also get some failures related to collation: "Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '='")

can someone confirm this?

lsoto: I think that's because BooleanField doesn't have the SubFieldBase metaclass which means that to_python isn't called

Because that should ensure the correct coersion

I might be wrong though

wow, I was always asuming that to_python was being called on django builtin field types

thanks for the tio

*tip

anyway, the current to_python doesn't consider the values 0 and 1

Does adding the __metaclass__ and 0, 1 to the choices solve the problem?

(Maybe try just adding 0 and 1 first, in class to_python is being called)

after reading the comments of #7190, I don't think so

seems like it must be fixed on the mysql backend

"Note: the approach in the mysql.patch attachment handles some cases that the approach in the boolfield.diff attachment does not: for example, the former fixes results returned by calling Model.objects.value()"

IOW, __metaclass__ and (0, 1) won't fix the test failure

jacobkm: I've confirmed that the serializer_regress test follows a code path which end with field values containing strings for TimeField

hm

so I think that such string support is there for serializers too

anyway, I won't break that, I think Honza is in a better position do whatever cleanup is needed as part of his model validation

brosner: wha?

one second while i formulate this

brosner: this better not be some sort of intervention.

I swear I only used Perl that one time.

haha

And she told me she was Ruby.

http://dpaste.com/59125/ won't work in newforms-admin. it works fine with a standard outside the admin form. i have tracked it down to RelatedFieldWidgetWrapper and the only thing that i *think* it might be is its __deepcopy__ method. firstly i dont understand what memo is?

RelatedFieldWidgetWrapper adds the nice add ability to fk in the admin fyi

it is at that point apparently storing a cached version of the queryset which i am not entirely sure how

brosner: fuck if I know, I leave double-underscore methods to jkocherhans.

:)

jacobkm: Are all the unit tests and everything done?

Alex_Gaynor: for #2070, you mean?

Yeah

Alex_Gaynor: as much as I can, yeah... there's still a few code paths that are hard to test -- mostly because the test client actually gets in the way by helping too much.

brosner: memo is passed around to track the objects that have already been copied IIRC

jacobkm: Ok, I got all confused when you removed the TODO, but there was no code up there :D

Alex_Gaynor: ha, yeah; I removed it because it's somewhere between hard and impossible.

The actual UploadHandler / UploadedFile classes get a good workout from the tests, but testing the parser is really difficult.

Though actually the one thing that *should* still be done is a few test cases involving custom upload handlers and adding/remving them on the fly.

jacobkm: is my git diff screwed, or does you 2070 branch add a blank line in django/core/files/__init__.py ¿

s/you/your/

telenieko: yeah, silly, there's indeed a blank line there. I just killed it.

I love the shit out of TextMate, but it has a habit of adding whitespace where I didn't expect it.

jacobkm: on pulling, the way my published master is done now is fine or should I find a better way to make it easier to spot what I've done there? (git cherry -v does not seem very helpfull from this side)

jacobkm: Use vim :P (now somebody will say "use emacs")

telenieko: it's pretty clear -- I'm using format-patch instead since I find it easier to review that way.

telenieko: use vim inside of emacs.

does it get which comes from me and what from merges of trunk?

telenieko: personally, I prefer single topic branches (all mine are named NNNN-some-label where NNNN is the Trac ticket number) but however you want to organize your repos is fine with me.

jacobkm: I have topic branches, but used git merge --squash, I'll publish then from now on ;)

ubernostrum: nice to know :))

I'm still not *really* using git all that much, so don't expect me to do much from your repos really... I still find dealing with patches *far* easier.

telenieko: have you seen git-trac?

jacobkm: yes, and I ended up setting up redmine, but I needed the multiple projects things

gotcha