Re: [Koha-devel] Invalid local time for date in time zone: America/Santiago
Hey Barton, Yes, the year 2038 for timestamps is a problem even on 64 bit machines. Here’s a test you can do on a 64 bit MySQL install on a 64 bit OS: select unix_timestamp('2038-01-20'); select from_unixtime(2548990800); Both will return null. You can also try storing a value of ‘2039-01-01’ into a timestamp column, and you’ll get the following message : Out of range value for column 'timestamp' at row 1 Here’s some good explanations: <http://askubuntu.com/questions/675116/timestamp-year-2038-problem-for-64-bit-ubuntu-system> http://askubuntu.com/questions/675116/timestamp-year-2038-problem-for-64-bit... <https://bugs.mysql.com/bug.php?id=12654> https://bugs.mysql.com/bug.php?id=12654 Also, I’d never heard of date_t before your email, and the only references I can see to it are for specific projects which have implemented it as a new data structure themselves. I think time_t is the standard and that depends on your system architecture, although as you can see in those above links – it’s more complicated than that in MySQL. -- issues.date_due is already a DATETIME data type, so we don’t need to worry about 2038. As for time zones, why would it be tricky storing it in UTC? At the moment, you can only have due dates in one time zone. That means you can’t have library systems spanning more than one timezone. Maybe that’s a rare or uncommon scenario, but here in the state of New South Wales, we actually do have 2 different timezones. In that case, when you’re doing a check out, you choose a local time, convert it to UTC for storage, and then render it in local time for the library user and library staff. I imagine the trickiest part would be the Reports module, as users wouldn’t know to use something like the CONVERT_TZ() function. Calculating overdues against holidays might be tricky… although that’s a case of just getting your timezone right as well. I reckon your timezone should be set per library, so you just fetch your UTC datetime, convert it to your local library timezone, probably convert it to floating to do math, and then convert it back to local timezone, and then whatever else you need. It sounds complex, but it’s really not that bad when you have it structured well. -- Barton, I’m not sure I understand your reaction :p. I also field support tickets for a living, and I think it would be a lot easier to field tickets for a re-architected system that is created off years of experience rather than years of technical debt. Of course, I know re-architecting a system is also a lot easier said than done. But I think that’s why you see folks like EBSCO thinking about creating a microservice based system which can integrate with Koha. Incremental change. I suppose dates are most important when it comes to circulation and patrons… and I think that’s what people probably think about when they think of core Koha. I suppose acquisitions and serials require a lot of careful date handling as well… I was actually thinking about Koha and “asset tracking” in general this morning on the way to the office. I don’t know. Maybe Koha is fine just being a 1 timezone system. I imagine library systems spanning over multiple timezones are probably the exception rather than the rule. We need to re-think some of our DateTime logic though. Whether we use ‘floating’ timezones more consistently, or are careful with creating local times that don’t exist… David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Office: 02 9212 0899 Direct: 02 8005 0595 From: Barton Chittenden [mailto:barton@bywatersolutions.com] Sent: Thursday, 5 May 2016 12:04 AM To: David Cook <dcook@prosentient.com.au> Subject: Re: [Koha-devel] Invalid local time for date in time zone: America/Santiago On Thu, Apr 28, 2016 at 7:27 PM, David Cook <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: I don’t think using “timestamp” would be preferable to “datetime”. First, MySQL’s timestamp seems to suffer from the Year 2038 problem ( <https://en.wikipedia.org/wiki/Year_2038_problem> https://en.wikipedia.org/wiki/Year_2038_problem)(http://bugs.mysql.com/bug.p...), while datetime does not. Even on 64 bit machines? Oh MySQL, MySQL, knowest thou not of date_t? Second, you’d be handling timezones at the data level rather than at the application level. I’d rather have a convention of storing DATETIME as UTC, having Koha fetch that DATETIME as UTC, and then doing timezone conversions from there. I think that could be handled *relatively* transparently in DateUtils.pm, but there are a lot of corner cases like due dates that get ... tricky. As for the same server running different time zones, you can change the timezone on a per connection setting I think, but that could be a hassle. I think it would be easier just to store things clearly as UTC. This is done on a different project I work on, and it works well. Oh, I think it’ll be a nightmare changing timezone/timestamp/datetime handling in Koha. I don’t even know if it’s possible, to be honest. Well, it would be possible, but it would likely be an error-prone process. It’s things like this which make me think that we need to start thinking about a version of Koha that doesn’t have a clear upgrade path. A Koha with some major architectural changes which will need a skilled hand to migrate to. It won’t be nice for a lot of people… but I don’t know how else we’re going to make some major changes without dramas.
From a design standpoint, I agree.
From the point of view of someone who fields support tickets for a living:
NOOOOOOOOOOOOOOOOOOOOO!!!! ... but that's just an emotional response. Cheers, --Barton
Greetings, Sorry if I took something out of context to create a different thread. David Cook wrote:
“Maybe Koha is fine just being a 1 timezone system.”
What I would prefer is a 1 timezone backend, with user specific timezone front-end with a default timezone. Why? Global consortia. The organization I work with has libraries across... 1,2,3, perhaps 4 time zones maybe 5. I could easily imagine it increasing in the future. GPML, Mark Tompsett P.S. Back to your regularly scheduled Y2038 doomsday discussion. From: David Cook<mailto:dcook@prosentient.com.au> Sent: Wednesday, May 04, 2016 8:23 PM To: 'Barton Chittenden'<mailto:barton@bywatersolutions.com> Cc: koha-devel@lists.koha-community.org<mailto:koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Invalid local time for date in time zone:America/Santiago Hey Barton, Yes, the year 2038 for timestamps is a problem even on 64 bit machines. Here’s a test you can do on a 64 bit MySQL install on a 64 bit OS: select unix_timestamp('2038-01-20'); select from_unixtime(2548990800); Both will return null. You can also try storing a value of ‘2039-01-01’ into a timestamp column, and you’ll get the following message : Out of range value for column 'timestamp' at row 1 Here’s some good explanations: http://askubuntu.com/questions/675116/timestamp-year-2038-problem-for-64-bit... https://bugs.mysql.com/bug.php?id=12654 Also, I’d never heard of date_t before your email, and the only references I can see to it are for specific projects which have implemented it as a new data structure themselves. I think time_t is the standard and that depends on your system architecture, although as you can see in those above links – it’s more complicated than that in MySQL. -- issues.date_due is already a DATETIME data type, so we don’t need to worry about 2038. As for time zones, why would it be tricky storing it in UTC? At the moment, you can only have due dates in one time zone. That means you can’t have library systems spanning more than one timezone. Maybe that’s a rare or uncommon scenario, but here in the state of New South Wales, we actually do have 2 different timezones. In that case, when you’re doing a check out, you choose a local time, convert it to UTC for storage, and then render it in local time for the library user and library staff. I imagine the trickiest part would be the Reports module, as users wouldn’t know to use something like the CONVERT_TZ() function. Calculating overdues against holidays might be tricky… although that’s a case of just getting your timezone right as well. I reckon your timezone should be set per library, so you just fetch your UTC datetime, convert it to your local library timezone, probably convert it to floating to do math, and then convert it back to local timezone, and then whatever else you need. It sounds complex, but it’s really not that bad when you have it structured well. -- Barton, I’m not sure I understand your reaction :p. I also field support tickets for a living, and I think it would be a lot easier to field tickets for a re-architected system that is created off years of experience rather than years of technical debt. Of course, I know re-architecting a system is also a lot easier said than done. But I think that’s why you see folks like EBSCO thinking about creating a microservice based system which can integrate with Koha. Incremental change. I suppose dates are most important when it comes to circulation and patrons… and I think that’s what people probably think about when they think of core Koha. I suppose acquisitions and serials require a lot of careful date handling as well… I was actually thinking about Koha and “asset tracking” in general this morning on the way to the office. I don’t know. Maybe Koha is fine just being a 1 timezone system. I imagine library systems spanning over multiple timezones are probably the exception rather than the rule. We need to re-think some of our DateTime logic though. Whether we use ‘floating’ timezones more consistently, or are careful with creating local times that don’t exist… David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Office: 02 9212 0899 Direct: 02 8005 0595 From: Barton Chittenden [mailto:barton@bywatersolutions.com] Sent: Thursday, 5 May 2016 12:04 AM To: David Cook <dcook@prosentient.com.au> Subject: Re: [Koha-devel] Invalid local time for date in time zone: America/Santiago On Thu, Apr 28, 2016 at 7:27 PM, David Cook <dcook@prosentient.com.au<mailto:dcook@prosentient.com.au>> wrote: I don’t think using “timestamp” would be preferable to “datetime”. First, MySQL’s timestamp seems to suffer from the Year 2038 problem (https://en.wikipedia.org/wiki/Year_2038_problem)(http://bugs.mysql.com/bug.p...), while datetime does not. Even on 64 bit machines? Oh MySQL, MySQL, knowest thou not of date_t? Second, you’d be handling timezones at the data level rather than at the application level. I’d rather have a convention of storing DATETIME as UTC, having Koha fetch that DATETIME as UTC, and then doing timezone conversions from there. I think that could be handled *relatively* transparently in DateUtils.pm, but there are a lot of corner cases like due dates that get ... tricky. As for the same server running different time zones, you can change the timezone on a per connection setting I think, but that could be a hassle. I think it would be easier just to store things clearly as UTC. This is done on a different project I work on, and it works well. Oh, I think it’ll be a nightmare changing timezone/timestamp/datetime handling in Koha. I don’t even know if it’s possible, to be honest. Well, it would be possible, but it would likely be an error-prone process. It’s things like this which make me think that we need to start thinking about a version of Koha that doesn’t have a clear upgrade path. A Koha with some major architectural changes which will need a skilled hand to migrate to. It won’t be nice for a lot of people… but I don’t know how else we’re going to make some major changes without dramas. From a design standpoint, I agree. From the point of view of someone who fields support tickets for a living: NOOOOOOOOOOOOOOOOOOOOO!!!! ... but that's just an emotional response. Cheers, --Barton ________________________________ _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
On Wed, May 4, 2016 at 11:15 PM, Mark Tompsett <mtompset@hotmail.com> wrote:
Greetings,
Sorry if I took something out of context to create a different thread.
I think that I may have highjacked the original thread. Apologies to David Cook and Larry Letelier, and thank you for creating a thread specific to this conversation.
David Cook wrote:
“Maybe Koha is fine just being a 1 timezone system.”
What I would prefer is a 1 timezone backend, with user specific timezone front-end with a default timezone. Why? Global consortia. The organization I work with has libraries across... 1,2,3, perhaps 4 time zones maybe 5. I could easily imagine it increasing in the future.
Instances with branches that span time zones is certainly not the most common case, but it's an exception that's likely to bite someone in the hind quarters. "Do you have branches in separate time zones" is not a question that's likely to be asked in a pre-migration interview, and could very possibly be a show-stopper. I could easily see this causing a lot of wasted time, money and effort. Koha is supposed to be a general purpose ILS, and I don't think that limiting it to a
GPML, Mark Tompsett
P.S. Back to your regularly scheduled Y2038 doomsday discussion.
*From:* David Cook <dcook@prosentient.com.au> *Sent:* Wednesday, May 04, 2016 8:23 PM *To:* 'Barton Chittenden' <barton@bywatersolutions.com> *Cc:* koha-devel@lists.koha-community.org *Subject:* Re: [Koha-devel] Invalid local time for date in time zone:America/Santiago
Hey Barton,
Yes, the year 2038 for timestamps is a problem even on 64 bit machines.
Here’s a test you can do on a 64 bit MySQL install on a 64 bit OS:
select unix_timestamp('2038-01-20');
select from_unixtime(2548990800);
Both will return null.
You can also try storing a value of ‘2039-01-01’ into a timestamp column, and you’ll get the following message :
Out of range value for column 'timestamp' at row 1
Here’s some good explanations:
http://askubuntu.com/questions/675116/timestamp-year-2038-problem-for-64-bit...
https://bugs.mysql.com/bug.php?id=12654
Also, I’d never heard of date_t before your email, and the only references I can see to it are for specific projects which have implemented it as a new data structure themselves. I think time_t is the standard and that depends on your system architecture, although as you can see in those above links – it’s more complicated than that in MySQL.
--
issues.date_due is already a DATETIME data type, so we don’t need to worry about 2038. As for time zones, why would it be tricky storing it in UTC?
At the moment, you can only have due dates in one time zone. That means you can’t have library systems spanning more than one timezone. Maybe that’s a rare or uncommon scenario, but here in the state of New South Wales, we actually do have 2 different timezones. In that case, when you’re doing a check out, you choose a local time, convert it to UTC for storage, and then render it in local time for the library user and library staff.
I imagine the trickiest part would be the Reports module, as users wouldn’t know to use something like the CONVERT_TZ() function.
Calculating overdues against holidays might be tricky… although that’s a case of just getting your timezone right as well. I reckon your timezone should be set per library, so you just fetch your UTC datetime, convert it to your local library timezone, probably convert it to floating to do math, and then convert it back to local timezone, and then whatever else you need. It sounds complex, but it’s really not that bad when you have it structured well.
--
Barton, I’m not sure I understand your reaction :p. I also field support tickets for a living, and I think it would be a lot easier to field tickets for a re-architected system that is created off years of experience rather than years of technical debt. Of course, I know re-architecting a system is also a lot easier said than done. But I think that’s why you see folks like EBSCO thinking about creating a microservice based system which can integrate with Koha. Incremental change.
I suppose dates are most important when it comes to circulation and patrons… and I think that’s what people probably think about when they think of core Koha. I suppose acquisitions and serials require a lot of careful date handling as well… I was actually thinking about Koha and “asset tracking” in general this morning on the way to the office.
I don’t know. Maybe Koha is fine just being a 1 timezone system. I imagine library systems spanning over multiple timezones are probably the exception rather than the rule.
We need to re-think some of our DateTime logic though. Whether we use ‘floating’ timezones more consistently, or are careful with creating local times that don’t exist…
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Office: 02 9212 0899
Direct: 02 8005 0595
*From:* Barton Chittenden [mailto:barton@bywatersolutions.com] *Sent:* Thursday, 5 May 2016 12:04 AM *To:* David Cook <dcook@prosentient.com.au> *Subject:* Re: [Koha-devel] Invalid local time for date in time zone: America/Santiago
On Thu, Apr 28, 2016 at 7:27 PM, David Cook <dcook@prosentient.com.au> wrote:
I don’t think using “timestamp” would be preferable to “datetime”.
First, MySQL’s timestamp seems to suffer from the Year 2038 problem ( https://en.wikipedia.org/wiki/Year_2038_problem)( http://bugs.mysql.com/bug.php?id=12654), while datetime does not.
Even on 64 bit machines?
Oh MySQL, MySQL, knowest thou not of date_t?
Second, you’d be handling timezones at the data level rather than at the application level. I’d rather have a convention of storing DATETIME as UTC, having Koha fetch that DATETIME as UTC, and then doing timezone conversions from there.
I think that could be handled *relatively* transparently in DateUtils.pm, but there are a lot of corner cases like due dates that get ... tricky.
As for the same server running different time zones, you can change the timezone on a per connection setting I think, but that could be a hassle. I think it would be easier just to store things clearly as UTC. This is done on a different project I work on, and it works well.
Oh, I think it’ll be a nightmare changing timezone/timestamp/datetime handling in Koha. I don’t even know if it’s possible, to be honest. Well, it would be possible, but it would likely be an error-prone process.
It’s things like this which make me think that we need to start thinking about a version of Koha that doesn’t have a clear upgrade path. A Koha with some major architectural changes which will need a skilled hand to migrate to. It won’t be nice for a lot of people… but I don’t know how else we’re going to make some major changes without dramas.
From a design standpoint, I agree.
From the point of view of someone who fields support tickets for a living:
NOOOOOOOOOOOOOOOOOOOOO!!!!
... but that's just an emotional response.
Cheers,
--Barton
------------------------------ _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
On Wed, May 4, 2016 at 8:23 PM, David Cook <dcook@prosentient.com.au> wrote:
Hey Barton,
...
Also, I’d never heard of date_t before your email, and the only references I can see to it are for specific projects which have implemented it as a new data structure themselves. I think time_t is the standard and that depends on your system architecture, although as you can see in those above links – it’s more complicated than that in MySQL.
First, let me say that I was in a slap-dash mood this morning -- technical accuracy was playing second fiddle to whatever stands in for a sense of humor in my head. In that context: ... Yeah. I meant time_t. The point being that time_t should, by default, compile to a 64 bit int on a 64 bit machine, and therefore not cause 2038 bugs. Given that MySQL *does* have a 2038 bug in its implementation of timestamp, I guess that using datetime with the timezone set to UCT makes sense ... or, if we *really* want to make a dramatic move, we could move away from MySQL --
I’m not sure I understand your reaction :p. I also field support tickets for a living, and I think it would be a lot easier to field tickets for a re-architected system that is created off years of experience rather than years of technical debt. Of course, I know re-architecting a system is also a lot easier said than done. But I think that’s why you see folks like EBSCO thinking about creating a microservice based system which can integrate with Koha. Incremental change.
Once again, I wasn't being entirely serious. I'm very much in favor of working to make Koha internally consistent, removing un-intentional limitations such as global time zone settings, and generally removing technical debt, but having said that, I'm a bit ambivalent about it, simply because moving too aggressively in that direction does create a lot of change in a short period of time, and that's rough on the support folks. You mentioned creating a version of Koha that would break the current schema, requiring something more akin to a data migration than an upgrade... that suggests a large change and a lot of additional work... channeling Darth Vader's "Noooooooooooo!" was just my over-dramatic way of saying that. ... there was a ';-)' missing at the end of that statement. I've got other thoughts, but I think they're better suited to Mark Tompsett's "1 timezone system?" thread. To be taken with a large grain of salt, --Barton
participants (3)
-
Barton Chittenden -
David Cook -
Mark Tompsett