Externalizing PO files (Was Re: externalising translator and etc)
Hi, [Splitting the thread, there are two distinct topics being discussed.] On Mon, Nov 15, 2010 at 4:16 AM, LAURENT Henri-Damien <laurenthdl@alinto.com> wrote:
Le 14/11/2010 22:51, Galen Charlton a écrit :
From my understanding of Scott Chacon (a git developer) book and git talk, git stores the files in its database, since it is a DAG oriented SCM. It may send the diffs, but actualy stores the files.
A cursory reading of a Git book reveals that a statement that Git "actually stores the files" is simplistic, at particular as far as the implications for storage are concerned. Please take a look at: http://book.git-scm.com/7_how_git_stores_objects.html Git's object storage takes the form of compressed version of each object (commit, file, tag, etc.) a files and (and this is the important part) packfiles. Packfiles, roughly speaking, encode only the changes to objects, so a significant amount of space is saved. If one disabled automatic garbage collection in a git repo and never ran git gc, you might end up with something close to storing the (compressed) version of each changed file, but in practice that doesn't happen.
I don't think all the release tags would be broken.
As it turns out, they would. I have actually tried running a git filter-branch to remove the PO directory. As I suspected, it does rewrite all of the commits (thereby changing the commit IDs), and it would be necessary to rewrite the tags. But doing so would invalidate the release tags that have a GPG signature. Rewriting history like this is a sure way to get months of complaints and calls for help on koha-devel with questions like "why does git rebase origin no longer work".
And it would allow to release localisation at a different pace... When there is a need.
There's is nothing stopping releasing localizations on a different schedule now. Packaging does not have any necessary connection with how the Git repository is laid out. I will grant that all other things being equal, it would have been a valid choice to have the PO files live in a separate repository. In fact, we perhaps can still do so. One thing that Chris Cormack mentioned to me yesterday is that Pootle has a Git plugin which can automatically commit changes to PO files to a repository. That would be a bit dangerous to allow for the main Koha repository, but safe enough for a separate translations file. Consequently, there is a possible compromise: [1] We seed a new repository for PO files. [2] We do a simple git rm misc/translator/po but *don't* rewrite history. This won't reduce the size of the main repository, of course, but achieves the separation. And, to be honest, it's mostly the possibility of integration with Pootle that would make me interested in this. However, since it is the translation manager who would be primarily affected by the split on a day-to-day basis, I would be interested in hearing from Frédéric as well as past TMs. Regards, Galen -- Galen Charlton gmcharlt@gmail.com
Le 15/11/2010 13:18, Galen Charlton a écrit :
Hi,
[Splitting the thread, there are two distinct topics being discussed.]
On Mon, Nov 15, 2010 at 4:16 AM, LAURENT Henri-Damien <laurenthdl@alinto.com> wrote:
Le 14/11/2010 22:51, Galen Charlton a écrit :
From my understanding of Scott Chacon (a git developer) book and git talk, git stores the files in its database, since it is a DAG oriented SCM. It may send the diffs, but actualy stores the files.
A cursory reading of a Git book reveals that a statement that Git "actually stores the files" is simplistic, at particular as far as the implications for storage are concerned. Please take a look at:
http://book.git-scm.com/7_how_git_stores_objects.html
Git's object storage takes the form of compressed version of each object (commit, file, tag, etc.) a files and (and this is the important part) packfiles. Packfiles, roughly speaking, encode only the changes to objects, so a significant amount of space is saved.
If one disabled automatic garbage collection in a git repo and never ran git gc, you might end up with something close to storing the (compressed) version of each changed file, but in practice that doesn't happen.
I don't think all the release tags would be broken.
As it turns out, they would. I have actually tried running a git filter-branch to remove the PO directory. As I suspected, it does rewrite all of the commits (thereby changing the commit IDs), and it would be necessary to rewrite the tags. But doing so would invalidate the release tags that have a GPG signature. OK. I knew it would rewrite history. But I thought it would be more efficient and cleaner though. That said, my proposal was only a proposal.
Rewriting history like this is a sure way to get months of complaints and calls for help on koha-devel with questions like "why does git rebase origin no longer work".
If you think so.
And it would allow to release localisation at a different pace... When there is a need.
There's is nothing stopping releasing localizations on a different schedule now. Packaging does not have any necessary connection with how the Git repository is laid out.
I will grant that all other things being equal, it would have been a valid choice to have the PO files live in a separate repository. In fact, we perhaps can still do so. One thing that Chris Cormack mentioned to me yesterday is that Pootle has a Git plugin which can automatically commit changes to PO files to a repository. That would be a bit dangerous to allow for the main Koha repository, but safe enough for a separate translations file.
Consequently, there is a possible compromise:
[1] We seed a new repository for PO files. [2] We do a simple git rm misc/translator/po but *don't* rewrite history.
Nice for me. This would break the commit history of previous po files. But who cares ?
This won't reduce the size of the main repository, of course, but achieves the separation. And, to be honest, it's mostly the possibility of integration with Pootle that would make me interested in this.
However, since it is the translation manager who would be primarily affected by the split on a day-to-day basis, I would be interested in hearing from Frédéric as well as past TMs.
Regards,
Galen
Thanks for your hints and thoughts. Regards. -- Henri-Damien LAURENT
Hi, On Mon, Nov 15, 2010 at 7:48 AM, LAURENT Henri-Damien <henridamien.laurent@gmail.com> wrote:
[1] We seed a new repository for PO files. [2] We do a simple git rm misc/translator/po but *don't* rewrite history. Nice for me. This would break the commit history of previous po files. But who cares ?
That doesn't follow. The PO file repository could be seeded from the main repository as you had originally proposed, keeping the history intact in the new repository. And, of course, doing a git rm of that directory in the main repository will still keep the history intact there as well. Regards, Galen -- Galen Charlton gmcharlt@gmail.com
participants (2)
-
Galen Charlton -
LAURENT Henri-Damien