Hi devs, Some of you may know, we got an issue with our Koha gitlab repo, it was not syncing because of too big (no detail here, but that did not really make sense anyway). It seems that it is syncing now but we removed all the branches that are not protected (so only master) I had a new try at splitting our repo, removing the po files from the core repo (https://wiki.koha-community.org/wiki/Git_Splitting_and_Shrinking). It's not a trivial process as it requires several changes, but I noticed that: After a git clone: `git count-objects -vH` returns 3.67/4.23 GiB (cannot remember the correct number, but high!) After a repack [1]: ~755 MiB The command I used: `git gc --aggressive --prune=now` I think we should schedule that command on the git server (it took around 3 hours on a small VPS) to reduce its size. I am available if you need any helps for that task. Cheers, Jonathan PS: you may need to increase the windowMemory value with: `git config --global pack.windowMemory 256m` [1] https://git-scm.com/docs/git-repack Spoiler: After removing the po file, and a repack, the repo index size is 127MiB
Ok, what time of day in UTC do you like for this? On Tue, Apr 28, 2020 at 3:19 AM Jonathan Druart < jonathan.druart@bugs.koha-community.org> wrote:
Hi devs,
Some of you may know, we got an issue with our Koha gitlab repo, it was not syncing because of too big (no detail here, but that did not really make sense anyway). It seems that it is syncing now but we removed all the branches that are not protected (so only master)
I had a new try at splitting our repo, removing the po files from the core repo ( https://wiki.koha-community.org/wiki/Git_Splitting_and_Shrinking). It's not a trivial process as it requires several changes, but I noticed that:
After a git clone: `git count-objects -vH` returns 3.67/4.23 GiB (cannot remember the correct number, but high!) After a repack [1]: ~755 MiB
The command I used: `git gc --aggressive --prune=now`
I think we should schedule that command on the git server (it took around 3 hours on a small VPS) to reduce its size.
I am available if you need any helps for that task.
Cheers, Jonathan
PS: you may need to increase the windowMemory value with: `git config --global pack.windowMemory 256m`
[1] https://git-scm.com/docs/git-repack
Spoiler: After removing the po file, and a repack, the repo index size is 127MiB _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
My feeling would be to: 0. stop pushing to the repo, so wait for the RM and RMaints to confirm they are aware of the task 1. copy the original repo directory (backup, just in case) 2.`git gc --aggressive --prune=now` I personally don't have preference about the time. Le mar. 28 avr. 2020 à 17:28, Liz Rea <wizzyrea@gmail.com> a écrit :
Ok, what time of day in UTC do you like for this?
On Tue, Apr 28, 2020 at 3:19 AM Jonathan Druart <jonathan.druart@bugs.koha-community.org> wrote:
Hi devs,
Some of you may know, we got an issue with our Koha gitlab repo, it was not syncing because of too big (no detail here, but that did not really make sense anyway). It seems that it is syncing now but we removed all the branches that are not protected (so only master)
I had a new try at splitting our repo, removing the po files from the core repo (https://wiki.koha-community.org/wiki/Git_Splitting_and_Shrinking). It's not a trivial process as it requires several changes, but I noticed that:
After a git clone: `git count-objects -vH` returns 3.67/4.23 GiB (cannot remember the correct number, but high!) After a repack [1]: ~755 MiB
The command I used: `git gc --aggressive --prune=now`
I think we should schedule that command on the git server (it took around 3 hours on a small VPS) to reduce its size.
I am available if you need any helps for that task.
Cheers, Jonathan
PS: you may need to increase the windowMemory value with: `git config --global pack.windowMemory 256m`
[1] https://git-scm.com/docs/git-repack
Spoiler: After removing the po file, and a repack, the repo index size is 127MiB _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Greetings, I checked out the wiki page. Thank you for typing that up, Jonathan. I vote for keeping tags (hey GB to MB -- we can keep tags), and I really like: https://wiki.koha-community.org/wiki/Git_Splitting_and_Shrinking#Howto_use_K... It's still pretty clean in terms of getting all of Koha. GPML, Mark Tompsett
The wiki page is outdated, I will try and update it soon. I managed to get rid of po files and l kept the history and tags (using BFG instead of git filter-branch). Le mar. 28 avr. 2020 à 20:02, Mark Tompsett <mtompset@hotmail.com> a écrit :
Greetings,
I checked out the wiki page. Thank you for typing that up, Jonathan.
I vote for keeping tags (hey GB to MB -- we can keep tags), and I really like: https://wiki.koha-community.org/wiki/Git_Splitting_and_Shrinking#Howto_use_K...
It's still pretty clean in terms of getting all of Koha.
GPML, Mark Tompsett _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
participants (3)
-
Jonathan Druart -
Liz Rea -
Mark Tompsett