[Koha-bugs] [Bug 23073] wiki.koha-community.org needs updating to a later version

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073

--- Comment #29 from Thomas Dukleth <td-koha-bugs at agogme.com> ---
Preserving version control changes after changing to the convention for hyphens
or dashes instead of underscores in program directories and filenames has
required replacing the Git archive with filenames in the history adjusted for
dashes.  Otherwise, moving files loses version control.

Code and configuration files to test migrating the MediaWiki database and
upgrading the wiki are currently available at the same location which was
originally created with hyphens but previously contained directories and
filenames with underscores,
git://test01.agogme.com/koha-migrate-mwiki-db-and-upgrade-test.git .  Files
excluded as inappropriate for version control are currently available with
hyphens having replaced underscores at
https://test01.agogme.com/koha-migrate-mwiki-db-and-upgrade-test.tgz .

Anyone interested in testing the code would need to separately obtain a full
dump of the Koha Wiki to avoid publicly compromising Koha Wiki usernames and
passwords which are not contained in the published archive.  The issue of
resetting wgSecretKey in LocalSettings.php could be avoided by obtaining a copy
of the Koha Wiki LocalSettings.php .  I have prepared a special nonpublic tar
archive for Koha developers interested in testing. 

Test instances were down for some extended periods in the past few days while I
was fixing bugs and regenerating the test instances.  Please note as before
that newly reinstalled MediaWiki instances take time for indexing etc. to catch
up over multiple days and indexing requires actual use to weight the result set
properly.
.  They may frequently go down or become slow when: regenerated for code
testing; or excess RAM or CPU use running some memory intensive processes
unrelated to Koha on the VPS.  Non-Koha use of the VPS starting too many
processes still occasionally leads the OOM killer to stop mysqld for which I
should add a monitoring and restart service for when I forget to check after
such excess RAM usage.

Reinstalled test instance of the Koha Wiki using Postgres at
https://koha-mw-pg-test01.agogme.com/ .

Koha test wiki migrated to MySQL at https://koha-mw-my-test01.agogme.com/ .

Koha MySQL test wiki upgraded to MediaWiki 1.35 LTS at
https://koha-mw-my-test01-upgr.agogme.com/ .

Please note that some adjustment of the upgraded wiki, such as the superseded
language change template at the bottom of the home page, may be outside the
ordinary scope of an automated upgrade process and require manual correction. 
The job queue may be full for an extended period after reinstalling or
upgrading a test wiki instance because reindexing is necessary.  Ordering of
search query result set improves with actual use but appears to be ordered by
page creation initially after reindexing rather than page modification or other
potentially more useful relevancy ranking which develops over time as the wiki
instance is used.

>From the current changelog.

In README.txt
-------------

Changed use of underscores to hyphens for file and directory names
for compliance with common conventions.

Clarified language and instruction variously.

In koha_migrate_mwiki_db_and_upgrade.ini
---------------------------------------

Changed use of underscores to hyphens for file and directory names
for compliance with common conventions.

Corrected location of debugging output logs for
koha-mediawiki-pgsqltomysql.py .

Added operating system version configuration for improving installation.

In koha-migrate-mwiki-db-and-upgrade-startup.sh
-----------------------------------------------

Changed use of underscores to hyphens for file and directory names
for compliance with common conventions.

Decreased verbosity of bash startup script when calling without
arguments for which koha-migrate-mwiki-db-and-upgrade.pl responds with a
usage reminder leaving response from the bash startup script unnecessary.

In koha-migrate-mwiki-db-and-upgrade.pl
---------------------------------------

Changed use of underscores to hyphens for file and directory names
for compliance with common conventions.

Corrected for temporary file inadvertantly created when creating download
directory for upgraded MediaWiki installation archive.

Corrected for setting permissions of pgpass file too early for newly
created pgpass file which led to failure of pg_restore.

Added removeal of pgpass file after use by pg_restore.

Corrected typo for binary in mysql command option
--default-character-set .

Corrections for excessive copying of files from the production archive to
the MediaWiki upgrade test instance when scope of binary files and other
files not suitable for version control had been moved to a separate
archive from version control files.

Added additional section heading comments for demarcating more code
sections for better readability of code and logs.

Added operating system version configuration for improving installation.

Removed redundant code section previously included with the intention
of chaning the section for additional MediaWiki testing instances.

In koha-mediawiki-pgsqltomysql.py
---------------------------------

Changed use of underscores to hyphens for file and directory names
for compliance with common conventions.

Corrected location of debugging logs.

In koha-mwiki-instances-postinstall.sh
--------------------------------------

Changed use of underscores to hyphens for file and directory names
for compliance with common conventions.

Added calls to runJobs.php.

Clarified section heading comments demarcating code
sections for better readability of code and logs.

In koha-migrate-mwiki-db_and-upgrade-forexport.sh
-------------------------------------------------

Substituted lowercase in filename for compliance with common
conventions.

Changed use of underscores to hyphens for file and directory names
for compliance with common conventions.

In get-mwiki-mbox-templates.sh
------------------------------

Changed use of underscores to hyphens for file and directory names
for compliance with common conventions.

Added explanatory header.

------------------------------------------

Pending Bug Fixes.

Full View of Saved Files Not Displaying.

I am investigating why the full view of at least some image files in the form
of /wiki/File:some_file are not displaying for even the direct restore of the
Koha MediaWiki Postgres instance.

PHP LTS on sury.org Only Supports Debian Stable and Old Stable.

Debian PHP maintainer, Ondřej Surý, has been continually supplying PHP LTS
updates for PHP 5.6 and PHP 7.4 as well as other versions of PHP with ongoing
patches which have been helpful when testing database migration and upgrades
using Debian 9.  Sometime in the past two weeks, Debian 9 support was dropped
on sury.org .  The sury.org packages for Debian 9 are still available at
archive.org and other possible archives and continue to run on my Debian 9 test
server.  The sury.org packages for Debian 11 stable and 10 old stable continue
to be available.  Debian 9 had been especially good for testing because it was
much less likely to have too many changes in database versions or other
dependencies which might break an old version of MediaWiki which we have stuck
in Postgres which first needs conversion to MySQL. 

As with any project, MediaWiki uses stable dependencies and takes time to adopt
new versions of dependencies which may introduce breaking changes.  The current
Koha Wiki using MediaWiki 1.16 mistakenly using Postgres was designed for PHP
5.3 but seems to be managing fine with PHP 5.6.  MediaWiki 1.35 LTS uses PHP
7.3 or 7.4.  No versions of MediaWiki yet use PHP 8.  The main issue is the
utility of having PHP 5.6 on a system where to help verify the correctness of
database migration before upgrading MediaWiki.

There are several workarounds.  Please keep in mind that moving the Koha wiki
to Debian 10 or 11 at least after upgrading to MediaWiki 1.35 LTS is presumed
in any case.

The installations of PHP 5.6 and PHP 7.4 continue to work on my test instance
of Debian 9 but cannot be as easily replicated on another instance of Debian 9.
 An alternative archive might be used in the place of sury.org .  Nevertheless,
my test instance of Debian 9 could be used for both MediaWiki database
migration from Postgres to MySQL and MediaWiki upgrade to 1.35 LTS if really
necessary. 

The best option is probably the following:  Debian 8 with PHP 5.6 will work for
MediaWiki 1.16 database migration from Postgres to MySQL and may even be better
than Debian 9 for database migration.  Debian 10 or 11 might be used for
upgrade to MediaWiki 1.35 LTS from a a migrated MySQL database dump and
associated files from the Koha MediaWiki instance.  Ondřej Surý continues to
provide PHP LTS support for the current Debian stable 11 and Debian old stable
10.  The scripts which currently run database migration and upgrade
sequentially would need at least functional division to support upgrading on a
different server from the database migration.  In the past few days, I have
improved section headings for easier log reading and to help more easily
identify code sections which would need at least a functional distinction
between migration and upgrade tasks if running distinctly on different virtual
servers with different Debian versions.  

An easier option might still work with migration and upgrade.  Running PHP 5.6
and 7.4 for everything starting on Debian 10 might work but the risk increases
for Postgres or MySQL versions in Debian 10 being too far out of sink with
MediaWiki 1.16 for database migration to work correctly especially considering
post-database migration MediaWiki maintenance scripts required to have a
working instances of MediaWiki even if merely used to assure working MediaWiki
instances before upgrading to MediaWiki 1.35 LTS.

Ondřej Surý provides commercial support with Freexian for PHP LTS releases for
Debian 8 and 9 but that is probably not a proper option for Koha.  See
https://php.freexian.com/ .

We could always compile PHP versions from source as needed.

pg_restore ftell Warning.

I had not remembered well my previous investigation into the ftell warning when
running pg_restore.  The database restore completes and appears to be correct.

PGPASSFILE=$pgpass_file_path pg_restore -h localhost -p 5432 -U postgres -w -d
database_name < $pgdump_file_path

"pg_restore: [custom archiver] WARNING: ftell mismatch with expected position
--ftell used"

The pg_restore command ftell warning may be an incorrect presumption of data
and dump size when the dump file is piped.  The warning may be likely to be a
false alarm.

The workaround of specifying the dump file for pg_restore with the -f argument
gives an error when also specifying the database to which to restore
everything.  The database restore fails.

PGPASSFILE=$pgpass_file_path pg_restore -h localhost -p 5432 -U postgres -w -d
database_name -f $pgdump_file_path

"pg_restore: options -d/--dbname and -f/--file cannot be used together"

Not specifying the restore database has complicated restore in long past
testing partly because we had made a mistake initially of not following the
database convention of specifying the database and the database user as the
same name and even changed the database user to be distinct from the database
name which can lead to tricky issues in restore where the database has two
users with imperfectly overlapping permissions which need to be specified with
excess care.

There may not be a completely satisfactory workaround for the warning which may
merely be a false alarm.  Some test might be devised to verify that the issue
is a false alarm.

Maintenance Job Control.

I added some support for running MediaWiki maintenance/runJobs.php to the post
install shell script for MediaWiki test instances but I need to add control for
maintenance/runJobs.php as an additional separate recurring process.

-- 
You are receiving this mail because:
You are watching all bug changes.