https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #2 from David Nind <david@davidnind.com> --- I am happy to help with testing, including doing a test upgrade locally to see what the issues are, and mapping out a plan for updating. This would require a database dump and the site files - not sure what the security implications are of this as the database would include user name and password information. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@bugs.koha-c | |ommunity.org -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #4 from Thomas Dukleth <td-koha-bugs@agogme.com> --- [Correcting my hurried mistaken reading, I had misread the units for the Koha wiki database dump. It is only about 100 MB not GB irrespective of the verbosity of the dump format. I had incorrectly reported the units in the 1 August Documentation meeting, https://meetings.koha-community.org/2019/documentation_irc_meeting_1_august_... .] In preparing to test migrating the MediaWiki database from Postgres to MySQL, I am now reverting to Postgres 9.6 in Debian Squeeze LTS for migration testing as postgres-common and postgresql-11 in Debian Buster currently have some bugs which we do not need trip over or otherwise pollute the results. "pg_upgradecluster writes data_directory to postgresql.auto.conf, and gets confused by it on the next upgrade [DATA LOSS]", https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=931635 . "ALTER TABLE statements causing "relation already exists" errors when some indexes exist", https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=932247 . It is possible to work around those bugs in Debian Buster but it seems preferable to avoid issues where things are less widely tested and where staying closer to the old version with an old migration script may work better. After migrating to MySQL, Debian Buster may be less problematic. Maria DB MySQL packages in Buster for use after migrating to MySQL are probably much more widely tested than Postgres packages have apparently been. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #6 from Thomas Dukleth <td-koha-bugs@agogme.com> --- I am preparing scripts to automate the conversion of the Koha MediaWiki database from Postgres to MySQL. Multiple methods for reproducing the MediaWiki Postgres database from different forms of dump files are working well. Using the full dump gives me more confidence with the requirement that more preparation of database users and associated username databases is necessary to avoid errors. The Postgres database restoration with the separate SQL structure and dump files require constraint options such as --data-only --disable-triggers --superuser to avoid errors with pg_restore. Philipp Spitzer's psqltomysql.py is written in a very generic abstracted matter and easy to modify with a few changes for both his database names, usernames, and adjustmemts where in Postgres we do not have the conventional form of username is the same as database name, http://www.winterrodeln.org/trac/browser/servermediawiki/trunk/pgsqltomysql/... . The are a few problematic MediaWiki Postgres database tables for migration to MySQL where an obvious generic or MySQL centric SQL structure file is not present. It may be necessary to partly convert from the Postgres SQL structure dump. Table anonymized_mwuser is owned by one particular database username which might be a legacy from the very first day of testing MediaWiki for Koha for which I started over the next day. All other tables have a different common username for the owner. I have not found any proper source for the anonymized_mwuser table. I have not found an easy to use MySQL structure file for the Semantic MediaWiki tables which have been essentially unused in consequence of Semantic MediaWiki not being well supported with Postgres. Semantic MediaWiki was released with SMW_Postgres_Schema.sql but no corresponding MySQL centric file unlike other extensions which add SQL tables. MySQL Semantic MediaWiki tables seem to be built from the Semantic MediaWiki installation script but that cannot be run on an empty database for the structure and I have not found an obvious database structure component in the includes files in the code. A workaround for preserving any existing data is to retrieve the structure from a MySQL SQL structure dump after installing Semantic MediaWiki on an otherwise empty MySQL based MediaWiki installation being careful of course to match the old versions we have for successful database migration. A similar workaround might be to temporarily drop the Semantic MediaWiki tables from the temporary Postgres database restoration; reinstall Semantic MediaWiki from the working MySQL conversion where no Semantic MediaWiki tables have been converted; dump the MySQL SQL structure; and then completely restart the database conversion using a newly restored temporary Postgres database and the newly obtained MySQL SQL structure dump. We might always choose not to preserve Semantic MediaWiki data under the presumption that there was no significant work done to make use of Semantic MediaWiki under Postgres where it would not work well: drop the Semantic MediaWiki tables from database conversion; uninstall Semantic MediaWiki; and then start again with Semantic MediaWiki by installing Semantic MediaWiki with no pre-existing Semantic MediaWiki data after upgrading MediaWiki to the current version in MySQL. The non-preservation option for Semantic MediaWiki might be best given known limitations of Semantic MediaWiki under Postgres, the peculiar dichotomy with how Postgres is treated, and the potential importance of Semantic MediaWiki for a flexible faceted system for finding content. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #8 from Thomas Dukleth <td-koha-bugs@agogme.com> ---

From printing variables, I seem to have confirmed that the problem with the Postgres to MySQL conversion script is failure to use proper quotation of the data and/or SQL statement as a whole, http://www.winterrodeln.org/trac/browser/servermediawiki/trunk/pgsqltomysql/... . It had been evident at glance that quoting had been originally considered for column names. Nothing seems to be magically quoting the data so the originator of the script presumably had some well trusted content.

Consequently, I seem to need to escape any quotes within the column data; quote the data from each column; possibly avoid quoting numeric values to avoid a data type problem; and possibly quote the entire SQL statement. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #10 from Thomas Dukleth <td-koha-bugs@agogme.com> --- Database migration from MySQL now functions with not only no errors but also with no warnings. I have resolved all 'data truncated' warnings for various columns. In some cases, setting a default value appropriate to a particular MySQL column as NULL, 0, or '' was sufficient. In a few cases, I have altered the MySQL column sizes to accommodate a larger data size than the defaults set by Mediawiki for MySQL. The difference which has allowed Postgres columns having larger data size than the size defined for for the same column in MySQL is that the Postgres columns at issue have a type text without any size constraint while the same column in MySQL often has a small size constraint. No longer having 'data truncated' warnings is very good but a little more checking should be done. For the cases of default value issues, it will be better to recheck my particular choices in the problematic columns for whether NULL, 0, or '' correctly align with default values set for the columns for MySQL. Despite now avoiding all warnings, I initially took a trial and error approach to solve as much as I could in a sprint without sleep shortly after eliminating all errors and it is possible a choice or choices might not be quite correctly aligned or at least after doing my course of jury duty I cannot remember perfectly well how thoroughly I had checked before jury duty. For the cases of Postgres having a larger data size than defined for the column in MySQL all the alterations I made to the matching MySQL columns size limits were made by trial and error and should at least be more closely aligned with what is actually necessary. Unlike the earlier errors, the 'data truncated' warnings do not give helpful information to identify offending rows. Particular MediaWiki implementations might need modification of the column size defaults to accommodate actual use and the case of Wikipedia has many but the modifications should not be unnecessary it is possible that some particular rows might have some buggy data from a code bug or a buggy user action. The columns for which I altered the maximum size may not generally have great consequence but there is one key column or key-like column affected, categorylinks.cl_sortkey for sorting titles properly with efficiency within a category. The cause of exceeding the size limit could be how we use the wiki with some informative verbose titles, however, it may that any extra long page titles are an accident or traces left over from spammers where the content has presumably been deleted but traces are left in the database. Running updateCollation.php to recompute sort keys for all pages would allow setting the column size back to VARCHAR(70) without an associated sorting problem. In two other cases, the column size limits which I found to work are well more than an order of magnitude greater than the column size defined for MediaWiki. I will try to query for rows which have excessive data size in the particular columns if I can find an efficient method to do so. It may be helpful to use some combination of aligning the column size better for the actual data and/or and choosing to truncate some column data without triggering any warning where it is obvious that excessive data was created by some obvious accident, such as leaning on the keyboard without intent. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #12 from David Nind <david@davidnind.com> --- That's awesome! Thanks Thomas for all your work on this. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmc@esilibrary.com |td-koha-bugs@agogme.com Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #15 from Thomas Dukleth <td-koha-bugs@agogme.com> --- As is well known, databases such as MySQL and Postgres do not have any magical diff friendly dump which would have appropriate granularity to use in an automated comparison to find any actual problem without being overwhelmed with noise from necessary differences between databases where the essential data is the same for relevant columns but where there are necessary database specific differences such as an extra indexing column, etc. Similar to what I had done for some part of debugging the Python script for the database migration, I am working on API based automated select of for every column and every row to write out each row equivalent of data between the two databases so that actual data is compared with appropriate granularity and not whole rows. Such an approach should be helpful to automatically check that the migrated data is complete and spot any unlikely but possible issues, such as with encoding of names from CJK languages which some users have, which would never be a problem for true UTF8 in Postgres but for which the MediaWiki solution to historic MySQL 3 byte limit for MySQL uft8 character set encoding is using binary data type for MySQL columns. [Encoding differences have been well addressed by insertion to MediaWiki supported binary data type columns for use in MySQL some time ago as will be explained retrospectively in a more detailed comment complete with references to an old version Debian time machine allowing the MySQL database to be created correctly for where we landed with MediaWiki when apprehending the database choice mistake.] The issue here is about an automated test for data comparison in which data has been read from both the originating Postgres database and the migrated MySQL database, then granularly saved in full UTF8 for diff comparison. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #18 from David Nind <david@davidnind.com> --- Thanks Thomas for your continued work on this! -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #20 from David Nind <david@davidnind.com> --- Hi Yolanda. I don't think your query relates to this bug, which is about updating the version of the software used for wiki.koha-community.org I don't know the answer to your query, but others on the general mailing list (https://koha-community.org/support/koha-mailing-lists/), or IRC (https://koha-community.org/get-involved/irc/) may be able to help. It does sound odd.. David -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #22 from Thomas Dukleth <td-koha-bugs@agogme.com> --- Currently correcting some minor bugs revealed when replacing archive files distinguished between binary files and files otherwise inappropriate for version control and version controlled files. Previous testing rerunning script changes but without replacing files excluded from version control had missed such bugs. The issue makes it more obvious that my previous conclusion over several iterations testing version control file inclusion that more unmodified non-binary files our existing installation archive are inappropriate for inclusion in version control. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #24 from Thomas Dukleth <td-koha-bugs@agogme.com> --- Important bug fix for installing a MediaWiki compliant stable version of PHP Composer completed a couple of months ago and finally enough incremental changes for excluding files inappropriate for version control through present and past commits. Carefully examining past versions for uniformity of version control exclusion was tedious but there should be no significant distractions for a file disappearing as outside the scope of version control and then reappearing in a later version when exclusion was missed. Files excluded as inappropriate for version control are currently available at https://test01.agogme.com/koha_migrate_mwiki_db_and_upgrade_test.tgz . Code and configuration files to test migrating the MediaWiki database and upgrading the wiki are currently available at git://test01.agogme.com/koha-migrate-mwiki-db-and-upgrade-test.git . Anyone interested in testing the code would need to separately obtain a full dump of the Koha Wiki to avoid publicly compromising Koha Wiki usernames and passwords which are not contained in the published archive. The issue of resetting wgSecretKey in LocalSettings.php could be avoided by obtaining a copy of the Koha Wiki LocalSettings.php . I have prepared a special nonpublic tar archive for Koha developers interested in testing. Mason James has been working on code to automatically validate the database migration. Test instances have been mostly up for over a year. They may frequently go down or become slow when: regenerated for code testing; or excess RAM or CPU use running some memory intensive processes unrelated to Koha on the VPS. Non-Koha use of the VPS starting too many processes still occasionally leads the OOM killer to stop mysqld for which I should add a monitoring and restart service for when I forget to check after such excess RAM usage. Reinstalled test instance of the Koha Wiki using Postgres at https://koha-mw-pg-test01.agogme.com/ . Koha test wiki migrated to MySQL at https://koha-mw-my-test01.agogme.com/ . Koha MySQL test wiki upgraded to MediaWiki 1.35 LTS at https://koha-mw-my-test01-upgr.agogme.com/ . Please note that some adjustment of the upgraded wiki, such as the superseded language change template at the bottom of the home page, may be outside the ordinary scope of an automated upgrade process and require manual correction. The job queue may be full for an extended period after reinstalling or upgrading a test wiki instance because reindexing is necessary. Ordering of search query result set improves with actual use but appears to be ordered by page creation initially after reindexing rather than page modification or other potentially more useful relevancy ranking which develops over time as the wiki instance is used.

From the current changelog. In README.txt

Clarified language and instruction variously. Added an explanation in installation instructions about obtaining files from a current git repository, an archive of files inappropriate for version control, database dump, etc. Added guidance in installation instructions to clarify what files currently need to be copied manually and what are examples needing modification to previous guidance about what is installed automatically. Added clarification for webserver configuration including testing Apache and Nginx at the same time with different webserver ports. Added the importance of starting the Perl script koha_migrate_mwiki_db_and_upgrade.pl from the shell script koha_migrate_mwiki_db_and_upgrade_startup.sh to provide better management and needed logging. Added a note to the configuration section noting that each of the main scripts has minimal internal configuration variables for finding the main configuration and other scripts. Clarified function of MediaWiki RESTBase in relation to VisualEditor extension. VisualEditor works fine without any of the subtle difficulties of configuring RESTBase to respond when proxied through the webserver. The core extension Parsoid is sufficient to run VisualEditor with no installation or configuration needed. The only disadvantage without RESTBase is that diffs may have some extra whitespace or such for imperfect diffs if switching back and forth between VisualEditor and WikiText editor before saving. Given the difficulties of configuring the wiki editor and Wikitext source editor. Choose elude many people. Added a section for configuration sharing security giving guidance and a script for substituting example values for security sensitive values in some configuration files when committing to public source code repositories. In koha_migrate_mwiki_db_and_upgrade_startup.sh --------------------------------------- Added bash script to control startup of migration and installation to log STDOUT and STDERR. In koha_migrate_mwiki_db_and_upgrade.pl --------------------------------------- Corrected for temporary file inadvertantly created when creating download directory for upgraded MediaWiki installation archive. Corrections for moving binary files and other files not suitable for version control to a separate archive from version control files. Improved elapsed time reporting. Standard date and time reporting. In LocalSettings.php -------------------- Commented out RESTBase configuration which has subtle configuration difficulties and substituted Parsoid configuration without RESTBase as sufficient for VisualEditor extension. Added localhost specification for page editing as the editor runs on localhost. Configuring page editing to specify the IP address from which the editor runs corrects for a problem otherwise preventing editing of spam protected pages with editing restrictions such as we have on the home page to prevent newly created accounts from immediately editing the home page. SearchBox.mustache ------------------ Changed the category excluded from the SimpleSearch form for dynamically archived content to Obsolete which has already been in use. koha_mwiki_instances_postinstall.sh ----------------------------------- Corrected installation of PHP composer to function more resiliantly in a script and set version to a stable point which avoids errors with MediaWiki. In etc/apache2 -------------- Changed listen port numbers for example Apache 2 configuration files to allow testing along with testing Nginx. In etc/nginx ------------ Example nginx configuration files. In koha_migrate_mwiki_db_and_upgrade_FOREXPORT.sh --------------------------------------- Added bash script to remove nonpublic elements from config files substituting public configuration examples for each of the nonpublic values in the following configuration line. The preceding line with a public configuration example starts with "# FOREXPORT # ". -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #26 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Why do we need «-"[[Category:Obsolete]]" AND» added automatically on searches again? I already heard about that a long time ago but forgot. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #27 from Thomas Dukleth <td-koha-bugs@agogme.com> --- The intent of adding an automatic exclusion for obsolete pages from the simple search box with -"[[Category:Obsolete]]" is for the simplest searches to exclude old obsolete pages so as not to confuse users or be found inappropriately in the most basic search result set. Old obsolete content pages can still be found using the advanced search including found explicitly by searching for "[[Category:Obsolete]]" without the exclusion operator prefix and thus acts as an archive for such content. Old obsolete pages are noted with a prominent notice via a template if they are encountered. Such pages should be updated if they can be, but are otherwise available to consult most importantly for valuable information they often contain which is not yet present in current pages. There should be a more elegant method to address the issue but migrating the database and upgrading has priority. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #29 from Thomas Dukleth <td-koha-bugs@agogme.com> --- Preserving version control changes after changing to the convention for hyphens or dashes instead of underscores in program directories and filenames has required replacing the Git archive with filenames in the history adjusted for dashes. Otherwise, moving files loses version control. Code and configuration files to test migrating the MediaWiki database and upgrading the wiki are currently available at the same location which was originally created with hyphens but previously contained directories and filenames with underscores, git://test01.agogme.com/koha-migrate-mwiki-db-and-upgrade-test.git . Files excluded as inappropriate for version control are currently available with hyphens having replaced underscores at https://test01.agogme.com/koha-migrate-mwiki-db-and-upgrade-test.tgz . Anyone interested in testing the code would need to separately obtain a full dump of the Koha Wiki to avoid publicly compromising Koha Wiki usernames and passwords which are not contained in the published archive. The issue of resetting wgSecretKey in LocalSettings.php could be avoided by obtaining a copy of the Koha Wiki LocalSettings.php . I have prepared a special nonpublic tar archive for Koha developers interested in testing. Test instances were down for some extended periods in the past few days while I was fixing bugs and regenerating the test instances. Please note as before that newly reinstalled MediaWiki instances take time for indexing etc. to catch up over multiple days and indexing requires actual use to weight the result set properly. . They may frequently go down or become slow when: regenerated for code testing; or excess RAM or CPU use running some memory intensive processes unrelated to Koha on the VPS. Non-Koha use of the VPS starting too many processes still occasionally leads the OOM killer to stop mysqld for which I should add a monitoring and restart service for when I forget to check after such excess RAM usage. Reinstalled test instance of the Koha Wiki using Postgres at https://koha-mw-pg-test01.agogme.com/ . Koha test wiki migrated to MySQL at https://koha-mw-my-test01.agogme.com/ . Koha MySQL test wiki upgraded to MediaWiki 1.35 LTS at https://koha-mw-my-test01-upgr.agogme.com/ . Please note that some adjustment of the upgraded wiki, such as the superseded language change template at the bottom of the home page, may be outside the ordinary scope of an automated upgrade process and require manual correction. The job queue may be full for an extended period after reinstalling or upgrading a test wiki instance because reindexing is necessary. Ordering of search query result set improves with actual use but appears to be ordered by page creation initially after reindexing rather than page modification or other potentially more useful relevancy ranking which develops over time as the wiki instance is used.

From the current changelog.

In README.txt ------------- Changed use of underscores to hyphens for file and directory names for compliance with common conventions. Clarified language and instruction variously. In koha_migrate_mwiki_db_and_upgrade.ini --------------------------------------- Changed use of underscores to hyphens for file and directory names for compliance with common conventions. Corrected location of debugging output logs for koha-mediawiki-pgsqltomysql.py . Added operating system version configuration for improving installation. In koha-migrate-mwiki-db-and-upgrade-startup.sh ----------------------------------------------- Changed use of underscores to hyphens for file and directory names for compliance with common conventions. Decreased verbosity of bash startup script when calling without arguments for which koha-migrate-mwiki-db-and-upgrade.pl responds with a usage reminder leaving response from the bash startup script unnecessary. In koha-migrate-mwiki-db-and-upgrade.pl --------------------------------------- Changed use of underscores to hyphens for file and directory names for compliance with common conventions. Corrected for temporary file inadvertantly created when creating download directory for upgraded MediaWiki installation archive. Corrected for setting permissions of pgpass file too early for newly created pgpass file which led to failure of pg_restore. Added removeal of pgpass file after use by pg_restore. Corrected typo for binary in mysql command option --default-character-set . Corrections for excessive copying of files from the production archive to the MediaWiki upgrade test instance when scope of binary files and other files not suitable for version control had been moved to a separate archive from version control files. Added additional section heading comments for demarcating more code sections for better readability of code and logs. Added operating system version configuration for improving installation. Removed redundant code section previously included with the intention of chaning the section for additional MediaWiki testing instances. In koha-mediawiki-pgsqltomysql.py --------------------------------- Changed use of underscores to hyphens for file and directory names for compliance with common conventions. Corrected location of debugging logs. In koha-mwiki-instances-postinstall.sh -------------------------------------- Changed use of underscores to hyphens for file and directory names for compliance with common conventions. Added calls to runJobs.php. Clarified section heading comments demarcating code sections for better readability of code and logs. In koha-migrate-mwiki-db_and-upgrade-forexport.sh ------------------------------------------------- Substituted lowercase in filename for compliance with common conventions. Changed use of underscores to hyphens for file and directory names for compliance with common conventions. In get-mwiki-mbox-templates.sh ------------------------------ Changed use of underscores to hyphens for file and directory names for compliance with common conventions. Added explanatory header. ------------------------------------------ Pending Bug Fixes. Full View of Saved Files Not Displaying. I am investigating why the full view of at least some image files in the form of /wiki/File:some_file are not displaying for even the direct restore of the Koha MediaWiki Postgres instance. PHP LTS on sury.org Only Supports Debian Stable and Old Stable. Debian PHP maintainer, Ondřej Surý, has been continually supplying PHP LTS updates for PHP 5.6 and PHP 7.4 as well as other versions of PHP with ongoing patches which have been helpful when testing database migration and upgrades using Debian 9. Sometime in the past two weeks, Debian 9 support was dropped on sury.org . The sury.org packages for Debian 9 are still available at archive.org and other possible archives and continue to run on my Debian 9 test server. The sury.org packages for Debian 11 stable and 10 old stable continue to be available. Debian 9 had been especially good for testing because it was much less likely to have too many changes in database versions or other dependencies which might break an old version of MediaWiki which we have stuck in Postgres which first needs conversion to MySQL. As with any project, MediaWiki uses stable dependencies and takes time to adopt new versions of dependencies which may introduce breaking changes. The current Koha Wiki using MediaWiki 1.16 mistakenly using Postgres was designed for PHP 5.3 but seems to be managing fine with PHP 5.6. MediaWiki 1.35 LTS uses PHP 7.3 or 7.4. No versions of MediaWiki yet use PHP 8. The main issue is the utility of having PHP 5.6 on a system where to help verify the correctness of database migration before upgrading MediaWiki. There are several workarounds. Please keep in mind that moving the Koha wiki to Debian 10 or 11 at least after upgrading to MediaWiki 1.35 LTS is presumed in any case. The installations of PHP 5.6 and PHP 7.4 continue to work on my test instance of Debian 9 but cannot be as easily replicated on another instance of Debian 9. An alternative archive might be used in the place of sury.org . Nevertheless, my test instance of Debian 9 could be used for both MediaWiki database migration from Postgres to MySQL and MediaWiki upgrade to 1.35 LTS if really necessary. The best option is probably the following: Debian 8 with PHP 5.6 will work for MediaWiki 1.16 database migration from Postgres to MySQL and may even be better than Debian 9 for database migration. Debian 10 or 11 might be used for upgrade to MediaWiki 1.35 LTS from a a migrated MySQL database dump and associated files from the Koha MediaWiki instance. Ondřej Surý continues to provide PHP LTS support for the current Debian stable 11 and Debian old stable 10. The scripts which currently run database migration and upgrade sequentially would need at least functional division to support upgrading on a different server from the database migration. In the past few days, I have improved section headings for easier log reading and to help more easily identify code sections which would need at least a functional distinction between migration and upgrade tasks if running distinctly on different virtual servers with different Debian versions. An easier option might still work with migration and upgrade. Running PHP 5.6 and 7.4 for everything starting on Debian 10 might work but the risk increases for Postgres or MySQL versions in Debian 10 being too far out of sink with MediaWiki 1.16 for database migration to work correctly especially considering post-database migration MediaWiki maintenance scripts required to have a working instances of MediaWiki even if merely used to assure working MediaWiki instances before upgrading to MediaWiki 1.35 LTS. Ondřej Surý provides commercial support with Freexian for PHP LTS releases for Debian 8 and 9 but that is probably not a proper option for Koha. See https://php.freexian.com/ . We could always compile PHP versions from source as needed. pg_restore ftell Warning. I had not remembered well my previous investigation into the ftell warning when running pg_restore. The database restore completes and appears to be correct. PGPASSFILE=$pgpass_file_path pg_restore -h localhost -p 5432 -U postgres -w -d database_name < $pgdump_file_path "pg_restore: [custom archiver] WARNING: ftell mismatch with expected position --ftell used" The pg_restore command ftell warning may be an incorrect presumption of data and dump size when the dump file is piped. The warning may be likely to be a false alarm. The workaround of specifying the dump file for pg_restore with the -f argument gives an error when also specifying the database to which to restore everything. The database restore fails. PGPASSFILE=$pgpass_file_path pg_restore -h localhost -p 5432 -U postgres -w -d database_name -f $pgdump_file_path "pg_restore: options -d/--dbname and -f/--file cannot be used together" Not specifying the restore database has complicated restore in long past testing partly because we had made a mistake initially of not following the database convention of specifying the database and the database user as the same name and even changed the database user to be distinct from the database name which can lead to tricky issues in restore where the database has two users with imperfectly overlapping permissions which need to be specified with excess care. There may not be a completely satisfactory workaround for the warning which may merely be a false alarm. Some test might be devised to verify that the issue is a false alarm. Maintenance Job Control. I added some support for running MediaWiki maintenance/runJobs.php to the post install shell script for MediaWiki test instances but I need to add control for maintenance/runJobs.php as an additional separate recurring process. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #31 from Thomas Dukleth <td-koha-bugs@agogme.com> --- Using Debian 8 for database migration from Postgres to MySQL avoids warnings and errors from pg_restore which is better than having some potential unidentified problem. The MySQL wiki dump is copied from Debian 8 to Debian 10 or 11. Debian 10 has been tested for wiki upgrade using the MySQL wiki dump from and should work with Debian 11. A test run of the Canasta MediaWiki Docker image for fairly fulsome use of MediaWiki pending. If the test run with Canasta works, we should upgrade for production the next available day. Code and configuration files to test migrating the MediaWiki database and upgrading the wiki are currently available at the same location, git://test01.agogme.com/koha-migrate-mwiki-db-and-upgrade-test.git . Files excluded as inappropriate for version control are currently available at the same location, https://test01.agogme.com/koha-migrate-mwiki-db-and-upgrade-test.tgz . Anyone interested in testing the code would need to separately obtain a full dump of the Koha Wiki to avoid publicly compromising Koha Wiki usernames and passwords which are not contained in the published archive. The issue of resetting wgSecretKey in LocalSettings.php could be avoided by obtaining a copy of the Koha Wiki LocalSettings.php . I have prepared a special nonpublic tar archive for Koha developers interested in testing. Please note as before that newly reinstalled MediaWiki instances take time for indexing etc. to catch up over multiple days and indexing requires actual use to weight the result set properly. They may frequently go down or become slow when: regenerated for code testing; or excess RAM or CPU use running some memory intensive processes unrelated to Koha on the VPS. Non-Koha use of the VPS starting too many processes still occasionally leads the OOM killer to stop mysqld for which I should add a monitoring and restart service for when I forget to check after such excess RAM usage. Reinstalled test instance of the Koha Wiki using Postgres on Debian 8 with Nginx at https://koha-mw-pg-test02.agogme.com/ . Koha test wiki migrated to MySQL on Debian 8 with Nginx at https://koha-mw-my-test02.agogme.com/ . Koha MySQL test wiki upgraded to MediaWiki 1.35 LTS on Debian 10 with Nginx at https://koha-mw-my-test00-upgr.agogme.com/ . We will likely continue to use Apache in production until or unless we have the resolution for wiki full image page working in Ngnix which currently returns a 404 error for such pages. For any deficiency of Nginx, try Apache by adding port 9443 to any page such as https://koha-mw-my-test00-upgr.agogme.com:9443/wiki/Some_Page . MediaWiki configuration will force the port back to the port back to the configured HTTPS port 443 and does not support running the same wiki instance simultaneously for different webservers on different ports. Please note that some adjustment of the upgraded wiki, such as the superseded language change template at the bottom of the home page, may be outside the ordinary scope of an automated upgrade process and require manual correction. The job queue may be full for an extended period after reinstalling or upgrading a test wiki instance because reindexing is necessary. Ordering of search query result set improves with actual use but may be partly ordered by page creation initially after reindexing rather than page modification or other potentially more useful relevancy ranking which develops over time as the wiki instance is used. Postgres based queries to which we may be accustomed do automatic word stemming and index page titles. MySQL based MediaWiki allows support for many options including Apache Lucene indexing via Elastic Search or Open Search. Dynamic archiving to exclude obsolete pages using "-[[Category:Obsolete]]" requires all pages to have at least one category which is generally necessary in any case even if there is insufficient time to select an appropriate category assigning a placeholder category for category unselected such as [[Category:Empty]] would be helpful which can later be replaced with an appropriate choice.

From the current changelog.

In /etc examples ---------------- Moved example files in the /etc directory to os-variants directory for Debian version specific use. In README.txt ------------- Added apt instruction for packages libconfig-simple-perl rsync gnupg postgresql-contrib libdbd-pg-perl python python-psycopg2 python-mysqldb python3 . Added additional package installation for several php and other packages which had been configured manually in advance of previous testing where they had been selected from packages filing requirements for the Debian distribution mediawiki package. Correction for Debian 10 and later substitute libnode-dev for nodejs-dev . Added os-variants. Added tested operating system version divided use where database migration from Postgres to MySQL is done in Debian 8 to avoid warnings and errors from using pg_restore for restoring the Postgres production database starting point. Wiki upgrade is then done on Debian 10 which has been tested using the MySQL databse dumped from Debian 8. Upgrade could also be done on Debian 11 untested. Clarified language and instruction variously. In koha_migrate_mwiki_db_and_upgrade.ini --------------------------------------- Added human readable postupgrade MySQL dump path. In koha-migrate-mwiki-db-and-upgrade-startup.sh ----------------------------------------------- Added support for additional necessary automatic package installation for libconfig-simple-perl . Added option for using database dump from old operating system version. Added upgrade option now used for automated upgrade. In koha-migrate-mwiki-db-and-upgrade.pl --------------------------------------- Changed use of underscore to hyphen for automatically created Apache Casandra apt repository list file. Added support for additional necessary automatic package installation for rsync gnupg libdbd-pg-perl python-psycopg2 python-mysqldb. Corrected misspelling with extra dash in automated installation of packages php5.6-mysql php5.6-pgsql php7.4-mysql php7.4-pgsql . Added support for additional necessary automatic package installation for several php and other packages which had been configured manually in advance of previous testing where they had been selected from packages filing requirements for the Debian distribution mediawiki package. Correction for Debian 10 and later substitute libnode-dev for nodejs-dev . Corrected for some instances missing apt-get -y option to ensure package installation works non-interactively. Corrected automated Apache configuration for Debian 8 using php-fpm. Commented out call to koha-mediawiki-mysql-alter-col-for-migration-from-postgres.sql . Added os-variants directory for installation use. Added option for using database dump from old operating system version. Added upgrade option now used for automated upgrade. Added additional postupgrade MySQL dumps. In koha-mediawiki-pgsqltomysql.py --------------------------------- Corrected for file exists error when creating a directory which already exists for log files. Activated additional file truncation code to avoid need to increase any column sizes as the Perl script call to koha-mediawiki-mysql-alter-col-for-migration-from-postgres.sql is now commented out. In koha-mwiki-instances-postinstall.sh -------------------------------------- Added prospective --server argument to update.php calls to avoid possible "PHP Notice: Undefined index: SERVER_NAME". In LocalSettings.php -------------------- Added !isset condition for REMOTE_ADDR to update.php reporting "PHP Notice: Undefined index: REMOTE_ADDR". Moved LocalSettings.php files to os-variants directory. In koha-migrate-mwiki-db_and-upgrade-forexport.sh ------------------------------------------------- Added path variables for files in os-variants directory. ------------------------------------------ Pending Bug Fixes. Full View of Saved Files Not Displaying. The full view of image files in the form of /wiki/File:some_file are not displaying. The suggested Nginx fix requires more attention. The easy workaround is to revert to Apache. Maintenance Job Control. I added some support for running MediaWiki maintenance/runJobs.php to the post install shell script for MediaWiki test instances but I need to add control for maintenance/runJobs.php as an additional separate recurring process which is actually fairly trivial. Using the Canasta Docker image for MediaWiki provides some convenient default configuration for some things such as job control. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |caroline.cyr-la-rose@inlibr | |o.com --- Comment #33 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- (In reply to Owen Leonard from comment #32)

...

https://lists.katipo.co.nz/pipermail/koha/2022-October/058585.html

Is it a known issue that searching fails? I see this message: "An error has occurred while searching: We could not complete your search due to a temporary problem. Please try again later."

I also had this problem. - Entered "Meeting" in the search box at the top of the main page - Pressed "Enter" - Got a message "An error has occurred while searching: We could not complete your search due to a temporary problem. Please try again later." -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #35 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- When I tested an earlier version of the updated wiki, I fell in love with the category editor. I see no way to edit/fix the categories on the pages now. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #36 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Katrin Fischer from comment #35)

When I tested an earlier version of the updated wiki, I fell in love with the category editor. I see no way to edit/fix the categories on the pages now.

Ok, maybe just the optics changed a little? you need to edit the page in order to get to it. Will experiment more. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #37 from David Nind <david@davidnind.com> --- Hi Thomas. Could you have a look at the account request settings? Lately, very few new account requests are coming through for approval. In the Recent changes page, there are lots of accounts being created (see the recent changes page - User creation log entries). Most are red linked, but these have not been coming through the Confirm account requests page (https://wiki.koha-community.org/wiki/Special:ConfirmAccounts). Also, there are some spam pages that need deleting - see 4 May 2023 on the recent changes page. See also, the discussion on the development mailing list: https://lists.koha-community.org/pipermail/koha-devel/2023-May/048145.html Thanks! David -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23073 --- Comment #39 from David Nind <david@davidnind.com> --- Message sent to the general mailing list: https://lists.katipo.co.nz/pipermail/koha/2023-May/059454.html -- You are receiving this mail because: You are watching all bug changes.