[Bug 21031] New: Apache Rewrite rules don't work for API when using anything but Debian package Plack configuration
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Bug ID: 21031 Summary: Apache Rewrite rules don't work for API when using anything but Debian package Plack configuration Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: REST api Assignee: koha-bugs@lists.koha-community.org Reporter: dcook@prosentient.com.au If you install Koha using the Debian packages and you enable Plack, you should be able to reach the API using the following URL: "/api/v1/" If you install Koha using the Debian packages and you don't enable Plack (or you install from source or you're using a git install or whatever else), you can only get the same API response by visiting "/api/v1/app.pl/api/v1/spec" It appears to me that the Rewrite rules in etc/koha-httpd.conf, debian/templates/apache-shared-opac.conf, and debian/templates/apache-shared-intranet.conf don't work. Fixing this doesn't seem quite as easy as just copying over the Apache configuration for Plack-enabled instances, unfortunately, but I'm looking at this... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 --- Comment #1 from David Cook <dcook@prosentient.com.au> --- Created attachment 76645 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=76645&action=edit Bug 21031 - Apache Rewrite rules don't work for API when using anything but Debian package Plack configuration The Rewrite rules for Apache don't work unless you're using debian/templates/apache-shared-opac-plack.conf or debian/templates/apache-shared-intranet-plack.conf. This patch fixes the Rewrite rules for the non-Plack Debian Apache configuration templates as well as the standard Apache configuration file that comes with Koha. __BEFORE APPLYING__ 1. Visit /api/v1/app.pl/api/v1/spec on your git dev install 2. This should display a large page of JSON 3. Visit /api/v1/spec on your git dev install 4. This should generate a 404 error __APPLY PATCH__ __AFTER APPLYING__ 5. Visit /api/v1/app.pl/api/v1/spec on your git dev install 6. This should display a large page of JSON 7. Visit /api/v1/spec on your git dev install 8. This should display a large page of JSON (identical to the one from earlier steps) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Jon Knight <J.P.Knight@lboro.ac.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |J.P.Knight@lboro.ac.uk --- Comment #2 from Jon Knight <J.P.Knight@lboro.ac.uk> --- On my kohadevbox I seem to get 404 errors for all the URLs in the test plan? Do I need to turn something on first? Going to an known API end point seems to work - for example http://localhost:8080/api/v1/app.pl/api/v1/patrons/50 or http://localhost:8080/api/v1/patrons/50 I'd sort of assumed that kohadevbox is a git install, so should be able to test this? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- (In reply to Jon Knight from comment #2)
On my kohadevbox I seem to get 404 errors for all the URLs in the test plan? Do I need to turn something on first? Going to an known API end point seems to work - for example
http://localhost:8080/api/v1/app.pl/api/v1/patrons/50
or
http://localhost:8080/api/v1/patrons/50
I'd sort of assumed that kohadevbox is a git install, so should be able to test this?
Hi Jon, Thanks for taking the time to try and test this out! I think the kohadevbox can't be used for testing this patch though. The kohadevbox is actually a package install which gets converted into using git, but it's not a git install per se. kohadevbox uses koha-gitify which re-writes the installed stable package Apache configuration, so using kohadevbox you're not actually testing these patches, unfortunately. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ere.maijala@helsinki.fi --- Comment #4 from Ere Maijala <ere.maijala@helsinki.fi> --- Is there a reason for only accepting v0 or v1 in the RewriteCond? Especially since the rule uses 0-9? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 --- Comment #5 from David Cook <dcook@prosentient.com.au> --- (In reply to Ere Maijala from comment #4)
Is there a reason for only accepting v0 or v1 in the RewriteCond? Especially since the rule uses 0-9?
Very good question. I was just keeping it consistent with what's in "debian/templates/apache-shared-opac-plack.conf" and "debian/templates/apache-shared-intranet-plack.conf". I imagine they all should be updated to 0-9 for the RewriteCond too. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #76645|0 |1 is obsolete| | --- Comment #6 from Ere Maijala <ere.maijala@helsinki.fi> --- Created attachment 76725 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=76725&action=edit Bug 21031 - Apache Rewrite rules don't work for API when using anything but Debian package Plack configuration The Rewrite rules for Apache don't work unless you're using debian/templates/apache-shared-opac-plack.conf or debian/templates/apache-shared-intranet-plack.conf. This patch fixes the Rewrite rules for the non-Plack Debian Apache configuration templates as well as the standard Apache configuration file that comes with Koha. __BEFORE APPLYING__ 1. Visit /api/v1/app.pl/api/v1/spec on your git dev install 2. This should display a large page of JSON 3. Visit /api/v1/spec on your git dev install 4. This should generate a 404 error __APPLY PATCH__ __AFTER APPLYING__ 5. Visit /api/v1/app.pl/api/v1/spec on your git dev install 6. This should display a large page of JSON 7. Visit /api/v1/spec on your git dev install 8. This should display a large page of JSON (identical to the one from earlier steps) Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 --- Comment #7 from David Cook <dcook@prosentient.com.au> --- Thanks for the sign off! -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl Assignee|koha-bugs@lists.koha-commun |dcook@prosentient.com.au |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |m.de.rooy@rijksmuseum.nl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> ---
From the plack counterpart: RewriteCond %{REQUEST_URI} !^/api/v[0-1]+/app.pl RewriteRule ^/api/(v[0-9]+)/(.*)$ /api/$1/app.pl/api/$1/$2 [L,PT]
From your patch: RewriteCond %{REQUEST_URI} !^/api/v[0-1]+/app.pl RewriteRule ^(v[0-9]+)/(.*)$ /api/$1/app.pl/api/$1/$2 [L]
Not the same. You start with v[0-9] which seems to be wrong ;) How did you test? Note btw that v[0-9]+ may not be what we want. It covers v987654321 for instance. We do not have v0, but only v1. Hoping for v2 :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 --- Comment #9 from David Cook <dcook@prosentient.com.au> --- (In reply to Marcel de Rooy from comment #8)
From the plack counterpart: RewriteCond %{REQUEST_URI} !^/api/v[0-1]+/app.pl RewriteRule ^/api/(v[0-9]+)/(.*)$ /api/$1/app.pl/api/$1/$2 [L,PT]
From your patch: RewriteCond %{REQUEST_URI} !^/api/v[0-1]+/app.pl RewriteRule ^(v[0-9]+)/(.*)$ /api/$1/app.pl/api/$1/$2 [L]
Not the same. You start with v[0-9] which seems to be wrong ;) How did you test?
It might seem wrong but it's not wrong ;). The reason is that these directives are happening in a Directory block with a RewriteBase of /api/ (note that in Apache 2.4 without an explicit RewriteBase an implicit one will be created anyway), so Apache strips off the "/api/" part after it matches the RewriteCond. You can check this using a logging directive like "LogLevel alert rewrite:trace3". I tested by putting in the Apache config with the directives I've included in these patches along with the logging I mention above and I observed that my test plan worked.
Note btw that v[0-9]+ may not be what we want. It covers v987654321 for instance. We do not have v0, but only v1. Hoping for v2 :)
That may be true but I don't think that's within the scope of this bug report. This patch is just trying to bring the CGI config in line with the Plack config. I'd like to reset this status back to Signed Off. I can provide some logging output if that would be helpful in regards to the RewriteRule? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off --- Comment #10 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to David Cook from comment #9)
(In reply to Marcel de Rooy from comment #8)
From the plack counterpart: RewriteCond %{REQUEST_URI} !^/api/v[0-1]+/app.pl RewriteRule ^/api/(v[0-9]+)/(.*)$ /api/$1/app.pl/api/$1/$2 [L,PT]
From your patch: RewriteCond %{REQUEST_URI} !^/api/v[0-1]+/app.pl RewriteRule ^(v[0-9]+)/(.*)$ /api/$1/app.pl/api/$1/$2 [L]
Not the same. You start with v[0-9] which seems to be wrong ;) How did you test?
It might seem wrong but it's not wrong ;).
The reason is that these directives are happening in a Directory block with a RewriteBase of /api/ (note that in Apache 2.4 without an explicit RewriteBase an implicit one will be created anyway), so Apache strips off the "/api/" part after it matches the RewriteCond. You can check this using a logging directive like "LogLevel alert rewrite:trace3".
I tested by putting in the Apache config with the directives I've included in these patches along with the logging I mention above and I observed that my test plan worked.
Note btw that v[0-9]+ may not be what we want. It covers v987654321 for instance. We do not have v0, but only v1. Hoping for v2 :)
That may be true but I don't think that's within the scope of this bug report. This patch is just trying to bring the CGI config in line with the Plack config.
I'd like to reset this status back to Signed Off. I can provide some logging output if that would be helpful in regards to the RewriteRule? I did it for you now. Yeah, your log might be helpful to prove :)
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 --- Comment #11 from David Cook <dcook@prosentient.com.au> --- (In reply to Marcel de Rooy from comment #10)
I did it for you now. Yeah, your log might be helpful to prove :)
Thanks, Marcel. I'll redo my tests in the morning and copy some of the log output here. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 --- Comment #12 from David Cook <dcook@prosentient.com.au> --- I haven't forgotten but I've been totally snowed under by other things. Will get to this when I get a chance... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 --- Comment #13 from David Cook <dcook@prosentient.com.au> --- Created attachment 77148 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=77148&action=edit Apache_rewrite_log.txt Apologies for the delay on getting this out. As you can see, because we're in a directory ending in "api", the "api" gets stripped out of the uri: [Fri Jul 20 13:43:01.358964 2018] [rewrite:trace1] [pid 11341] mod_rewrite.c(477): [client 192.168.1.100:15501] 192.168.1.100 - - [192.168.1.200/sid#7f1ca31258b0][rid#7f1c9d3a50a0/initial] pass through /api/v1/cities [Fri Jul 20 13:43:01.359158 2018] [rewrite:trace3] [pid 11341] mod_rewrite.c(477): [client 192.168.1.100:15501] 192.168.1.100 - - [192.168.1.200/sid#7f1ca31258b0][rid#7f1c9d3a50a0/initial] [perdir /home/dcook/koha/api/] strip per-dir prefix: /home/dcook/koha/api/v1/cities -> v1/cities [Fri Jul 20 13:43:01.359189 2018] [rewrite:trace3] [pid 11341] mod_rewrite.c(477): [client 192.168.1.100:15501] 192.168.1.100 - - [192.168.1.200/sid#7f1ca31258b0][rid#7f1c9d3a50a0/initial] [perdir /home/dcook/koha/api/] applying pattern '^(v[0-9]+)/(.*)$' to uri 'v1/cities' [Fri Jul 20 13:43:01.359271 2018] [rewrite:trace2] [pid 11341] mod_rewrite.c(477): [client 192.168.1.100:15501] 192.168.1.100 - - [192.168.1.200/sid#7f1ca31258b0][rid#7f1c9d3a50a0/initial] [perdir /home/dcook/koha/api/] rewrite 'v1/cities' -> '/api/v1/app.pl/api/v1/cities' [Fri Jul 20 13:43:01.359302 2018] [rewrite:trace2] [pid 11341] mod_rewrite.c(477): [client 192.168.1.100:15501] 192.168.1.100 - - [192.168.1.200/sid#7f1ca31258b0][rid#7f1c9d3a50a0/initial] [perdir /home/dcook/koha/api/] trying to replace prefix /home/dcook/koha/api/ with /api/ [Fri Jul 20 13:43:01.359334 2018] [rewrite:trace2] [pid 11341] mod_rewrite.c(477): [client 192.168.1.100:15501] 192.168.1.100 - - [192.168.1.200/sid#7f1ca31258b0][rid#7f1c9d3a50a0/initial] [perdir /home/dcook/koha/api/] trying to replace context docroot /home/dcook/koha/api with context prefix /api [Fri Jul 20 13:43:01.359353 2018] [rewrite:trace1] [pid 11341] mod_rewrite.c(477): [client 192.168.1.100:15501] 192.168.1.100 - - [192.168.1.200/sid#7f1ca31258b0][rid#7f1c9d3a50a0/initial] [perdir /home/dcook/koha/api/] internal redirect with /api/v1/app.pl/api/v1/cities [INTERNAL REDIRECT] I admit that I don't 100% understand some of the later lines. I get the "strip per-dir prefix", "applying pattern", "rewrite" and "internal redirect", but I don't understand why "trying to replace prefix" and "trying to replace context docroot" happen. Maybe I need to increase the trace level to 5 from 3 - I don't know. But yeah the important thing is that the RewriteRule for non-Plack Apache configuration shouldn't include the "api" path since the directory-level configuration means "api" gets stripped out of the incoming uri. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #76725|0 |1 is obsolete| | --- Comment #14 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 77300 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=77300&action=edit Bug 21031 - Apache Rewrite rules don't work for API when using anything but Debian package Plack configuration The Rewrite rules for Apache don't work unless you're using debian/templates/apache-shared-opac-plack.conf or debian/templates/apache-shared-intranet-plack.conf. This patch fixes the Rewrite rules for the non-Plack Debian Apache configuration templates as well as the standard Apache configuration file that comes with Koha. __BEFORE APPLYING__ 1. Visit /api/v1/app.pl/api/v1/spec on your git dev install 2. This should display a large page of JSON 3. Visit /api/v1/spec on your git dev install 4. This should generate a 404 error __APPLY PATCH__ __AFTER APPLYING__ 5. Visit /api/v1/app.pl/api/v1/spec on your git dev install 6. This should display a large page of JSON 7. Visit /api/v1/spec on your git dev install 8. This should display a large page of JSON (identical to the one from earlier steps) Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Passed QA with few notes posted separately to Bugzilla. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 --- Comment #15 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- QA Comment: Thanks David. Improving parts not touched by most users (Debian/Plack). Just a few remarks (close to nitpicking): [1] A difference with the rule in shared-opac-plack is that you do not include the PT flag. Note that this is fine. You do not need the PT flag since you have the rule in a <Directory> section. (The PT flag is implied in per-directory contexts.) [2] The implicit PT may explain why you got the two additional lines in the apache trace. Might be possible to finetune that, but needs more Apache background. [3] The rule is a bit hybrid. As you explained, it is in the context of a RewriteBase with /api/. So paths are relative. But the replacement is not relative but absolute. No reason to change, but a relative replace without /api in front would have looked good to me too. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 --- Comment #16 from David Cook <dcook@prosentient.com.au> --- (In reply to Marcel de Rooy from comment #15)
QA Comment: Thanks David. Improving parts not touched by most users (Debian/Plack).
Just a few remarks (close to nitpicking): [1] A difference with the rule in shared-opac-plack is that you do not include the PT flag. Note that this is fine. You do not need the PT flag since you have the rule in a <Directory> section. (The PT flag is implied in per-directory contexts.) [2] The implicit PT may explain why you got the two additional lines in the apache trace. Might be possible to finetune that, but needs more Apache background. [3] The rule is a bit hybrid. As you explained, it is in the context of a RewriteBase with /api/. So paths are relative. But the replacement is not relative but absolute. No reason to change, but a relative replace without /api in front would have looked good to me too.
Thanks for the notes! I appreciate the feedback and it gives me food for thought! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com Status|Passed QA |Pushed to Master --- Comment #17 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Patch pushed to master. Thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable CC| |martin.renvoize@ptfs-europe | |.com --- Comment #18 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Pushed to 18.05.x for 18.05.03. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #19 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Pushed to 17.11.x for 17.11.10 BTW here is the Nginx configuration for this rewrite : rewrite ^/api/(v[0-9]+)/(.*)$ /api/$1/app.pl/api/$1/$2; -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Stable |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21031 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=18175 CC| |jonathan.druart@bugs.koha-c | |ommunity.org -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org