New subject: [Bug 43137] Add system health monitoring with status indicators in the staff interface

22 Jul 2026

      https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43137

            Bug ID: 43137
           Summary: Add system health monitoring with status indicators in
                    the staff interface header
   Initiative type: ---
        Sponsorship ---
            status:
           Product: Koha
           Version: Main
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5 - low
         Component: Architecture, internals, and plumbing
          Assignee: koha-bugs@lists.koha-community.org
          Reporter: tomascohen@gmail.com
        QA Contact: testopia@bugs.koha-community.org
  Target Milestone: ---

Koha depends on several services (search engine, background job workers, SFTP
endpoints) but provides no at-a-glance visibility into their health. Staff
discover problems only when operations fail -- a patron search returns nothing,
a batch job silently stops, or EDI transfers accumulate.

Proposed solution:

A system health framework that checks services, collects warnings and critical
messages, and surfaces a summary indicator in the staff interface header.

API response structure:

GET /api/v1/health

{
    "core": [
        {
            "id": "search_engine",
            "label": "Search engine",
            "warnings": [
                "Index koha_biblios last reindexed 3 days ago"
            ],
            "critical": [
                "Elasticsearch cluster is red: 2 shards unassigned"
            ]
        },
        {
            "id": "background_jobs",
            "label": "Background jobs",
            "warnings": [],
            "critical": []
        },
        {
            "id": "sftp",
            "label": "SFTP connections",
            "warnings": [
                "EDI vendor X: connection timeout on last attempt"
            ],
            "critical": []
        }
    ],
    "plugins": {}
}

- 'core' is an array of check results from Koha's own service checks
- 'plugins' is a hash keyed by plugin class -- populated by companion bug
(plugin hook)
- Each check has 'warnings' and 'critical' arrays of human-readable messages
- Empty arrays = healthy

Header indicator logic:

- Red: any 'critical' list is non-empty (core or plugins)
- Yellow: any 'warnings' list is non-empty, no criticals
- Green: all lists empty
- Click navigates to detail view (new tab in about.pl System information, or
standalone page)

Core health checks to implement initially:

- Search engine (Elasticsearch/Zebra): cluster/daemon reachability, index
staleness
- Background jobs: oldest pending job exceeds threshold (workers not picking
up), failed jobs accumulating
- SFTP connections: configured EDI/acquisition targets failing to connect

Architecture:

1. Health check classes (Koha::Health::SearchEngine,
Koha::Health::BackgroundJobs, Koha::Health::SFTP):
   - Each implements check() returning { id, label, warnings => [...], critical
=> [...] }
   - Checks must be cheap -- query local state only

2. API endpoint GET /api/v1/health:
   - Runs all core checks
   - Calls health_check plugin hook (see companion bug)
   - Caches the full assembled response (core + plugins) in Memcached with 30s
TTL
   - A single cache key serves all users since health status is global
   - 60 terminals polling every 60s = ~1 req/s; with 30s cache only 2 actual
check runs per minute
   - Permission-gated

3. Staff interface:
   - JS poller in header.inc, hits endpoint every 60s
   - Renders summary dot with tooltip showing count of warnings/criticals
   - Detail view groups checks by source (core section, then per-plugin
sections)

Display considerations:

- Detail view could be a new tab on about.pl alongside existing system
information
- Or a standalone page -- either way, should render core and plugin sections
grouped
- Healthy checks still appear (as OK) so staff can see what is being monitored

Caching:

The endpoint caches the full assembled response in Memcached with a 30s TTL. A
single cache key (koha:health) serves all users since health status is global
-- not per-user or per-branch. 60 staff terminals polling every 60s means
roughly 1 request per second on average. With a 30s cache, at most 2 actual
health check runs happen per minute regardless of terminal count. Cache is
TTL-based only -- no explicit invalidation needed since 30s staleness is
acceptable for a status dot.

-- 
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.

[Bug 43137] New: Add system health monitoring with status indicators in the staff interface header

tags

participants (1)