Mimir v1.0.25

There's been many updates since Mimir 1.0.2, this is just an overview:

Mimir 1.0.2 → 1.0.25: hardening a tiny database for real WordPress

Mimir is a ~1 MB MySQL-wire-compatible server written in C, running in a scratch container next to the sites it serves. Everything since 1.0.2 has chased one goal: never fall over on a production database.

The releases below weren't driven by a roadmap - they were driven by mysqldump files from live sites. The ones with a wp_options table bloated to a hundred megabytes, a decade of post revisions, plugin schemas full of things the SQL parser had never seen, and every so often a genuinely corrupt page left behind by an old crash.

Read back to front, a few clear arcs emerge: no single query can pin a CPU core anymore; the SQL surface grew to cover what WordPress and its plugins actually emit; databases dump and restore through the standard tooling; connections can be encrypted; and the storage engine learned to reclaim its own bloat.

The five big arcs

• Nothing pins a core anymore. Corrupt B-tree pointers, cyclic overflow chains, MVCC tombstone bloat, and an oversized row wedged mid-leaf could each spin one thread at 100% CPU while holding the write lock - freezing every other connection. Each was found on a real site and bounded for good. (1.0.18, 1.0.23, 1.0.24, 1.0.25)
• The SQL WordPress actually writes. CTEs and ROW_NUMBER(), advisory locks, the BINARY operator, UTC time functions, CHECK constraints, two-word foreign-key actions, quoted numeric lookups, columns literally named count - the surface grew to match what core and plugins emit. (1.0.3, 1.0.9, 1.0.12, 1.0.16, 1.0.17, 1.0.20, 1.0.22)
• Dump and restore, for real. Full mysqldump / mariadb-dump round-trips: SHOW CREATE TABLE, executable comments, importer comment handling, multi-statement rejection, and a lexer that no longer needs gigabytes to parse a big multi-row insert. (1.0.8, 1.0.10, 1.0.11, 1.0.13, 1.0.15)
• A store that heals itself. MVCC leaves dead row versions behind under churn; a busy wp_options could bloat until a cleanup query went quadratic. Mimir now compacts tombstoned tables on startup and online, and seeks the transient-cleanup self-join through an index. (1.0.24)
• Encrypted connections, bounded memory, visible internals. In-band StartTLS with a zero-config cert; a commit log that no longer allocated gigabytes; scans that free scratch as they go; coalesced writes; indexed prefix LIKE; and SHOW MIMIR STATUS to account for every byte. (1.0.2, 1.0.4–1.0.7, 1.0.19)

Every release, newest first   

1.0.25 - big rows & spatial columns

• Fixed: a large row wedged between large rows spun the B-tree split forever. When three biggish rows landed adjacent in a leaf, no two-way split fit and the balance loop oscillated at 100% CPU. The split is now bounded, and only the one wedging value spills to overflow - ordinary large values stay inline, no space cost. Surfaced by a 74 MB import that pinned CPU and never finished.
• Added: spatial column types (POINT, POLYGON, GEOMETRY, …) import as blobs, so WP Google Maps tables load and their bytes round-trip.

1.0.24 - self-healing storage

• Fixed: MVCC tombstone bloat wedged churn-heavy tables. Rows deleted while any snapshot was open kept their B-tree key forever, so a transient-churning wp_options bloated until the cleanup self-join went quadratic. Startup compaction now rebuilds bloated tables into fresh trees - an in-place upgrade self-heals on first restart.
• Added: online auto-vacuum - a badly-bloated table is also compacted while the server runs, so a long-lived instance never re-bloats between restarts.
• Added: index-assisted self-join - WordPress's expired-transient cleanup DELETE a, b FROM wp_options … now point-looks-up the inner row instead of a full inner scan (O(n²) → O(n log n)).

1.0.23 - anti-hang + slow-query visibility

• Fixed: a corrupt overflow chain spun one thread at 100% CPU. Freeing an overflow value followed the chain with no limit; a cyclic pointer looped forever on the write path. The walk is now bounded by the page count.
• Added: in-flight slow-query log. --slow-query-ms (and --debug at 2 s) reports a query while it's still running, so a stuck or slow query surfaces without waiting for it to finish.

1.0.22 - plugin features that were hammering the server

• Fixed: advisory locks (GET_LOCK/RELEASE_LOCK/…), the BINARY operator, and SHOW COLUMNS … WHERE. Three unimplemented features broke plugin logic and made them repeat expensive work every request, pushing some sites to 80%+ CPU. The locks are now real process-global named locks; the rest now parse.

1.0.21 - FULLTEXT / SPATIAL indexes

• Fixed: FULLTEXT/SPATIAL indexes are parsed and ignored. Mimir was building them as ordinary B-trees over the whole column, so a multi-KB value failed with "row too large" (e.g. WPML's mail log).

1.0.20 - import parse & NULL fixes

• Fixed: two-word foreign-key actions (ON DELETE SET NULL) and CHECK constraints now parse, and a UNIQUE index allows many NULLs (e.g. WooCommerce's wp_wc_customer_lookup). Real duplicates are still rejected.

1.0.19 - TLS + memory & data-loss fixes

• Added: TLS via in-band StartTLS. Clients requiring SSL now connect. Zero-config ephemeral self-signed cert (or supply a PEM pair); mbedTLS statically linked, so the image stays ~1 MB. On by default in the container.
• Fixed: large scans/dumps exhausted memory - a 300k-row / 60 MB table now dumps and reloads peaking at ~160 MB instead of multiple GB.
• Fixed: multi-table DELETE ignored the JOIN condition. DELETE t FROM t JOIN u ON … was a cross product that wiped the target table; ON is now evaluated first.

1.0.18 - backward-compat + traversal guards

• Fixed: databases from ≤1.0.14 loaded a corrupt schema and hit 100% CPU. 1.0.15 changed the on-disk index format without a compat path; the reader now detects the format and parses old and new correctly.
• Hardened: B-tree traversals are bounded by the page count - a cyclic pointer returns a storage-corruption error instead of looping forever.

1.0.17 - CTEs, window numbering, index hints

• Added: WITH, ROW_NUMBER() OVER, and USE/FORCE/IGNORE INDEX. Together they power Yoast SEO's sitemap query, which had failed as an unsupported statement.

1.0.16 - UTC time functions

• Added: UTC_TIMESTAMP() / UTC_DATE() / UTC_TIME(). Gravity Forms and others timestamp in UTC; these had failed as unknown functions.

1.0.15 - the big import-hardening release

• Fixed: large dump imports OOM-killed the server. The lexer sized every string literal to the whole remaining statement - O(statement²). A ~1 MB / 8,000-row insert peaked at 7.6 GB RSS; it now peaks at ~49 MB. Memory and time are linear in dump size.
• Fixed: prefix indexes are honored - KEY (meta_value(32)) no longer indexes the whole multi-KB value and rejects the row as "row too large".
• Fixed: backtick reserved words (columns named key/value), index options (USING BTREE), >16 MB packets (verified to 159 MB), charset introducers (_binary'…'), and per-row scan scratch. Threads also got an explicit 8 MB stack.

1.0.14 - real table sizes

• Fixed: SHOW TABLE STATUS reports real DATA_LENGTH/INDEX_LENGTH. phpMyAdmin/Adminer had shown 16 KB for every table; sizes now scale with the data at no extra cost.

1.0.13 - multi-statement packets

• Fixed: a multi-statement packet is rejected, not silently truncated. An importer posting a whole .sql file in one query used to get "success" while importing nothing (the leading SET matched a shortcut).

1.0.12 - quoted-number lookups (data-correctness)

• Fixed: WHERE int_col = '19' returned zero rows on an indexed column. PHP and WordPress routinely quote integers, so this returned wrong results, not errors. Verified by index-vs-full-scan differential testing.

1.0.11 - importer comment handling

• Fixed: leading -- / # / /* … */ comments are skipped before dispatch. Adminer/phpMyAdmin leave a dump's header comments on the first statement, which had failed the whole import.

1.0.10 - duplicate index names (dbDelta)

• Fixed: ADD KEY rejects a duplicate name with error 1061. WordPress/WooCommerce re-run index creation on every activation and rely on that error to be idempotent; Mimir had been silently stacking copies of each key.

1.0.9 - function-name columns

• Fixed: columns named count, sum, … now parse (they're aggregates only when followed by (). WordPress's wp_term_taxonomy.count had blocked dumping any WooCommerce install.

1.0.8 - SQL dump support

• Added: mariadb-dump / mysqldump round-trips. SHOW CREATE TABLE reconstructs loadable DDL, MySQL executable comments are unwrapped, and dumps reload into both stock MariaDB and a fresh Mimir.

1.0.7 - indexed prefix LIKE

• Performance: col LIKE 'prefix%' uses an index range scan. The wp_options transient sweeps WordPress hammers went ~45× faster at 8k rows and ~470× at 50k. The full LIKE is still re-checked on candidates, so results stay exact.

1.0.6 - counters & DB size

• Added: lifetime counters and on-disk size in SHOW MIMIR STATUS (total_queries, total_connections, uptime_seconds, db_bytes/wal_bytes) - enough for an external poller over the same socket.

1.0.5 - memory accounting

• Added: SHOW MIMIR STATUS and a SIGUSR1 memory dump - resident vs accounted memory and the gap, broken down by pool / scratch / commit-log / WAL, plus the transaction horizon. The signal path works even when the server is saturated.

1.0.4 - Site Health SQL + a critical memory fix

• Fixed: the commit log no longer grows with the lifetime transaction count. Indexed by absolute txid, it allocated a multi-GB array once the persisted counter reached the billions - one WordPress container held ~1.8 GB against a 20 MB database. Now indexed relative to a moving base; memory stays flat. Recommended upgrade for any long-lived deployment.
• Added: DATABASE(), USER() in expressions, and quoted-string aliases, so WordPress Site Health's database-size query parses.

1.0.3 - USING joins + version visibility

• Added: JOIN … USING (col) - WooCommerce's product-filter joins parse.
• Added: @@version_comment reports Mimir <version>, so you can confirm which build a server runs over SQL. VERSION() stays 8.0.35-Mimir so WordPress version_compare() is unaffected.

1.0.2 - hot-path performance

• Performance: coalesced result writes (one write() per statement), concurrent-read scaling (the transaction manager's global lock became a read/write lock - +20% point-read throughput at 8 clients), and a deferred checkpoint that makes inserts ~18% faster with half the per-commit write volume.

Every server reports its build over SQL: SELECT @@version_comment; returns Mimir 1.0.25.