The data-loss bug my tests could not see
It said “Processed 10 records.” The table was empty. Every test passed.
What happened
I had just finished a plugin that pulls data from an API into WordPress. The test suite was green — over a hundred assertions at that point, covering the mapping, the storage, the display. So I installed it on a real WordPress site, pointed it at a live API, and ran a sync.
It reported success: processed => 10. Then I looked in the database table, and there was nothing in it.
The cause, which is embarrassingly small
The storage layer wraps a batch of inserts in a database transaction, so a half-finished batch can be rolled back. It started that transaction like this:
$started = (bool) $wpdb->query( 'START TRANSACTION' );WordPress’s database wrapper returns, for a statement like this, the number of affected rows. START TRANSACTION affects zero rows. So it returns 0 — and (bool) 0 is false.
My code therefore believed no transaction had begun. So it never issued the matching COMMIT. MySQL had genuinely opened a transaction, dutifully accepted ten inserts into it, and then — when the connection closed at the end of the request — rolled the whole thing back, exactly as it is supposed to.
The fix is one character’s worth of thinking: only an explicit false means failure.
$started = ( false !== $wpdb->query( 'START TRANSACTION' ) );Why every test passed anyway
This is the part worth keeping.
Inside a transaction, the connection that opened it can see its own uncommitted rows. That is how transactions are meant to work. So within a single request — insert, then read back, then assert — everything looked perfect. My tests inserted records and immediately found them, because they were the same connection asking.
The data only vanished at a boundary the tests never crossed: the end of the request. And the fake database I used in tests applied every insert immediately, so it had no concept of “committed” versus “merely written”. My test double was more forgiving than reality, which meant it was quietly lying to me.
The change that would have caught it
I rewrote the test double to model what MySQL actually does: writes inside a transaction go to a pending buffer, only COMMIT moves them into the real set, and a new method — simulate_disconnect() — throws the pending buffer away, just as closing a connection does.
Then the regression test reads like the bug itself:
- Run a sync. Assert it says two records were processed.
- Assert a
COMMITwas actually issued. - Simulate the connection closing.
- Assert the records are still there, and that a later request can read them.
Step three is the whole test. Without it, steps one, two and four all pass on broken code.
What I take from it
The lesson isn’t “check return types”, though I do now. It’s that a test double is a theory of how the real system behaves, and when the theory is kinder than reality, your tests measure your optimism.
The bug had been shipped in an earlier version, and would have silently discarded data for every user who installed it. What caught it was not a smarter test — it was thirty seconds of looking in the actual table on an actual site, after the code had already told me it was fine.
That habit has now paid for itself several times over. If a feature has never been observed working from outside the process that claims it works, it isn’t finished.