AL+

Language: RO (română) EN (English)

Email us: paul@alplustech.com

Returned database statistics are application data too

DuckDB's DECIMAL statistics fix shows why metadata belongs in an application's correctness contract, even when the stored database values remain intact.

Citiți articolul în română

A database can store the right numbers while returning the wrong description of those numbers. Once an application uses that description to make a decision, metadata correctness becomes part of the application's contract. Calling a result a statistic does not make its consequences less real. The useful question is which decisions depend on it, and what happens when that dependency gives an incorrect answer.

DuckDB's 1.5.5 announcement on 22 July 2026 gives a narrow example: swapped minimum and maximum statistics for multi-row-group, 128-bit DECIMAL values in RETURN_STATS. Looking back in November, the lesson concerns the interface applications consume. It is not evidence that every decimal query returned incorrect rows or that the underlying values were corrupted.

Follow the description to its consumer

The original pull request, opened on 8 July, explains an incorrect comparison while combining statistics across row groups. The important boundary is between the written values and the returned file-level description. A patch discussion and a later release announcement are different events. Neither date establishes when a particular application's installed dependency changed.

Consider a hypothetical export service that records the returned bounds in a catalogue. Another component might use that catalogue to display a range or decide which files deserve inspection. Those are application decisions even if nobody directly opens the statistics response. The catalogue has carried the dependency onward. Correct data in the file cannot automatically repair a misleading description copied into another system.

That example is an architectural possibility, not a report of an affected customer. Exposure depends on using the affected feature, encountering the relevant data shape and relying on its output. Keeping those conditions together prevents dismissing metadata as harmless or treating a scoped statistics defect as general database unreliability. Both shortcuts obscure the work that actually needs investigation.

Give each field a meaning

A consumer contract should explain what a minimum describes. Is it the entire exported file, a selected subset, or a group inside that file? Is the value exact, approximate or unavailable? These questions should be settled before a caller uses the field. A numeric type alone cannot communicate the population over which a statistic was produced or the cases where it has no meaning.

Absence also needs a deliberate representation. A missing bound should not silently become zero merely because a dashboard expects a number. Zero can be a legitimate observation. If a caller cannot distinguish unavailable information from a measured value, a correct producer may still feed a misleading application. The return format and the consuming decision need to agree on that distinction.

Basic relationships provide useful checks. For an ordered, nonempty set of comparable values with both bounds present, the minimum cannot exceed the maximum. That relationship is worth checking where the application accepts statistics. It remains a limited check: two incorrect bounds can satisfy the ordering rule. Passing it does not establish that either bound describes the intended data.

Verify the dependency that matters

Useful regression evidence should connect the producer's output to the caller's purpose. If an application selects files using bounds, the relevant question is whether that selection remains correct for representative inputs. Merely showing that the database returned two decimal-looking strings misses the behavior being protected. Verification should include the conditions under which the caller is allowed to trust the description.

Multiple row groups matter to this particular defect. A tiny example with a single group would not address the same combination path. Representation matters too: the assigned fix concerns 128-bit decimals. A successful check of an unrelated numeric type cannot stand in for the affected case. These are reasons to choose relevant evidence, not a claim that AL+ ran a database reproduction.

A dependency update and a downstream repair are separate tasks. If incorrect metadata was persisted, upgrading the producer does not by itself replace earlier catalogue entries. A team should establish whether such entries exist, how they were generated and whether consumers still use them. Regeneration may be appropriate, but its scope should follow that evidence rather than an assumption that all historical exports are suspect.

When there is no application exposure

The strongest counterargument is straightforward: many applications never request or consume these statistics. If the affected output does not enter an application decision, this defect may create no application-level exposure there. A routine maintenance update can still be sensible without inventing a customer-impact story. Prioritisation should reflect the actual dependency path and the consequences of a wrong result.

Even an exposed application may use the values only as a visual hint. That differs from excluding files from a critical calculation. The same upstream defect can warrant different responses because consumers have different responsibilities. Documenting that distinction helps a maintainer choose between refreshing a display, rebuilding metadata and examining decisions already made from it.

Maintain the whole interface

For software maintenance, the practical unit of work is the dependency plus its consumers. The owner should be able to explain which returned fields the product trusts and where that trust ends. That explanation makes future patch notes easier to assess. It also prevents an upgrade from being mistaken for proof that every copied result has been corrected.

Returned statistics deserve the attention their uses demand. Keep the defect's scope narrow, follow the information beyond its first response and verify the decision that depends on it. Intact stored values and correct application behavior are related claims, but each needs its own evidence.