UK merchandising weeks still have a Tuesday. Feeds land, titles get a seasonal prefix, and someone in engineering mentions “the model retrained.” Product then asks whether search got worse. Without a snapshot, you are holding a feeling.
In Query Signal Lab module 05 we freeze forty queries that already matter: your ugly strings, your hero categories, two competitor names people type anyway, and five queries that currently convert. Same device class if you can. Capture the top eight positions with SKU ids, not screenshots of a styled grid that will be restyled next month.
After the refresh, run the same forty. Drift is a moved SKU, a vanished colourway, or a dress that overtook a jacket because the title gained a token. You do not need NDCG to say that sentence. NDCG is useful later, with labels you actually collected. Most teams do not have those labels. Pretending they do is how ranking conversations become theology.
Date the two runs in the diary. If legal worries about storing queries, hash the session and keep the string. The string is the point. A board pack that says “relevance improved 0.04” with no query quoted is how you end up arguing about feelings again in March.
This habit will not stop a bad feed. It will stop you from discovering the bad feed in a customer complaint two weeks late.