The first week of reading 45 pricing pages every morning
What a daily check actually runs into: pages that fill in prices after you stop looking, sliders that hide most of the ladder, bot walls, a captcha that pretends to be a page, an archive that will not take deliveries, and a schedule that fires twelve hours late.
Daily tracking started on 23 August. The idea is simple: every morning, open 45 pricing pages from a server in the United States, find the prices seen the day before, and log anything that moved. Here is what that simple idea ran into in its first six days.
Pages that fill in the prices after you stop looking
Cloudways delivers plan cards with placeholder numbers and swaps in the real ones a moment later. Reading too early produced five days of wrong prices on this site (the whole story). Mailchimp prices by contact count through a dropdown; the page shows one band at a time, so a script that reads once sees one price out of forty-four. The check now drives the dropdown and reads every band, and Mailchimp's rows on the email calculator no longer carry an "uncaptured slider" warning. Flodesk does something similar with a cross-fade animation, and the full ladder, 64 tiers, is now read from the page's own matrix.
Walls
InterServer, UltaHost and Kit answer the script with a Cloudflare interstitial. GoDaddy answers with Akamai's Access Denied. SiteGround sometimes answers with a 182-byte page whose only content is a redirect to its own captcha, which for a while the check misread as "the prices changed" rather than "I was shown a door". I do not try to get past any of these. What I do is ask the Internet Archive for its newest copy of the same page and check the prices against that, saying so on the company page. That is how GoDaddy got onto the site. InterServer, UltaHost and Kit have no recent archive copies, so their pages say "blocked our check" and their last-verified dates stay honest.
A page that moved
EasyWP's homepage stopped printing prices and started sending visitors to two other sites. The check said "changed" every day until I looked (what I found).
The archive is not a service you can rely on either
I ask the archive to save each source page so that every price on this site has a receipt someone else holds. This week the archive's save endpoint refused or timed out on 9 of every 10 requests, from the US server and from here. So a receipt is now the newest capture the archive already holds, dated, which for most pages is within the last few days, and the receipts are kept in the same append-only history file as the prices instead of a database that gets rebuilt every morning, which is where the first batch of them quietly disappeared.
The schedule
The morning job is scheduled for 06:17 UTC. On 28 August it ran at 18:45 UTC, twelve and a half hours late, on GitHub's clock rather than mine, and landed on top of a run I had started by hand. Both tried to write the day's history at once and one of them lost. Runs now queue behind each other, the history file merges by appending, and there is a second scheduled slot in the afternoon as a backstop.
The tally
This morning: 40 of 45 pages re-verified, one of them through the archive, three blocked, none awaiting a second look. That line is on the front page every day, computed from the run, because a site that says "checked daily" should be able to show its homework.
The figures in this post are what the pages said on the dates given. The tables linked from it are re-read every morning and may have moved since; the changes log has every move with a date and a receipt.