Skip to main content
A compliance team at a bank watches 60 pages of 4 regulators. A tool that compares whole pages sends an alert for a new banner, a new date in the footer, and a menu in another order, and after a month nobody opens the alerts. Then a regulator lowers one threshold in one table from 10% to 8%, and the alert for it looks like all the others. This page builds a watch that is silent while the text is the same and exact when it is not.

The run in 5 steps

Fetch the watch list with the fingerprints of the night before

Every record has a fingerprint, which is a hash of its text. Send each page with the fingerprint of its last record. A batch takes 20 pages, so 60 pages are 3 calls.
Request body of fetch_contents_batch, cut to 3 of 20 items
Replace each placeholder that ends in _URL with the address of a page, and each that ends in _FINGERPRINT with the fingerprint of that page’s record from the night before.

Skip what is unchanged

The response starts with a tally, and each result says how its fetch ended. This response is cut to the tally and to the outcome of each record.
Part of the response
Of the 3 pages, 2 have the text of the night before, and your agent drops them. The page on capital requirements has another text, and its new record came back as an outline. A new banner, a new footer, or a changed menu does not make a page count as changed. The fingerprint is a hash of the page’s main content, and those parts are not in it.

Compare the changed page with its last record

Send the new record to compare_contents, with the id of the record from the night before.
Request body of compare_contents
Replace BASELINE_CONTENTS_ID with the id of the earlier record of that page. The comparison lists what changed, block by block. This response is cut to the fields that say what changed.
Part of the response
The alert that your team gets has 2 lines: one cell of one table went from 10% to 8%, and one sentence is new. No model made this comparison, so the same 2 records give the same list on every call.

Read the row of the changed cell

A cell alone is a number. To say which threshold it is, read the table of the new record and take the row with its headings. The table comes cell by cell, so the alert can say “the large exposure limit for unrated counterparties” and not “row 3, column 2”.

Quote from the version that binds

A regulator in a country with 2 official languages publishes each rule twice. A record lists the page’s own versions in other languages, and where the page declares it, which version binds.
Part of the record
Watch the version that binds, and quote from it. authoritative is there only where the page itself declares which versions bind.

What the compliance officer can check

Know the limits

A watch runs when your code calls. No call watches a page for you. A record is stored for 48 hours after its last use and for at most 30 days. For a page that is fetched each night, the record of the night before is still stored. For a longer history, store the Markdown of each record with its fingerprint, and send both as the baseline of a later comparison. A fingerprint and a comparison work only between 2 records of one version of getclean. When a new version comes, send the earlier one as pinned_version and the watch goes on as before, for as long as that version is served. Or fetch every page once under the new version, and start from those records. The comparison says which words changed. What the change means for the bank is for your agent and your lawyers to say. For the calls, see Find the one cell that changed and Fetch 20 sources at once and know what came back before you read a word.