Skip to main content
Every call after a fetch or an upload takes the id of a record, which starts with cnt_, and reads the stored copy. So the source is visited once, and every reader of the record sees the same text.

What a record has

A record is made once, and its facts do not change after that. A source that could not be read is a record too. Its outcome says what happened and what can work next, and it has no content. For the outcomes, see Outcomes and notices.

The same source gives the same record

Under one version and one configuration of getclean, the same page or the same file gives the same text and the same ids of sections and blocks, on every fetch. That sameness is what lets a place in a record be cited: the id of a block means the same passage for everyone who reads the record.
The fingerprint makes the sameness checkable. It is the SHA-256 hash of the record’s whole Markdown, written as sha-256=:VALUE: with the value in Base64. Anyone who reads that whole text with get_contents can compute the same value: in the view the fetch stored, with no block_kinds, no highlight marks, and no front matter, after Unicode normalization to NFC and with LF line ends. A later fetch that returns the same value has the same text. Fingerprints can be compared only within one view, one version, one configuration, and one kind of source, which is a live page, an archive copy, or an upload.

A page has 2 views

A web page is stored with everything that was on it, and the field scope chooses the view. The view main is the default: the page’s main content alone. The view whole_page has every part of the page, the menus and the footer included. Both views come from the same fetch, so asking for the second one later does not visit the source again. A document has one view, the whole document.

How long a record is stored

A record is stored for 48 hours after the last call that used it, and for at most 30 days after it was made. Each call on the record moves the 48 hours forward, and expires_at in a response has the moment. What you need for longer than 30 days, you store yourself, with the fingerprint. After a record expires, a call on it returns the problem contents-expired for 30 days and not-found after that. A request that sends retention as none stores nothing after its response. The policy of an account can make that the default.

What costs nothing

Reading a stored record costs nothing, in any format and at any size: the whole record, one section, the blocks of one kind, the links, and a comparison with another record. A highlight that is found word for word costs nothing either. A summary, a filled schema, or the highlights that a model has to judge cost one model step each, the highlights once per call.

A search is stored too

A search makes a record of its own, with an id that starts with srch_. It has the whole list of results with the receipt, and it is stored by the same rule: 48 hours after its last use, and at most 30 days. For reading it again, see See what a search covered and what it missed.