> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getclean.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> getclean's own MCP server is at https://mcp.getclean.ai/mcp. The /mcp path of this docs site is Mintlify's search over these pages, not getclean.
> Every answer of the API opens with outcome. A page that could not be read is a record whose outcome says why, never an HTTP error; read outcome before anything else.
> The API's contract is the OpenAPI 3.1 file at https://docs.getclean.ai/openapi.yaml; the SDKs are generated from it.
> The product's name is written getclean, in lower case.

# A time machine for backtests

> Build a dataset in which every page is read from the last archive copy made on or before its trade date. Each row has the time its copy was made, a page without a copy is marked and not guessed, and model validation can check every row.

A signal built from web pages can do well in a backtest and lose money live. One cause is a page that was published or edited after the trade date and read as if it had been there before. Model validation then asks one question of every input: how do you know that this was visible on that day? This page builds a dataset in which every row has that proof.

## The run in 5 steps

### Read each page as of its trade date

In a batch, each address has its own day. One call reads 20 pages as of 20 trade dates.

```json Request body of fetch_contents_batch, cut to 2 of 20 items theme={"theme":"css-variables"}
{
  "items": [
    { "url": "SOURCE_URL", "as_of_date": "2026-03-03" },
    { "url": "SECOND_SOURCE_URL", "as_of_date": "2026-06-09" }
  ]
}
```

Replace `SOURCE_URL` and `SECOND_SOURCE_URL` with the addresses of the pages.

Each record is the latest copy that an archive made on or before the end of that day, in UTC.

### Store the time of each copy

The record says when its copy was made. That time, and not the day you asked for, is what your dataset stores. This part of a record is cut to the 2 times, and `archive` also has the name of the archive and the address of the copy.

```json Part of one record theme={"theme":"css-variables"}
{
  "source": {
    "carrier": "archive",
    "archive": { "captured_at": "2026-02-27T14:11:36Z" },
    "next_capture_at": "2026-03-09T06:02:10Z"
  }
}
```

The page of this record was seen on 2026-02-27 and again on 2026-03-09. What it said on the trade date, 2026-03-03, is the earlier copy if the page did not change in between, and the record does not claim more than that.

A copy from later on the trade date itself can come back. For a signal that trades at the open, ask for the day before.

### Tell a claimed date from an observed one

A page prints its own date, and `published_date` reports it with where it was read. That date is a claim of the page. A page that was edited last week can still print the date of its first version. Only the time of a copy is an observation, so a rule against look-ahead should use that time.

### Refuse a page that cannot be dated

If no archive has a copy on or before the day, the record's outcome is `failed` with the reason `no_history`, and it names the archives that were asked. Mark the row as excluded. The live page is not a substitute, because nothing shows that it said the same then.

### Find when a sentence appeared

To date one sentence, step through the copies. Each record names the time of the next copy. Send the sentence to each copy as a highlight with `deterministic_only` set to `true`. The last copy without the sentence and the first copy with it give an interval, and the sentence appeared inside it. The dataset stores both ends.

## What model validation can check

| Validation asks | What each row has |
| - | - |
| When was this text seen? | The time the copy was made. |
| Who made the copy? | The archive, by name, with the address of the copy. |
| Is the text complete? | The record's outcome, with its fingerprint to show later that the text is unchanged. |
| What about pages without a copy? | They are in the dataset as excluded, with the reason. |

## Know the limits

An archive has only the copies it made. Between 2 copies, the state of a page is not known.

A copy is the page as the archive served it. The country and the language of the visit cannot be chosen for a copy.

For the calls, see [Time machine: read a page as it was on a past day](https://docs.getclean.ai/read/time-machine.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.