Skip to main content
A research agent learns from what a search and a read return. If the web changes between 2 rollouts, the same action gets 2 different observations, and the agent learns noise. If the reward comes from a judge model, the agent learns to please the judge. An environment for research needs 5 things that the live web does not give: observations that repeat, time fixed on one day, actions with consequences, feedback after each action, and a reward that a script can compute.

Make the observations repeat

A record is the stored copy of a source, and every read of it returns the same text with the same ids. Fetch the sources of a task once, and every rollout that reads them sees the same observation, in the same parts: the outline, a section, a table. These 3 fields keep the rest the same. A record is stored for 48 hours after its last use, and for at most 30 days. For a training run that pauses or runs longer, store the text of each record with its fingerprint on your side, and serve the environment from there.

Fix the time on one day

A task such as “what will the regulator decide?” is only fair if the agent cannot read what was published after the question was asked. as_of_date fixes the environment on one day: a fetch reads each page from the last archive copy made on or before that day, and a search asks each datasource that has history for its state on that day. This search asks one such datasource in 3 wordings, runs no model step, and fetches the outlines of its first 3 results.
Request body of search
Replace DATASOURCE_ID with the id of a datasource whose entry in the catalog has history set to true. A page without a copy on or before that day comes back with the reason no_history. That outcome is an observation too, and an agent can learn what to do with it. To step forward in time, a record names the time of the next copy of its page in next_capture_at, where a later copy exists.

Give the agent actions worth learning

A search tool with one field, the query, teaches an agent one skill. Here the request is the action, and it has many parts that a good researcher decides on. Each decision changes what comes back, so a policy has something real to learn.

Return feedback after every action

A sparse reward at the end of an episode says little about which search was the weak one. Every response here says how the action went.
  • A search says how many results each wording found on each datasource, what was dropped and why, why the list ended, and what could not be searched.
  • A fetch says what happened at the source before any content: delivered, partial, failed, refused, or in need of a person, each with its reason.
  • A filled template says of each field without a value why it has none: empty, absent, unread, or failed.
These parts of a response are structured fields, so they can be turned into reward terms or shown to the agent as its next observation.

Compute the reward without a judge model

A script can check these 5 things against the records. The first check costs nothing, so a reward can run it on every sentence of every rollout.

Keep text aimed at models out of the observations

A web page can hide text that only a model would read. Such text is returned apart from the content, with the way it was hidden. Your environment decides whether to show it: leave it out to train on what a reader sees, or put it in to test whether a policy follows a planted instruction.

Know the limits

A search of the live web can return other results on another day, because the datasources change. To give every rollout the same search, store the response of the search and serve it again. A search is stored for 48 hours after its last use, and for at most 30 days. as_of_date reads archive copies, and an archive has only the copies it made. A datasource can be searched as of a past day only if its entry in the catalog has history set to true. The rate limits of your key apply to rollouts. get_limits returns them.