getclean package calls the getclean API from Python. It’s generated from the API’s OpenAPI contract, so each of the 22 operations is a method of the client with the operation’s own name, its fields are keyword arguments with the API’s own names, and its answer is a typed object you read with dots. Every method’s docstring is the operation’s description from the contract, so help in your editor says the same thing as the API reference. On top of the generated code, a small hand-written layer does the things a research agent would otherwise write itself: it follows an upload’s job to the record, pages a long section, drops repeated items from a list, and retries a request whose first attempt is still running.
Create the client
GETCLEAN_API_KEY. If there’s no key there and you pass none, the constructor raises ConfigurationError before any request leaves, with the message “api_key is required: pass it to the client or set the environment variable GETCLEAN_API_KEY.” A missing key never looks like a network problem.
You can also pass the key, or a function that returns an access token:
https://api.getclean.ai. To point it elsewhere, pass base_url or set GETCLEAN_BASE_URL. The client holds a connection pool, so close it when you’re done, or use it as a context manager:
AsyncGetClean, with the same 22 methods as coroutines and the same hand-written helpers:
The first fetch, and how to read it
outcome first, every time. A page that couldn’t be read is still a record, returned normally, and its outcome says why. Nothing is raised for a failed page:
record.cost.actual.amount is "0.0000". Notices come right after the outcome and are worth a look before you use the result:
Requests and types
Every field of a request is a keyword argument. The ids from the path come first, positionally:client.get_contents(record.id, max_characters=0), client.get_section(record.id, "s2"). A field whose value is an object takes the SDK’s model of it, such as Money for a spending limit:
scope is a Literal type, and an open set such as outcome.reason is a plain string, so a value added in a later version arrives without breaking anything. A member that doesn’t apply to a record is None, and so is a member the API sends as null; the member’s description in the reference tells you which it can be.
The research loop
This is the loop a research agent runs, in Python: search, fetch the best result, find where it says the thing, read the rest for free, and keep a pointer that survives.Reading a record again
get_contents returns the same record in any format, at any size, under either view, with the highlights, summary and schema values the fetch produced. It’s free, so the record it returns has no cost:
iter_section walks them for you, one page of the section at a time, until there’s no more:
SectionContents, the same object get_section returns, so you can stop whenever you have enough.
Pagers
The seven list operations return a pager. Loop over it and it reads the next page when it needs to:response. A page of blocks, for example, repeats the record’s fingerprint and version:
list_links, list_datasources, get_search and compare_contents page the same way. On list_contents and list_jobs, a record or a job created while you page can make an item appear twice, and the SDK drops the repeat for you, by id.
Uploading a document
Pass the document as a path, as bytes, or as an open stream. A long document runs as a job, and withwait=True the SDK follows the job to the record:
wait_seconds, the API answers 201 and answer.contents is the record straight away. When it runs on as a job, the API answers 202, and with wait=True the SDK reads the job with get_job, which waits on the server for up to a minute per request, until the job is final, then reads the record under the upload’s own max_characters, formats and block_kinds. A job that fails before any record exists raises the error class of its problem’s status, the same class the API would have raised had it refused the upload directly. A job canceled before any page was done raises JobCanceled, which carries the job. A job still running after the API’s own limit of 30 minutes comes back as it stands, so you’re never stuck.
Without wait, you get the 202 as it is, with answer.job set, and you follow it yourself:
max_cost on every upload: a long scan is the most expensive request there is. An upload requires an idempotency key, and the SDK generates one when you give none. Turn PDFs, Office files, and scans into evidence you can cite by page covers what a document gives you.
Errors
A problem, the API’s own refusal, raises an error. Each HTTP status has its own class, named after the status, and the error carries the problem asbody:
body.type and not on the class alone: the eleven 422 types all raise UnprocessableEntityError. The error’s message starts with the problem’s title and detail, so a log line says what happened without unpacking anything. body.retryable says whether asking again can help, and body.change carries the fix when it’s data. Every class derives from ApiError, which also carries the status code and the headers. ApiError, ConfigurationError and JobCanceled are all importable from getclean. Errors lists every type.
Retries and the idempotency key
The SDK retries on its own when the API answers429, 502, 503 or 504: up to five attempts in all, with a backoff that starts at one second and doubles to a ceiling of 30 seconds, plus jitter. When the response has Retry-After within that ceiling, the SDK waits exactly that long. A Retry-After beyond it ends the retries, and the error reaches your code as it arrived, with retry_after_seconds in its body.
On the eight operations that take an idempotency key, the SDK generates one per request when you give none and sends the same key on every attempt, so a retry is never charged twice. That covers one more case: 409 idempotency-key-in-flight, which means the first attempt with this key is still running on the server. The SDK waits the retry_after_seconds the API asks for and sends the same request with the same key again, up to five attempts under the same ceiling, and once the first attempt has answered, the same key returns its response. Any other 409 is raised at once.
Give your own key when your program may repeat a request after a crash or a restart, so the second run finds the first response:
Budgets and time
Two arguments keep a research agent inside its budget.max_cost is the most one request may cost, and the API drops optional work from the end of a published order rather than refuse the request; cost.ceiling on the response shows the limit that applied. wait_seconds is how long one request may work on the server. Set a spending limit that no call can pass explains how the money is reserved and cut.
The client’s own timeout is a separate thing: it bounds one attempt on your side. It starts at 120 seconds. Before the first request that can be charged, the client reads your limits once and sets the timeout for each family to the API’s own call_seconds bound plus ten seconds, so a request gets the time the API itself allows and no more. A timeout you pass for one request in request_options stands.
Per-request options
Every operation accepts a W3Ctraceparent header, which getclean logs with the request and never forwards to a site or a source. It goes through request_options, together with a per-request timeout when one request needs more than the client’s default: