getclean package calls the getclean API from TypeScript and JavaScript, in Node and in the other runtimes that have fetch. It’s generated from the API’s OpenAPI contract, so each of the 22 operations is a method of the client named in camelCase, such as fetchContents, taking one request object whose members keep the API’s own names, such as max_characters. The answer is typed with the record’s members, and a member that only sometimes applies is optional, so the compiler makes you check before you read it. Every method’s JSDoc is the operation’s description from the contract. On top of the generated code, a small hand-written layer does the things a research agent would otherwise write itself: it follows an upload’s job to the record, pages a long section, drops repeated items from a list, and retries a request whose first attempt is still running.
Create the client
apiKey, the client reads your API key from GETCLEAN_API_KEY. If there’s no key there and you pass none, the constructor throws right away, with the message “apiKey is required: pass it to the client or set the environment variable GETCLEAN_API_KEY.” A missing key never looks like a network problem.
You can pass the key, or a function that returns an access token:
https://api.getclean.ai. To point it elsewhere, pass baseUrl or set GETCLEAN_BASE_URL.
The first fetch, and how to read it
outcome first, every time. A page that couldn’t be read is still a record, returned normally, and its outcome says why. Nothing is thrown for a failed page:
offer.cost?.actual.amount is "0.0000". Notices come right after the outcome and are worth a look before you use the result:
Requests and types
Every request is one object, including the ids from the path:client.getContents({ contents_id: record.id, max_characters: 0 }), client.getSection({ contents_id: record.id, section_id: "s2" }). A field whose value is an object takes an object literal with the API’s names:
GetClean namespace, and a closed set is both a type and a set of values, so GetClean.Candidate.DroppedReason.FoldedCopy is a value you can compare against. An open set such as outcome.reason is typed as a string, so a value added in a later version arrives without breaking anything.
The optional members follow the API’s own rule. A record always has id, outcome, notices, fetched_at, truncated, source, execution, retention, run_id, expires_at and request_id. Everything else is there only while its condition holds, and the SDK types it with ?, so with strict null checks on you check before you read. A member the API sends as null is typed | null, which is a different thing from absent: a candidate’s relevance_score is number | null, and it’s null under deterministic_only.
The research loop
This is the loop a research agent runs, in TypeScript: search, fetch the best result, find where it says the thing, read the rest for free, and keep a pointer that survives.Reading a record again
getContents returns the same record in any format, at any size, under either view, with the highlights, summary and schema values the fetch produced. It’s free, so the record it returns has no cost:
iterSection walks them for you, one page of the section at a time, until there’s no more:
SectionContents, the same object getSection returns, so you can break out whenever you have enough.
Pagers
The seven list operations return a page that you can iterate withfor await, and it reads the next page when it needs to:
response. A page of blocks, for example, repeats the record’s fingerprint and version:
listLinks, listDatasources, getSearch and compareContents page the same way. On listContents and listJobs, a record or a job created while you page can make an item appear twice, and the SDK drops the repeat for you, by id.
Uploading a document
document takes a path, a Buffer, a Blob or File, or a stream. A long document runs as a job, and with wait: true the SDK follows the job to the record:
wait_seconds, the API answers 201 and answer.contents is the record straight away. When it runs on as a job, the API answers 202, and with wait: true the SDK reads the job with getJob, which waits on the server for up to a minute per request, until the job is final, then reads the record under the upload’s own max_characters, formats and block_kinds. A job that fails before any record exists throws the error class of its problem’s status, the same class the API would have thrown had it refused the upload directly. A job canceled before any page was done throws JobCanceled, which carries the job. A job still running after the API’s own limit of 30 minutes comes back as it stands, so you’re never stuck.
Without wait, you get the 202 as it is, with answer.job set, and you follow it yourself. The call’s raw response says which status you got:
max_cost on every upload: a long scan is the most expensive request there is. An upload requires an idempotency key, and the SDK generates one when you give none. The file is read into memory before it’s sent, so that a retry can send the same bytes. Turn PDFs, Office files, and scans into evidence you can cite by page covers what a document gives you.
Errors
A problem, the API’s own refusal, makes the call throw. Each HTTP status has its own error class under theGetClean namespace, and the error carries the problem as body:
body.type and not on the class alone: the eleven 422 types all throw GetClean.UnprocessableEntityError. The error’s message starts with the problem’s title and detail, so a log line says what happened without unpacking anything. body.retryable says whether asking again can help, and body.change carries the fix when it’s data. A status the contract doesn’t declare throws the plain GetCleanError. JobCanceled is exported from the package and under the namespace as GetClean.JobCanceled. Errors lists every type.
Retries and the idempotency key
The SDK retries on its own when the API answers429, 502, 503 or 504: up to five attempts in all, with a backoff that starts at one second and doubles to a ceiling of 30 seconds, plus jitter. When the response has Retry-After within that ceiling, the SDK waits exactly that long. A Retry-After beyond it ends the retries, and the error reaches your code as it arrived, with retry_after_seconds in its body.
On the eight operations that take an idempotency key, the SDK generates one per request when you give none and sends the same key on every attempt, so a retry is never charged twice. That covers one more case: 409 idempotency-key-in-flight, which means the first attempt with this key is still running on the server. The SDK waits the retry_after_seconds the API asks for and sends the same request with the same key again, up to five attempts under the same ceiling, and once the first attempt has answered, the same key returns its response. Any other 409 is thrown at once.
Give your own key when your program may repeat a request after a crash or a restart, so the second run finds the first response. It’s a member of the request, named like the header:
Budgets and time
Two members keep a research agent inside its budget.max_cost is the most one request may cost, and the API drops optional work from the end of a published order rather than refuse the request; cost.ceiling on the response shows the limit that applied. wait_seconds is how long one request may work on the server. Set a spending limit that no call can pass explains how the money is reserved and cut.
The client’s own timeout is a separate thing: it bounds one attempt on your side. It starts at 120 seconds. Before the first request that can be charged, the client reads your limits once and sets the timeout for each family to the API’s own call_seconds bound plus ten seconds, so a request gets the time the API itself allows and no more. A timeoutInSeconds you pass for one request in its options stands.
Per-request options
Every operation accepts a W3Ctraceparent header, which getclean logs with the request and never forwards to a site or a source. It goes in the options after the request, together with a per-request timeout when one request needs more than the client’s default: