> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.instabase.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.instabase.com/_mcp/server.

# Calling LLMs from custom functions in flows

> Call an LLM from Python in flow legacy scripts using CLIENTS.llm_client, registration, and step configuration on single-tenant deployments.

Enterprise Single-tenant

In single-tenant environments with advanced view enabled, you can call an LLM from custom function code to get text or structured output—for example, to drive extraction, classification, or validation with AI. Implement Python in the project's **legacy scripts** folder or module, register entrypoints, and reference functions by name in the step configuration. Model and provider come from the tenant's configured LLM provider and the AI runtime model.

Calling LLMs from custom functions works differently in the app editor, see [Calling LLMs from custom functions](/automate/llms-custom-functions).

## Availability

In the flow editor, `llm_client` is supported in these steps and event hooks:

* [Pre-flow UDF](/flow/custom-functions/custom-functions-flow#pre-flow-udf)

* [Post-flow UDF](/flow/custom-functions/custom-functions-flow#post-flow-udf)

* [Apply classifier](/flow/step-config-reference/apply-classifier)

* [Apply refiner](/flow/step-config-reference/apply-refiner)

* [Map UDF](/flow/custom-functions/custom-functions-flow#map-udf)

* [Reduce UDF](/flow/custom-functions/custom-functions-flow#reduce-udf)

> **Note**
>
> The *Agent classifier* and *Agent extract* steps natively support calling LLMs.

### Design considerations

When you use the LLM client in flow steps, consider the following:

* The LLM client bypasses the platform's built-in guardrails. Use it only where the dedicated agent steps can't meet the requirement -- for example, where custom preprocessing logic must run before an LLM call, or where an external data source must be queried and incorporated into a prompt. In all other cases, use the *Agent classifier* and *Agent extract* steps.

* Contain LLM client calls to a single, well-defined step. Don't scatter custom LLM calls across multiple UDFs throughout the flow.

* When you call an LLM directly through the client rather than through a dedicated agent step, the following don't apply automatically:

  * **Grounding** -- The model isn't constrained to return values from the source document. Enforce grounding through careful prompting. For example, if using the LLM client for classification, the prompt must explicitly instruct the model to return only one of the defined class names and handle cases where the document doesn't match any category.

  * **Schema validation** -- The platform doesn't validate that LLM output matches an expected structure. Output parsing and error handling are the developer's responsibility. Error handling must account for both external API failures and LLM output validation failures.

  * **Rate limit management** -- Multiple LLM client calls across a flow at high document volume can quickly exhaust Azure OpenAI or Vertex AI rate limits, causing flows to fail.

## Using the LLM client in flow steps

Calling an LLM from a flow step involves the following components and processes.

* **Legacy scripts** -- Add your Python function to the project's legacy scripts folder or module, then reference the function by name in the step configuration.

* **Registration** -- Decorate the entrypoint with `@register_fn(provenance=False)` from `instabase.provenance.registration` so the platform can register and invoke it.

* **Clients** -- Declare `CLIENTS` as a positional argument in your function signature. The flow runtime passes it in automatically. Use attribute access to reference clients -- for example, `CLIENTS.llm_client` and `CLIENTS.ibfile` -- rather than dict-style access with `.get('llm_client')`.

* **LLM usage** -- Call `generate_content` on `llm_client` using the same parameters described in the **Call generate\_content** subsection below.

```python
from instabase.provenance.registration import register_fn


@register_fn(provenance=False)
def llm_step(CLIENTS, *args, **kwargs):
    llm_client = CLIENTS.llm_client
    resp_text = llm_client.generate_content(
        prompt="<Task and source text the model must use.>",
    )
    return resp_text
```

When you wire the function in the flow--for example for a pre-flow or post-flow event--invoke it by passing `CLIENTS` according to your step's formula, such as `llm_step(CLIENTS)`. Use the function name you registered and the variable order your step expects (see [Custom functions in flow](/flow/custom-functions/custom-functions-flow)).

### Get llm\_client from CLIENTS

When the LLM client is available, the `CLIENTS` object exposes an LLM client (`llm_client`) and a file client (`ibfile`). Use the file client's `read_file` to read document content. Use the LLM client's `generate_content` method to send a prompt to the model and get a response (optionally with file data or a response schema for structured output). If the LLM client isn't available for a given run (for example, the step doesn't support it), guard the call before use, for example `if CLIENTS.llm_client is None`.

### Call generate\_content

When you have the client, call `generate_content` for either text-only or file-aware generation:

```python
generate_content(
    prompt='str'
    file_data=None,
    mime_type=None,
    file_path=None,
    response_schema=None,
    enable_thinking=True,
    enable_logprobs=False,
)
```

| Parameter         | Required? | Type    | Description                                                                                                          |
| ----------------- | --------- | ------- | -------------------------------------------------------------------------------------------------------------------- |
| `prompt`          | Yes       | string  | The text prompt to send to the model.                                                                                |
| `file_data`       | No        | bytes   | File data. Must be provided with `mime_type`. Default: None.                                                         |
| `mime_type`       | No        | string  | The multipurpose internet mail extensions (MIME) type of the file. Must be provided with `file_data`. Default: None. |
| `file_path`       | No        | string  | The file path. Default: None.                                                                                        |
| `response_schema` | No        | dict    | Schema for structured output. Default: None.                                                                         |
| `enable_thinking` | No        | boolean | Whether to enable thinking/reasoning mode. Default: True.                                                            |
| `enable_logprobs` | No        | boolean | Whether to include log probabilities (per-token confidence scores from the model) in the response. Default: False.   |

## Example: Map UDF with structured output

The following example uses `@register_fn`, takes `CLIENTS` as a positional argument together with *Map UDF* variables, reads a file with `CLIENTS.ibfile`, and writes model output into `out_files`. Set the *Map UDF* step formula to match your declared parameters—for example `map_with_llm(INPUT_RECORD, STEP_FOLDER, CLIENTS)`.

```python
import json

from instabase.provenance.registration import register_fn


@register_fn(provenance=False)
def map_with_llm(input_record, step_folder, CLIENTS, *args, **kwargs):
    if CLIENTS.llm_client is None:
        return {"out_files": []}

    input_filepath = input_record["input_filepath"]
    file_content, err = CLIENTS.ibfile.read_file(input_filepath)
    if err:
        raise IOError("Could not read file {}".format(input_filepath))

    resp = CLIENTS.llm_client.generate_content(
        prompt="Return the deductions table. Remove any extra symbols from the amount deducted.",
        file_data=file_content,
        mime_type="application/pdf",
        file_path=input_filepath,
        response_schema={
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "description": {"type": "string"},
                    "amount deducted": {"type": "float"},
                },
            },
            "description": "deductions table",
        },
    )

    if isinstance(resp, (dict, list)):
        payload = json.dumps(resp).encode("utf-8")
    else:
        payload = str(resp).encode("utf-8")

    return {
        "out_files": [
            {
                "filename": "deductions.json",
                "content": payload,
            }
        ]
    }
```

Adjust filenames, MIME type, schema, formula, and return shape to match your step.