Writing custom functions

Custom functions extend project and deployment capabilities by letting you write Python code to perform specialized extraction, cleaning, validation, and integration. You can use custom functions to calculate values, connect to external APIs, implement complex validation logic, or transform data in ways specific to your business requirements.

Custom functions are available in these variations:

• Custom function fields

  Shared functions are a way to reuse custom functions across custom function fields.

• Cross-class custom function fields

• Cleaning functions

• Validation functions

• Cross-class validation functions

• Deployment integration functions

Supported libraries

The standard Python library is supported, except for the logging service. As an alternative to logging, you can use print() statements totaling up to 4 KB in length. If needed, you can split results into multiple print statements.

In addition to the standard Python library, you can import these external libraries at the version indicated.

• datetime 4.4

• dateparser 1.1.4

• fuzzywuzzy 0.18.0

• numpy 1.21.6

• pandas1.5.3

• regex 2022.10.31

• requests 2.25.1

• typing 3.6.2

• typing-extensions 4.5.0

For example:

1
import pandas as pd
2
print(pd.Series([1,2,3]))

Using keys in custom functions

Keys enable access to various types of runtime data, configurations, and sensitive values within custom functions.

• System keys — Built-in values about files, documents, users, and execution environments available through the context parameter. System keys are supported in custom function fields, cleaning functions, and validation functions, but not deployment integration functions.

  • context['document_text'] — Contains the text content of the current document.

  • context['file_path'] — Retrieves the path to the uploaded file in the underlying file system. Often used in conjunction with the filesystem API to access file content.

• Custom keys — Runtime variables you define for apps and deployments, accessed through the keys parameter using keys['custom']['<key-name>']. Values can change between environments. System keys are supported in custom function fields, cleaning functions, validation functions, and deployment integration functions.

• Secret keys — Organization-level encrypted values like API keys or credentials, accessed through the keys parameter using keys['secret']['<key-name>']. Secrets are managed by administrators and scoped to specific workspaces. System keys are supported in custom function fields, cleaning functions, validation functions, and deployment integration functions.

For custom and secret keys, when writing custom functions in an automation project, you specify test values to use during development. After you create an app, you replace test values with runtime values when you run the app.

When you create a deployment based on an app that uses custom or secret keys, you can define default runtime values for the keys. You can’t delete keys or modify key names or types that are defined in an app. Keys persist across app versions and remain in the deployment even if modified in newer app versions. In addition to app-specified keys, you can create custom and secret keys specifically for use in deployment integration functions.

When you run a deployment, you can use the runtime values specified in the deployment, or you can override the values.

Custom and secret keys used in app or deployment runs are included in logs. For secret keys, only the key names are displayed, not the actual secret values.

Secrets are automatically redacted when printed to logs with print(keys['secrets']['my_secret']). To bypass this protection, you can use print(keys['secrets']['my_secret'].get_value()).

Adding custom and secret keys in custom functions

You can add custom and secret keys in custom functions, including extraction, cleaning, validation, and integration functions.

1. From a custom function editor, click Show keys (project editor) or expand the Keys section (deployment integration functions).

2. Select the Custom keys or Secret keys tab, then click to add a key.

3. Specify details about the key:

   • Key — Specify a unique name for the key that indicates its purpose.

   • Test value — Specify a test value for use during development. Ideally, test values represent realistic production data. Test values are replaced with production values when you run the app or create a deployment that uses the app.

     For secret keys, you can select None, choose an organization-level secret available in the workspace, or enter a custom test value.

   • Type (Custom keys only) — If necessary, modify the custom key value type by clicking the type indicator (T by default) and selecting from available options: string, number, boolean, or JSON.

   • Description — Optionally select the overflow icon , then select Edit description. Specify details about the key that can help differentiate it and help other users understand its purpose, then click Save.

4. (Optional) Close or collapse the Keys pane to continue editing the custom function.

Example custom function

Here’s an example of a validation function that verifies file size and content.

The example uses a secret key to authorize a call to the filesystem API, retrieving the file using the system key context['file_path']. A custom key sets the maximum allowed file size. And the system key context['document_text'] is used to check that the file isn’t empty.

1
def validate_file_size(context, keys):
2
    """Validate that an uploaded file is under the maximum file size and isn't empty."""
3
    import requests
4
5
    API_ROOT = 'https://aihub.instabase.com/api'
6
    FILES_API_ENDPOINT = API_ROOT + '/v2/files/'
7
8
    # Get max file size from max_file_size_mb custom key
9
    max_size_mb = keys['custom']['max_file_size_mb']
10
11
    # Get file using file_path context variable
12
    url = FILES_API_ENDPOINT + context['file_path']
13
    params = {'expect-node-type': 'file'}
14
15
    # Use api_token secret key for API authentication
16
    headers = { 'Authorization': f'Bearer {keys["secret"]["api_token"]}' }
17
18
    response = requests.get(url, headers=headers, params=params)
19
    file_size_mb = int(response.headers['Content-Length']) / (1024 * 1024)
20
    if file_size_mb > max_size_mb:
21
        return (f'File size ({file_size_mb:.1f}MB) exceeds max size ({max_size_mb}MB)')
22
23
    # Get document content with document_text context variable
24
    if not context['document_text'].strip():
25
        return 'File is empty'
26
27
    # Both validations passed
28
    return None

Shared functions

Shared functions are custom Python functions you create one time and can then reuse across multiple fields and projects within an organization. Instead of duplicating function code in each custom function field, import a shared function wherever needed. Functions are stored at the organization level in the function library.

Users with developer permissions or higher in any workspace can create and update shared functions.

Follow these best practices when creating shared functions.

• Use descriptive names — Choose names that clearly indicate purpose, such as format_date or validate_postal_code.

• Document with descriptions — Add descriptions to help team members understand when and how to use each function.

• Test before publishing — Use test values to verify functionality before publishing.

• Version carefully — Test new versions thoroughly before switching existing fields to use them.

• Organize by purpose — Create functions for common operations like date formatting, data validation, or API calls.

Creating shared functions

Create shared functions from the function library.

1. In Workspaces, click Function Library > Create.

2. Specify function details:

   • Name — Enter a unique, descriptive name.

   • Description — Describe what the function does to help other users understand its purpose.

   • Arguments — Click Add argument to specify parameters. Enter argument names separated by commas.

   • Code — Write the Python function code. The function must accept the specified arguments and return a value.

   • Test values — Specify test values for each argument to validate the function before publishing.

3. Click Run code to test with your test values.

4. Click Publish function to save to the function library.

Using shared functions

After creating a shared function, import it into any custom function field within a project.

1. In a project, create or edit a custom function field.

2. In the custom function editor, click Show function library.

3. Select the function and click Import function.

4. Call the function directly with the appropriate arguments. The function code isn’t added to the field editor. If you need to modify the code, copy and paste it without importing it.

Updating shared functions

Shared functions are versioned, meaning any change is saved as a new version of the function. Fields referencing the function don’t automatically update to the latest published version, but can be manually updated.

1. In Workspaces, click Function Library. Select and open the function to edit.

2. Make your changes to the code, arguments, or other properties. Click Run code to test your changes.

3. When finished editing, click Publish function.

4. Select whether the change is a patch, minor, or major update, then click Add version.

Updating shared functions in projects

After updating a shared function, you can update projects that reference the function to use the new version. When you update the version for one custom function field in a project, the version is updated across all other usages in the project.

1. In a project, open the field editor for any custom function field using the shared function.

2. Expand the Imported shared functions panel and find the function to update.

3. Click the overflow icon and select Change version.

4. Select a function version, then click Update version.