Packet-processing flows
Packet-processing flows
This guide covers how to manually build packet-processing flows when your automation project defines cross-class fields — that is, packet-level fields consolidated across document classes.
To run the flow and inspect cross-class field output end to end, publish your flow as an advanced app.
For schema and validation concepts in the app editor view, see Extracting data from packets and Validating packets.
Step order
A typical packet-processing pipeline runs in the following step order:
-
Process files — OCR and digitization.
-
Unified extractor — Structured extraction across the packet using your configured schema.
-
Apply checkpoint — Class-level validations (when your project defines them).
-
Combine — Merges all file records into one stream before packet-level refinement. See Combine classes.
-
Process case — Runs the packet refiner program (
case.ibrefiner) that materializes cross-class fields into the IB stream. See Process case. -
Apply checkpoint (optional) — Cross-class validations, only when the published project defines rules on cross-class fields. See Cross-class checkpoint.
-
Combine — Final merge when you need all classes and cross-class output in one stream. See Combine all.
Process case
The Process case step loads a packet refiner program from the flow bundle and writes output that includes materialized cross-class fields.
For UI field descriptions, folder keys, kwargs, and the full reference for authoring case.ibrefiner, see Process case.
Module layout
The packet refiner program lives in the flow bundle at:
Process case resolves case_program_path to that JSON program. The file does not have to be named case.ibrefiner — the name is determined by the path you set in case_program_path. Per-class refiner modules and .ibvalidations files for class-level checkpoints live in separate module folders.
modules/<STEP-NAME>.refiner/<NAME>.ibrefiner. To match the expected path, use the file explorer to create a prog folder inside the .refiner folder, then move your .ibrefiner file into it. If you create your schema in a project, then export the project to the flow editor, the prog folder is created for you.Authoring packet refiner programs
The fixed prefix builds the inputs for packet-level logic: a class-field manifest and one bridge column per referenced (class name, field name) pair. Bridge labels (typically __ClassName_FieldName_<uuid>) are what downstream cross-class formulas reference.
The cross-class field columns follow the bridges, one object per cross-class field in project order. Each uses a choice_type: DERIVED, AGGREGATED, or UDF. For schema details and native function descriptions, see Process case.
The JSON examples below are drawn from a loan application project with four document classes: Bank_Statement, Loan_Application, Pay_Stub, and W2. Substitute your real class and field names when authoring your program.
Fixed-prefix skeleton
Two patterns exist for the fixed prefix, depending on how your program was generated.
Pattern A: get_intra_class_field per bridge (common)
Each bridge column calls get_intra_class_field directly and includes its own field_selection_strategy. There is no separate shared aggregation column. This is the pattern generated when starting from an automation project.
Each bridge block represents one input class field. In this example, the first bridge exposes Loan_Amount from Loan_Application and the second exposes Wages_Tips from W2. For each bridge, three values are project-specific and must be updated together: label (the stable identifier downstream formulas reference), class_name, and field_name.
Add one bridge block for each (class, field) pair your cross-class fields depend on. Append cross-class field columns after all the bridges.
The class_fields value on get_input_fields_per_class_field is project-specific — replace {} with your own class-to-field mapping (such as {"Loan_Application": ["Loan__Amount"], "W2": ["Wages__Tips"]}).
The kwargs input on each bridge line is required by the refiner schema but ignored at runtime; keep "value": "{}".
Pattern B: get_all_intra_class_fields + get_intra_class_field_from_calculated (two-step)
An alternative pattern uses a single shared aggregation column (get_all_intra_class_fields) followed by individual bridge columns that each call get_intra_class_field_from_calculated to extract one (class, field) pair from it. The field_selection_strategy is set one time on the aggregation column rather than per bridge.
In this pattern, kwargs on get_all_intra_class_fields and each get_intra_class_field_from_calculated line is required by the refiner schema but ignored at runtime; keep "value": "{}". For custom_udf strategies, pass an extra udf_json argument on get_all_intra_class_fields.
Derived cross-class fields (choice_type: DERIVED)
Prompt-based columns use refine_field_v2. The first prompt line sets previous_line to null; later lines chain previous_line to the prior line identifier (such as Is_Reasonable_Loan@0). prompt_arg_vals reference bridge label strings.
Derived field example (Is_Reasonable_Loan)
Three inputs use project-specific values that must be updated together: the prompt string in prompt (which embeds human-readable class field labels), prompt_arg_names (which lists those same human-readable labels), and prompt_arg_vals (which references the corresponding bridge labels from the fixed prefix). In this example, the field asks whether a loan amount from Loan_Application is reasonable given the wages from W2.
Ranked/aggregated cross-class fields (choice_type: AGGREGATED)
selection_logic (such as FIRST_VALID) maps to the first line’s native function name in lowercase (first_valid). input_fields order drives input_field_names and input_field_vals. The second line is always ranked_value_unwrap, referencing the first line by label and index (such as Applicant_Name@0).
Other ranking natives follow the same two-line pattern with a different function_id.name: ocr_confidence, field_confidence, first_ranked.
Aggregated field example (Applicant_Name)
This example picks the applicant name by trying each class in priority order — Loan_Application, then Pay_Stub, then Bank_Statement, then W2 — and returning the first valid value. input_field_names and input_field_vals must be listed in the same order and reference bridge labels from the fixed prefix.
Custom function cross-class fields (choice_type: UDF)
The primary line calls run_case_udf_in_lambda_v2 with: script path, serialized function_args, udf_id, lambda_udf_id, generated code_arg and fn_name, a class_fields map of human-readable names to bridge labels, udf_imports, and args for the lambda. ENUM fields may add select_options and output_type: "SELECT" on the column object.
UDF field example (user__has__enough__money__in__bank)
This example calls a Python function that checks whether the loan amount is less than ten times the bank account’s ending balance. It takes Is_Reasonable_Loan — a cross-class field computed earlier in the same program — as a positional input via args. class_fields maps human-readable ClassName.FieldName keys to their corresponding bridge labels, making those values available to the function via the class_fields parameter. function_args lists any cross-class fields passed as named inputs.
Column ordering — Emit fixed-prefix columns first, then cross-class field columns in project order.
Cross-class checkpoint
This step is optional and applies only when the published project defines validation rules scoped to cross-class fields (case_field_validation_rules non-empty at publish). It runs after Process case on the case step’s output folder, so rules see both class extraction and materialized cross-class fields.
The checkpoint uses a validations module.
This validation file embeds cross-class settings from the project (project.settings.cross_class), so rules behave consistently with the app editor experience. Supported rule types include ALL_INPUTS_VALID, ALL_INPUTS_MATCH, CHOSEN_FIELD_VALID, and UDF.
Validation file top-level shape:
This module is distinct from class-level checkpoints such as modules/all-classes.checkpoint/validations.ibvalidations, which run earlier on single-class extraction.
