Reviewing results

Human review enables validating and correcting results from deployment runs.

When a run from a review-enabled deployment includes failed validations, results are added to the Review tab in Workspaces. The scope of queued results depends on the review settings specified in the deployment: either the entire run is queued for review, or only files that failed validation. By default, files can be reviewed by any member of the deployment workspace.

Enterprise organizations can configure additional review options for each deployment:

• Review and escalation queues assign reviews to specific groups within the workspace.

• Service-level agreements let you specify efficiency targets for human review.

Working in the review interface

Understanding the review interface can help you quickly process reviews.

The review interface includes these components.

1. File list. The file list displays the files under review. If a file includes multiple documents, as shown, the entire file opens together so that you can review documents with full context. Additionally, if you manually select multiple files from the Review tab, all files you select are displayed. The selected document is highlighted and displayed in the document view.

2. Document view. Displays the document selected in the file list.

3. Document view tabs.

   • Single document. Use this view to modify extraction data. For single-page documents, you can also use this view to modify classification data.

   • Documents grid. For files that contain multiple documents, use this view to modify mapping and classification data. The grid lets you move pages into the correct document and modify document class.

   • Results table. For files that contain multiple documents, use this view to modify or compare extracted data across all documents.

4. Document view toolbar. Use these options to modify the current document view to suit your needs, including:

   • Switch between the document image and text only.

   • Search for text.

   • Toggle bounding boxes to highlight source data in the document.

   • Pop-out the document view to a separate window.

   • Zoom and navigate pages.

5. Fields list. The fields list displays fields associated with the displayed document. You can search, sort, and filter fields. For example, you can filter to display only fields with validation failures.

6. Document classification. If the run assigns a class—or document type—a label displays the document class, along with an Edit icon to modify classification.

7. Mark document as reviewed button. Click to mark the document as reviewed. All documents in a run must be marked as reviewed before the review can be closed.

8. Submit and continue button. Click to mark all the documents in the file list as reviewed and advance to the next review available to you.

9. Review options. Use review options to modify human review settings, including:

   • Show execution errors — Indicates errors in the deployment run, such as validation logic with missing field references.

   • Show source for selected field only — Highlights source data in the document for the selected field only. By default, source data for all fields is highlighted.

Reviewing results

Your core task as a reviewer is to check and correct data from automated document processing workflows. You can review files from any workspace or workflow queue you have access to.

If the documents you’re reviewing were submitted as part of a multipage file, verify that the file was correctly parsed into individual documents. If necessary, remap documents from multipage files so that they’re split correctly.

Next, if your workflows assigns classes—or document types—review classification, because the data that’s extracted varies by class.

Finally, review extracted data, focusing particularly on fields that are flagged as failing validation.

1. In Workspaces, select the Review tab, then click Start review.

2. For each file you’re reviewing, verify and correct data if needed, then mark the file as reviewed. Alternatively, if the file needs additional review and an escalation queue is configured, click Escalate file then Confirm escalation.

   • To correct mapping data where multipage files were incorrectly parsed into individual documents, select the documents grid. Select pages and use the button controls to move or delete pages or create additional documents.

   • To correct classification data, use one of these methods:

     • In the fields list, click the Edit classification icon near the assigned class. Select the correct class, then click Confirm.

     • In the documents grid, click the class name. Select the correct class, then click Confirm.

       When you change document classification, you can specify how the schema for the new class is applied. By default, the app reprocesses the document to identify field results for the new class. Reprocessing incurs usage charges at the same rates as regular app runs. To apply the schema for the new class without reprocessing, deselect Extract fields for the updated class.

   • To correct text fields, in the fields list, select a field. Enter a new value or, in the document viewer, select the area of the document that contains the information for that field. You can click to select text, or use your mouse to draw a box around the information.

     If validations apply to the selected document, all fields are automatically revalidated when you modify a value.

   • To correct tables, in the fields list, select a table to open it for editing. Select any table cell, then in the document viewer, select the area of the document that contains the information for that cell. You can click to select text, or use your mouse to draw a box around the information.

3. Review any additional files in the file list.

4. Click Submit and continue.

   This action marks everything in the file list as reviewed and advances you to the next file available for review.

Completing and closing reviews

Human review involves two steps:

1. Reviewing files and marking them reviewed.

2. Closing reviews. When all required reviews are complete for a given run, a workspace manager can close the review. Closing reviews sends results to downstream integrations.

Only workspace managers can close reviews. When you have reviews to close, a tally appears on the Review tab. Click Close reviews to close all available reviews or select specific reviews to close. To ensure you see reviews across all workspaces you manage, select All workspaces in the sidebar.

Viewing review metrics

Review metrics help you monitor human review performance, handling time, and workload distribution, giving you insight into review efficiency.

Workspace managers can access review metrics for their workspaces.

On the Review tab, a performance summary displays key metrics for the selected workspace, including number of files currently awaiting review, number of files by status, and average review time over the past seven days. To see additional metrics with visualizations, click Metrics.

From the review dashboard, you can view detailed metrics over a specified period and view by file or by run.

Review metrics include two key measures:

• Total handling time is the amount of time it takes to process a file or run, including both automation and human review. Time is measured from the start of the run to completion of the run or, if review is required, when the file(s) are marked as reviewed.

• Human review handling time is the amount of time it takes to review a file or run. Time is measured from when reviews are queued to when file(s) are marked as reviewed.

Within each chart, use the toggle to display total handling time or human review times only.

The Handling time trends chart displays processing times over your selected time range. The chart includes a blue line representing total handling time, including both automated processing and human review, and a yellow line representing review time only. Spikes in the chart indicate longer processing times, which might represent complex documents or process bottlenecks. Use this chart to identify trends and understand the relationship between automation and human review times.

The Handling time distribution chart presents a histogram of processing times. The x-axis shows time intervals in minutes, while the y-axis displays the number of files or runs processed within each interval. The chart includes key statistics such as average handling time and a trimmed mean that excludes outliers above a specified percentile. A vertical red line represents the percentile cutoff. Use this chart to understand the distribution of handling times, identify common durations and outliers, and assess overall efficiency.

The Handling time by class chart compares processing times across document types. This chart is displayed only when you select Review time by file. Additionally, you can search by class name or sort the data by various criteria. A vertical dashed line indicates the overall average handling time across all classes. Use this chart to identify classes that require more processing time, which might suggest the need for automation improvements or additional reviewer training.

The Reviewer performance table provides detailed metrics for individual reviewers. The table shows key performance indicators for each reviewer, including number of files or runs reviewed and average review time per file or run. Use the search field to find specific reviewers, and click any column header to sort the data by different metrics. Use this table to understand reviewer workload and efficiency, identify potential bottlenecks in the review process, and ensure balanced task distribution across review teams.