Multipart file upload

To upload a file larger than 10 MB in size to a batch, you must use multipart upload. Multipart upload involves three endpoints and the following steps:

1. Start a multipart upload session, specifying the batch ID, file name, and file size. This call returns a session ID and maximum part_size to reference when uploading each part.

2. Split the file into parts, according to the specified part_size, and upload each part individually.

3. When all parts are uploaded, commit the session.

Multipart upload example request (Python)

The following Python script performs each of the tasks required for multipart file upload, when <LOCAL_FILEPATH> is defined as the full path to the file in the machine that’s running the script. The API_ROOT and API_TOKEN must also be defined.

1
headers = {
2
  'Authorization': f'Bearer {API_TOKEN}'
3
}
4
# 1. create session
5
local_filepath = '<LOCAL_FILEPATH>'
6
size = os.path.getsize(local_filepath)
7
resp = requests.post(f'<API_ROOT>/v2/batches/multipart-upload',
8
                     headers=headers,
9
                     data=json.dumps({'path': destination_filepath, 'file_size': size}))
10
session_endpoint = resp.headers['location']
11
part_size = resp.json()['part_size']
12
13
# 2. upload parts
14
parts = []
15
part_num = 1
16
with open(local_filepath, 'rb') as input_file:
17
    part = input_file.read(part_size)
18
while part:
19
    part_resp = requests.put(f'{session_endpoint}/parts/{part_num}', headers=headers, data=part)
20
    parts.append({'part_num': part_num, 'part_id': part_resp.json()['part_id']})
21
    part = input_file.read(part_size)
22
    part_num += 1
23
24
# 3. Commit all the uploaded parts.
25
commit_resp = requests.post(session_endpoint, headers=headers, data=json.dumps({'action': 'commit', 'parts': parts}))

Start multipart upload session

Description

Start a multipart upload session. Use this endpoint when you need to upload a file larger than 10 MB to a batch.

Request body

Response status

Response headers

Response schema

Examples

See the full multipart upload request.

Upload part to session

Description

Upload part of a file to the multipart upload session, where each part’s size matches the part_size returned by the Start multipart upload session call. Each part should match the part_size, except for the final part, which can be smaller than the part_size.

<SESSION_ID> can be obtained in the Location header from the Start multipart upload session response, and <PART_NUM> is an increasing consecutive integer sequence starting at 1 for every part uploaded.

Request body

Raw content of the part to be uploaded.

Response status

Response schema

Examples

See the full multipart upload request.

Commit session

Description

After uploading all parts to a multipart upload session, use this endpoint to commit and close the multipart upload session, or to cancel the session.

Request body

Response status

Response schema

There is no response body.

Examples

See the full multipart upload request.