Upload

The sync tasks upload service allows you to upload files directly to an uploadable data source via its sync task ID. This is the recommended way to upload files programmatically.

Finding the sync task ID

Before uploading, find the sync task ID for the data source you want to upload to. Use the ?uploadable=true filter to narrow the results:

GET /sync_tasks/?uploadable=true

Example

Request:

curl "https://nexus.stellaspark.com/api/v1/sync_tasks/?uploadable=true&token={web_api_token}"

Response:

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 123,
            "name": "Dataset export",
            "owner": "Organization Y",
            "task_type": "data_source",
            "uploadable": true,
            ...
        }
    ]
}

The id field is the sync task ID used in the upload URL.

Uploading a file

POST /sync_tasks/{id}/upload/

Upload a new version of the data source. The request must be a multipart form upload with the file in the upload_file field. In case of timeseries measurements, rasters or files (such as photos, documents or videos) the data will be appended to the currently stored data. Any overlapping data will be overwritten with the newest version (this upload). Non-timestamped data (regular spatial datasets) will replace the data currently in Nexus. As such, the upload is considered to contain the 'latest version' of the data source.

Example

Request:

curl -F "upload_file=@C:/path/to/new/dataset.csv" "https://nexus.stellaspark.com/api/v1/sync_tasks/123/upload/?token={web_api_token}"

Response:

{
    "uploader": "Data supplier X",
    "filename": "dataset.csv",
    "upload_date": "2021-01-18T11:55:11Z",
    "download_link": "{temp_download_link}"
}

Check the tutorial Uploading data with Python for a full Python example.

Upload limits

The upload limit is 100 MB. Several files zipped together count as one upload.

Warning

JSON and Excel files are an exception: each of these is limited to 100 MB uncompressed. Since these formats compress well, an upload that is small enough can still contain a file that is too large to process.

Tip

If your file is too large, you can split the data into smaller files and upload them one after another. Wait for each upload to be processed before you send the next one; as long as the previous upload is still being processed, a new one is refused with a 409 response.

Response codes

Code	Meaning
201	File uploaded successfully and processing has started
400	The sync task has no associated data source, or the data source is not marked as uploadable
403	Authentication is missing or the user does not have the Data Supplier role
404	The sync task does not exist
409	The data source is currently being processed; retry once the previous upload has finished
413	The file exceeds the maximum upload size; see Upload limits

Upload processing

Nexus processes uploads asynchronously: your file is stored right away, and a task is queued that reads its contents into Nexus. The uploads dialog in the Viewer shows the status of that task.

Nexus also sanitizes the filename for security reasons. Only alphanumeric characters (A-Z, a-z and 0-9), hyphens (-), dots (.) and underscores (_) are retained; any other character will be removed. For example; a filename like evaporation#2020-1-1.tif will become evaporation2020-1-1.tif after upload.

Warning

This is generally not a problem, as only the file contents matter. However, if the filename itself carries special meaning that is to be interpreted by Nexus, then this might get lost in the upload process.

Tip

If you have files with special characters in the filename and you want to retain them, consider zipping the file before uploading to ensure that the filename stays the same, or rename the files before uploading.