Introduction

Our SFTP solution allows you to send us data to ingest. When you send us the files, we process them in real time and subsequently delete the files from the server.

Connection requirements

You can use any SFTP client (for example, FileZilla) that supports a SSH Private Key for authorization.

Credentials

Once the contract is signed, you will generate an SSH public/private key pair, and share the public key with us. Banyan will then provide you with the hostname and user name for your SFTP server, which you will access with your private SSH key.

In general:

Server address: YOUR_COMPANY_NAME.sftp.getbanyan.com
User: sftpuser
Authentication: SSH Private Key

Where to upload

Once you are logged in to the server, please use the input folder under the data folder to upload the files. Within this folder, create subdirectories for each "type" of file you are uploading. If you have flattened your data to all be included into one file you do not have to do this step. Examples of file types:

Transactions
Items
Product Catalog
Tenders
Stores

Within the /data directory, also include a subdirectory called "manifest" that will contain a daily manifest file that gets written at the end of your upload process. This file should contain the names of all files that were attempted to be uploaded on that day. This will ensure the Banyan ingestion process can check for any missing files and delay the ingestion.

Schedule

Depending on your infrastructure we would prefer to receive the file at least once per day or as otherwise mentioned in your agreement with Banyan.

File format

We recommend files sent should be no bigger than 2GB in size.
Files should have the date of upload in the name, file type, as well as a part number if broken into multiple files within a single day.
- example: 2022-03-04-transactions-0001.csv

Caveats

When you’ll write your custom implementation to upload data to our SFTP server, make sure to take the following scenarios in to account:

The server can become unavailable for a short period of time. Make sure to have in place a retry mechanism.
The signature of the server might change due to hardware failure or changes in the hardware configuration. Make sure to take this into account (the URI won’t change).
Due to a limitation of the cloud storage backend, files cannot be updated in-place: they must first be deleted, and then re-uploaded.