Our SFTP solution allows you to send us data to ingest. When you send us the files, we process them in real time and subsequently delete the files from the server.
You can use any SFTP client (for example, FileZilla) that supports a SSH Private Key for authorization.
Once the contract is signed, you will generate an SSH public/private key pair, and share the public key with us. Banyan will then provide you with the hostname and user name for your SFTP server, which you will access with your private SSH key.
- Server address:
- Authentication: SSH Private Key
Once you are logged in to the server, please use the
input folder under the
data folder to upload the files. Within this folder, create subdirectories for each "type" of file you are uploading. If you have flattened your data to all be included into one file you do not have to do this step. Examples of file types:
- Product Catalog
Within the /data directory, also include a subdirectory called "manifest" that will contain a daily manifest file that gets written at the end of your upload process. This file should contain the names of all files that were attempted to be uploaded on that day. This will ensure the Banyan ingestion process can check for any missing files and delay the ingestion.
Depending on your infrastructure we would prefer to receive the file at least once per day or as otherwise mentioned in your agreement with Banyan.
- We recommend files sent should be no bigger than 2GB in size.
- Files should have the date of upload in the name, file type, as well as a part number if broken into multiple files within a single day.
- example: 2022-03-04-transactions-0001.csv
When you’ll write your custom implementation to upload data to our SFTP server, make sure to take the following scenarios in to account:
- The server can become unavailable for a short period of time. Make sure to have in place a retry mechanism.
- The signature of the server might change due to hardware failure or changes in the hardware configuration. Make sure to take this into account (the URI won’t change).
- Due to a limitation of the cloud storage backend, files cannot be updated in-place: they must first be deleted, and then re-uploaded.
Updated 4 months ago