Best Practices for File Integration in Boards on Fire

This guide outlines the essential steps to prepare, upload, and manage CSV files for integration. Following these recommendations will ensure smooth processing, accurate data, and minimal troubleshooting.

1. Preparing Your File for Integration

Ensuring your file is prepared correctly is the first step to smooth and error-free integration. Following these guidelines will save time and avoid delays in your data import process.

• Save the file as CSV in UTF-8 format.
• Make sure you file is less than 5000 rows.
• Use the date format YYYY-MM-DD.
• Keep the same column order in all uploads.
• Always include the required fields:
  For data sources: Date and Organization.
  For entities: Organization
• Test with a small file before full upload.

Tip: Test your file before full upload to confirm formatting and prevent data errors. Create a test data source or entity to upload the test data to.

2. Secure File Transfer via SFTP

Using a secure transfer method ensures your data is protected during transmission. Follow these steps to maintain security and reliability. 

• Use the unique SFTP credentials provided by Boards on Fire.
• Credentials are time-sensitive, store them securely.
• Use a secure network when uploading files.
• Remove old or unused accounts regularly.

3. Mapping Your Data Correctly

Mapping ensures your data is understood by the system and ends up in the right place. Accuracy here prevents reporting errors later.

• Make sure you select the correct CSV separator for your file (semicolon or comma).
• Map required fields first.
• Map the correct CSV columns to the required fields in the system. Make sure you select the right column (e.g. Date, Organization) to avoid data ending up in the wrong place.
• Select "No mapping" for unused columns to avoid errors.
• Map Organizations using External Ids
• When using a data source, do not use the same timestamp twice for the same organization and group. Every row must have its own unique time value based on the time precision you have selected (e.g. day, hour, minute).

4. Avoiding Duplicate or Missing Data

Keeping your database clean improves reporting accuracy and avoids confusion. These steps help you control data quality.

• When uploading to an entity, choose whether the new file should replace all existing data or only add/update specific records.
• Use a unique External ID column for updates.
• If replacing data, ensure the file contains all required rows.
• Always check the log after uploading to confirm row count.
  Note: Logs are automatically deleted after 90 days.
• Schedule regular file audits to remove duplicates.

Tip: Test data updates on a small file before running them on full datasets.

5. Updating an existing file integration

• If you remove a column from your file, also remove its mapping.
• If you add a new column to your file, remember to create a mapping for it.

6. Troubleshooting Common Issues

If something goes wrong, use these checks to quickly find the cause and fix it.

• File doesn’t upload? Check the separator setting (semicolon or comma) or the number of rows in your file is not bigger than 5000.
• Rows missing? Verify date format and organization mapping.
• Unexpected results? Confirm your CSV is in UTF-8 format.
• File inactive? Activate it under 'Actions' in the menu.
• Compare with your last successful file for clues.