CSV format for file integration into Boards on Fire

This guide outlines the essential steps to prepare, upload, and manage CSV files for integration. Following these recommendations will ensure smooth processing, accurate data, and minimal troubleshooting.

What is a CSV file?

A CSV file (Comma-Separated Values) is a text-based file format used to store tabular data. Unlike an Excel file, data is not stored in a spreadsheet but as text, where each row represents a data record and each column is separated by a delimiter, usually a comma or semicolon. The format is an established standard and can be automatically generated by many systems.

How the data is structured in the file

In a CSV file, each row represents a data record for a specific organization on a specific date. The required fields specify the date and organization, while the other columns contain the values ​​to be sent to Boards on Fire.

All value columns in a file should belong to the same data source or entity.

A CSV file should only contain data that belongs to the same data source or entity. Do not mix different types of key figures in the same file (e.g. production figures and HR data). Each data source in Boards on Fire that will receive data needs its own CSV file.

Rules for values ​​in rows

The values ​​in the CSV file must follow certain formatting rules to be read correctly:
The value must not contain semicolons (;).
The value must not contain quotation marks (").
Each value must be compatible with the field in Boards on Fire that it maps to.

File naming

The file name must be consistent throughout the life of the integration.
The file name must not be changed after the integration is configured.
The file name should not contain dates or time stamps.
Use a name that corresponds to the data source or entity (for example, OEE.csv).
If multiple files are used, each file must have a unique name.

1. Preparing Your File for Integration

Ensuring your file is prepared correctly is the first step to smooth and error-free integration. Following these guidelines will save time and avoid delays in your data import process.

• Save the file as CSV in UTF-8 format.
• Make sure you file is less than 50 000 rows.
• Use the date format YYYY-MM-DD.
• Keep the same column order in all uploads.
• Always include the required fields:
  For data sources: Date and Organization.
  For entities: Organization
• Test with a small file before full upload.

Tip: Test your file before full upload to confirm formatting and prevent data errors. Create a test data source or entity to upload the test data to.

2. Secure File Transfer via SFTP

Using a secure transfer method ensures your data is protected during transmission. Follow these steps to maintain security and reliability. 

• Use the unique SFTP credentials provided by Boards on Fire.
• Credentials are time-sensitive, store them securely.
• Once a file has been uploaded to the server, it will typically load automatically within 30-60 seconds. In some cases, it may take up to 5 minutes for the data to become visible in Boards on Fire.
• Use a secure network when uploading files.
• For regular or frequent data transfers, an automated script can be used to generate and upload CSV files to the SFTP server.
• Remove old or unused accounts regularly.

3. Mapping Your Data Correctly

Mapping ensures your data is understood by the system and ends up in the right place. Accuracy here prevents reporting errors later.

• Make sure you select the correct CSV separator for your file (semicolon or comma).
• Map required fields first.
• Map the correct CSV columns to the required fields in the system. Make sure you select the right column (e.g. Date, Organization) to avoid data ending up in the wrong place.
• Select "No mapping" for unused columns to avoid errors.
• Map Organizations using External Ids
• When using a data source, do not use the same timestamp twice for the same organization and group. Every row must have its own unique time value based on the time precision you have selected (e.g. month/week/day).

4. Avoiding Duplicate or Missing Data

Keeping your database clean improves reporting accuracy and avoids confusion. These steps help you control data quality.

• When uploading to an entity, choose whether the new file should replace all existing data or only add/update specific records.
• Use a unique External ID column for updates.
• If replacing data, ensure the file contains all required rows.
• Always check the log after uploading to confirm row count.
  Note: Logs are automatically deleted after 90 days.
• Schedule regular file audits to remove duplicates.

Tip: Test data updates on a small file before running them on full datasets.

5. Updating an existing file integration

• If you remove a column from your file, also remove its mapping.
• If you add a new column to your file, remember to create a mapping for it.

6. Troubleshooting Common Issues

If something goes wrong, use these checks to quickly find the cause and fix it.

• File doesn’t upload? Check the separator setting (semicolon, tab or comma) or the number of rows in your file is not bigger than 50 000.
• Rows missing? Verify date format and organization mapping.
• Unexpected results? Confirm your CSV is in UTF-8 format.
• Are your Organizations external IDs correctly named?
• File inactive? Activate it under 'Actions' in the menu.
• Compare with your last successful file for clues.