Data Interface
Configuration File HAKOM.Config
All settings concerning the Data Interface in general are located in the HAKOM.Config file in the installation directory of the HAKOM Data Interface.
Data Interface Settings
Settings
Setting | Description | Default Value | Possible Values |
---|---|---|---|
<HAKOMConfigConnectionName> | Specifies which of the defined database connections will be used for importing or exporting data. | String, names of defined database connections | |
<HAKOMConfigConnectionNameProtocol> | Specifies which of the defined database connections will be used for logging purposes. | String, names of defined database connections | |
<ConfigFilesPath> | Path of directory where specific import and/or export configuration files are located. | Installation folder of Data Interface | String |
<ImportMaxDegreeOfParallelism> | Specifies how many parallel import processes are allowed to run. Requires licensing of parallel processing. | Integer |
Example
<HAKOMConfiguration>
<products>
<TimeSeriesDataInterfaceAssistent>
<settings>
<trace>0</trace>
<HAKOMConfigConnectionName>TSM</HAKOMConfigConnectionName>
<HAKOMConfigConnectionNameProtocol>TSM</HAKOMConfigConnectionNameProtocol>
<ConfigFilesPath>.</ConfigFilesPath>
</settings>
</TimeSeriesDataInterfaceAssistent>
</products>
</HAKOMConfiguration>
In addition to the settings listed here it is also necessary to configure the processing of import files and the creation of export files in order to use these features. It is possible to keep these import and/or export configurations in the HAKOM.Config file or create separate configuration files that correspond to specific import/export processes. For better clarity it is recommended to create separate configuration files.
If the import and export configuration is to be located in the HAKOM.Config the settings must be included as follows:
<HAKOMConfiguration>
<products>
<ConfigName> <!-- Name of the configuration section -->
<!-- Import/Export settings -->
</ConfigName>
</products>
</HAKOMConfiguration>
Dedicated Configuration Files
In case separate configuration files are prepared for specific import or export runs, the HAKOM.Config located in the Data Interface installation directory must contain the location of these configuration files in the setting "ConfigFilesPath" as described above.
These configuration files must be structured as follows:
<HAKOMConfiguration>
<products>
<ConfigName> <!-- Name of the configuration file -->
<!-- Import/Export settings -->
</ConfigName>
</products>
</HAKOMConfiguration>
CSV Import Configuration
The import configuration file has to be tailored to the import file it corresponds to. An import configuration file does not have to contain all settings but only settings needed to correctly process the contents of the import file and achieve the desired outcome.
Setting | Description | Example | Default Value | Possible Values | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
<Trace> | Specifies the log level of import processes. |
XML
| Following trace levels are available:
| |||||||||||||||||||
<ImportPath> | Specifies the location of the files to be imported. Note: A path to a file directory must be provided, paths to specific files are not supported. |
XML
| String, path to a directory | |||||||||||||||||||
<ArchivingPath> | Specifies the location where import files will be archived after the import. This settings defines the main archiving location, subfolders will be created for specific imports. This directory will be created if it does not already exist. |
XML
| String, path to a directory | |||||||||||||||||||
<ArchiveFileTimeStampFormat> | Allows to add time stamps to the archived import files in the format provided via this setting. If this setting is not used, no time stamp will be added. |
XML
| - | Date and time format string
| ||||||||||||||||||
<ArchiveFolderTimeStampFormat> | Allows to add time stamps to the automatically created archiving folders in the format provided via this setting. If this setting is not used, no time stamps will be added |
XML
| - | Date and time format string
| ||||||||||||||||||
<TimeSeriesNameFromFileName> | Read the name of time series created when importing from the import file name. Index and length of the desired name must be provided comma separated. |
XML
Will read four characters from the file name starting with the first character. | - | Comma separated integers | ||||||||||||||||||
<SuffixTimeSeriesNameFromFileName> | Read a string from the import file name and add it as a suffix to the name of the time series created when importing. Negative values are allowed, if the index is provided as a negative integer, the index will start at the end of the file name. |
XML
Will read the two characters in front of the character at position four from the file name. | - | Comma separated integers, negative values allowed | ||||||||||||||||||
<TimeSeriesNameSeparator> | Defines which character is used to separate prefixes, postfixes and suffixes from the time series name. |
XML
| String | |||||||||||||||||||
<PrefixTimeSeriesName> | Allows to define a fixed string to be prefixed to the time series name. |
XML
| - | String | ||||||||||||||||||
<PostfixTimeSeriesName> | Allows to define a fixed string to be postfixed to the time series name. |
XML
| - | String | ||||||||||||||||||
<ColumnIndexTimeSeriesName> | The column index of the import file where the time series name is provided. |
XML
| - | Integer | ||||||||||||||||||
<ColumnIndexTimeSeriesID> | The column index of the import file where the time series ID is provided. |
XML
| - | Integer | ||||||||||||||||||
<ColumnIndexFrom> | The column index of the import file where the from date of a data point is provided. |
XML
| - | Integer | ||||||||||||||||||
<ColumnIndexTo> | The column index of the import file where the to date of a data point is provided. |
XML
| - | Integer | ||||||||||||||||||
<ColumnIndexDate> | The column index of the import file where the date of a data point is provided. |
XML
| - | Integer | ||||||||||||||||||
<ColumnIndexTime> | The column index of the import file where the time of a data point is provided. |
XML
| - | Integer | ||||||||||||||||||
<ColumnIndexFlag> | The column index of the import file where the flag of a data point is provided. |
XML
| - | Integer | ||||||||||||||||||
<ColumnIndexValue> | The column index of the import file where the value of a data point is provided. |
XML
| - | Integer | ||||||||||||||||||
<ColumnIndexDataSource> | The column index of the import file where the data source of a data point is provided. Dedicated time series will be created for each data source and values will only be stored on the corresponding time series. |
XML
| - | Integer | ||||||||||||||||||
<HeadersIndexTimeSeriesName> | The column index of the header row where the time series name is provided. |
XML
| - | Integer | ||||||||||||||||||
<HeadersIndexUnit> | The column index of the header row where the unit is provided. |
XML
| - | Integer | ||||||||||||||||||
<MinValue> | The minimum value to be imported. Any values contained in the import file that are lower than the specified value will not be imported. |
XML
| - | Integer | ||||||||||||||||||
<MaxValue> | The maximum value to be imported. Any values contained in the import file that are higher than the specified value will not be imported. |
XML
| - | Integer | ||||||||||||||||||
<DataUnit> | Specifies the unit of the import data. If an unknown unit is provided it will be created in the database. |
XML
| String | |||||||||||||||||||
<TimeFormatString> | Specifies the format of time stamps contained in the import file. |
XML
| According to the regional settings of the machine. | Time format string
| ||||||||||||||||||
<DateFormatString> | Specifies the format of dates contained in the import file. |
XML
| According to regional settings of the machine. | Date format string
| ||||||||||||||||||
<DataSourceHeaders> | Allows to define a number of rows as header rows to be skipped during import. |
XML
| - | Integer | ||||||||||||||||||
<DataSourceTimeZone> | Specifies the time zone relevant to time stamps contained in the import file. |
XML
| According to time stamp format. |
| ||||||||||||||||||
<DataSourceTimeSeriesType> | Defines the type of time series that will be created during the import. |
XML
| A |
| ||||||||||||||||||
<DataSourceTimeSeriesInterval> | Defines the interval of the time series that will be created during the import. The interval may be provided in seconds (positive values) or as a HAKOM interval enumerator (negative values).
An interval must be provided for time series of type A and E. |
XML
| - |
| ||||||||||||||||||
<DataSourceTimeOffset> | Allows to set an offset in seconds for all imported data points. |
XML
| - | Integer | ||||||||||||||||||
<CreationTime> | Sets the creation date and time for the import data. |
XML
| - | Date and time stamp | ||||||||||||||||||
<CreationTimeFromFileName> | Allows to extract the creation date and time for the import data from the import file name. Index and format must be provided. |
XML
Will read the time stamp from the file name "Source1_20210101_1400" | - | See description | ||||||||||||||||||
<CreationTimeFromHeader> | Allows to extract the creation date and time for the import data from a header row in the import file. The header column, the index and formant must be provided. |
XML
Will read the time stamp from the header "K1; K2_20120101_1400" | - | See description | ||||||||||||||||||
<CreationTimeAsQuotationTime> | Allows to use the creation time as a quotation time stamp. |
XML
| false | Boolean | ||||||||||||||||||
<FlagMapping> | Allows to map content from the import file to HAKOM TSM flags. The numerical flag ID references must be used when mapping flags. |
XML
Will map values from the import file as shown below:
| - | See description | ||||||||||||||||||
<DataSeparator> | Defines the column separator used in the CSV import file. |
XML
| ; | String | ||||||||||||||||||
<FileFilter> | Allows to filter the files that will be imported. File names, file format extensions and placeholders may be used. |
XML
| - | String, wildcards are available | ||||||||||||||||||
<NumberDecimalSeparator> | Defines the character used as a decimal separator. |
XML
| According to regional settings of the machine. | String | ||||||||||||||||||
<NumberGroupSeparator> | Defines the character used as a thousands separator. |
XML
| According to regional settings of the machine. | String | ||||||||||||||||||
<SortCriteria> | Specifies the criteria used for sorting the import files. |
XML
| Sort by
| |||||||||||||||||||
<SortDirection> | Specifies the sorting order. |
XML
| descending |
| ||||||||||||||||||
<Encoding> | Specifies the character encoding. |
XML
| List of available options can be found here: Encoding table for imports |
CSV Export Configuration
Setting | Description | Example | Default Value | Possible Values | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
<Trace> | Specifies the log level of import processes. |
XML
| Following trace levels are available:
| |||||||||||||||||||
<ExportPath> | Specifies the path where exported files will be placed. This path will be created if it does not already exist. |
XML
| String | |||||||||||||||||||
<ExportFileName> | Specifies the file nname format of exported files. The following placeholders may be used:
|
XML
| String, placeholders available | |||||||||||||||||||
<ExportFileNameDateFormat> | Specifies the date format of the placeholder "##DATE##" mentioned above. |
XML
| Date format string | |||||||||||||||||||
<ExportFileNameTimeFormat> | Specifies the format of the placeholder "##TIME##" mentioned above. |
XML
| Time format string | |||||||||||||||||||
<CSVHeader> | Allows to define a header row for the export file. The following placeholders may be used:
|
XML
| - | String, placeholders are available | ||||||||||||||||||
<CSVHeaderDateFormat> | Specifies the format of dates in the header row of the export file. |
XML
| Date format string | |||||||||||||||||||
<ExportPeriod> | Defines the period relevant for the data export. |
XML
| - |
| ||||||||||||||||||
<ExportPeriodSeparator> | Specifies the start of the export period:
|
XML
| String/Integer depending on <ExportPeriod> setting | |||||||||||||||||||
<SkipMissingData> | Specifies if missing data should be skipped. |
XML
| false | Boolean | ||||||||||||||||||
<Interval> | Defines the interval relevant for the export. Positive integers will be interpreted as seconds, negative values will be interpreted as enumerators.
|
XML
| Integer or HAKOM interval enumerator | |||||||||||||||||||
<ExportCalculationFormula> | Allows to apply an ad hoc formula during the data export. |
XML
| HAKOM formula syntax applies | |||||||||||||||||||
<ExportCalculationInterval> | Defines the calculation interval for the ad hoc formula. Positive integers will be interpreted as seconds, negative values will be interpreted as enumerators.
|
XML
| Integer or HAKOM interval enumerator | |||||||||||||||||||
<CSVSeparator> | The character used for separating the columns in the export file. |
XML
| ; | String | ||||||||||||||||||
<ExportColumns> | Specifies the columns that will be exported. |
| See examples | |||||||||||||||||||
<Unit> | Specifies the unit of the values for the export file. |
XML
| - | String | ||||||||||||||||||
<UseDayLightSaving> | Controls whether day light saving will be considered.
|
XML
| False | Boolean | ||||||||||||||||||
<MapFlag> | Allows to export custom values instead of the HAKOM TSM Flag IDs. |
XML
| - | See examples | ||||||||||||||||||
<DayNames> | Allows to export data only for characteristic days. Multiple day types may be provided, separated by a semicolon ";". |
XML
| - |
|
Encoding
For CSV exports only UTF-8 character encoding is supported.
Export of Multiple Time Series
It is possible to export multiple time series in one export file. The following configuration settings may be extended for this purpose:
Value Format
In the export configuration file multiple formats for values can be defined:
<Field Name="Value" Format="0.0" DecimalSeparator="," GroupSeparator=""/>
<Field Name="Value1" Format="0.000" DecimalSeparator="," GroupSeparator=""/>
<Field Name="Value2" Format="0.0000000" DecimalSeparator="," GroupSeparator=""/>
For all exported time series a value format should be provided, for all additional time series without a specific configuration the format of "Value" will be applied. In case no "Value" entry exists only time series with an applicable format will be exported.
Interval
In the export configuration file multiple intervals can be defined:
<Interval>900</Interval>
<Interval1>3600</Interval1>
For all exported time series an interval should be provided, for all additional time series without a specific configuration the interval of the setting "Interval" will be applied. In case no "Interval" entry exists only time series with an applicable interval will be exported.
Individual Import/Export Parameters
For both import and export configuration files it is also possible to define placeholders for certain configuration settings and provide the values for these settings when starting the import or export process with the console command.
The relevant settings need to be configured as follows in the import or export onfiguration file:
<HAKOMConfiguration>
<products>
<ConfigName>
<Setting>##Placeholder##</Setting>
</ConfigName>
</products>
</HAKOMConfiguration>
The string used as a placeholder (in this example "Placeholder") may be freely chosen, the leading and trailing "##" must be present in order to use this functionality.
When starting an import or export run the value for the setting can be provided as shown here using the example of an import command:
HAKOM.Framework.DataInterface.Console.exe import csv -c {Name_of_configuration_file} -p Placeholder=3