Audit and Quotation

Audit

Auditing changes is essential in certain critical areas. HAKOM TSM supports you with the following functions:

• Auditing time series definition / master data: each modification of master data is persisted in HAKOM TSM databases

• Auditing time series data: the time range modified by a given user is tracked for all time series, while the history of changed data is not recorded by default

• Auditing time series data modifications: the data record changed by any user at a certain time is persisted in dedicated tables. The system seemlesly supports working with this data as they were live data.

Concept

Audit can be enabled for any given time series within the system.

While saving data to such a time series, a modification timestamp is automatically set to the respective system date time. All data included in the according change set is assigned to that given modification timestamp, which allows retrieving historical data for any specific modification.

Exact Vs. Not Exact

In the context of audit HAKOM TSM distinguishes between exact and non-exact values.

Exact

In order to retrieve only values changed at a given modification time, the time series data can be requested with an exact audit flag (in TSM App: main GUI → Time series → Audit → Change at. in WebTSM Services API: parameter exactAudit=true). In case there is no modification in the given time range no data will be returned (i.e. for each grid point on the display raster a missing 0 value will be displayed).

Tip

TSM App lists all historisation times stamps per time series in the main GUI - Time series section - Audit selection list - Change at:

WebTSM Services API all changes can be retrieved via path GET /repositories/:repository/timeseries/:id|name/data/modifications.

Not Exact

In order to retrieve the status of a time series at a given modification time, i.e. all changes at that time including all previous changes, the time series data can be requested without exact audit flag (in TSM App: main GUI → Time series → Audit → Status at. in WebTSM Services API: parameter exactAudit=false). In this case, even if for the given modification date there is no underlying dataset, the system will return values for changes prior to the selected modification date (if there is any existing data).

Example

The following figure shows three changes of a given same time series, with the oldest change date at the bottom ("Change date 1") and the newest change date ("Change date 3") on the top:

Comparing data requests at change date 3 and change date 2 exact and not exact:

Enabling Audit

To activate recording of historical data for specific time series follow the below instructions.

TSM App:

1. Click on Load in the TSM Ribbon

2. Press the Search... button in the Time series area of the TSM window

3. Search for the required time series by entering search parameters and pressing Search button

4. Select the time series in the results grid

5. Select the Edit tab in the Time Series Search window > the definition of the selected time series is being loaded

6. Select Archive checkbox in Standard section > the according historisation table will be set automatically

7. Press the Save button

WebTSM Services API:

In any time series definition path (/repositories/:repository/timeseries or /repositories/:repository/timeseriescollections/definition) set property Audit to true in the body.

{
    "Name": "MyFirstTimeSeries",
    "Type": 2,
    "Interval": {
        "Value": "Minute",
        "Multiplier": 15
    },
    "Unit": "KWh",
	"Audit": true
}

Writing Historical Data

In case audit is enabled for a given time series, any data write action will automatically result in new historical datasets.

Reading Historical Data

To read time series data with historical values, follow the below instructions.

TSM App:

1. Click on Load in the TSM Ribbon

2. Select the Name of the time series in the Time Series section, or use the time series search

3. Select Interval, Unit and if necessary further parameters (such as descending and / or missing as null)

4. Depending on whether exact or not exact data is needed choose one of the following options:

   1. For exact data: select Change at in Audit selection list and select modification date

   2. For last n exact changes prior to selected modification date: select Change before in Audit selection list, select modification date and the number of changes you want to see

   3. For not exact data: select Status at in Audit selection list and select modification date

   4. For last n not exact changes prior to selected modification date: select Status before in Audit selection list, select modification date and the number of changes you want to see

5. Press the Open button

WebTSM Services API:

Passing modification date for historised data is supported in all data requests (such as /repositories/:repsoitory/timeseries/:id|name/data) via parameter audit in ISO 8601 or as unix epoch date time stamp format. To get exact data, parameter exactAudit=true can be used.

To learn more about audit in TSM App, refer here: Audit

Quotation

Quotation Dimension Versus Audit

Time series with enabled audit already provide reading historical data. However the modification timestamps that represent the change history are assigned by the system automatically based on the actual system date. The user does not have control about those timestamps.

In certain cases the user does not want to rely on automatically generated timestamps, but wants to set them manually. For example, when some independent variables have to be linked together with calculation results sharing the same timestamp.

As an example this is the case with forecasts where forecast results should be linked together with weather data and other input factors, although their real modification time will be different, since saved at different times. As such using quotations the user can manually set the timestamp (quotation date) and link multiple processes to one business context.

On top, the system also keeps track of changes per quotation. I.e. for each quotation date multiple modification (history) streams can be accessed.

Enabling Quotation

Quotation should be enabled when creating a given time series, by the latest before saving the first data on the given time series. 

Note

If quotation is enabled for a time series with already existing data, the existing data (if necessary) needs to be migrated into quotation enabled tables.

TSM App:

1. Click on Load in the TSM Ribbon

2. Press the Search... button in the Time series area of the TSM window

3. Search for the required time series by entering search parameters and pressing Search button

4. Select the time series in the results grid

5. Select the Edit tab in the Time Series Search window > the definition of the selected time series is being loaded

6. Select Quotation checkbox in Standard section > the according quotation table will be set automatically

7. Press the Save button

WebTSM Services API:

In any time series definition path (/repositories/:repository/timeseries or /repositories/:repository/timeseriescollections/definition) set the property "Quotation" in the body to true.

{
    "Name": "MyFirstTimeSeries",
    "Type": 2,
    "Interval": {
        "Value": "Minute",
        "Multiplier": 15
    },
    "Unit": "KWh",
	"Quotation": true
}

Default Quotation Behaviour

In TSM App and WebTSM Services the setting "Default Quotation Behavior" can be selected when creating a time series. Please visit the documentation for TSM App: Define Time Series or for WebTSM Services: Open API Reference

The selection of the standard quotation behavior has an effect on which version of the time series is loaded when reading with quotation:

Writing Quotation Data

In case quotation is enabled for a given time series, new data can be only written on the time series, if the quotation date is being passed as well.

TSM App:

1. Open any existing TSM Excel time series template containing data of the required time series or load the required time series (see chapter "Read Data" on page: Working with Time Series Data) and adjust the data accordingly

2. In row 13 of the respective time series column enter the desired quotation date in your local Excel date time format (e.g. MM/dd/yyyy hh:mm:ss AM/PM for an english system or yyyy/MM/dd hh:mm:ss for a central european system)

3. In TSM Ribbon, click on Save → time series data will be saved.

WebTSM Services API:

• Save single time series data with quotations
  

  CODE

  POST /repositories/:reposiory/timeseries/:ID|Name/data?quotation=<quotation date time as ISO 8601 date>

  
  

• Save multiple time series data in bulk:
  

  CODE

  POST /repositories/:reposiory/timeseriescollections/data?quotation=<quotation date time as ISO 8601 date>

  
  

Reading Quotation Data

TSM App:

1. Click on Load in the TSM Ribbon

2. Select the Name of the time series in the Time Series section, or use the time series search

3. Select Interval, Unit and if necessary further parameters (such as descending and / or missing as null)

4. Depending on whether exact or not exact data is needed choose one of the following options:

   1. For exact data: select Most recent at in Quotation selection list and select quotation date

   2. For last n exact changes prior to selected modification date: select Most recent before in Quotation selection list, select quotation date and the number of changes you want to see

   3. For not exact data: select Version at in Quotation selection list and select quotation date

   4. For last n not exact changes prior to selected modification date: select Version before in Quotation selection list, select quotation date and the number of changes you want to see

5. Press the Open button

Tip

You may combine historisation parameters (such as Status at / Change at) with quotation requests, to get specific modifications of a given quotation dimension.

WebTSM Services API:

Passing a quotation date for quotation data is supported in all data requests (such as /repositories/:repsoitory/timeseries/:id|name/data) via parameter quotation in ISO 8601 or as unix epoch date time stamp format. To get exact data, parameter exactQuotation=true can be used.

To learn more about quotations in TSM App, refere here: Quotations

We recommend the following videos under Video Tutorials:

• Auditing

• Quotation Time Series

• Exact Versus NotExact