Attributes and Time Series

Attributes

Using attributes, time series can be categorized and easily aggregated. For example, you can use attributes to assign attributes to a given business entity (e.g. consumer and area) and calculate the corresponding aggregates (e.g. schedule forecasts per area).

Attribute Data Types

HAKOM TSM supports numerous attribute data types, such as Category, Text, Date, YesOrNo, Time series etc. (see also Creating, Editing and Deleting Time Series). In TSM App, all entred attributes are type validated against user input.

The same attribute types can also be managed and assigned via WebTSM Services API, however note, using the API attribute types are not enforced, as such in the API they all behave as text key-value-pairs.

Simple Attributes Vs. Trees

Besides common data types (as mentioned above) using data type Node hierarchical levels, so called Trees can be modelled as well. In the below chapters we will reference to all non-tree type attributes as "Simple attributes".

Attributes with Effective Dates

In certain scenarios attribute values might be required to change over time. For example, if an attribute "supplier" shall reflect a different supplier starting from a specific effective date. In HAKOM TSM attribute values can be assigned to effective dates from that time stamps the values are valid. This means that an attribute value is always valid until the next highest effective date. Such effective dates are also considered in the aggregation according to the selected attribute values and time line.

Example: changing supplier attribute and considering within aggregation

Let's assume we have two time series: A and B with the below attribute value and time series values.

Attribute "supplier" assignments:

Aggregation result for attribute value "Supplier A":

Create a New Attribute

To create a new attribute, follow these steps

TSM App:

1. Click on Load in the TSM Ribbon
2. Press the Search... button in the Time Series area of the TSM window
3. Select the Master data tab in the Time Series Search window
4. Select Attributes in the Table selection list
5. Add Name, Description and Data type of the attribute 
   

   

In case you selected List as Data type you will need to define now the list of entities the user may select from by follow these steps:

1. Change Table selection to Units
2. Define a Unit (that can be again a data type Text, if all list items will share the same unit, or List etc.).
   Let's assume, you selected Text
3. Change Table selection to Value -> Unit assignments
4. Select the previously created Unit, add a Value (that will represent the list item) and a Description (that will be displayed in TSM App attributes value list
5. Repeat step 4 for each item to be added to your list

More about managing attributes in TSM App can be found here: Creating, Editing and Deleting Time Series

Using WebTSM Services API attribute management functionalities are available under the following paths:

• /attributetypes
  • > GET: List all attribute types
• /repositories/:repository/attributes
  • > GET: List all attributes in system
  • > POST: Create a new attribute (see Body below)
• /repositories/:repository/attributes/:id|name
  • > GET: Get a specific attribute by name or id
  • > POST: Update a specific attribute by name or id

Request body of an attribute in WebTSM Services API:

{
  "ID": -1,
  "Name": "SensorName",
  "Description": "string",
  "Type": 2
}

Note

ID: -1 is to be used to create new objects.

Assign Attributes to Time Series

TSM App:

1. Click on Load in the TSM Ribbon
2. Press the Search... button in the Time series area of the TSM window
3. Search for the required time series by entering search parameters and pressing Search button
4. Select the time series in the results grid
5. Select the Edit tab in the Time Series Search window > the definition of the selected time series is being loaded
6. Click on Attributes tab
7. Click on green plus icon  to open the Add Attribute window
8. Choose the required Attribute(s) and press OK

Tip

You can also assign a predefined list of attributes at once using attribute templates.

Tip

TSM App currently does not display attribute values with multiple effective dates. To find out how to assign such attributes please refer to the TSM App documentation: Master Data Import and Export

More about assigning attributes to time series can be found here: Creating, Editing and Deleting Time Series

Using WebTSM Services API attributes can be assigned to time series using the following paths:

• /repositories/:repository/timeseries/:id|name/attributes
  • > GET: read all attribute assignments of a given time series

  • > POST:

    • in case you post a single attribute assignment in the body using a single attribute assignment object: the API assigns a new attribute and keeps all existing assignments

    • in case you post multiple attributes using an array of attribute assignment objects: the API overwrites all assigned attributes of the given time series with the posted list of objects!

      

      Note

      If you post an empty list ([]) to this endpoint all attribute assignments of the given time series will be deleted.

      Attribute assignement Body - single object

      XML

      {
        "AttributeID": 42,
        "Values": {
          "1971-01-01T00:00:00Z": "attribute value"
        }
      }

      Attribute assignement Body - list object

      XML

      [
        {
          "AttributeID": 42,
          "Values": {
            "1971-01-01T00:00:00Z": "attribute value"
          }
        },
        {
          "AttributeID": 43,
          "Values": {
            "1971-01-01T00:00:00Z": "53",
      	  "2020-01-01T00:00:00Z": null,
            "2020-06-01T00:00:00Z": "100",
          }
        }
      ]

      

      Tip

      The time stamps within the Values dictionary object represent the effective date of an attribute. More about effective dates, see chapter Attributes with effective dates on this page.

• /repositories/:repository/timeseries/:id|name/attributes/:id|name
  • > GET: reads a specific attribute of the given time series
  • > PUT: updates a specific attribute of the given time series
  • > DELETE: deletes a specific attribute of the given time series
• /repositories/:repository/timeseries/:id|name/attributes/:id|name/values
  • > GET: reads all attribute values of the specified attribute assigned
  • > PUT: overwrites all assigned values of the specified attribute
  • > DELETE: deletes all assigned values of the specified attribute
• /repositories/:repository/timeseries/:id|name/attributes/:id|name/values/:key
  • > GET: reads the assigned value of an attribute on the selected effective date (key)
  • > PUT: overwrites the assigned value of an attribute on the selected effective date
  • > DELETE: deletes the assigned value for the selected effective date including the effective date.
• /repositories/:repository/trees
  • > GET: reads the main nodes of all defined trees
  • > POST: creates a new main node as a basis for a new tree
• /repositories/:repository/trees/:id|name
  • > GET: reads the main node of the selected tree
  • > PUT: overwrites the main node of the selected tree
  • > DELETE: deletes the selected main node (tree)
• /repositories/:repository/trees/:id|name/nodes
  • > GET: reads the complete tree structure (subnodes) of the selected tree
  • > POST: overwrites the complete tree structure (subnodes) of the selected tree

Simple attributes

In case of simple attributes in HAKOM TSM we distinguish between different attribute types (such as category, number, text, date, time series reference etc.) where most of them represent a key-value-pair.

Assign Attributes to Time Series

A category is an attribute without a value, which is assigned to a time series like a simple tag. On the other hand, key-value-pair type attributes may (and in most cases will) have an attribute value. As such using attributes a time series can be categorized on two levels:

• Is a given attribute assigned or not (this is helpful, if I want to aggregate all time series having a specific type - e.g. all customers for all locations)
• By the value of a given attribute (e.g. to aggregate all time series of a given location)

Further Simple Attribute Types

There are several different attribute types (like Text, Numeric, Date etc.) available in the TSM App (see also Creating, Editing and Deleting Time Series). All entered attributes are type validated in TSM App against user input.

The same attribute types can also be managed and assigned via WebTSM Services API, however note, using the API attribute types are not enforced, as such they all behave as text key-value-pairs.

Simple Attributes in Time Series Search

In TSM App Time Series Search it is possible to include two attributes in one search at the same time. Attribute values can also be considered in the search:

1. Click on Load in the TSM Ribbon
2. Press the Search... button in the Time series area of the TSM window
3. Select the required attributes in the Attributes section of the Time Series Search window
4. Enter any further search tags if necessary
5. Press the Search button

Using the WebTSM Services API the number of attributes or attribute key-value-pairs is unlimited. You may add for each required attribute a "attribute=attribute name" or "attribute=attribute name|attribute value" parameter using following request:

• GET /repositories/:repository/timeseries?attribute=<attribute search string>

Tip

Using the WebTSM Services API a Tree attribute can be also included in the search with the same attribute=tree name|node id logic.

Aggregating by Simple Attribues

In HAKOM TSM there are several ways to aggregate time series values by simple attributes. You may use the built in attribute aggregate features (of TSM App or WebTSM Services API), or use attribute aggregation formulas (EvalComponents) to get aggregations in a formula calculation context.

TSM App - category aggregates:

Note

The TSM App main GUI Attribute Aggregate of time series section currently only offers category type aggregates. In the upcoming releases more advanced quick aggregates for key-value-pairs will come. In the meanwhile you may use ad-hoc formulas to get quick key-value-pair aggregates (see below).

1. Click on Load in the TSM Ribbon
2. Select Interval, Unit and if necessary further parameters (such as descending and / or missing as null)
3. Select an attribute in Attribute aggregate of time series section Category selection list
4. Press the Open button
   
   

TSM App - key-value-pair aggregates:

1. Click on Load in the TSM Ribbon
2. Select Interval, Unit and if necessary further parameters (such as descending and / or missing as null)

3. Select any time series as calculation template* in Time series section via Name selection list (or search functionality)

   

   Tip

   The evaluation engine will use the interval of the specified time series as calculation raster.

4. Enter following formula function:
   TSAA(unit, bestStatus, attribute_1.name|attribute_1.value, ..., attribute_n.name|attribute_n.value)
   

   

   Tip

   To aggregate all time series having attribute location with Unit with Unit "MWh" and setting all (value quality) flags to "Valid" if at least one time series with valid values could be found, use following formula:

   TSAA("MWh", TRUE, "location")

   You may also add attribute value using a separator pipe to the attribute name (attrinute name|attribute value) and add additional attributes to the aggregation by separating key-value-pairs by comma:

   TSAA("MWh", TRUE, "location|myLocation", "customer|myCustomer")

   1. Optional: select a calcultion raster using Calc. increment selection list
5. Press the Open button
   
   

WebTSM Services API:

• Get the Sum of multiple time series via assigned Attributes, or key-value-pairs (in attribute name|attribute value notation): 

  GET /repositories/:reposiory/timeseriescollections/data/sum
  ?from=<fromDate>&to=<toDate>&attribute=<attribute>|<value>&attribute=<attribute>|<value>&interval=<interval>&intervalMultiplier=<intervalMultiplier>

Trees

So-called node attributes allow the assignment of time series to complex tree hierarchies.

Note

One time series in the same hierarchy can only be assigned to one node. 

Trees in Time Series Search

TSM App:

1. Click on Load in the TSM Ribbon
2. Press the Search... button in the Time series area of the TSM window
3. Select the required node in the tree panel on the right
4. Enter any further search tags if necessary
5. Press the Search button

Tip

Note the tree panel is being only shown, if at least one attribute with type "node" has been defined in the given data source (repository).

If a node is selected, all time series that are assigned to the selected node will be considered in the search. Time series from sub nodes are currently not considered in TSM App - Time Series Search.

WebTSM Services API: 

• at the /repositories/:repository/timeseries path, trees can be searched exactly using the parameter "attribute=node attribute name|node ID".
• It is also possible to output all time series definitions assigned to a node or subnode:
  
  • Main node: /repositories/:repository/trees/:id|name/sum - see includeTimeSeries and includeSubNodes parameters
  • Subnodes: /repositories/:repository/trees/:id|name/nodes/:nodeId/sum - see includeTimeSeries and includeSubNodes parameters

Aggregating Trees

TSM App:

1. Click on Load in the TSM Ribbon
2. Select Interval, Unit and if necessary further parameters (such as descending and / or missing as null)
3. Select a node or subnode in Node aggregate section Node selection list
4. Press the Open button

Note

A node aggregate in TSM App will always contain all sub nodes (children) as well.

WebTSM Services API:

• Aggregate on main node level: /repositories/:repository/trees/:id|name/sum 
• Aggregate on sub node level: /repositories/:repository/trees/:id|name/nodes/:nodeId/sum

Tipp

Different to TSM App in WebTSM Services API you can control whether to include subnodes in the calculation (see parameter sumType). 

We recommend the following video under Video Tutorials:

• Attributes