Spontaneous Time Series

What Is a Spontaneous Time Series

Spontaneous time series, also known as asynchronous or acyclic time series, are time series with variable time intervals between two values on the time axis.

Spontaneous vs. Cyclic Time Series

The following comparison shows the differences between a spontaneous time series and cyclic time series:

Spontaneous Time Series

Cyclic Time Series

Time axis

The time axis is variable, the time span between two values can be freely chosen - with varying interval lengths.

The time axis follows a fixed raster, the time span between two values has always the same length.

Value validity range

A value is always valid up to the next value on the time axis. The last value on the time axis is valid until infinity (UniversalTime.Max).

Example: Spontaneous power time series

#	Timestamp	Valid until #	Value
1	01.01.2020 00:00	#2	10
2	01.01.2020 02:00	#3	30
3	01.01.2020 03:00	#4	35
4	01.01.2020 05:00	infinity	100

There is a fixed equidistant raster of time stamps on the time axis with. Each value is valid until the end of the interval assigned to the time series.

Example: Cyclical power time series with 1-hour interval

Timestamp	Valid until	Value
01.01.2020 00:00	01.01.2020 01:00	10
01.01.2020 01:00	01.01.2020 02:00	10
01.01.2020 02:00	01.01.2020 03:00	30
01.01.2020 03:00	01.01.2020 04:00	35
01.01.2020 04:00	01.01.2020 05:00	35
01.01.2020 05:00	01.01.2020 06:00	100
01.01.2020 06:00	01.01.2020 07:00	100
...	...	100

Note the values above replicate the spontaneous time series in the example on the left. (The last value would have to be updated for a long period of time).

Raster

As all HAKOM time series, spontaneous time series can be read and output to any raster.

As all HAKOM time series, cyclic time series can be read and output to any raster.

Storage space and compressibility

Spontaneous time series require less storage space than cyclic time series for the same time period (if not all potential time stamps have their own value), but due to their asynchronous nature, they cannot be compressed into time blocks of equal length. See also (3.11.1) Time Series Compression.

Cyclic time series require more storage space compared to spontaneous time series, but in order to reduce size, they can be compressed. See also (3.11.1) Time Series Compression.

When to Use Spontaneous Time Series?

Depending on the business context, data volume, performance requirements and how the interval between two values changes, the following bullet points might help you decide what time series type to use:

Use Spontaneous Time Series

Constant, never changing values (e.g. to model fixed volume deals and flexible deals using the same solution)
Values must keep their acyclic intervals upon saving and reading
Any given value must be valid until the next value change on the time line (without repeating values)
Number of occurring values, value changes is small enough, that a compressed cyclic time series will not provide a major performance improvement.

Use Cyclic Time Series

Values are frequently changing
Amount of values per compression block is higher than 75% of a given block, so that using compressed cyclic time series would result in higher performance (see (3.11.1) Time Series Compression)

Time Series Data With and Without Interval

Spontaneous time series can be used with or without an interval. When using an interval, all data points have to be aligned with the selected raster (so, for example, if the “Hour” interval is chosen for a spontaneous time series, all data points have to lie at full hours (like 01:00, 03:00 or 23:00, but not 01:23)).

Deep Dive

In order to further understand the intricacies of spontaneous time series with and without interval, we have to look at two different places where interval definitions are used:

Time Series Definition (Storage Format)

When creating a new time series, a type (Spontaneous, Begin, End) and an interval (No, Millisecond, Second, …) have to be defined. These settings define how data is saved in the database. Most features can be used regardless of what storage format has been chosen (there are some exceptions though). Generally, the storage format should align with the type of data the time series will store. For example, if new data becomes available every hour, an “Hour” interval would be the natural choice. If there are long stretches where data does not change, but then it changes rapidly a spontaneous storage format helps with saving storage space.

It is very important to keep in mind, that one can read and write any time series using any interval (except when using units that have an aggregation method set that is not “Average”, like “kWh”, those cannot be written without interval on a time series with interval).

The interval defined during time series creation, controls the granularity of the time series: While it is possible to read data with a smaller time unit as interval than defined as storage format, writing data this way will have unintended consequences as data will be stored aggregated to the interval defined in the time series. The smallest supported interval is either Millisecond or Second, as defined during database setup.

For spontaneous time series, the interval defined in the time series definition, defines with which interval data points have to be aligned with. For example, a time series with storage format “Hour” can store data points at 1:00 or 3:00, but not 1:23.

Since spontaneous time series without an interval are not supported in PowerTSM 4+, it is strongly recommended to always create time series with an interval!

Time Series Data (Reading and Writing / API Data Format)

The interval used for reading and writing, defines how the data should be interpreted:

If there is no interval, each data point is valid until the next data point, or infinity if it is the last data point and there is no “stopper value” (0 with flag 0 or 19) at the end.

If there is an interval different from 0, data points have to be aligned to the chosen interval and each data point is valid for exactly one interval length. Gaps are hereby treated as “leave as is”. For example: A time series has the value 200 from 01 Jan 2000 until infinity. When writing the value 100 with interval “1 Hour” at 01 Jan 200101:00, then, assuming that the time series storage format has a granularity of one hour or lower, the result will be:

2000/1/1 00:00 - 2001/1/1 01:00: 200
2001/1/1 01:00 - 2001/1/1 02:00: 100
2001/1/1 02:00 - infinity: 200

On the other hand, writing the same data without interval, the result would be:

2000/1/1 00:00 - 2001/1/1 01:00: 200
2001/1/1 01:00 - infinity: 100

Independence of Storage Format and API Data Format

In theory the choice of storage format and API data format (for reading/writing) are completely independent, however, chosing the same format for both will have a positive effect on overall performance. To summarize:

When chosing the interval for the storage format of a new time series, choose the finest granularity that you will need to support your use case. For future compatibility, an interval (different from 0/No) should always be chosen when creating new time series.

When chosing the interval for API data format when reading or writing data, choose the interval that matches the current task.

Future Compatibility of Time Series Without Interval

Starting with PowerTSM 4, time series without interval are no longer supported. Time series with "no interval" have to be converted to time series with second or millisecond interval (depends on database setup).

Important: When using a PowerTSM 4+ database with a PowerTSM 3.x service, time series with "no interval" are also not supported. The database-level changes introduced in PowerTSM 4+ prevent 3.x services from creating time series without intervals, even though the service version is older.

For more information, please check the documentation of PowerTSM 4+.

The following WebTSM Services API request examples will illustrate the differences between a fixed interval and no interval.

Time series definitions

We will use two time series, one with interval "No" (spontaneous time series without interval) and one with interval "Hour" (spontaneous time series with hour interval):

Name	Type	Interval.Value	Interval.Multiplier	Unit
Spontaneous_NoInterval	Spontaneous	No	1	kW
Spontaneous_HourInterval	Spontaneous	Hour	1	kW

WebTSM Services API example - time series definition

XML

POST {{baseUrl}}\repositories\ZAMS\timeseriescollections\definition
[
    {
        "ID": -1,
        "Name": "Spontaneous_NoInterval",
        "Type": 3,
        "Interval": {
            "Value": 0,
            "Multiplier": 1
        },
        "Unit": "kW"
    },
    {
        "ID": -1,
        "Name": "Spontaneous_HourInterval",
        "Type": 3,
        "Interval": {
            "Value": 3,
            "Multiplier": 1
        },
        "Unit": "kW"
    }
]

Spontaneous Data

Let's take a first look at how HAKOM TSM handles spontaneous data (without interval) under following circumstances:

Saving on spontaneous time series without interval
Saving on spontaneous time series with interval ("Hour")
Reading from a spontaneous time series without interval
Reading from a spontaneous time series with interval ("Hour")

Writing Data

To demonstrate spontaneous data, the following data has a gap of one day between the values:

Timestamp	Value
2019-12-31T23:00:00Z	22.8
2020-01-01T23:00:00Z	23.6

Note

In WebTSM Services API - using the default, so called "selfcontained" data format, you can specify unit and interval of the data, so that the system can interpret and convert data to the target time series interval and unit.

To set the data to spontaneous without interval, we will use Interval "No" which is represented by the Value "0":

XML

"Interval":{
	"Value": 0,
	"Multiplier": 1
}

With the following REST request this spontaneous data will be saved using the interval type 0 (“No” interval):

WebTSM Services API example - write spontaneous data without interval (interval: 0/No)

XML

POST {{baseUrl}}/repositories/ZAMS/timeseriescollections/data
[
	{
		"Name": "Spontaneous_NoInterval",
		"Data":
		{
		    "Interval": {
		        "Value": 0,
		        "Multiplier": 1
		    },
		    "Unit": "kW",
		    "Data": [
		        {
		            "From": "2019-12-31T23:00:00Z",
		            "Value": 22.8,
		            "Flag": 9
		        },
		        {
		            "From": "2020-01-01T23:00:00Z",
		            "Value": 23.6,
		            "Flag": 9
		        }
		    ]
		}
	},
	{
		"Name": "Spontaneous_HourInterval",
		"Data":
		{
		    "Interval": {
		        "Value": 0,
		        "Multiplier": 1
		    },
		    "Unit": "kW",
		    "Data": [
		        {
		            "From": "2019-12-31T23:00:00Z",
		            "Value": 22.8,
		            "Flag": 9
		        },
		        {
		            "From": "2020-01-01T23:00:00Z",
		            "Value": 23.6,
		            "Flag": 9
		        }
		    ]
		}
	}
]

API response

Since we are using a unit that is supported as unit for time series data without interval (it has aggregation type average), writing this data will succeed on both time series, without problems.

Time series	Spontaneous_NoInterval	Spontaneous_HourInterval
Request Status	201	201
Explanation	Successfully saved	Successfully saved

WebTSM Services API response - write spontaneous data

XML

[
    {
        "ID": 13131,
        "Name": "Spontaneous_NoInterval",
        "Status": 201,
        "Message": "Time series data successfully saved.",
        "Details": null
    },
    {
        "ID": 13132,
        "Name": "Spontaneous_HourInterval",
        "Status": 201,
        "Message": "Time series data successfully saved.",
        "Details": null
    }
]

Reading Data

To illustrate the results of the previous write process, we run a read request against both time series with the following results.

Timestamp	Spontaneous_NoInterval	Spontaneous_HourInterval
2019-12-31T23:00:00Z	22.8	22.8
2020-01-01T23:00:00Z	23.6	23.6

Note

When requesting time series data (e.g. using /repositories/:repository/timeseries/:id|name/data) besides specifying the time period (using from and to parameter) you may also specify the interval and interval multiplier for the output raster.

If you do not specify any interval, each time series data will be output in the original interval specified on the respective time series definition. See also Intervals and raster.

WebTSM Services API example - read spontaneous data

XML

GET {{baseUrl}}/repositories/ZAMS/timeseriescollections/data?name=Spontaneous_NoInterval&name=Spontaneous_HourInterval&from=2019-12-31T23:00:00Z&to=2020-01-02T23:00:00Z
[
    {
        "ID": 13131,
        "Name": "Spontaneous_NoInterval",
        "Status": 200,
        "Message": null,
        "Details": null,
        "Data": {
            "Interval": {
                "Value": 0,
                "Multiplier": 1
            },
            "Unit": "KWh",
            "Data": [
                {
                    "From": "2019-12-31T23:00:00Z",
                    "Value": 22.8,
                    "Flag": 9
                },
                {
                    "From": "2020-01-01T23:00:00Z",
                    "Value": 23.6,
                    "Flag": 9
                }
            ]
        }
    },
    {
        "ID": 13130,
        "Name": "Spontaneous_HourInterval",
        "Status": 200,
        "Message": null,
        "Details": null,
        "Data": {
            "Interval": {
                "Value": 0,
                "Multiplier": 1
            },
            "Unit": "KWh",
            "Data": [
                {
                    "From": "2019-12-31T23:00:00Z",
                    "Value": 22.8,
                    "Flag": 9
                },
                {
                    "From": "2020-01-01T23:00:00Z",
                    "Value": 23.6,
                    "Flag": 9
                }
            ]
        }
    }
]

(Complementary Info): Spontaneous Data With Not Supported Unit

Spontaneous time series with units other than "average" cannot always be calculated correctly. Therefore, correct results are not guaranteed when using non-"average" units. Starting with PowerTSM 4+, the behavior of acyclic time series has been improved, which makes the calculation clearer and thus enables support for units other than "average".

Data With Interval

Let's also take a look at how HAKOM TSM handles data with interval under the following circumstances:

Saving on spontaneous time series without interval
Saving on spontaneous time series with hour interval
Reading from a spontaneous time series without interval
Reading from a spontaneous time series with hour interval

Writing Data

To demonstrate data with interval, we will use the same data as above:

Timestamp	Value
2019-12-31T23:00:00Z	22.8
2020-01-01T23:00:00Z	23.6

In this example we will not keep each value valid until the next day's value (as it was the case with the spontaneous data). For better visibility we will close both values with a 0 value on the next available time stamp on the raster:

Timestamp	Value
2019-12-31T23:00:00Z	22.8
2020-01-01T00:00:00Z	0
2020-01-01T01:00:00Z	0
...	0
2020-01-01T23:00:00Z	23.6
2020-01-02T00:00:00Z	0

Note

To tell the API that the data is in an hourly granularity, we will set interval header to "Hour" (which is represented by the integer 3) in our self contained data format:

XML

"Interval":{
	"Value": 3,
	"Multiplier": 1
}

WebTSM Services API example - write data with hour interval

XML

POST {{baseUrl}}/repositories/ZAMS/timeseriescollections/data
[
	{
		"Name": "Spontaneous_NoInterval",
		"Data":
		{
		    "Interval": {
		        "Value": 3,
		        "Multiplier": 1
		    },
		    "Unit": "kWh",
		    "Data": [
		        {
		            "From": "2019-12-31T23:00:00Z",
		            "Value": 22.8,
		            "Flag": 9
		        },
		        {
		            "From": "2020-01-01T23:00:00Z",
		            "Value": 23.6,
		            "Flag": 9
		        }
		    ]
		}
	},
	{
		"Name": "Spontaneous_HourInterval",
		"Data":
		{
		    "Interval": {
		        "Value": 3,
		        "Multiplier": 1
		    },
		    "Unit": "kWh",
		    "Data": [
		        {
		            "From": "2019-12-31T23:00:00Z",
		            "Value": 22.8,
		            "Flag": 9
		        },
		        {
		            "From": "2020-01-01T23:00:00Z",
		            "Value": 23.6,
		            "Flag": 9
		        }
		    ]
		}
	}
]

API response

The above shown data write requests will result in the API responding that on both time series (spontaneous without and with interval) data with a fixed interval could be successfully saved.

Time series	Spontaneous_NoInterval	Spontaneous_HourInterval
Request Status	201	201
Explanation	Successfully saved	Successfully saved

WebTSM Services API response - write data with hour interval

XML

[
    {
        "ID": 13132,
        "Name": "Spontaneous_NoInterval",
        "Status": 201,
        "Message": "Time series data successfully saved.",
        "Details": null
    },
    {
        "ID": 13133,
        "Name": "Spontaneous_HourInterval",
        "Status": 201,
        "Message": "Time series data successfully saved.",
        "Details": null
    }
]

Reading Data

To illustrate the results of the previous write process, we run a read request against both time series with the following results.

Timestamp	Spontaneous_NoInterval	Spontaneous_HourInterval
2019-12-31T23:00:00Z	22.8	22.8
2020-01-01T00:00:00Z	0	0
2020-01-01T23:00:00Z	23.6	23.6
2020-01-02T00:00:00Z	0	0

WebTSM Services API example - read data

XML

{{baseUrl}}/repositories/ZAMS/timeseriescollections/data?name=Spontaneous_NoInterval&name=Spontaneous_HourInterval&from=2019-12-31T23:00:00Z&to=2020-01-02T23:00:00Z
[
    {
        "ID": 13132,
        "Name": "Spontaneous_NoInterval",
        "Status": 200,
        "Message": null,
        "Details": null,
        "Data": {
            "Interval": {
                "Value": 0,
                "Multiplier": 1
            },
            "Unit": "KWh",
            "Data": [
                {
                    "From": "2019-12-31T23:00:00Z",
                    "Value": 22.8,
                    "Flag": 9
                },
                {
                    "From": "2020-01-01T00:00:00Z",
                    "Value": 0.0,
                    "Flag": 19
                },
                {
                    "From": "2020-01-01T23:00:00Z",
                    "Value": 23.6,
                    "Flag": 9
                },
                {
                    "From": "2020-01-02T00:00:00Z",
                    "Value": 0.0,
                    "Flag": 19
                }
            ]
        }
    },
    {
        "ID": 13133,
        "Name": "Spontaneous_HourInterval",
        "Status": 200,
        "Message": null,
        "Details": null,
        "Data": {
            "Interval": {
                "Value": 0,
                "Multiplier": 1
            },
            "Unit": "KWh",
            "Data": [
                {
                    "From": "2019-12-31T23:00:00Z",
                    "Value": 22.8,
                    "Flag": 9
                },
                {
                    "From": "2020-01-01T00:00:00Z",
                    "Value": 0.0,
                    "Flag": 19
                },
                {
                    "From": "2020-01-01T23:00:00Z",
                    "Value": 23.6,
                    "Flag": 9
                },
                {
                    "From": "2020-01-02T00:00:00Z",
                    "Value": 0.0,
                    "Flag": 19
                }
            ]
        }
    }
]

Cases With and Without Interval Compared

In summary, the following side-by-side comparison shows the difference between data without and with interval:

Data Without Interval

Data With Interval

The respective value is valid until the next change

Timestamp	Value
2019-12-31T23:00:00Z	22.8
2020-01-01T23:00:00Z	23.6

The respective value is valid until the end of the underlying interval

Timestamp	Value
2019-12-31T23:00:00Z	22.8
2020-01-01T00:00:00Z	0
2020-01-01T23:00:00Z	23.6
2020-01-02T00:00:00Z	0

We recommend the following video under (3.11.1) Video Tutorials:

Types of Time Series