TransWikia.com

How to drop points from a data series for presentation?

Data Science Asked by Leonel on December 29, 2020

We monitor long running industrial engines and we have data series that we want to present on a line chart on a web page. For instance, we have sensors that monitor the oil temperature and pressure on the engine.

There are several other similar data series on the components of the equipment.

The objective is to have a human operator identify deviations in engines, for post-analysis. Our chart will display 24 or 48 hours of engine operation and the operator may identify peaks in temperature or pressure, or on the other measurements.

As such, it is a large amount of data to present on the chart on the web page, and we’re starting to hit limitations in several places.

At 24 hours * 3600seconds/hour * 1 data point/second = 86400 data points on the chart.

This amount of points is slowing down the rendering of the web page, and is a lot of data to transfer.

We want to reduce the count of data points presented on the chart, without losing much context. So I ask:

  • How can I drop data points without losing much precision?
  • What techniques are usually applied in this scenario?

A first (naive) thought was to group them in 5-second windows and only return one data point to represent the 5-second window on the chart;

  • should I average the points in the window?
  • should I consider the maximum in the window?

Are there other techniques than grouping data points in window, to reduce the loss of meaning for the monitoring?

One Answer

Indeed this problem isn't a very simple one to deal with, despite looking very easy to conceptualise. There exist a certain number of techniques for "reducing" the number of points of timeseries, one being called "downsampling".

A little litterature : https://skemman.is/bitstream/1946/15343/3/SS_MSthesis.pdf

hope this helps,

Cheers

Answered by antounes on December 29, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP