TransWikia.com

Ordering of analytical events

Software Engineering Asked by Toshakins on February 26, 2021

My team is working on an analytics system for the web app. The web app has a service-oriented architecture, and all events are gathered in one centralized analytics service. The system is sending events from the front-end and back-end; these are requirements of the product team. However, the engineering team has stuck into an ordering issue. For example, we expect that front-end event A must come before the back-end event B, but it does not happen. At the moment, we use client time as the time of the event, but the client’s clock can be completely wrong. There is a plan to avoid that issue by relying on the difference between the client’s event time and the time of the event arrival using the formula:

real event time = analytics service time – (client's event timeevent arrival time)

Still, this approach does not provide any guarantees, and we may continue to run into out-of-order events(though a small percentage is expected).

Is it even possible to guarantee the order of events while processing them on the analytics service? If yes, how? Maybe, tradeoffs I mentioned already provide a decent design. What is the state-of-the-art approach to the problem?

One Answer

Assuming your front end is not malicious and not purposefully manipulating the time, you can follow this procedure:

  1. Modify the message so that it contains two timestamps: (A) The time the event happened and (B) the current time-- both according to the client's own clock.

  2. If transmission of an event fails, the front end should keep the original value of A but keep updating B with each retry.

  3. When the event is received by the server, compute the difference between the actual system time (according to the server) and the current time reported by the client (field B). Call this difference C.

  4. Compute the actual event time as A - C. This gives you the time of the event, corrected for the difference between server time and client time. Call this D.

  5. Sort your events by D.

This procedure allows the client to use its own timestamp while truing up the client's clock to server time. It also neatly avoids any issue with time zone or DST.

Answered by John Wu on February 26, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP