ETSY: A rule-based approach to Event and Tracking data SYnchronization

From an analysis perspective, there are two key types of data for soccer:

1. Event data, which records information about on-the-ball actions. It provides semantic information about what is happening in the match. However, it ignores what happens off the ball (e.g., positions of other players).
2. Optical tracking data, which records the locations of all players and the ball multiple times per second. This provides important context about off-the-ball positioning. However, it lacks the semantic information about which actions are being performed on the pitch.

Consequently, the richest analyses require integrating information from both types of data sources.

Unfortunately, integrating these two data sources is challenging because they are often not time-synchronized. That is, an event with a certain timestamp does not correspond to the frame in the tracking data with the same timestamp. There are two main reasons for this. First, the clocks used in both collection processes might start at slightly different times, which introduces a constant bias. Second, because event data are manually annotated, inaccuracies in the event’s timestamps might also be due to the occasional human error. This is solved by “integrating or synchronizing” these data sources, which entails finding for each event the tracking frame that corresponds to the match situation at the moment of the event. This is a crucial albeit less discussed aspect of the analysis pipeline.

In this blog post, we describe our rule-based approach ETSY, which has a publicly available implementation. We compare and contrast it with the few existing approaches. Finally, we describe its empirical performance on a set of 313 matches.

The ETSY approach

We start with the following two data sources of a game:

1. Event data, with for each on-the-ball action its location on the pitch, its type, timestamp, result, acting player, etc. ETSY uses SPADL (i.e., a unified format to represent event data), which makes the approach applicable to any event data stream that can be transformed into this format.
2. Tracking data, with all x,y-coordinates of all players and the x,y,z-coordinates of the ball at 10 or 25 frames per second.

Intuitively, ETSY aims to find the matching frame for each event by comparing the match situation recorded in the tracking data with that recorded in the event data. It uses various simple rules to find the best match for each event. For example, “the ball should be at the same location in both data sources”, “the player that performs the action should be the same as the one that is close to the ball”, etc.

Concretely, each playing period is treated separately and ETSY uses the following two-step approach to synchronize the events with the tracking frames.

1. It considers the kickoff of each period separately and finds their matching frames. The matching tracking frame is the frame within the first 5 seconds of the game where the ball has the highest acceleration and is within 2 meters of the acting player. This step determines the constant bias between the timestamps in the event and the tracking data.¹ ETSY then adjusts the tracking data timestamps for this offset.
2. It synchronizes the remaining events of the playing period using the following three steps:
   • Retrieve the candidate set of frames within a window of $\pm t_a$ seconds of the tracking frame with the same timestamp as the event’s timestamp. The parameter $t_a$ is an action-specific hyperparameter.
   • Filter out frames from the candidate set with a timestamp that is after the last matched frame or that violate one of the event-specific rules. These event-specific rules are summarized in the table below.
   • Score each remaining frame in this window using the unweighted sum of three linear functions: a function that maps the distance between the ball and the acting player in the tracking frame to a score between 0 and 33.33, a function that maps the distance between the acting player’s location in the event and the tracking data to a score between 0 and 33.33, and a function that maps the distance between the ball’s location in the event and the tracking data to a score between 0 and 33.33. The frame with the highest total score is identified as the best matching one.

How does ETSY compare conceptually to existing approaches?

Only a few synchronization approaches have been publicly described. Below is a short summary of the main variants.

1. A rather straightforward approach is to simply assume that both data sources align after correcting for the constant bias introduced by the different clocks.
2. A more sophisticated approach by Anzer & Bauer uses a weighted sum of various features to determine the best matching frame for each event. A grid search on a manually labeled test set is used to optimize the weights.
3. A bio-informatics-inspired approach by Clark & Kwiatkowski (sync.soccer). This approach uses the Needleman-Wunsch algorithm with a self-defined scoring function to create a synchronized sequence including all events.

Both ETSY and the timestamp approach are fully automated whereas sync.soccer and the approach by Anzer & Bauer require (minimal) manual intervention. sync.soccer requires tuning the weights of the scoring function for each game because the approach does not correct for the constant bias introduced by the different clocks. This manual intervention can be removed by removing the constant bias before synchronizing. Anzer & Bauer’s approach requires an expert to manually label a test set before training the model.

Most approaches can synchronize all actions. However, Anzer & Bauer only reported results on shots and passes.

And what about experimentally?

We evaluate the approaches on 313 games of a European first-division league and compare the synchronization quality and runtime. Anzer & Bauer’s approach is left out of the experiments as we did not have access to expert labels.

The lack of ground truth labels makes it difficult to measure the quality of the resulting synchronization. Instead of accuracy, we consider two alternative metrics. First, we investigate the coverage of the resulting synchronization. This is the percent of events for which a matching frame can be found. Second, we look at an approach’s agreement with ETSY which is the percent of events for which the found frame is within s seconds from the one identified by ETSY. We analyze the agreement for different intervals and for each action category separately.

The timestamp-based approach is by far the fastest. It simply needs to retrieve the frame (if it exists) with the same timestamp as the event. However, this does not guarantee that the matched frame is actually correct. Sync.soccer is much slower due to the use of the Needleman-Wunsch algorithm which considers all frames when matching with an event. Every event is also assigned a matching frame, even in cases where there is no suitable one (e.g., due to errors in the data).

ETSY is roughly three times faster than sync.soccer and slightly slower than the trivial time-based solution. Coverage-wise, ETSY fails to match ~8% of all events. This happens when all frames in the candidate window are filtered out due to one (or multiple) of the action-specific filters.

An in-depth analysis of the failed matches reveals that, in most cases, either the filter on the timestamp or the filter on the distance between the ball and the acting player causes the failure. If there is a human error on the annotated timestamp, this could cause a drift in the data such that the candidate window does not include the correct frame at all. An error in the tracked locations might also cause a frame to be discarded. Thus, in case of errors in the data, ETSY prefers the safe option and creates no match. Avoiding mismatches is an important aspect of the synchronization as mismatched frames could negatively impact downstream analyses tasks. We believe this trade-off is worth the slightly lower coverage. A final cause of failed matches are cascading misses: in case the algorithm matches an event to a slightly incorrect frame, this error propagates and might prevent synchronizing the next couple of actions. This mainly happens when actions are performed very quickly one after another.

For most action types, we see a large agreement between sync.soccer and ETSY, indicating that ETSY’s result is similar. One exception is the set-piece category, which could indicate that some set-piece events might take much longer to perform than the fixed time window, causing ETSY to mismatch (or fail to match) the event.

The below figure shows one example of a matched pass by both sync.soccer (left) and ETSY (right). The resulting synchronization only differs by one frame, which is barely visible.

Overall, we can conclude that ETSY’s rule-based approach performs a satisfactory synchronization that is similar to the result of existing state-of-the-art approaches and safer towards errors in the data, while using less time, without manual intervention, and can be applied to all action types. The experimental analysis has highlighted some core points to improve upon in the future.

This research was presented at the Machine Learning and Data Mining for Sports Analytics workshop at ECML/PKDD 2023 and the ETSY source code is available online.