Valuing On-the-Ball Actions in Soccer: A Critical Comparison of xT and VAEP

Assessing the impact of the individual actions performed by soccer players is a crucial task in soccer analytics. Unfortunately, both the traditional metrics (i.e., shots and assists) and their context-dependent successors (i.e., xG and xA) fall short in addressing this task as they focus on rare actions like shots and goals alone. For most players, these constitute less than 1% of all the actions they undertake. Ideally, we want to assign a value to each action performed by a soccer player that captures how helpful that action was for winning the game.

As an illustrative example, consider the through ball by Laporte below in the game against Wolverhampton on January 12, 2019.

It is pretty clear that Laporte’s pass plays a crucial role in enabling Jesus’ goal. However, in traditional football statistics, only Sané (assist) and Jesus (goal) receive credit for their actions. Laporte’s excellent pass is not valued by xG, as no shot was taken, nor by xA, because it is unlikely that Sané will score himself from that position. This illustrates why we need more advanced metrics for valuing the contributions of soccer players.

Although this is an extremely challenging problem, several football analysts have recently introduced their own framework. Some notable examples are Sarah Rudd’s Markov chain model, Nils Mackay’s xG added model, Karun Singh’s Expected Threat (xT) metric, OptaPro’s Possession Value (PV) framework, and our own Valuing Actions by Estimating Probabilities (VAEP) framework. In this blog post, we give a high-level overview of these frameworks and compare two of them: VAEP and xT. These two frameworks are canonical exemplars for the two styles of approaches that characterize all these frameworks.

Assigning values to actions…

The contributions that we can measure are inherently limited by the data we have. This post focuses on approaches for event stream or play-by-play data. That implies that the data only contains the location of the player possessing the play. The locations of all other players is unknown. This is what the event stream looks like for Laporte’s pass (6):

While obviously limited, this data is much more widely available than optical-tracking data.

Constrained by this data, the task that we try to solve is assigning a numeric value to each individual on-the-ball action in this stream. While this is straightforward for shots, it is non-trivial for all other actions due to the following two reasons:

1. Just looking at the effect of an individual action is difficult as the action may have longer range effects on how the sequence plays out. A player should not just be rewarded for moving the ball into good shooting positions, but also for moving it into ‘threatening’ positions which can in turn lead to a high probability of the ball reaching a good position for shooting. For example, in the possession below, Laporte moves the ball into a position from which Sané is unlikely to score himself, but if Sané would manage to cross the ball Jesus or Sterling are in a good position to score.

2. Even knowing the longer range effects, it is non-obvious how to assign credit to each action in the sequence. It would seem undesirable to penalize a great pass that does not lead to a shot. For example, Laporte’s cross should not get a lower rating because Silva subsequently fails to reach De Bruyne.

Therefore, these action-valuing frameworks aim to assign a value to multiple types of actions (e.g., passes, crosses, dribbles, take-ons, and shots) that quantifies how much the action contributed towards winning the game. An action’s value should reflect both the circumstances under which it was performed as well as its longer-term effects.

…by assigning values to game states

Rather than directly assigning values to actions, the existing approaches all start by assigning values to game states. To illustrate the underlying intuition, consider the two photos below.

The one on the left shows the game right before Laporte passes to Sané and the one on the right shows the game just after Sané successfully controlled the pass. The effect of the pass was to change the game state. Intuitively, the game state entails everything that has happened in a game up until now, so the current score, time left, the current location of the ball, all prior actions, etc.

Consequently, a natural way to assess the usefulness of an action is to assign a value to each game state. Then an action’s usefulness is simply the difference between the post-action game state $S_{i}$ and pre-action game state $S_{i-1}$. This can be expressed as:

where $V$ captures the value of a particular game state.

The differences among the various approaches arise in terms of (1) how they represent a game state, that is, define features such as the ball’s location or score difference that capture relevant aspects of the game at a specific point in time; and (2) assign a value to a specific game state.

Now we will look at two illustrative examples for valuing actions: xT and VAEP.

xT

The expected threat or xT model is a possession-based model. That is, it divides matches into possessions, which are periods of the game where the same team has control of the ball. The key insights underlying xT are that (1) players perform actions with the intention to increase their team’s chance of scoring, and (2) the chance of scoring can be adequately captured by only considering the location of the ball.

Point (2) means that xT represents a game state solely by using the current location of the ball. Therefore, xT overlays a $M \times N$ grid on the pitch in order to divide it into $M \cdot N$ zones. Each zone $z$ is then assigned a value $\textrm{xT}(z)$ that reflects how threatening teams are at that location, in terms of scoring:

The value of each zone can be learned with a Markov decision process. For an intuitive explanation of how this works, we refer to Karun’s blog post.

VAEP

VAEP is based on a slightly different insight than xT, which is that players tend to perform actions with two possible intentions: (1) to increase their team’s chance of scoring a goal in the near future, and (2) to decrease their team’s chance of conceding a goal in the near future. Thus VAEP values a game state as:

where $P_{score}(S_i, t)$ and $P_{concede}(S_i, t)$ are the probabilities that team $t$ which possesses the ball in state $S_i$ will respectively score or concede in the next 10 actions.

VAEP also uses a more complex representation of the game state: it considers the three last actions that happened during the game: $S_i = \left\{a_{i-2}, a_{i-1}, a_{i}\right\}$. This is what the pre- and post-action game states look like for Laporte’s pass:

However, in reality, these game states are converted to a feature-vector representation such that the $P_{score}(S_i, t)$ and $P_{concede}(S_i, t)$ values can be “learned”. That is, a gradient boosted binary classifier is trained on historical data to predict how a game state will turn out based on what happened in similar game states that arose in past games.

Implications of xT’s and VAEP’s Design Choices

xT and VAEP are similar approaches in the sense that they both value individual on-the-ball actions of soccer players by computing the differences between the game state values before and after the action. However, they approach the problem in different ways that leads to three important practical differences.

1. Types of actions valued. xT is a “ball-progression model” and only values actions that move the ball from one zone to another (i.e., passes, dribbles and crosses). Hence, it ignores defensive actions like tackles and interceptions, as well as valuable offensive actions such as take-ons within the same zone of the pitch. In contrast, VAEP’s continuous game state representation allows it to consider defensive actions like tackles, interceptions, and recoveries that do not progress the ball.

2. Risk of actions. The xT model only values an action’s offensive contribution, that is, how it changes the team’s chance of scoring. In contrast, VAEP both (1) values a game state by considering the probability of conceding and (2) reasons about what happens after turnovers. Hence, it may better capture the risks associated with taking certain actions by additionally reasoning about how an action alters a team’s chance of conceding in addition to the action’s offensive contribution.

3. Interpretability. The xT framework assigns values to game states solely based on the location of the ball. Hence, one can explain why a state receives a certain value: it is the expected future xG of a possession starting in that location. In contrast, VAEP uses a large set of features to describe a game state, which requires using a learned model (e.g., a gradient boosted tree ensemble) to value game states. As such, game state values are derived from complex interactions among a large set of features. Explaining why a particular value is assigned to a specific game state is no longer straightforward in this framework.

Player ratings

These differences have an important impact on the players who are identified as stand-out performers. To illustrate these differences, the table below lists the 25 players recording the highest output in terms of goals, assists, xT and VAEP in the Premier League 2018/19 season. Since spending more time on the pitch offers more opportunities to contribute, player ratings are normalized per 90 minutes of game time.

Both xT and VAEP clearly succeed at identifying top players. Yet, there are some major differences between both rankings. The main difference is that the xT-based rankings are biased towards creative players that complete many key passes and dribbles, while VAEP is biased towards strikers. That is because VAEP generally assigns goals high action values such that players can boost their rankings by scoring many goals. This can also be seen from the correlation between the different metrics. For VAEP ($\rho_{goals/90} = 0.25$) we obtain a stronger correlation with goals per 90 minutes than xT ($\rho_{goals/90} = 0.11$), while assists per 90 minutes is correlated stronger with xT ($\rho_{assists/90} = 0.40$) than with VAEP ($\rho_{assists/90} = 0.37$). Most importantly, however, these correlations prove that both metrics produce rankings that deviate from those produced by considering traditional goal-and assist based metrics. Hence, they are valuable metrics that can provide additional insights into player performance.

The top of the list mainly contains attacking players, whereas defensive players are typically ranked lower. In both models, offensive actions have access to higher rewards. It is thus easier for offensive players to get higher ratings than it is for defensive players.

Action ratings

To get more insights into how xT and VAEP assess different actions in different contexts, we explored four actions:

1. A (risky) backward pass in the own half. Backward passes have an interesting risk-reward trade-off as they usually open up space (reward), but also move the ball closer to the team’s own goal (risk). Only VAEP can capture the risk and reward of these passes, assigning more diverse values.

2. Recovering the ball to set up a counter-attack. Recovering the ball on the own half while the opponent is still in an offensive position gives a team the opportunity of building a fast counter-attack that exploits the opponents’ unorganized defensive positioning and thus increasing the odds of scoring a goal. The fact that the ball was only very recently recovered (and how this differs from normal open play) is a contextual clue that can only be leveraged by VAEP’s more powerful reasoning on game states. Hence, VAEP rates these actions significantly higher than xT.

3. A forward dribble into the opponent’s penalty box. xT assigns a value of zero to a major part of all forward dribbles inside the penalty box. Since xT discretizes the pitch into relatively large zones, many short dribbles do not move the ball into a different zone and therefore do not increase the xT value. Yet, these short dribbles may suffice to take on a defender and — when the ball is extremely close to the goal — small differences in location can considerably increase the odds of scoring.

4. A through ball near the opponent’s penalty box. A successful forward pass to the border of the penalty box can considerably raise the odds of scoring a goal for two reasons: (1) it moves the ball closer to the goal, and (2) it often bypasses at least one player from the opposing team. On average, xT values the positional advantage gained by a through ball more than VAEP. Due to the detailed game state representation used by VAEP, it is hard to explain why VAEP assigns lower values to these types of actions than xT. Using only the reasoning behind xT, one possible explanation for this is that xT is better than VAEP at capturing the positional advantage.

Conclusion

xT and VAEP are two prominent approaches for the important task of valuing actions in a soccer match. Key differences arise in how each approach represents the game state and what actions are valued. These lead to interesting differences such as VAEP better capturing the risk-reward trade-off of actions and xT being more interpretable. Importantly, both metrics produce rankings that deviate from those produced by considering traditional metrics (goals or assists). Hence, they provide additional insights into player performance and are a useful statistical tool in soccer analytics.

A paper about this research will be presented at the AAAI-20 Workshop on AI in Team Sports. A Python implementation of xT and VAEP is available online.