Result Prediction for European Football Games

Xiaowei Liang Zhuodi Liu Rongqi Yan

A53204220 A53214122 A53203966
Apt 312, 9450 Gilman Drive Apt 615, 9450 Gilman Drive Apt 1106, One Miramar Street
La Jolla, California 92092 La Jolla, California 92092 La Jolla, California 43017-6221
xil568@eng.ucsd.edu zhl384@eng.ucsd.edu roy007@eng.ucsd.edu

KEYWORDS
Classification, Logistic Regression

1 INTRODUCTION
As one of the most popular sports among the world, football games
attracts huge attention and occupies a great share of gaming indus-
try. It is a difficult task to predict the result of one match for it may
involve a lot of circumstances: ability, chance and also luck! Even
the most famous bookmaker can make many wrong predictions. In
this paper, we aim to build a model that can give a comparatively ac-
curate prediction on game results based on data of previous matches
and relevant analysis. Our model shows ability of dealing with big
data and can beats bookmakers on game results classification task.
Figure 1: Probability of match result
2 DATABASE AND TASK
2.1 Database Introduction
Our data comes from a public database called European Soccer
Database. It contains information of more than 25,000 matches
and more than 10,000 players. There are 7 tables in this data-
base: Country, League, Match, Player, Player Attributes, Team,
and Team Attributes.
2.1.1 Country and League. In these tables, all of the 9 countries
and 9 leagues where match data were collected are recorded.
2.1.2 Match. The main information of the database is in Match
table. In every line, it contains detailed data for one match: country
and league, both teams, starting players, basic technical statistics,
betting odd and game result. It only contains id of team and player
that can be used to connect to other tables.
Figure 2: Probability of match result in different leagues
2.1.3 Player and Player Attributes. It contains the personal in-
formation for more than 10,000 players. All of these information
are from FIFA, which includes and does not restricts to: birthday, defeat 25% respectively.
overall rating, detailed ability, etc.
2.1.4 Team and Team Attributes. It contains team technique 2.2.2 League Difference. In different league, match results may
information for several hundred teams. These information can be vary by factors as coach, game schedule, etc. In Figure 2, we show
used to reveal the ability of every team. the match results difference by league.

2.2 Database Exploration

We explore the data-set in following aspects.
2.2.1 Label Information. The label for match is one of ’win’,
’draw’ and ’defeat’ for our task is to predict the result of the match.
We generate this label by compare the information of home-goal
and away-goal from the database. As shown in Figure 1, home team
wins around 46% games from all 25,000+ matches, draw 29% and
2.2.3 Odd. Bookmakers hire expert data analyst to generate
odds for matches. Thus, odd is a useful feature to judge the the
ability of both teams. In Figure 3, we show the odd distribution for
different match results. Apparently, games winning by home teams
are always with high odds and games winning by away teams are
with low odds.
 Figure 3: Odd distribution by probability
Figure 4: Player ability distribution
2.2.4 Starting Player Ability. The main factor to determine the
ability of one team is the ability of its eleven starting players. To
judge the ability of players, we get the overall rating for players in
table Player Attributes, which is from FIFA data. In Figure 4, we
show the variance of ability of different players.
2.2.5 Team Technical Ability. Best players cannot ensure best
team without a suitable build of team. Thus, we add some team
technical indicators to show the tactical ability of the team. We
measure it in seven aspects: build up play speed, build up play
passing, chance creation passing, chance creation crossing, chance
creation shooting, defense pressure and defense aggression. We
show these data in Figure 5.
2.2.6 Results of Last Several Matches. Results of last several
matches can reveal the competing condition of one team and his-
tory results between two teams can reveal . We calculate the average
match goal and average points win for both teams in last several
matches and calculate their average goal in the matches they met
before. In following figure, we show corresponding data.
2
Figure 5: Team ability distribution
Figure 6: Probability of match result in different leagues
2.2.7 Rest Time after Last Match. When a team encounters hard
match schedules, it can hardly perform its normal level in the match.
Even the best team may fall in struggling situation in tiredness.
In Figure 6, we show the average rest time for teams in different
league.
2.3 Task
We aim to predict the result of match with useful features we get
from the database. The prediction is in three labels: win, draw and
defeat. We evaluate models by the percentage of correct predictions.
2.4 Related Work
In Kaggle, there are many discussion and analysis about the foot-
ball match dataset. In general, they can be separated into two
categories: match outcome prediction and player ability prediction.
For match outcome prediction which is also what we focus on in
this paper, many classifiers such as Logistic Regression and Random
Forest are compared to each other to find the best classification
approach. Many different features are considered including betting
information, starting playersfi overall abilities, recent match results,
etc. There are also some work trying to predict goals or scores of
matches. Another category player ability prediction tries to predict
whether a football player can be a good player or not. In fiCan
you be a good football player?fi, the author analyzes playersfi FIFA
ratings according to different leagues, playersfi height and weight,
position, age and different technique ability such as crossing and
finishing. Then the author uses Decision Tree to classify good play-
ers whose FIFA rating is over 70. The author finally achieves 0.668
accuracy by using Decision Tree.
Some users in Kaggle also present interesting findings on this
dataset. In fiThe fans stay loyal, the players move onfi, the author
explores the player transfers among different teams in different
leagues. He also presents several graphs of player transfers in some
big football teams like Manchester United and Arsenal. He lists
some players who change teams frequently as well. Some research
concentrate on home advantage analysis and conclude that home
team has an obvious advantage over away team which satisfies
our intuition. In fiThe Most Predictable Leaguefi, the author uses
B365 betting information to calculate entropy of each league and
finds that Spain League (La Liga) is the most predictable league
and French League is the least predictable one. The main reason he
finds is that the match results of two giants of La Liga Barcelona
FCB and Real Madrid are very predictable.
3 MODEL
We select some features of match from the database, as we have
illustrated in last section, and use PCA method to extract a smaller
number of uncorrelated features from them. With these features,
we train a classifier based on logistic regression to predict the result
of match.
3.1 Selected Features
3.1.1 Feature of League. We use one-hot to express this feature.
The dimension of feature is 9. The entry of corresponding league
that the match is from equals 1 and others are 0.
3.1.2 Feature of Odd. We use a 6-dimensional feature to express
odd information. There are win by home team odd, draw odd and
win by away team odd from two main bookmaker BET365 and
BW. Notice that we change the odd into possibility when gener-
ating features instead of using origin odd number. This operation
can eliminate the interference of difference of benefits of different
bookmakers.
3.1.3 Feature of Players. This is a 22-dimensional feature, which
contains overall ability for all starting players of both teams.
3.1.4 Feature of Team. We measure the ability of team in a
7-dimensional feature. In each column, it is the minus ability of
corresponding team ability as we illustrate in last section from
home team to away team.
3.1.5 Feature of Last Matches. This is a 6-dimensional feature,
four of which are average goals and average points win for both
team in last 10 matches, and the other two considering their record
in matches directly between these two teams.
3.1.6 Feature of Rest Time. This is a feature with only one di-
mension that shows the rest time difference between two teams.
3
3.2 Principle Component Analysis
Principal component analysis (PCA) is a statistical procedure that
uses an orthogonal transformation to convert a set of observations
of possibly correlated variables into a set of values of linearly un-
correlated variables called principal components. The number of
principal components is less than or equal to the number of original
variables or the number of observations. This transformation is
defined in such a way that the first principal component has the
largest possible variance (that is, accounts for as much of the vari-
ability in the data as possible), and each succeeding component in
turn has the highest variance possible under the constraint that it
is orthogonal to the preceding components. The resulting vectors
are an uncorrelated orthogonal basis set. PCA is sensitive to the
relative scaling of the original variables.
The number of dimension of our feature is fifty-three, which is
relatively large. And since that we have features like odd from
BET365 and BW, or player ability and team ability, which may be
correlated to each other. Thus, we firstly do PCA operation on
our origin features to generate low-dimensional and uncorrelated
features, and make the model easier and quicker to be established.
In our experiment, we select top 10 principle components from the
features and use them as new features.
3.3 Logistic Regression
Logistic regression is a regression model that uses sigmoid function
to calculate the prediction of data point and map this to binary
classification. Logistic regression was developed by statistician
David Cox in 1958[4], and it is used to estimate the probability of
a binary response based on one or more predictor variables. It is
widely used in machine learning, medical fields and social science.
Logistic regression takes probability distribution into account. It
has good performance and is still cheap to train compared with
some complex models. Thus, we choose logistic regression as our
main method in this approach.
In our experiment, for we aim to do a prediction task with three
labels, we use multi-class logistic regression classifier. Multi-class
logistic regression is a classification method that generalizes logistic
regression to multi-class problems. We use one-vs-rest scheme to
solve the problem of multi-classification. That is, for each label, we
train a logistic regression model to determine the choice of using
this label and not-using this label. After that, we choose the most
possible label for the data point. Since we have three labels in our
prediction, we in fact train three logistic regression models behind
this multi-class logistic regression model.
We can use methods of gradient descent to train the model. In our
experiment, we use ’liblinear’ method in numpy to train the model.
4 EXPERIMENTS
4.1 Compared methods
[1]
4.1.1 Baseline. The dataset contains the gambling odds of sev-
eral gambling companies assigned to each football team. So if we
just naively predict the result of football game according to the odd
values, we can get an accuracy as the baseline of model, which is
about 46.04%.
4.1.2 Random Forest. Random Forest can be take as an ensem-
ble of decision trees, it use averaging to averaging to improve the
predictive accuracy and control over-fitting. Random forest are
good to use at the first stage when you donfit know the underlying
model, or when you want to build a decent model in a short time
because it has very few parameters to tune and can be used quite
efficiently with default parameter settings. By adjusting the num-
ber of trees that were used in the model, we found that the best
performance of 53.50% was obtained when the number of trees was
set to 200.
4.1.3 AdaBoost. AdaBoost algorithm trains a sequence of learn-
ers such as decision trees on repeatedly modified versions of data,
then combine the predictions of all learners through a weighted
majority sum to produce the final prediction. At first, we set equal
weights to each boosting iteration, in each successive iteration, the
weights are individually modified, those training examples that
were incorrectly predicted by the boosted model have their weights
increased, whereas the weights are decreased for those that were
predicted correctly. Thus, AdaBoost is a precise classifier but its
prediction will be affected by noise in dataset and it is more time-
consuming. In our experiment, when the maximum number of
estimators of AdaBoost is set to 150, we get the best prediction
accuracy of 51.04%
4.1.4 K-Nearest Neighbour. K-Nearest Neighbors algorithm is
an instanced-based method used for classification and regression by
calculating the distance between points, its output is a class mem-
bership. An object is classified by majority vote of its neighbors,
with the object being assigned to the class most common among
its k nearest neighbors. There is no explicit training phase or it is
very minimal, which means the training phase is pretty fast. All
the training data is needed during the testing phase. We run K-NN
model by varying the number of neighbours that were used and
we found that the best accuracy of 43.47% was obtained when the
number of neighbours was equal to 20.
4.1.5 Gaussian Naive Bayes. Bayes theorem[3] is based on con-
ditional probability. The naive Bayes classifier assumes all the
features are independent to each other. Even if the features de-
pend on each other or upon the existence of the other features. A
Gaussian Naive Bayes algorithm is a special type of NB algorithm.
Itfis specifically used when the features have continuous values.
Itfis also assumed that all the features are following a gaussian
distribution i.e, normal distribution. Our Gaussian Naive Bayes
model gives an accuracy of 47.55%.
4.1.6 Support Vector Machine. Support Vector Machine is a su-
pervised machine learning technique used for both both classifica-
tion or regression problems.[2] The linear SVM can be extended
to a nonlinear classifier by first using a kernel function to map the
input pattern into a higher dimensional space. The nonlinear SVM
classifier so obtained is linear in terms of the transformed data but
nonlinear in terms of the original data. The best accuracy of SVM
is 53.40%.
4
Figure 7: Prediction Probability
4.2 Result
After comparing the accuracy of different models[5], we get the
best performance when using logistic regression. After analyzing
and comparing the features of different models, we get some disad-
vantages of other models in this problem. The prediction of random
forest is kind of unbalanced because we have unbalanced rate of
’Win’, ’Draw’ and ’Default’ labels in dataset, the classifier tends
to assign more weights to the larger class. The performance of
AdaBoost classifier is limited as the dataset contains some noise
like the data from FIFA and ability data of players. K-Nearest Neigh-
bours assume that every feature in dataset has equal weight, thus
will case some error when making prediction. Gaussian Naive Bayes
didn’t produce satisfied result because some features in dataset may
not be independent to each other, and our dataset is a high dimen-
sional dataset. For support vector machine model, as we have a
large dataset and our problem is likely to be a nonlinear problem,
it is hard to find a suitable sigmoid function of SVM, even though
the accuracy of SVM is close to the accuracy of logistic regression.
5 CONCLUSIONS AND FURTHER WORK
In this paper, we present a efficient multi-class classifier to predict
the result of European football matches. Firstly, we explore the
database and generate useful features for training model. We takes
the features of league difference, odd, player ability, team ability,
result of last several matches and rest time after last match into
account and appropriately modify these features to make them suit
training. Secondly, we use PCA method to compress the data to a
low-dimensional scheme. After all, we train a multi-class logistic
regression classifier to make prediction of the match. Our method
outperforms some other models and get satisfying results. We beats
the bookmakers by about 9 percent. In fact, if we buy odd as our
prediction, when we spend one dollar at one match, we can earn
about 3,360 dollar after 6,000 attempts.
Due to time limit, we cannot implement all of our ideas in the
experiment. Thus, there are still several aspects that we can improve
our prediction. Firstly, we can take rest time of player into account.
Some essential players may rest by rotation so they can perform
better in hard schedules. Secondly, ability of coach, weather, and
formation can be features that influence the result of match. Thirdly,
in our experiment, we consider the ability of player independently.
However, in real match, players should be judged with player in
corresponding place in the other team (for example one striker
versus defenders in the other team). Fourthly, we can use data from
Football Manager instead of FIFA, which may be more detailed and
more precise. Besides, we can de some prediction on the second
round of two-round knockout games, which are more attracting in
football games.
However, even if we design a extraordinary classifier, we can never
predict a miracle like FC Barcelona did in ECL quarter final.

REFERENCES
[1] Thomas G Dietterich. 2000. Ensemble methods in machine learning. In Interna-
tional workshop on multiple classifier systems. Springer, 1–15.
[2] Josip Hucaljuk and Alen Rakipović. 2011. Predicting football scores using ma-
chine learning techniques. In MIPRO, 2011 Proceedings of the 34th International
Convention. IEEE, 1623–1627.
[3] Anito Joseph, Norman E Fenton, and Martin Neil. 2006. Predicting football results
using Bayesian nets and other machine learning techniques. Knowledge-Based
Systems 19, 7 (2006), 544–553.
[4] David S Stoller. 1958. Some queuing problems in machine maintenanc. Naval
Research Logistics (NRL) 5, 1 (1958), 83–87.
[5] Ian H Witten, Eibe Frank, Mark A Hall, and Christopher J Pal. 2016. Data Mining:
Practical machine learning tools and techniques. Morgan Kaufmann.