|
Big Data is as much a buzz word as an apt description of a real problem: the amount of data generated per day is growing faster than our processing abilities. Hence the need for algorithms and data structures which allow us, e.g., to store, retrieve and analyze vast amounts of widely varied data that streams in at high velocity.
In this course we will limit ourselves to data mining aspects of the Big Data problem, more specifically to the problem of classification in a Big Data setting. To make algorithms viable for huge amounts of data they should have low complexity, in fact it is easy to think of scenarios where only sublinear algorithms are practical. That is, algorithms that see only a (vanishingly small) part of the data: algorithms that only sample the data.
We start by studying PAC learning, where we study tight bounds to learn (simple) concepts almost always almost correctly from a sample of the data; both in the clean (no noise) and in the agnostic (allowing noise) case. The concepts we study may appear to allow only for very simple – hence, often weak – classifiers. However, the boosting theorem shows that they can represent whatever can be represented by strong classifiers.
PAC learning algorithms are based on the assumption that a data set represents only one such concept, which obviously isn’t true for almost any real data set. So, next we turn to frequent pattern mining, geared to mine all concepts from a data set. After introducing basic algorithms to compute frequent patterns, we will look at ways to speed them up by sampling using the theoretical concepts from the PAC learning framework.
Course form
Lectures and tutorials.
Literature
The slides completed by your own lecture notes are in principle all you need. Background reading material is, however, also available:
- For the first part of the course, we largely follow the first 8 chapters of the book "Understanding Machine Learning, From Theory to Algorithms" by Shai Shalev-Shwartz and Shai Ben-David
- you can legally download the book from a webpage of the first author
- you can, of course, also buy this book.
- It is a good book, so if you want to become a data scientist, buying it is a sensible choice
- For the later parts of the course we will point to the papers that the lectures are based on. You can download these papers (again legally) from anywhere in the UU network/
|