Posted on Sat 21 February 2015 in Statistics
In the last part we have looked at the basic problem of discrete entropy estimation. In this article we will see a number of proposals of improved estimators.
In 1955 George Miller proposed a simple correction to the naive plugin estimator \(\hat{H}_N\) by adding the constant …
Posted on Sat 07 February 2015 in Statistics
Estimation of the entropy of a random variable is an important problem that has many applications. If you can estimate entropy accurately, you can also estimate mutual information, which allows you to find dependent random variables in large data sets. There are numerous applications.
The setting of discrete entropy estimation …
Posted on Thu 29 January 2015 in Machine Learning, Structured Prediction
In December 2014, just in time for NIPS, MIT Press released an edited volume on structured prediction models and their applications in natural language processing, computer vision, and computational biology.
Advanced Structured Prediction, Editors Sebastian Nowozin, Peter V. Gehler, Jeremy Jancsary, Christoph H. Lampert, (MIT Press, Amazon)
The volume offers …
Posted on Sun 25 January 2015 in Algorithms
Given a sequence of observed data we would often like to estimate simple quantities like the mean and variance.
Sometimes the data is available in a streaming setting, that is, we are given one sample at a time. For example, this is the case when
Posted on Sun 25 January 2015 in Personal
This is the start of my blog. This will be a quite technical blog and therefore address a more specialized audience.
The articles will cover topics in the area of machine learning, statistics, maybe some computer vision, let's see. I plan to publish one article every two weeks, but let …