Sebastian Nowozins slow bloghttp://www.nowozin.net/sebastian/blog/Wed, 05 Dec 2018 08:30:00 +0100Debiasing Approximate Inferencehttp://www.nowozin.net/sebastian/blog/debiasing-approximate-inference.html<p>This year at <a href="http://neurips.cc/">NeurIPS 2018</a> the
<a href="http://approximateinference.org/">Symposium on Advances in Approximate Bayesian
Inference</a> discussed challenges and advances
in approximating probabilistic inference in rich models. It was a genuinely
exciting program!</p>
<p>I was lucky enough to give an invited talk at …</p>Sebastian NowozinWed, 05 Dec 2018 08:30:00 +0100tag:www.nowozin.net,2018-12-05:/sebastian/blog/debiasing-approximate-inference.htmlMLSS 2018 in Madridhttp://www.nowozin.net/sebastian/blog/mlss-2018-in-madrid.html<p>The <a href="http://mlss.cc/"><em>Machine Learning Summer Schools</em> (MLSS)</a> is the largest
and most popular machine learning summer school series. For two weeks in
August and September the <a href="http://mlss.ii.uam.es/mlss2018/index.html">MLSS 2018 is held in Madrid</a>.</p>
<p>I am happy to speak on the topics of …</p>Sebastian NowozinSun, 02 Sep 2018 23:00:00 +0200tag:www.nowozin.net,2018-09-02:/sebastian/blog/mlss-2018-in-madrid.htmlDo Bayesians Overfit?http://www.nowozin.net/sebastian/blog/do-bayesians-overfit.html<p><em>TLDR:</em> Yes, and there are precise results, although they are not as well known
as they perhaps should be.</p>
<p>Over the last few years I had many conversations in which the statement was
made that Bayesians methods are generally immune …</p>Sebastian NowozinWed, 11 Jul 2018 22:30:00 +0200tag:www.nowozin.net,2018-07-11:/sebastian/blog/do-bayesians-overfit.htmlStable GAN Models and Creative Machineshttp://www.nowozin.net/sebastian/blog/stable-gan-models-and-creative-machines.html<p>We just published an <a href="https://www.microsoft.com/en-us/research/blog/will-machines-one-day-creative-humans/">article discussing our recent work on stabilizing generative adversarial networks</a>.</p>Sebastian NowozinMon, 04 Dec 2017 15:00:00 +0100tag:www.nowozin.net,2017-12-04:/sebastian/blog/stable-gan-models-and-creative-machines.htmlNIPS 2016 Generative Adversarial Training workshop talkhttp://www.nowozin.net/sebastian/blog/nips-2016-generative-adversarial-training-workshop-talk.html<p>The biggest AI conference of the year has just ended:
<a href="http://nips.cc/">NIPS</a> in Barcelona broke all records this year and the
program was exciting as always. It certainly remains my favorite conference
to attend.</p>
<p>One of the best things about NIPS …</p>Sebastian NowozinSat, 10 Dec 2016 22:30:00 +0100tag:www.nowozin.net,2016-12-10:/sebastian/blog/nips-2016-generative-adversarial-training-workshop-talk.htmlBook Review: Computer Age Statistical Inferencehttp://www.nowozin.net/sebastian/blog/book-review-computer-age-statistical-inference.html<p><img alt="Book cover: Computer Age Statistical
Inference" src="http://www.nowozin.net/sebastian/blog/images/computer-age-statistical-inference.jpg"></p>
<p>A new book, <a href="http://www.cambridge.org/us/academic/subjects/statistics-probability/statistical-theory-and-methods/computer-age-statistical-inference-algorithms-evidence-and-data-science">Computer Age Statistical Inference: Algorithms, Evidence, and Data
Science</a> by
<a href="http://statweb.stanford.edu/~ckirby/brad/">Bradley Efron</a> and
<a href="http://web.stanford.edu/~hastie/">Trevor Hastie</a>, was released in July this
year. I finished reading it a few weeks ago and this is a short review from
the …</p>Sebastian NowozinWed, 23 Nov 2016 21:00:00 +0100tag:www.nowozin.net,2016-11-23:/sebastian/blog/book-review-computer-age-statistical-inference.htmlStreaming Log-sum-exp Computationhttp://www.nowozin.net/sebastian/blog/streaming-log-sum-exp-computation.html<p>A common numerical operation in statistical computing is to compute</p>
<div class="math">$$\log \sum_{i=1}^n \exp x_i,$$</div>
<p>where <span class="math">\(x_i \in \mathbb{R}\)</span>, and <span class="math">\(n\)</span> is potentially very large.</p>
<p>We can implement the above computation by exponentiating each number, then
summing …</p>Sebastian NowozinSun, 08 May 2016 21:30:00 +0200tag:www.nowozin.net,2016-05-08:/sebastian/blog/streaming-log-sum-exp-computation.htmlWhere will Artificial Intelligence come from?http://www.nowozin.net/sebastian/blog/where-will-artificial-intelligence-come-from.html<p>Artificial Intelligence (AI) is making progress in great strides, or at least
it appears so!
Almost no week passes by without some major announcements of new challenges
solved by AI technology or new products powered by AI.</p>
<p>Indeed many quantifiable …</p>Sebastian NowozinWed, 20 Apr 2016 23:30:00 +0200tag:www.nowozin.net,2016-04-20:/sebastian/blog/where-will-artificial-intelligence-come-from.htmlThe Best of Unpublished Machine Learning and Statistics Bookshttp://www.nowozin.net/sebastian/blog/the-best-of-unpublished-machine-learning-and-statistics-books.html<p>Nowadays authors in the fields of statistics and machine learning often choose
to write their books openly by publishing early draft versions.
For popular books this creates a lot of feedback and in the end clearly
improves the final book …</p>Sebastian NowozinTue, 09 Feb 2016 23:00:00 +0100tag:www.nowozin.net,2016-02-09:/sebastian/blog/the-best-of-unpublished-machine-learning-and-statistics-books.htmlThe Fair Price to Pay a Spy: An Introduction to the Value of Informationhttp://www.nowozin.net/sebastian/blog/the-fair-price-to-pay-a-spy-an-introduction-to-the-value-of-information.html<p><img alt="Spy image" src="http://www.nowozin.net/sebastian/blog/images/voi-spy-illustration.png"></p>
<p>(This article covers the decision-theoretic concept of <em>value of information</em>
through a classic example.)</p>
<p>What is the value of a piece of information?</p>
<p>It depends.
Two factors determine the value of information:
first, whether the information is new to you …</p>Sebastian NowozinSat, 09 Jan 2016 22:30:00 +0100tag:www.nowozin.net,2016-01-09:/sebastian/blog/the-fair-price-to-pay-a-spy-an-introduction-to-the-value-of-information.htmlICCV 2015, Day 4http://www.nowozin.net/sebastian/blog/iccv-2015-day-4.html<p>This article summarizes the fourth day of the <a href="http://pamitc.org/iccv15/">ICCV
2015</a> conference, the International Conference on
Computer Vision.
A summary of the <a href="http://www.nowozin.net/sebastian/blog/iccv-2015-day-1.html">first day</a>,
<a href="http://www.nowozin.net/sebastian/blog/iccv-2015-day-2.html">second day</a>, and
<a href="http://www.nowozin.net/sebastian/blog/iccv-2015-day-3.html">third day</a> is also available.</p>
<h2>ICCV 2017 and 2019</h2>
<p>ICCV 2017 will be in …</p>Sebastian NowozinWed, 16 Dec 2015 23:50:00 +0100tag:www.nowozin.net,2015-12-16:/sebastian/blog/iccv-2015-day-4.htmlICCV 2015, Day 3http://www.nowozin.net/sebastian/blog/iccv-2015-day-3.html<p>This article summarizes the third day of the <a href="http://pamitc.org/iccv15/">ICCV
2015</a> conference, the International Conference on
Computer Vision.
A summary of the <a href="http://www.nowozin.net/sebastian/blog/iccv-2015-day-1.html">first day</a> and
<a href="http://www.nowozin.net/sebastian/blog/iccv-2015-day-2.html">second day</a> is also available.</p>
<h2>Interesting Papers</h2>
<h3>Registering Images to Untextured Geometry Using Average Shading Gradients …</h3>Sebastian NowozinWed, 16 Dec 2015 01:00:00 +0100tag:www.nowozin.net,2015-12-16:/sebastian/blog/iccv-2015-day-3.htmlICCV 2015, Day 2http://www.nowozin.net/sebastian/blog/iccv-2015-day-2.html<p>This article summarizes the second day of the <a href="http://pamitc.org/iccv15/">ICCV
2015</a> conference, the International Conference on
Computer Vision.
A summary of the <a href="http://www.nowozin.net/sebastian/blog/iccv-2015-day-1.html">first day</a> is also available.</p>
<h2>Awards</h2>
<p>The following awards were given at ICCV 2015.</p>
<h3>Achievement awards</h3>
<ul>
<li>PAMI Distinguished Researcher …</li></ul>Sebastian NowozinTue, 15 Dec 2015 01:20:00 +0100tag:www.nowozin.net,2015-12-15:/sebastian/blog/iccv-2015-day-2.htmlICCV 2015, Day 1http://www.nowozin.net/sebastian/blog/iccv-2015-day-1.html<p><a href="http://pamitc.org/iccv15/">ICCV 2015</a>, the International Conference on
Computer Vision, is one of the premier venues for computer vision research,
together with the CVPR conference.
This ICCV is happening in Santiago, Chile, a beautiful city with amazing food.</p>
<p>The computer vision community …</p>Sebastian NowozinMon, 14 Dec 2015 23:00:00 +0100tag:www.nowozin.net,2015-12-14:/sebastian/blog/iccv-2015-day-1.htmlTen Tips for Writing CS Papers, Part 2http://www.nowozin.net/sebastian/blog/ten-tips-for-writing-cs-papers-part-2.html<p>This continues the <a href="http://www.nowozin.net/sebastian/blog/ten-tips-for-writing-cs-papers-part-1.html">first part</a> on tips to write
computer science papers.</p>
<h2>6. Ideal Structure of a Paragraph</h2>
<p>A paper has different levels of formal structure:
sections, subsections, paragraphs, sentences.
It is important to ensure that the structure of the …</p>Sebastian NowozinThu, 10 Dec 2015 22:30:00 +0100tag:www.nowozin.net,2015-12-10:/sebastian/blog/ten-tips-for-writing-cs-papers-part-2.htmlTen Tips for Writing CS Papers, Part 1http://www.nowozin.net/sebastian/blog/ten-tips-for-writing-cs-papers-part-1.html<p>As a non-native English speaker I can relate to the challenge of writing
concise and clear English.
Scientific writing is particularly challenging because the audience is only
partially known at the time of writing: at best, the paper will still …</p>Sebastian NowozinSun, 29 Nov 2015 21:00:00 +0100tag:www.nowozin.net,2015-11-29:/sebastian/blog/ten-tips-for-writing-cs-papers-part-1.htmlHistory of Monte Carlo Methods - Part 3http://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-3.html<p>This is the third part of a three part post.
The <a href="http://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-1.html">first part</a> covered the early history of Monte
Carlo and the rejection sampling method, the <a href="http://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-2.html">second
part</a> covered sequential Monte Carlo.</p>
<h1>Part 3</h1>
<p>In this part we are going …</p>Sebastian NowozinFri, 13 Nov 2015 21:30:00 +0100tag:www.nowozin.net,2015-11-13:/sebastian/blog/history-of-monte-carlo-methods-part-3.htmlHistory of Monte Carlo Methods - Part 2http://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-2.html<p>This is the second part of a three part post.
The <a href="http://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-1.html">last part</a> covered the early history of Monte
Carlo and the rejection sampling method.</p>
<h1>Part 2</h1>
<p>In this part we are going to look at importance sampling and sequential …</p>Sebastian NowozinFri, 30 Oct 2015 20:00:00 +0100tag:www.nowozin.net,2015-10-30:/sebastian/blog/history-of-monte-carlo-methods-part-2.htmlHistory of Monte Carlo Methods - Part 1http://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-1.html<p>Some time ago in June 2013 I gave a <em>lab tutorial</em> on Monte Carlo methods at
Microsoft Research. These tutorials are seminar-talk length (45 minutes) but
are supposed to be light, accessible to a general computer science audience,
and fun …</p>Sebastian NowozinFri, 16 Oct 2015 21:30:00 +0200tag:www.nowozin.net,2015-10-16:/sebastian/blog/history-of-monte-carlo-methods-part-1.htmlThe Julia language for Scientific Computinghttp://www.nowozin.net/sebastian/blog/the-julia-language-for-scientific-computing.html<p><a href="http://julialang.org/">Julia</a> is a relatively new programming language with
the <a href="http://julialang.org/blog/2012/02/why-we-created-julia/">declared goal</a>
to become the leading language for scientific computing.</p>
<p>I have probably annoyed half of my colleagues by raving about how great the
language is and what it is good …</p>Sebastian NowozinFri, 02 Oct 2015 22:30:00 +0200tag:www.nowozin.net,2015-10-02:/sebastian/blog/the-julia-language-for-scientific-computing.htmlHow good are your beliefs? Part 2: The Quizhttp://www.nowozin.net/sebastian/blog/how-good-are-your-beliefs-part-2-the-quiz.html<p>This post continues the previous post, <a href="http://www.nowozin.net/sebastian/blog/how-good-are-your-beliefs-part-1-scoring-rules.html">part 1</a> on
scoring rules. However, today we will be more hands on, testing your skill of
making good and well-calibrated predictions.</p>
<p>To this end, I will ask you several questions about numerical quantities …</p>Sebastian NowozinFri, 18 Sep 2015 21:00:00 +0200tag:www.nowozin.net,2015-09-18:/sebastian/blog/how-good-are-your-beliefs-part-2-the-quiz.htmlHow good are your beliefs? Part 1: Scoring Ruleshttp://www.nowozin.net/sebastian/blog/how-good-are-your-beliefs-part-1-scoring-rules.html<p>This article is the first of two on <em>proper scoring rules</em>,
a specific type of loss function defined on probability distributions or
functions of probability distributions.</p>
<p>If this article sparks your interest, I recommend the gentle introduction to
scoring rules …</p>Sebastian NowozinFri, 04 Sep 2015 22:00:00 +0200tag:www.nowozin.net,2015-09-04:/sebastian/blog/how-good-are-your-beliefs-part-1-scoring-rules.htmlMachine Learning for Intelligent Image and Video Processing (ICCV 2015 Workshop)http://www.nowozin.net/sebastian/blog/machine-learning-for-intelligent-image-and-video-processing-iccv-2015-workshop.html<p><a href="http://ei.is.tuebingen.mpg.de/person/mhirsch">Michael Hirsch</a> and myself are
organizing a <a href="http://ml4ip-iccv2015.is.tuebingen.mpg.de">workshop on the topic of machine learning for image and video
processing</a>
as part of the <a href="http://pamitc.org/iccv15/">ICCV 2015 programme</a>.</p>
<p>The workshop takes place on the 17th December 2015 in Santiago, Chile, right
after …</p>Sebastian NowozinWed, 02 Sep 2015 23:30:00 +0200tag:www.nowozin.net,2015-09-02:/sebastian/blog/machine-learning-for-intelligent-image-and-video-processing-iccv-2015-workshop.htmlEffective Sample Size in Importance Samplinghttp://www.nowozin.net/sebastian/blog/effective-sample-size-in-importance-sampling.html<p>In this article we will look at a practically important measure of efficiency
in importance sampling, the so called <em>effective sample size</em> (ESS) estimate.
This measure was proposed by <a href="http://www.decode.com/management/">Augustine
Kong</a> in 1992 in a technical report which
until recently …</p>Sebastian NowozinFri, 21 Aug 2015 21:30:00 +0200tag:www.nowozin.net,2015-08-21:/sebastian/blog/effective-sample-size-in-importance-sampling.htmlReverse Searchhttp://www.nowozin.net/sebastian/blog/reverse-search.html<p>One of my all-time favorite algorithms is <em>reverse search</em> proposed by
<a href="http://cgm.cs.mcgill.ca/~avis/">David Avis</a> and
<a href="http://www.inf.ethz.ch/personal/fukudak/">Komei Fukuda</a> in 1992,
<a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.26.4487&rep=rep1&type=pdf">PDF</a>.</p>
<p>Reverse search is an algorithm to solve enumeration problems, that is,
problems where you would like to list a finite set …</p>Sebastian NowozinFri, 07 Aug 2015 21:30:00 +0200tag:www.nowozin.net,2015-08-07:/sebastian/blog/reverse-search.htmlStochastic Computation Graphshttp://www.nowozin.net/sebastian/blog/stochastic-computation-graphs.html<p>This post is about a recent arXiv submission entitled
<a href="http://arxiv.org/abs/1506.05254">Gradient Estimation Using Stochastic Computation
Graphs</a>, and authored by
<a href="http://www.eecs.berkeley.edu/~joschu/">John Schulman</a>,
Nicolas Heess,
<a href="http://thphn.com/">Theophane Weber</a>, and
<a href="http://www.cs.berkeley.edu/~pabbeel/">Pieter Abbeel</a>.</p>
<p>In a nutshell this paper generalizes the <a href="https://en.wikipedia.org/wiki/Backpropagation">backpropagation
algorithm</a> to allow
<em>differentiation through …</em></p>Sebastian NowozinFri, 24 Jul 2015 22:00:00 +0200tag:www.nowozin.net,2015-07-24:/sebastian/blog/stochastic-computation-graphs.htmlMultilevel Splittinghttp://www.nowozin.net/sebastian/blog/multilevel-splitting.html<p>This article is about <em>multilevel splitting</em>, a method for estimating the
probability of rare events.</p>
<p>Estimating the probability of <em>rare events</em> is important in many fields.
One vivid example is in the study of reliability of systems; imagine for
example …</p>Sebastian NowozinFri, 10 Jul 2015 22:50:00 +0200tag:www.nowozin.net,2015-07-10:/sebastian/blog/multilevel-splitting.htmlBayesian P-Valueshttp://www.nowozin.net/sebastian/blog/bayesian-p-values.html<p><a href="https://en.wikipedia.org/wiki/P-value">P-Values</a> (see also <a href="https://stat.duke.edu/~berger/p-values.html">Jim Berger's page
on p-values</a>) are probably one of
the most misunderstood concepts in statistics and certainly have been abused
in statistical practice.
Originally proposed as an informal diagnostic by <a href="http://en.wikipedia.org/wiki/Ronald_Fisher">Ronald Fisher</a>, there are many
reasons for …</p>Sebastian NowozinSat, 27 Jun 2015 00:15:00 +0200tag:www.nowozin.net,2015-06-27:/sebastian/blog/bayesian-p-values.htmlThe Entropy of a Normal Distributionhttp://www.nowozin.net/sebastian/blog/the-entropy-of-a-normal-distribution.html<p>The <a href="http://en.wikipedia.org/wiki/Multivariate_normal_distribution">multivariate normal
distribution</a>
is one of the most important probability distributions for multivariate data.
In this post we will look at the entropy of this distribution and how to
estimate the entropy given an iid sample.</p>
<p>For a multivariate …</p>Sebastian NowozinSat, 13 Jun 2015 23:30:00 +0200tag:www.nowozin.net,2015-06-13:/sebastian/blog/the-entropy-of-a-normal-distribution.htmlA quick summary of CVPR 2015http://www.nowozin.net/sebastian/blog/a-quick-summary-of-cvpr-2015.html<p><a href="http://www.pamitc.org/cvpr15/">CVPR 2015</a>, "Computer Vision and Pattern
Recognition" is the main conference of the computer vision community and just
finished.
I unfortunately was only able to stay for the three main conference days, but
here is my short subjective summary.</p>
<p>For …</p>Sebastian NowozinThu, 11 Jun 2015 22:00:00 +0200tag:www.nowozin.net,2015-06-11:/sebastian/blog/a-quick-summary-of-cvpr-2015.htmlDemosaicinghttp://www.nowozin.net/sebastian/blog/demosaicing.html<p>This article describes the basic problem of image demosaicing and a recent work
of mine providing a research dataset for demosaicing research.</p>
<p><a href="http://en.wikipedia.org/wiki/Demosaicing">Image demosaicing</a> is a procedure
used in almost all digital cameras.
From your smartphone camera to the top-of-the-line …</p>Sebastian NowozinFri, 29 May 2015 23:00:00 +0200tag:www.nowozin.net,2015-05-29:/sebastian/blog/demosaicing.htmlBecoming a Bayesian, Part 3http://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-3.html<p>This post continues the previous post, <a href="http://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-1.html">part 1</a> and
<a href="http://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-2.html">part 2</a>,
outlining my criticism towards a ''naive'' subjective Bayesian viewpoint:</p>
<ol>
<li><a href="http://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-1.html">The consequences of model misspecification</a>.</li>
<li><a href="http://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-2.html">The ''model first computation last'' approach</a>.</li>
<li>Denial of methods of classical statistics, in this post …</li></ol>Sebastian NowozinFri, 15 May 2015 21:00:00 +0200tag:www.nowozin.net,2015-05-15:/sebastian/blog/becoming-a-bayesian-part-3.htmlBecoming a Bayesian, Part 2http://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-2.html<p>This post continues the previous post, <a href="http://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-1.html">part 1</a>,
outlining my criticism towards a ''naive'' subjective Bayesian viewpoint:</p>
<ol>
<li><a href="http://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-1.html">The consequences of model misspecification</a>.</li>
<li>The ''model first computation last'' approach, in this post.</li>
<li><a href="http://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-3.html">Denial of methods of classical statistics</a>.</li>
</ol>
<h2>The ''Model First …</h2>Sebastian NowozinSat, 02 May 2015 18:30:00 +0200tag:www.nowozin.net,2015-05-02:/sebastian/blog/becoming-a-bayesian-part-2.htmlBecoming a Bayesian, Part 1http://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-1.html<p>I have used probabilistic models for a number of years now and over this time
I have used different paradigms to build my models, to estimate them from
data, and to perform inference and predictions.</p>
<p>Overall I have slowly become …</p>Sebastian NowozinSun, 19 Apr 2015 17:30:00 +0200tag:www.nowozin.net,2015-04-19:/sebastian/blog/becoming-a-bayesian-part-1.htmlExtended Formulationshttp://www.nowozin.net/sebastian/blog/extended-formulations.html<p>An amazing fact in high dimensions is this:
<em>Projecting a simple convex set described by a small number of inequalities
can create complicated convex set with an exponential number of inequalities.</em></p>
<p>It is amazing because it contradicts our everyday human …</p>Sebastian NowozinSun, 05 Apr 2015 16:30:00 +0200tag:www.nowozin.net,2015-04-05:/sebastian/blog/extended-formulations.htmlHow to report uncertaintyhttp://www.nowozin.net/sebastian/blog/how-to-report-uncertainty.html<p><a href="http://en.wikipedia.org/wiki/Error_bar">Error bars</a> and the
<a href="http://en.wikipedia.org/wiki/Plus-minus_sign"><span class="math">\(\pm\)</span>-notation</a>
are used to quantitatively convey uncertainty in experimental results.
For example, you would often read statements like <span class="math">\(140.7 \textrm{Hz} \pm 2.8
\textrm{ Hz SEM}\)</span> in a paper to report both an …</p>Sebastian NowozinThu, 19 Mar 2015 22:30:00 +0100tag:www.nowozin.net,2015-03-19:/sebastian/blog/how-to-report-uncertainty.htmlEstimating Discrete Entropy, Part 3http://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-3.html<p>In the last two parts (<a href="http://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-1.html">part one</a>,
<a href="http://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-2.html">part two</a>) we looked at the problem of
entropy estimation and several popular estimators.</p>
<p>In this final article we will take a look at two Bayesian approaches to the
problem.</p>
<h2>Bayesian Estimator due …</h2>Sebastian NowozinSat, 07 Mar 2015 16:00:00 +0100tag:www.nowozin.net,2015-03-07:/sebastian/blog/estimating-discrete-entropy-part-3.htmlMachine Learning in Cambridge 2015http://www.nowozin.net/sebastian/blog/machine-learning-in-cambridge-2015.html<p>This year we (<a href="http://mlg.eng.cam.ac.uk/zoubin/">Zoubin</a>, together with
<a href="http://lopezpaz.org/">David</a> and myself) are again organizing a workshop
event for the local Cambridge (UK) machine learning community.
The schedule is available at the workshop homepage,
<a href="http://research.microsoft.com/en-us/um/cambridge/events/CamML2015/">Machine Learning in Cambridge
2015</a>, and
we also plan …</p>Sebastian NowozinThu, 26 Feb 2015 20:00:00 +0100tag:www.nowozin.net,2015-02-26:/sebastian/blog/machine-learning-in-cambridge-2015.htmlEstimating Discrete Entropy, Part 2http://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-2.html<p>In the <a href="http://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-1.html">last part</a> we have looked at the
basic problem of discrete entropy estimation.
In this article we will see a number of proposals of improved estimators.</p>
<h3>Miller Correction</h3>
<p>In 1955 George Miller proposed a simple correction to the …</p>Sebastian NowozinSat, 21 Feb 2015 19:00:00 +0100tag:www.nowozin.net,2015-02-21:/sebastian/blog/estimating-discrete-entropy-part-2.htmlEstimating Discrete Entropy, Part 1http://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-1.html<p>Estimation of the
<a href="http://en.wikipedia.org/wiki/Entropy_%28information_theory%29">entropy</a> of a
random variable is an important problem that has many applications.
If you can estimate entropy accurately, you can also estimate <a href="http://en.wikipedia.org/wiki/Mutual_information">mutual
information</a>, which allows
you to find dependent random variables in large data sets …</p>Sebastian NowozinSat, 07 Feb 2015 14:00:00 +0100tag:www.nowozin.net,2015-02-07:/sebastian/blog/estimating-discrete-entropy-part-1.htmlAdvanced Structured Predictionhttp://www.nowozin.net/sebastian/blog/advanced-structured-prediction.html<p>In December 2014, just in time for NIPS, <a href="http://mitpress.mit.edu/">MIT Press</a>
released an edited volume on structured prediction models and their
applications in natural language processing, computer vision, and
computational biology.</p>
<p><img alt="Advanced Structured Prediction cover image" src="http://www.nowozin.net/sebastian/blog/images/asp-cover.jpg"></p>
<p><a href="http://mitpress.mit.edu/books/advanced-structured-prediction">Advanced Structured Prediction</a>,
Editors Sebastian Nowozin, Peter V. Gehler, Jeremy …</p>Sebastian NowozinThu, 29 Jan 2015 22:30:00 +0100tag:www.nowozin.net,2015-01-29:/sebastian/blog/advanced-structured-prediction.htmlStreaming Mean and Variance Computationhttp://www.nowozin.net/sebastian/blog/streaming-mean-and-variance-computation.html<p>Given a sequence of observed data we would often like to estimate simple
quantities like the mean and variance.</p>
<p>Sometimes the data is available in a <em>streaming</em> setting, that is, we are
given one sample at a time. For example …</p>Sebastian NowozinSun, 25 Jan 2015 21:30:00 +0100tag:www.nowozin.net,2015-01-25:/sebastian/blog/streaming-mean-and-variance-computation.htmlThe Beginninghttp://www.nowozin.net/sebastian/blog/the-beginning.html<p>This is the start of my blog.
This will be a quite technical blog and therefore address a more specialized
audience.</p>
<p>The articles will cover topics in the area of machine learning, statistics,
maybe some computer vision, let's see.
I …</p>Sebastian NowozinSun, 25 Jan 2015 21:00:00 +0100tag:www.nowozin.net,2015-01-25:/sebastian/blog/the-beginning.html