Sebastian Nowozins slow bloghttps://www.nowozin.net/sebastian/blog/Tue, 09 Aug 2022 19:00:00 +0100Thoughts on Trace Estimation in Deep Learninghttps://www.nowozin.net/sebastian/blog/thoughts-on-trace-estimation-in-deep-learning.html<p>Efficiently estimating the <em>trace</em> <span class="math">\(\textrm{tr}(A) = \sum_{i=1}^d A_{ii}\)</span> of a
square matrix <span class="math">\(A \in \mathbb{R}^{d \times d}\)</span> is an important problem required
in a number of recent deep learning and machine learning models …</p>Sebastian NowozinTue, 09 Aug 2022 19:00:00 +0100tag:www.nowozin.net,2022-08-09:/sebastian/blog/thoughts-on-trace-estimation-in-deep-learning.htmlStatistics, Machine LearningLongevity and Supplementshttps://www.nowozin.net/sebastian/blog/longevity-and-supplements.html<p><em>TLDR</em>: in the past decade longevity has emerged as a serious research field.
There are now a number of studies that indicate that a number of safe
supplements may likely extend lifespan and health in adult humans.</p>
<p><em>Note</em>: I normally …</p>Sebastian NowozinWed, 03 Feb 2021 22:00:00 +0000tag:www.nowozin.net,2021-02-03:/sebastian/blog/longevity-and-supplements.htmlLongevityDebiasing Approximate Inferencehttps://www.nowozin.net/sebastian/blog/debiasing-approximate-inference.html<p>This year at <a href="http://neurips.cc/">NeurIPS 2018</a> the
<a href="http://approximateinference.org/">Symposium on Advances in Approximate Bayesian
Inference</a> discussed challenges and advances
in approximating probabilistic inference in rich models. It was a genuinely
exciting program!</p>
<p>I was lucky enough to give an invited talk at …</p>Sebastian NowozinWed, 05 Dec 2018 08:30:00 +0000tag:www.nowozin.net,2018-12-05:/sebastian/blog/debiasing-approximate-inference.htmlApproximate Inference, Machine LearningMLSS 2018 in Madridhttps://www.nowozin.net/sebastian/blog/mlss-2018-in-madrid.html<p>The <a href="http://mlss.cc/"><em>Machine Learning Summer Schools</em> (MLSS)</a> is the largest
and most popular machine learning summer school series. For two weeks in
August and September the <a href="http://mlss.ii.uam.es/mlss2018/index.html">MLSS 2018 is held in Madrid</a>.</p>
<p>I am happy to speak on the topics of …</p>Sebastian NowozinSun, 02 Sep 2018 23:00:00 +0100tag:www.nowozin.net,2018-09-02:/sebastian/blog/mlss-2018-in-madrid.htmlMachine LearningDo Bayesians Overfit?https://www.nowozin.net/sebastian/blog/do-bayesians-overfit.html<p><em>TLDR:</em> Yes, and there are precise results, although they are not as well known
as they perhaps should be.</p>
<p>Over the last few years I had many conversations in which the statement was
made that Bayesians methods are generally immune …</p>Sebastian NowozinWed, 11 Jul 2018 22:30:00 +0100tag:www.nowozin.net,2018-07-11:/sebastian/blog/do-bayesians-overfit.htmlStatistics, Machine LearningStable GAN Models and Creative Machineshttps://www.nowozin.net/sebastian/blog/stable-gan-models-and-creative-machines.html<p>We just published an <a href="https://www.microsoft.com/en-us/research/blog/will-machines-one-day-creative-humans/">article discussing our recent work on stabilizing generative adversarial networks</a>.</p>Sebastian NowozinMon, 04 Dec 2017 15:00:00 +0000tag:www.nowozin.net,2017-12-04:/sebastian/blog/stable-gan-models-and-creative-machines.htmlMachine LearningNIPS 2016 Generative Adversarial Training workshop talkhttps://www.nowozin.net/sebastian/blog/nips-2016-generative-adversarial-training-workshop-talk.html<p>The biggest AI conference of the year has just ended:
<a href="http://nips.cc/">NIPS</a> in Barcelona broke all records this year and the
program was exciting as always. It certainly remains my favorite conference
to attend.</p>
<p>One of the best things about NIPS …</p>Sebastian NowozinSat, 10 Dec 2016 22:30:00 +0000tag:www.nowozin.net,2016-12-10:/sebastian/blog/nips-2016-generative-adversarial-training-workshop-talk.htmlMachine LearningBook Review: Computer Age Statistical Inferencehttps://www.nowozin.net/sebastian/blog/book-review-computer-age-statistical-inference.html<p><img alt="Book cover: Computer Age Statistical
Inference" src="https://www.nowozin.net/sebastian/blog/images/computer-age-statistical-inference.jpg"></p>
<p>A new book, <a href="http://www.cambridge.org/us/academic/subjects/statistics-probability/statistical-theory-and-methods/computer-age-statistical-inference-algorithms-evidence-and-data-science">Computer Age Statistical Inference: Algorithms, Evidence, and Data
Science</a> by
<a href="http://statweb.stanford.edu/~ckirby/brad/">Bradley Efron</a> and
<a href="http://web.stanford.edu/~hastie/">Trevor Hastie</a>, was released in July this
year. I finished reading it a few weeks ago and this is a short review from
the …</p>Sebastian NowozinWed, 23 Nov 2016 21:00:00 +0000tag:www.nowozin.net,2016-11-23:/sebastian/blog/book-review-computer-age-statistical-inference.htmlMachine Learning, Statistics, Book ReviewStreaming Log-sum-exp Computationhttps://www.nowozin.net/sebastian/blog/streaming-log-sum-exp-computation.html<p>A common numerical operation in statistical computing is to compute</p>
<div class="math">$$\log \sum_{i=1}^n \exp x_i,$$</div>
<p>where <span class="math">\(x_i \in \mathbb{R}\)</span>, and <span class="math">\(n\)</span> is potentially very large.</p>
<p>We can implement the above computation by exponentiating each number, then
summing …</p>Sebastian NowozinSun, 08 May 2016 21:30:00 +0100tag:www.nowozin.net,2016-05-08:/sebastian/blog/streaming-log-sum-exp-computation.htmlAlgorithmsWhere will Artificial Intelligence come from?https://www.nowozin.net/sebastian/blog/where-will-artificial-intelligence-come-from.html<p>Artificial Intelligence (AI) is making progress in great strides, or at least
it appears so!
Almost no week passes by without some major announcements of new challenges
solved by AI technology or new products powered by AI.</p>
<p>Indeed many quantifiable …</p>Sebastian NowozinWed, 20 Apr 2016 23:30:00 +0100tag:www.nowozin.net,2016-04-20:/sebastian/blog/where-will-artificial-intelligence-come-from.htmlArtificial IntelligenceThe Best of Unpublished Machine Learning and Statistics Bookshttps://www.nowozin.net/sebastian/blog/the-best-of-unpublished-machine-learning-and-statistics-books.html<p>Nowadays authors in the fields of statistics and machine learning often choose
to write their books openly by publishing early draft versions.
For popular books this creates a lot of feedback and in the end clearly
improves the final book …</p>Sebastian NowozinTue, 09 Feb 2016 23:00:00 +0000tag:www.nowozin.net,2016-02-09:/sebastian/blog/the-best-of-unpublished-machine-learning-and-statistics-books.htmlMachine Learning, StatisticsThe Fair Price to Pay a Spy: An Introduction to the Value of Informationhttps://www.nowozin.net/sebastian/blog/the-fair-price-to-pay-a-spy-an-introduction-to-the-value-of-information.html<p><img alt="Spy image" src="https://www.nowozin.net/sebastian/blog/images/voi-spy-illustration.png"></p>
<p>(This article covers the decision-theoretic concept of <em>value of information</em>
through a classic example.)</p>
<p>What is the value of a piece of information?</p>
<p>It depends.
Two factors determine the value of information:
first, whether the information is new to you …</p>Sebastian NowozinSat, 09 Jan 2016 22:30:00 +0000tag:www.nowozin.net,2016-01-09:/sebastian/blog/the-fair-price-to-pay-a-spy-an-introduction-to-the-value-of-information.htmlDecision Theory, Machine LearningICCV 2015, Day 4https://www.nowozin.net/sebastian/blog/iccv-2015-day-4.html<p>This article summarizes the fourth day of the <a href="http://pamitc.org/iccv15/">ICCV
2015</a> conference, the International Conference on
Computer Vision.
A summary of the <a href="https://www.nowozin.net/sebastian/blog/iccv-2015-day-1.html">first day</a>,
<a href="https://www.nowozin.net/sebastian/blog/iccv-2015-day-2.html">second day</a>, and
<a href="https://www.nowozin.net/sebastian/blog/iccv-2015-day-3.html">third day</a> is also available.</p>
<h2>ICCV 2017 and 2019</h2>
<p>ICCV 2017 will be in …</p>Sebastian NowozinWed, 16 Dec 2015 23:50:00 +0000tag:www.nowozin.net,2015-12-16:/sebastian/blog/iccv-2015-day-4.htmlComputer VisionICCV 2015, Day 3https://www.nowozin.net/sebastian/blog/iccv-2015-day-3.html<p>This article summarizes the third day of the <a href="http://pamitc.org/iccv15/">ICCV
2015</a> conference, the International Conference on
Computer Vision.
A summary of the <a href="https://www.nowozin.net/sebastian/blog/iccv-2015-day-1.html">first day</a> and
<a href="https://www.nowozin.net/sebastian/blog/iccv-2015-day-2.html">second day</a> is also available.</p>
<h2>Interesting Papers</h2>
<h3>Registering Images to Untextured Geometry Using Average Shading Gradients …</h3>Sebastian NowozinWed, 16 Dec 2015 01:00:00 +0000tag:www.nowozin.net,2015-12-16:/sebastian/blog/iccv-2015-day-3.htmlComputer VisionICCV 2015, Day 2https://www.nowozin.net/sebastian/blog/iccv-2015-day-2.html<p>This article summarizes the second day of the <a href="http://pamitc.org/iccv15/">ICCV
2015</a> conference, the International Conference on
Computer Vision.
A summary of the <a href="https://www.nowozin.net/sebastian/blog/iccv-2015-day-1.html">first day</a> is also available.</p>
<h2>Awards</h2>
<p>The following awards were given at ICCV 2015.</p>
<h3>Achievement awards</h3>
<ul>
<li>PAMI Distinguished Researcher …</li></ul>Sebastian NowozinTue, 15 Dec 2015 01:20:00 +0000tag:www.nowozin.net,2015-12-15:/sebastian/blog/iccv-2015-day-2.htmlComputer VisionICCV 2015, Day 1https://www.nowozin.net/sebastian/blog/iccv-2015-day-1.html<p><a href="http://pamitc.org/iccv15/">ICCV 2015</a>, the International Conference on
Computer Vision, is one of the premier venues for computer vision research,
together with the CVPR conference.
This ICCV is happening in Santiago, Chile, a beautiful city with amazing food.</p>
<p>The computer vision community …</p>Sebastian NowozinMon, 14 Dec 2015 23:00:00 +0000tag:www.nowozin.net,2015-12-14:/sebastian/blog/iccv-2015-day-1.htmlComputer VisionTen Tips for Writing CS Papers, Part 2https://www.nowozin.net/sebastian/blog/ten-tips-for-writing-cs-papers-part-2.html<p>This continues the <a href="https://www.nowozin.net/sebastian/blog/ten-tips-for-writing-cs-papers-part-1.html">first part</a> on tips to write
computer science papers.</p>
<h2>6. Ideal Structure of a Paragraph</h2>
<p>A paper has different levels of formal structure:
sections, subsections, paragraphs, sentences.
It is important to ensure that the structure of the …</p>Sebastian NowozinThu, 10 Dec 2015 22:30:00 +0000tag:www.nowozin.net,2015-12-10:/sebastian/blog/ten-tips-for-writing-cs-papers-part-2.htmlComputer ScienceTen Tips for Writing CS Papers, Part 1https://www.nowozin.net/sebastian/blog/ten-tips-for-writing-cs-papers-part-1.html<p>As a non-native English speaker I can relate to the challenge of writing
concise and clear English.
Scientific writing is particularly challenging because the audience is only
partially known at the time of writing: at best, the paper will still …</p>Sebastian NowozinSun, 29 Nov 2015 21:00:00 +0000tag:www.nowozin.net,2015-11-29:/sebastian/blog/ten-tips-for-writing-cs-papers-part-1.htmlComputer ScienceHistory of Monte Carlo Methods - Part 3https://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-3.html<p>This is the third part of a three part post.
The <a href="https://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-1.html">first part</a> covered the early history of Monte
Carlo and the rejection sampling method, the <a href="https://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-2.html">second
part</a> covered sequential Monte Carlo.</p>
<h1>Part 3</h1>
<p>In this part we are going …</p>Sebastian NowozinFri, 13 Nov 2015 21:30:00 +0000tag:www.nowozin.net,2015-11-13:/sebastian/blog/history-of-monte-carlo-methods-part-3.htmlStatisticsHistory of Monte Carlo Methods - Part 2https://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-2.html<p>This is the second part of a three part post.
The <a href="https://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-1.html">last part</a> covered the early history of Monte
Carlo and the rejection sampling method.</p>
<h1>Part 2</h1>
<p>In this part we are going to look at importance sampling and sequential …</p>Sebastian NowozinFri, 30 Oct 2015 20:00:00 +0000tag:www.nowozin.net,2015-10-30:/sebastian/blog/history-of-monte-carlo-methods-part-2.htmlStatisticsHistory of Monte Carlo Methods - Part 1https://www.nowozin.net/sebastian/blog/history-of-monte-carlo-methods-part-1.html<p>Some time ago in June 2013 I gave a <em>lab tutorial</em> on Monte Carlo methods at
Microsoft Research. These tutorials are seminar-talk length (45 minutes) but
are supposed to be light, accessible to a general computer science audience,
and fun …</p>Sebastian NowozinFri, 16 Oct 2015 21:30:00 +0100tag:www.nowozin.net,2015-10-16:/sebastian/blog/history-of-monte-carlo-methods-part-1.htmlStatisticsThe Julia language for Scientific Computinghttps://www.nowozin.net/sebastian/blog/the-julia-language-for-scientific-computing.html<p><a href="http://julialang.org/">Julia</a> is a relatively new programming language with
the <a href="http://julialang.org/blog/2012/02/why-we-created-julia/">declared goal</a>
to become the leading language for scientific computing.</p>
<p>I have probably annoyed half of my colleagues by raving about how great the
language is and what it is good …</p>Sebastian NowozinFri, 02 Oct 2015 22:30:00 +0100tag:www.nowozin.net,2015-10-02:/sebastian/blog/the-julia-language-for-scientific-computing.htmlProgrammingHow good are your beliefs? Part 2: The Quizhttps://www.nowozin.net/sebastian/blog/how-good-are-your-beliefs-part-2-the-quiz.html<p>This post continues the previous post, <a href="https://www.nowozin.net/sebastian/blog/how-good-are-your-beliefs-part-1-scoring-rules.html">part 1</a> on
scoring rules. However, today we will be more hands on, testing your skill of
making good and well-calibrated predictions.</p>
<p>To this end, I will ask you several questions about numerical quantities …</p>Sebastian NowozinFri, 18 Sep 2015 21:00:00 +0100tag:www.nowozin.net,2015-09-18:/sebastian/blog/how-good-are-your-beliefs-part-2-the-quiz.htmlStatistics, Machine LearningHow good are your beliefs? Part 1: Scoring Ruleshttps://www.nowozin.net/sebastian/blog/how-good-are-your-beliefs-part-1-scoring-rules.html<p>This article is the first of two on <em>proper scoring rules</em>,
a specific type of loss function defined on probability distributions or
functions of probability distributions.</p>
<p>If this article sparks your interest, I recommend the gentle introduction to
scoring rules …</p>Sebastian NowozinFri, 04 Sep 2015 22:00:00 +0100tag:www.nowozin.net,2015-09-04:/sebastian/blog/how-good-are-your-beliefs-part-1-scoring-rules.htmlStatistics, Machine LearningMachine Learning for Intelligent Image and Video Processing (ICCV 2015 Workshop)https://www.nowozin.net/sebastian/blog/machine-learning-for-intelligent-image-and-video-processing-iccv-2015-workshop.html<p><a href="http://ei.is.tuebingen.mpg.de/person/mhirsch">Michael Hirsch</a> and myself are
organizing a <a href="http://ml4ip-iccv2015.is.tuebingen.mpg.de">workshop on the topic of machine learning for image and video
processing</a>
as part of the <a href="http://pamitc.org/iccv15/">ICCV 2015 programme</a>.</p>
<p>The workshop takes place on the 17th December 2015 in Santiago, Chile, right
after …</p>Sebastian NowozinWed, 02 Sep 2015 23:30:00 +0100tag:www.nowozin.net,2015-09-02:/sebastian/blog/machine-learning-for-intelligent-image-and-video-processing-iccv-2015-workshop.htmlComputer Vision, Image Processing, Machine LearningEffective Sample Size in Importance Samplinghttps://www.nowozin.net/sebastian/blog/effective-sample-size-in-importance-sampling.html<p>In this article we will look at a practically important measure of efficiency
in importance sampling, the so called <em>effective sample size</em> (ESS) estimate.
This measure was proposed by <a href="http://www.decode.com/management/">Augustine
Kong</a> in 1992 in a technical report which
until recently …</p>Sebastian NowozinFri, 21 Aug 2015 21:30:00 +0100tag:www.nowozin.net,2015-08-21:/sebastian/blog/effective-sample-size-in-importance-sampling.htmlStatistics, Machine LearningReverse Searchhttps://www.nowozin.net/sebastian/blog/reverse-search.html<p>One of my all-time favorite algorithms is <em>reverse search</em> proposed by
<a href="http://cgm.cs.mcgill.ca/~avis/">David Avis</a> and
<a href="http://www.inf.ethz.ch/personal/fukudak/">Komei Fukuda</a> in 1992,
<a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.26.4487&rep=rep1&type=pdf">PDF</a>.</p>
<p>Reverse search is an algorithm to solve enumeration problems, that is,
problems where you would like to list a finite set …</p>Sebastian NowozinFri, 07 Aug 2015 21:30:00 +0100tag:www.nowozin.net,2015-08-07:/sebastian/blog/reverse-search.htmlAlgorithmsStochastic Computation Graphshttps://www.nowozin.net/sebastian/blog/stochastic-computation-graphs.html<p>This post is about a recent arXiv submission entitled
<a href="http://arxiv.org/abs/1506.05254">Gradient Estimation Using Stochastic Computation
Graphs</a>, and authored by
<a href="http://www.eecs.berkeley.edu/~joschu/">John Schulman</a>,
Nicolas Heess,
<a href="http://thphn.com/">Theophane Weber</a>, and
<a href="http://www.cs.berkeley.edu/~pabbeel/">Pieter Abbeel</a>.</p>
<p>In a nutshell this paper generalizes the <a href="https://en.wikipedia.org/wiki/Backpropagation">backpropagation
algorithm</a> to allow
<em>differentiation through …</em></p>Sebastian NowozinFri, 24 Jul 2015 22:00:00 +0100tag:www.nowozin.net,2015-07-24:/sebastian/blog/stochastic-computation-graphs.htmlStatistics, Machine LearningMultilevel Splittinghttps://www.nowozin.net/sebastian/blog/multilevel-splitting.html<p>This article is about <em>multilevel splitting</em>, a method for estimating the
probability of rare events.</p>
<p>Estimating the probability of <em>rare events</em> is important in many fields.
One vivid example is in the study of reliability of systems; imagine for
example …</p>Sebastian NowozinFri, 10 Jul 2015 22:50:00 +0100tag:www.nowozin.net,2015-07-10:/sebastian/blog/multilevel-splitting.htmlStatisticsBayesian P-Valueshttps://www.nowozin.net/sebastian/blog/bayesian-p-values.html<p><a href="https://en.wikipedia.org/wiki/P-value">P-Values</a> (see also <a href="https://stat.duke.edu/~berger/p-values.html">Jim Berger's page
on p-values</a>) are probably one of
the most misunderstood concepts in statistics and certainly have been abused
in statistical practice.
Originally proposed as an informal diagnostic by <a href="http://en.wikipedia.org/wiki/Ronald_Fisher">Ronald Fisher</a>, there are many
reasons for …</p>Sebastian NowozinSat, 27 Jun 2015 00:15:00 +0100tag:www.nowozin.net,2015-06-27:/sebastian/blog/bayesian-p-values.htmlMachine Learning, StatisticsThe Entropy of a Normal Distributionhttps://www.nowozin.net/sebastian/blog/the-entropy-of-a-normal-distribution.html<p>The <a href="http://en.wikipedia.org/wiki/Multivariate_normal_distribution">multivariate normal
distribution</a>
is one of the most important probability distributions for multivariate data.
In this post we will look at the entropy of this distribution and how to
estimate the entropy given an iid sample.</p>
<p>For a multivariate …</p>Sebastian NowozinSat, 13 Jun 2015 23:30:00 +0100tag:www.nowozin.net,2015-06-13:/sebastian/blog/the-entropy-of-a-normal-distribution.htmlStatisticsA quick summary of CVPR 2015https://www.nowozin.net/sebastian/blog/a-quick-summary-of-cvpr-2015.html<p><a href="http://www.pamitc.org/cvpr15/">CVPR 2015</a>, "Computer Vision and Pattern
Recognition" is the main conference of the computer vision community and just
finished.
I unfortunately was only able to stay for the three main conference days, but
here is my short subjective summary.</p>
<p>For …</p>Sebastian NowozinThu, 11 Jun 2015 22:00:00 +0100tag:www.nowozin.net,2015-06-11:/sebastian/blog/a-quick-summary-of-cvpr-2015.htmlComputer Vision, ConferencesDemosaicinghttps://www.nowozin.net/sebastian/blog/demosaicing.html<p>This article describes the basic problem of image demosaicing and a recent work
of mine providing a research dataset for demosaicing research.</p>
<p><a href="http://en.wikipedia.org/wiki/Demosaicing">Image demosaicing</a> is a procedure
used in almost all digital cameras.
From your smartphone camera to the top-of-the-line …</p>Sebastian NowozinFri, 29 May 2015 23:00:00 +0100tag:www.nowozin.net,2015-05-29:/sebastian/blog/demosaicing.htmlImage Processing, Digital CamerasBecoming a Bayesian, Part 3https://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-3.html<p>This post continues the previous post, <a href="https://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-1.html">part 1</a> and
<a href="https://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-2.html">part 2</a>,
outlining my criticism towards a ''naive'' subjective Bayesian viewpoint:</p>
<ol>
<li><a href="https://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-1.html">The consequences of model misspecification</a>.</li>
<li><a href="https://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-2.html">The ''model first computation last'' approach</a>.</li>
<li>Denial of methods of classical statistics, in this post …</li></ol>Sebastian NowozinFri, 15 May 2015 21:00:00 +0100tag:www.nowozin.net,2015-05-15:/sebastian/blog/becoming-a-bayesian-part-3.htmlMachine LearningBecoming a Bayesian, Part 2https://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-2.html<p>This post continues the previous post, <a href="https://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-1.html">part 1</a>,
outlining my criticism towards a ''naive'' subjective Bayesian viewpoint:</p>
<ol>
<li><a href="https://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-1.html">The consequences of model misspecification</a>.</li>
<li>The ''model first computation last'' approach, in this post.</li>
<li><a href="https://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-3.html">Denial of methods of classical statistics</a>.</li>
</ol>
<h2>The ''Model First …</h2>Sebastian NowozinSat, 02 May 2015 18:30:00 +0100tag:www.nowozin.net,2015-05-02:/sebastian/blog/becoming-a-bayesian-part-2.htmlMachine LearningBecoming a Bayesian, Part 1https://www.nowozin.net/sebastian/blog/becoming-a-bayesian-part-1.html<p>I have used probabilistic models for a number of years now and over this time
I have used different paradigms to build my models, to estimate them from
data, and to perform inference and predictions.</p>
<p>Overall I have slowly become …</p>Sebastian NowozinSun, 19 Apr 2015 17:30:00 +0100tag:www.nowozin.net,2015-04-19:/sebastian/blog/becoming-a-bayesian-part-1.htmlMachine LearningExtended Formulationshttps://www.nowozin.net/sebastian/blog/extended-formulations.html<p>An amazing fact in high dimensions is this:
<em>Projecting a simple convex set described by a small number of inequalities
can create complicated convex set with an exponential number of inequalities.</em></p>
<p>It is amazing because it contradicts our everyday human …</p>Sebastian NowozinSun, 05 Apr 2015 16:30:00 +0100tag:www.nowozin.net,2015-04-05:/sebastian/blog/extended-formulations.htmlOptimizationHow to report uncertaintyhttps://www.nowozin.net/sebastian/blog/how-to-report-uncertainty.html<p><a href="http://en.wikipedia.org/wiki/Error_bar">Error bars</a> and the
<a href="http://en.wikipedia.org/wiki/Plus-minus_sign"><span class="math">\(\pm\)</span>-notation</a>
are used to quantitatively convey uncertainty in experimental results.
For example, you would often read statements like <span class="math">\(140.7 \textrm{Hz} \pm 2.8
\textrm{ Hz SEM}\)</span> in a paper to report both an …</p>Sebastian NowozinThu, 19 Mar 2015 22:30:00 +0000tag:www.nowozin.net,2015-03-19:/sebastian/blog/how-to-report-uncertainty.htmlMachine LearningEstimating Discrete Entropy, Part 3https://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-3.html<p>In the last two parts (<a href="https://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-1.html">part one</a>,
<a href="https://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-2.html">part two</a>) we looked at the problem of
entropy estimation and several popular estimators.</p>
<p>In this final article we will take a look at two Bayesian approaches to the
problem.</p>
<h2>Bayesian Estimator due …</h2>Sebastian NowozinSat, 07 Mar 2015 16:00:00 +0000tag:www.nowozin.net,2015-03-07:/sebastian/blog/estimating-discrete-entropy-part-3.htmlStatisticsMachine Learning in Cambridge 2015https://www.nowozin.net/sebastian/blog/machine-learning-in-cambridge-2015.html<p>This year we (<a href="http://mlg.eng.cam.ac.uk/zoubin/">Zoubin</a>, together with
<a href="http://lopezpaz.org/">David</a> and myself) are again organizing a workshop
event for the local Cambridge (UK) machine learning community.
The schedule is available at the workshop homepage,
<a href="http://research.microsoft.com/en-us/um/cambridge/events/CamML2015/">Machine Learning in Cambridge
2015</a>, and
we also plan …</p>Sebastian NowozinThu, 26 Feb 2015 20:00:00 +0000tag:www.nowozin.net,2015-02-26:/sebastian/blog/machine-learning-in-cambridge-2015.htmlMachine LearningEstimating Discrete Entropy, Part 2https://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-2.html<p>In the <a href="https://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-1.html">last part</a> we have looked at the
basic problem of discrete entropy estimation.
In this article we will see a number of proposals of improved estimators.</p>
<h3>Miller Correction</h3>
<p>In 1955 George Miller proposed a simple correction to the …</p>Sebastian NowozinSat, 21 Feb 2015 19:00:00 +0000tag:www.nowozin.net,2015-02-21:/sebastian/blog/estimating-discrete-entropy-part-2.htmlStatisticsEstimating Discrete Entropy, Part 1https://www.nowozin.net/sebastian/blog/estimating-discrete-entropy-part-1.html<p>Estimation of the
<a href="http://en.wikipedia.org/wiki/Entropy_%28information_theory%29">entropy</a> of a
random variable is an important problem that has many applications.
If you can estimate entropy accurately, you can also estimate <a href="http://en.wikipedia.org/wiki/Mutual_information">mutual
information</a>, which allows
you to find dependent random variables in large data sets …</p>Sebastian NowozinSat, 07 Feb 2015 14:00:00 +0000tag:www.nowozin.net,2015-02-07:/sebastian/blog/estimating-discrete-entropy-part-1.htmlStatisticsAdvanced Structured Predictionhttps://www.nowozin.net/sebastian/blog/advanced-structured-prediction.html<p>In December 2014, just in time for NIPS, <a href="http://mitpress.mit.edu/">MIT Press</a>
released an edited volume on structured prediction models and their
applications in natural language processing, computer vision, and
computational biology.</p>
<p><img alt="Advanced Structured Prediction cover image" src="https://www.nowozin.net/sebastian/blog/images/asp-cover.jpg"></p>
<p><a href="http://mitpress.mit.edu/books/advanced-structured-prediction">Advanced Structured Prediction</a>,
Editors Sebastian Nowozin, Peter V. Gehler, Jeremy …</p>Sebastian NowozinThu, 29 Jan 2015 22:30:00 +0000tag:www.nowozin.net,2015-01-29:/sebastian/blog/advanced-structured-prediction.htmlMachine Learning, Structured PredictionStreaming Mean and Variance Computationhttps://www.nowozin.net/sebastian/blog/streaming-mean-and-variance-computation.html<p>Given a sequence of observed data we would often like to estimate simple
quantities like the mean and variance.</p>
<p>Sometimes the data is available in a <em>streaming</em> setting, that is, we are
given one sample at a time. For example …</p>Sebastian NowozinSun, 25 Jan 2015 21:30:00 +0000tag:www.nowozin.net,2015-01-25:/sebastian/blog/streaming-mean-and-variance-computation.htmlAlgorithmsThe Beginninghttps://www.nowozin.net/sebastian/blog/the-beginning.html<p>This is the start of my blog.
This will be a quite technical blog and therefore address a more specialized
audience.</p>
<p>The articles will cover topics in the area of machine learning, statistics,
maybe some computer vision, let's see.
I …</p>Sebastian NowozinSun, 25 Jan 2015 21:00:00 +0000tag:www.nowozin.net,2015-01-25:/sebastian/blog/the-beginning.htmlPersonal