Saving matplotlib animations in libx264 viewable by Quicktime

In order to save quicktime viewable libx264 videos using the matplotlib animation package.

anim.save("circly.mp4",dpi=120,codec="libx264",extra_args=['-pix_fmt','yuv420p'])

the key is the

-pix_fmt

business.

Convincing BibLatex to handle collaborations properly

In HEP, I end up with a lot of citations that look like:

@article{Banner:1983jy,
	Author = {Banner, M. and others},
	Collaboration = {UA2 Collaboration},
	Date-Added = {2013-04-10 19:41:58 +0000},
	Date-Modified = {2013-04-10 19:42:58 +0000},
	Doi = {10.1016/0370-2693(83)91605-2},
	Journal = {Phys.Lett.},
	Pages = {476-485},
	Reportnumber = {CERN-EP-83-25},
	Slaccitation = {%%CITATION = PHLTA,B122,476;%%},
	Title = {{Observation of Single Isolated Electrons of High Transverse Momentum in Events with Missing Transverse Energy at the CERN anti-p p Collider}},
	Volume = {B122},
	Year = {1983},
	Bdsk-Url-1 = {http://dx.doi.org/10.1016/0370-2693(83)91605-2}}

(These come from the excellent inSPIRE)

I'd like to have the author listed as the collaboration if the collaboration field is present. Otherwise, it should fall back to the usual author field.

Here's what I came up with.

\usepackage[backend=biber]{biblatex}
\DeclareSourcemap{
 \maps[datatype=bibtex,overwrite=true]{
  \map{
    \step[fieldsource=Collaboration, final=true]
    \step[fieldset=usera, origfieldval, final=true]
  }
 }
}

\renewbibmacro*{author}{%
  \iffieldundef{usera}{
    \printnames{author}
  }{
    \printfield{usera}
  }
}

Basically, we're loading up biblatex with the biber backend. Then, we use a hack to copy the collaboration field, which biblatex doesn't recognize into the usera field, which it does. Finally, we update the author macro to print the usera field if it exists. Otherwise, it prints the usual author.

LaTeX Makefile

I've taken to using rubber and a Makefile to help in my LaTeX projects.  I create a main directory with a .tex file, and put any additional ones in a tex/ directory.


Saved for prosterity

XPRA, mosh, and tmux - a little slice of heaven

XPRA is basically mosh for X environments.  It lets you connect to an X environment in a persistent way.

So, my new workflow uses mosh, tmux and XPRA to accomplish great things.

On work machine:

 xpra start :7 && DISPLAY=:7 tmux new -s remote

which will start an xpra server on display hook 7, and launch a new tmux instance with name 'remote' that is attached to said display device. This ensures that whenever we try to launch an X instance (i.e. 'ipython --pylab' ) it will be able to attach to an X device

On laptop:

 xpra attach ssh:<hostname>:7 & mosh <hostname> -- tmux attach -t remote

This will have xpra attach to the remote X session display, and run mosh for a persistent ssh connection to my work machine, and once connected attach to my remote tmux session which already is connected to the xpra display, and I have a working, fast persistent ssh + X forwarding connection to my work computer.

 What this means is that I have a connection to my work computer that behaves transparently as if it were my work computer, plus will fix itself if I put my laptop to sleep and wake it back up. So I can plot things in ipython all I want without a care in the world.

For extra fun, alias those commands on the respective machines to make your life easier.

Making mpeg4

So, instead of making animated gifs, making mpeg4 with libx264 results in faster, smaller higher quality images.

 avconv -i pics/%05d.png -c:v libx264 -tune stillimage [-crf 23 -preset fast] output.mp4 

Note that you have to use the C style syntax %05d for the file names, if you use glob (*) you'll overwrite all of your images, beware!

-crf sets the quality from 0 to 50, 0 being the best
-preset does some presets, ultrafast, fast, slow, etc
-qp 0  for lossless

But for 1000 pngs, this gives a nice 4 M movie of high quality.  E.g.

Welford's Algorithm

Computing means and standard deviations for data can be hard.  Naively implementing the formula can result in large numerical errors.

A better way is to use Welford's Algorithm, which is numerically stable.

Below is a gist for doing just that.

Python Email Notification

So, I occasionally want to run a long simulation, and not really pay attention to when it finishes.

Ideally, I could have python email me when it was done.  Even more ideal, I could have it use gmail to send the email notification.

Well, I've done just that, along with using netrc to hold the password configuration so I know its secure.

The Gist is up at https://gist.github.com/2150894, or copied below.