Welcome
-
Monitoring Per-Process Memory Usage with Munin
Something Emporium and a few associated websites are run off my VPS server from Rimuhosting. (I used to work for Rimuhosting: they're awesome.) Recently, I moved from running a traditional mod_php/mpm_prefork setup to a multi-threaded mod_fcgid/mpm_worker setup.
In a mod_fcgid PHP setup, a few persistant php-cgi workers are set up and then requests for dynamic content are farmed out to them. By stripping PHP out of the apache workers, you get a setup that uses far less memory to serve each connection, and you're free to use a multi-threaded MPM (PHP is not thread-safe, so you can't with mod_php).
OK, so, I've set this up, which is a process with spotty documentation. I've got it working nicely, and I've been able to reduce the stack's memory footprint considerably and increase its performance.
The snag so far is this:
This graph shows total memory usage on the server, catagorised by type, as we transitioned from mod_php to mod_fcgid. It's a graph provided by Debian's default install of Munin. This VPS has 224MB of memory, so it's something I have to watch fairly closely. You'll notice the sawtooth shape of the graph after the changeover. Here's a closeup of that:
This is pretty good evidence of some sort of memory leak. There's a process there that's allocating memory and can't (or won't) deallocate it. It's being reset at fairly regular intervals, either by monit or perhaps by some other worker handler: apache, mod_fcgid. Actually, it could be almost any process. Damn.
So, my next task was to identify the culprit. Which comes down to the following question: which processes are using memory in this sawtooth pattern? Sounds like a task for Munin, huh. But out of the box, Debian's Munin doesn't have a recipe for graphing the memory usage of individual processes.
Which lead me to write a Munin "plugin", multimemory. Based off multips, this plugin lets you specify a bunch of process names, and it'll graph the total memory usage of processes matching each one. It's available at GitHub if anyone wants it.
It gave me the following graph, which, twelve hours later, makes the culprit in the case of the leaking memory pretty clear.
Next steps: wait for 5.3's new garbage collector. Well, probably more likely I'll just jump ship to 5.3 sooner, stable version be damned. That, and reduce the number of requests each php-cgi gets to serve before it's replaced.
Apparently, this isn't PHP's fault:
PHP is glue. It is the glue used to build cool web applications by sticking dozens of 3rd-party libraries together and making it all appear as one coherent entity through an intuitive and easy to learn language interface. The flexibility and power of PHP relies on the stability and robustness of the underlying platform. It needs a working OS, a working web server and working 3rd-party libraries to glue together. When any of these stop working PHP needs ways to identify the problems and fix them quickly. When you make the underlying framework more complex by not having completely separate execution threads, completely separate memory segments and a strong sandbox for each request to play in, feet of clay are introduced into PHP's system.
Sure, I get it, you like having your own process to play around in. But dammit, PHP, why don't you clean up after yourself each request? Is this the experience of everyone who runs PHP with fcgid?
-
Bibliography/Footnotes with LaTeX and BibLaTeX
Here's how I do referencing in LaTeX:
Set up
- If you're using something like MiKTeX, the BibLaTeX package should be downloaded for you when you first use it. If not, you'll need to install it yourself. There's a biblatex package for Ubuntu, if you're using that.
- In my document's preamble, I include the BibLaTeX package, with some nice options for traditional style "ibidem" references:
\usepackage[style=verbose-trad2,natbib=true]{biblatex} - I don't like my endnotes being called "References" or the like, and I don't like the section being numbered, so I make sure I set the heading text in the preamble:
\defbibheading{bibliography}{\section*{Bibliography}} - I use the open-source JabRef reference manager to produce a .bib file. Most of the time, I call it "main.bib" and include it like this (again, in the preamble):
\bibliography{main}
Footnotes
- Footnotes are done with
\autocite[prenote][page]{bibtex key}:It doesn't seem to matter which side of the punctuation you put your\autocite{Popper1963}
\autocite[163]{Popper1963}
\autocite[Related to][163]{Popper1963}\autocite. - Footnotes will then automatically appear at the bottom of the page.
Bibliography
- I like my bibliography on a seperate page at the end. I include it like this:
\pagebreak
\printbibliography - By default, the bibliography includes all the references you've already cited using other commands, like
\autocite. But I often have a number of books that I want in the bibliography, but don't want to use as a footnote somewhere. So, I make sure my entire bibliography of references is included by citing them all with\nocite, which doesn't print anything:That line can go anywhere in the document.\nocite{Chalmers1999,Chalmers1990,Popper1982,Popper1963,Hempel1945,Schick1999}
Building it all
- First, build your document with LaTeX once. LaTeX should warn you about empty biblography files.
- Then, run BibTeX (note: BibTeX != BibLaTeX) on the .aux file LaTeX should have generated. This will usually generate a few further files (.bbl, .blg, -blx.bib).
- Then build your document with LaTeX twice. LaTeX should warn you when it needs to be run again.
- After that third run, you should have a clean build, and a beautiful bibliography.
- If you add extra references, you might need to repeat this process by deleting all the extra files (.aux, .bbl, .blg, -blx.bib) and doing a clean build. I've also found this necessary when doing some weird stuff in my IDE (TeXlipse).
CTAN has more information about the BibLaTeX package, along with the nicely formatted documentation. If you use Lyx, there's some BibLaTex help.
-
News Editors
SE now supports submitting news in Markdown and XHTML along with the existing WYSIWYG editor.
That means you should theoretically be able to get MathML going in news items. However, I haven't been able to myself. I really don't want to, and shouldn't have to, change the doctype of the entire page to insert a MathML fragment, especially given that's what namespaces are for, no?
-
Using WolframAlpha as a Firefox keyword search
Up until recently, if I needed to do a quick unit conversion or sum, I'd have used Google Calculator. It has pretty basic features, but it's very convenient (on most browsers thesedays, Google Search is just up there ↗)
Now I have a new tool: WolframAlpha. You might have heard of it. During its recent release, a lot of journalists portrayed it as a search engine and said it was competing with Google. That's not quite true: it's a computation knowledge engine, and I'd say it competes with Google Calculator, not Google Search proper.
If you haven't already given it a try, definately check it out. Their list of sample inputs is pretty impressive, but here's one to get you started: x ^ 2 sin(x)
However, WolframAlpha isn't quite as handy as Google Calculator: you'd have to visit their site to use it. So, I created a Firefox keyword search from their main input field. The bookmark looked like this:
But that didn't let me use "+" in my search. That's pretty vital if you're going to use it as a calculator. The solution is to use a spot of Javascript to escape your search first. Change the bookmark URL to:
javascript:document.location="http://www.wolframalpha.com/input/?i="+encodeURIComponent("%s");
Ahh. Much better. I can now do searches for things like $17 + 10%.
-
The Kember Identity May Not Exist
I used to work with Elliot Kember at Eventfinder. He recently proposed a search for the MD5 "Kember identity", a string which, when hashed, returns itself.
Something I've been thinking about: it's entirely possible such an identity doesn't exist.
Imagine a hashing algorithm that takes a two bit input and returns a two bit output. That is, the domain and range is {0, 1, 2, 3}. If the algorithm perfectly distributes its hashes over the range then, for each input value, there's a 0.25 probability of it being an identity. So, the probability of no identity being produced by the hash function over any of its domain is (0.75 ^ 4) = 0.32; about a one in three chance.
MD5 has a range of 2 ^ 128. For finding the identity, the domain must be the same as a range. So, there's a 1 in 2 ^ 128 chance of a random input being the identity. The probability of an input not being an identity is ((2 ^ 128) - 1 / (2 ^ 128)). And the probability of there not being any Kember Identity at all is ((2 ^ 128) - 1 / (2 ^ 128)) ^ (2 ^ 128).
Which, according to some quick mpmath, is:
0.367879441171442321595523770161
46086744527058077566697138371152
93233486842268612093790091730736
201865…So, there's a decent chance that there is no Kember identity. Good to know if you're considering starting to churn away on the problem.
-
PCBs!
I've been working on my GSSL Control project lately. I've designed a PCB for it, and got it back from fabrication. I managed to solder all the SMD stuff without incident so far. I've got a debug LED flashing, which is a good start. I'm using an AT90USB1287, and was planning on just using the DFU bootloader from Atmel, but it turns out that the USB bootloader can't program the fuse bytes. I had to dig out my USBasp and do some dodgy wiring to program the fuse bits to turn of the CLK/8 bit. Now I've done that though, the USB bootloader should be fine. I'll try and update the project page when I get some time.Justin
-
Everything Sounds Better With Autotune
Another (as yet) undiscovered Youtube gem. 3000 odd views, and less on the first one. Take it Katie!
-
In the Aeroplane Over the Sea
From Neutral Milk Hotel's In the Aeroplane Over the Sea:
There are lights in the clouds
Anne's ghost all around
Hear her voice as it's rolling and ringing through me
Soft and sweet
How the notes all bend and reach above the trees
Now how I remember you
How I would push my fingers through
Your mouth to make those muscles move
That made your voice so smooth and sweet