Turns out this is a limitation of VirtualBox: http://www.virtualbox.org/ticket/3925