OS X

Setting the OS X kernel asynchronous I/O limits to avoid VirtualBox crashing OS X

I had the need to setup a new VM for software testing today, and I kept running into intermittent problems where VirtualBox would freeze and then an OS X kernel panic, freezing/crashing the entire machine.

Luckily, I had made a snapshot in the OS moments earlier to the crash so I had a safe place to revert to, but the crashes kept happing at seemingly random times.

I setup a looped execution of 'dmesg' to see what was going on just before the crash and saw this at the next freeze:

VBoxDrv: host_vmxon  -> vmx_use_count=1 rc=0
VBoxDrv: host_vmxoff -> vmx_use_count=0
VBoxDrv: host_vmxon  -> vmx_use_count=1 rc=0
aio_queue_async_request(): too many in flight for proc: 16.
aio_queue_async_request(): too many in flight for proc: 16.
aio_queue_async_request(): too many in flight for proc: 16.
aio_queue_async_request(): too many in flight for proc: 16.
aio_queue_async_request(): too many in flight for proc: 16.
aio_queue_async_request(): too many in flight for proc: 16.

The first VBoxDrv messages didn't pull anything interesting in Google, but the other messages did: Virtual Box ticket #11219 and this blog post.

It would appear that the default limits for the OS X kernel's asynchronous I/O are very, very low. VirtualBox likely exceeds them when your VM(s) are performing heavy disk I/O, hence the 'too many in flight' message in the logs.

Luckily for us, there's a quick and easy solution:

sudo sysctl -w  kern.aiomax=512 kern.aioprocmax=128 kern.aiothreads=16

then restart VirtualBox. These settings will apply until you reboot. To make the changes permanent, add/update the following lines in /etc/sysctl.conf:

kern.aiomax=512
kern.aioprocmax=128
kern.aiothreads=16

Note: you can probably set those limits even higher, as documentation for Sybase (by SAP) recommends values 2048 / 1024 / 16 when using its software.

Reset sync history in OS X Lion without using iSync

In previous version of OS X, the iSync application could be used to reset sync history. This technique often resolved a bunch of miscellaneous bugs and sync-related problems. The iSync application has since been removed in OS X Lion and Apple's support documents aren't of much help. This one claims that resetting sync services can be performed from iSync in Leopard and from the Sync menu extra in later version of OS X. This document describes how to enable the menu extra in the MobileMe settings. Besides the fact that MobileMe is deprecate and no longer in use, the option described in that document to enable the Sync menu extra only appears after signing in to MobileMe... Which is deprecated.

Fortunately, there's a hidden folder you can open to enable any menu extra manually! Open Finder and select Go > Go to Folder (or press Command+Shift+G). In the prompt that appears, type: /System/Library/CoreServices/Menu Extras/. This should bring you to a folder with a bunch of menu extras that you can double-click to add to the menu bar. The one of interest in this case is Sync.menu, which should appear in your menu bar a second or two after opening it.

Once it's in your menu bar, hold Alt/Option and click on it to expose the advanced options and click "Reset Sync Services". After that's done, you can remove it from your menu bar if you'd like by holding Command and then clicking & dragging it out from the menu bar.

Printing ODT documents from the command line (and mass PDF exporting)

I recently had to print a collection of documents I had created over the year and I really was not thrilled at the prospect of having to open the file, Command+P, Enter, Command+W, Command+Tab, open new document, rinse, repeat for some 60 odd documents. All the documents in question were ODT documents created using either OpenOffice or LibreOffice, depending on how old they were.

A bit of research on command-line printing using OpenOffice and LibreOffice led me to some outdated posts that didn't quite give me the information I was looking for, but gave me a good place to start: the Writer application's binary is named 'soffice' and accepts various command-line arguments. A quick search revealed that the file does indeed exist inside the OS X application file:

$ find /Applications/LibreOffice.app -name soffice
/Applications/LibreOffice.app/Contents/MacOS/soffice

(for those that don't know, OS X applications are actually just plain folders that hold the application's metadata, executables and related files)

What's even better is that LibreOffice now has an option to do exactly what I was looking for:

$ /Applications/LibreOffice.app/Contents/MacOS/soffice -h
LibreOffice 3.5

Usage: soffice [options] [documents...]

Options:
--minimized    keep startup bitmap minimized.
[...]
--print-to-file [-printer-name printer_name] [--outdir output_dir] files
      Batch print files to file.
      If --outdir is not specified then current working dir is used as output_dir.
      Eg. --print-to-file *.doc
          --print-to-file --printer-name nasty_lowres_printer --outdir /home/user *.doc

Using the "--print-to-file" parameter will convert any document that LibreOffice can read into a raw postscript file by the same name.

Although I could easily send the resulting PostScript files to the printer using CUPS' lp utility, I wanted to take this one step further. Each of these documents contained the same colour logo and I wanted to avoid wasting any ink. Another search brought me to this question on SuperUser.com, where ysis's answer demonstrated that the gs (GhostScript) utility can do this and even merge all the documents into one file:

gs -sOutputFile=converted.pdf -sDEVICE=pdfwrite -sColorConversionStrategy=Gray -dProcessColorModel=/DeviceGray -dCompatibiltyLevel=1.4 -dNOPAUSE -dBATCH input1.ps input2.ps input3.ps

After about 30 seconds of processing all of the 60 files were there, in blank and white, concatenated into converted.pdf. Printed it using OS X's Preview application using the "fast draft" setting and I had all my 60 documents printed with minimal ink used in less than 5 minutes!

Disable Time Machine's local snapshots feature

Lion introduced a new Time Machine feature the OS would automatically create local snapshots of files so that even without a backup disk attached, you could still enter time machine and recover or undo changes to certain files. As your disk space becomes low, this space is automatically freed and the rate of snapshot creation is reduced.

This is great on desktops and I would assume the newer Macbook Pros that have better disks, but on my older MacBook Pro I've noticed that Safari has had trouble loading web pages and during this time, I hear the disk thrashing for about a minute or so and then all of a sudden the page loads. After a bit of researching and some help from the 'fs_usage' command, I discovered that a process called 'mtmd' was accessing the filesystem often and it turns out that mtmd is responsible for creating these local time machine snapshots. A quick Google revealed how to disable the local snapshot functionality:

sudo tmutil disablelocal

We'll see if that helps the disk thrashing! I did immediately recover 1GB of disk space though which is a good sign.

Cryptic MySQL error

Today as I was attempting to test one of my PHP applications, I received this error after attempting to connect to a MySQL database:

Warning:  mysql_connect() [function.mysql-connect]: OK packet 6 bytes shorter
than expected in index.php on line 29

Warning:  mysql_connect() [function.mysql-connect]: mysqlnd cannot connect to
MySQL 4.1+ using old authentication in index.php on line 29

The script giving the error was running on OS X 10.6.4 with the stock PHP 5.3.1. After doing a bit of searching and reading the MySQL documentation on the old password format, I was a bit confused because I ran this on the server:

[user@host ~]# rpm -q mysql mysql-server
mysql-5.0.77-4.el5_5.3
mysql-server-5.0.77-4.el5_5.3

Both the server and client should support the new authentication version, which was introduced all the way back in MySQL 4.1. So why wouldn't it connect?

It turns out that CentOS 5 disables the new password hashes by default in favour of remaining compatible with 3.x (and earlier) MySQL clients. All you have to do is edit /etc/my.cnf and comment the old_passwords=1 line. After restarting the server, you should notice that running SELECT PASSWORD('foobar'); in a MySQL prompt will return 41-character hashes, not the old-style 16 character hashes. Reset the user passwords to start using the new hashes and you'll be good to go.