I followed the suggestions in this blog article Encrypted partitions with Ubuntu/Debian. In Wheezy, KDE4 finds the USB key prompts for the password, then shows you the unencrypted volume and you can mount/eject like any other USB key.

The problem is that this doesn't work in Squeeze. KDE4 prompts you for the password, and it creates the device-mapper entry, but it doesn't allow you to mount/eject the USB key from the GUI. While its a irritation, it's easy to manually mount/umount the drive from the command line, but it's a problem for desktop users.

I don't know enough about the innards of udev, udisks, D-Bus, HAL and KDE4 to know where to look for a fix. A quick hunt through the Debian bug reports suggest that this is a problem and it's not really a KDE4 problem as it also impacts GNOME users too - but so far no solution.