Multibooting Linux, Windows, and Mac OS X on a Mac and using them as Virtual Machines

Recently, I set up my Macbook to boot OS X, Ubuntu, and Windows XP, with a large common partition that they all can natively read and write.  Then I decided it’d be nice to be able to interact with these other systems through OS X in VMware Fusion, so I set that up as well.  This took many tries, and a lot of experimentation.  In the spirit of science, after I had everything working, I documented it, and then started from scratch to verify my instructions.  I wrote up my instructions along with a dabble of theory.  They follow.

Windows XP, Ubuntu Linux, and Mac OS X

This is what shows up when I boot.

rEFIt showing Mac OS X, Ubuntu Linux, and Windows XP

If you try this, make sure to have a backup of everything.

To multiboot a Mac with OS X 10.5.5, Ubuntu 8.10, and Windows XP SP3 circa November 2008:

Ingredients:

  • Intel Macintosh
  • OS X 10.5 Media
  • Ubuntu 8.10
  • Windows XP SP2 or higher media
  • VMWare Fusion (If you want to be able to run Windows and/or Linux inside of OS X.)

There are many ways to get this working, but also many ways that do not.  One way that works is this:

  1. Boot the Mac from the OS X install media.  Instead of installing right away, go Disk Utility.  Use Disk Utility to partition the hard drive into four partitions, Linux, Share, Windows, and Mac.  Set the first three to be MS-DOS (FAT).  Set the Mac partition to be Mac OS Extended (Journaled).  Apply this.  Quit Disk Utility.
  2. Install OS X on the Mac partition.
  3. Reboot and set up your administrator user.  Do all software updates.  Reboot.  Check for software updates again.  Reboot and repeat until no updates are available.
  4. Install rEFIt.  rEFIt is an EFI boot menu.
  5. Set the windows partition to be active (bootable).  This can be done in OS X through the terminal.  Open the terminal (/Applications/Utilities/Terminal.app) and run the command: sudo fdisk -e /dev/rdisk0
  6. You will likely get the error, “fdisk: could not open MBR file /usr/standalone/i386/boot0: No such file or directory”.  That’s ok.  Type p to print. You should see four partitions.  If you look carefully, the first partition is a 200 megabyte one and what you thought was partition one is actually partition two.  OS X put an EFI System partition in the first spot when you formatted the disk, so you actually have five partitions on the disk.  Your last partition, the Mac one, shouldn’t be visible from this screen.  The disk is a GPT disk, which contains inside of it a Master Boot Record (MBR). This MBR is how older OSes find partitions and bootloaders.  With Apple, this MBR doesn’t support extended partitions, so you can only have your four primary partitions.  That’s what you’re seeing here.  You have to type f 4 to set the Windows partition as bootable.  After that, press q to quit and then y to save changes.
  7. Insert the Windows XP media and reboot.  Select the Windows CD when rEFIt asks how it should boot.
  8. After loading drivers, Windows will ask where it should install.  Once again, it’s going to see four partitions.  The 200 meg EFI partition, and the first three partitions you defined.  I’ve tried this step in many, many ways, and the only times I’ve been successful I’ve done the following:  Make sure that the install partition is labeled C: and is the last partition shown.
  9. Select it, and format the partition.  A FAT32 quick format and long format both work.  I haven’t tested NTFS.  If I didn’t format the partition, after Windows reboots and tries to continue the install, I have gotten “Missing Operating System” errors, “hal.dll not found” errors, as well as any other excuse Windows can find for why it cannot boot.  If a different partition is labeled C: than the Windows partition that we created at the beginning, quit the install, reboot into OS X, and go back to step 5.  See if there is a little asterisk next the partition we’re trying to install to.  That means active and bootable.  From my experiments, it looks like Windows sets C: to the the active partition if there is one, otherwise it is first FAT/NTFS partition.  Why don’t we make this easy and install Windows to the first FAT/NTFS partition?  If Windows isn’t installed on the last partition in the MBR, it fails to install properly.  I’ve tried mucking around in boot.ini and other files with no luck. Windows will need to reboot twice while installing.  Finish the install.
  10. Insert your OSX 10.5 disk.  This has all the drivers you need for your system.  If you don’t have it, or you have a developer edition that doesn’t have those drivers built in, you can extract the drivers from the Boot Camp Assistant application or find them on the internet.  Go through the “Boot Camp” driver auto-installer that autoruns, and then reboot.  When rEFIt shows up, select the Windows install on the hard drive, and do all the Windows updates. Reboot and continue to do updates until there are no further updates.
  11. At rEFIt, select the partitioning tool to sync the GPT and MBR.  If it doesn’t need syncing, it will tell you.  Then select OS X and verify it boots properly.  Insert your Ubuntu media and reboot into the disk using rEFIt.
  12. Install Ubuntu as you normally would, with two exceptions: partitioning, and the boot loader.  At partitioning, select the manual partition option.  It will scan your disks, and show you yet another view of the drive.  Check the format box on the Linux partition, and set it to mount at /.  This will be sda2 if you’re following the partitioning layout.  Then proceed.  The installer will complain about a lack of swap space.  If you need swap space, you can set it up as a swapfile after the installer.  Acknowledge the complaint and move on. At the last step before you let the install finish, there is a little button labeled “Advanced”.  Clicking that button opens a window that includes a dropdown box for setting the location where the GRUB bootloader is installed.  If you leave this as the default, you’ll have to go through GRUB for both Windows and Linux, which you probably don’t want.  Instead, set this to be the Linux partition (/dev/sda2).  You won’t have to tell Windows about this bootloader because rEFIt will use it directly.  After clicking next on this screen, Ubuntu will install and prompt you to remove the media and reboot the machine.
  13. Verify Ubuntu, Windows, and OS X all boot by choosing them at the rEFIt menu.  Boot into OS X last, and remember which non-OS X OS you booted into last before choosing OS X. (You shouldn’t need to, but if you have issues see if you need to sync your partition tables between GPT and the MBR using the partitioning tool in rEFIt.)

At this point, if you don’t want to be able to use those installs in VMWare while booted into OS X, you’re done!  Otherwise, trek on, my faithful reader.

In OS X, install VMWare Fusion.  There’s nothing special about VMWare Fusion over any of the other products, its just the one I already have.  Start VMWare Fusion, and ignore the “Boot Camp Partition” entry.  There’s a bug filed at VMWare to allow people to remove it.  There’s not a real reason why you need to ignore it, but I’m going to teach you how to do it manually. You’re going to have to do it manually at least once to get Linux working, and its pretty easy.

To create a VM that points to a partition on the disk, so we can use a partition to boot into as well as run in a VM, we need to do the following.

  1. Use vmware-rawdiskcreator to create a disk file that points to the partition.
  2. Create a new VM, and remove any autocreated harddrives and add our raw disk file.
  3. Edit the vmx preferences file to disable snapshots and suspend.  With these enabled, its too easy to corrupt your partition.

vmware-rawdiskcreator is a command-line program that allows VMware VMs to directly address partitions instead of using a file as a hard drive.  It can only address the MBR, so there are the same restrictions on which partitions can be pointed to as with Windows.  At this point, I’m going to copy and paste the help.

tredegarh:~ wolf$ /Library/Application Support/VMware Fusion/vmware-rawdiskCreator –help
Usage:

/Library/Application Support/VMware Fusion/vmware-rawdiskCreator print <diskDev>
Print the partition table of a physical disk.

/Library/Application Support/VMware Fusion/vmware-rawdiskCreator hasBootCamp <diskDev>
Check if a physical disk has an Apple Boot Camp partition (exit
code is 0) or not (exit code is non-0).

/Library/Application Support/VMware Fusion/vmware-rawdiskCreator create <diskDev> <partNum> <virtDiskPath> <adapterType>
Create a VMware virtual disk backed by a partition of a physical
disk.

<diskDev> is a disk device, e.g. “/dev/disk0”.

<partNum> is the partition number as printed by the “print”
command, e.g. 3. 0 is special and means “the Apple Boot Camp
partition”.

<virtDiskPath> is the path of the virtual disk description file to
create, e.g. “~/Virtual Machines/My VM/My Raw Disk”. Two files
will be created: “<virtDiskPath>.vmdk” and
“<virtDiskPath>-pt.vmdk”.

<adapterType> is the virtual disk type. It must be one of “ide”,
“buslogic”, or “lsilogic”.

<virtDiskPath>.vmdk is a human-readable parameters file.  It identifies the vmdk as a raw disk and points to the device representing the partition.  When we’re done with this, the booted VM is going to see this raw partition as the only partition on a virtual device.  What’s going to happen when the VM wants to read the MBR and partition tables of the virtual hard drive?  Those aren’t stored in the partition we’re pointing to.  This is where the second file comes in–the <virtDiskPath>-pt.vmdk.  According to my experiments, this file is a copy of the drive’s (<diskDev>) MBR and some modified partition tables.  This is created when vmware-rawdiskCreator is run.  It isn’t dynamic.  This is why we have to remember which non-OS X partition we booted last.  rEFIt marks a partition active/bootable when we select it from rEFIt’s menu.  If we make a raw disk file pointing to a non-bootable partition, the bootloader won’t start right when loaded by the virtual machine.

Back to the doing!

Assuming we’re following my partitioning layout, and assuming we booted Linux last, we run
/Library/Application Support/VMware Fusion/vmware-rawdiskCreator create /dev/disk0 2 linux ide

Now we need to create a VM.  Open VMware Fusion, and create a new VM.  When it asks about install media, skip with “Continue without disk,” then “Create a custom virtual machine.”  Pick the closest OS you can, then finish the wizard.  Then VMware opens up the System Settings.  In the Hard Disks section, remove the automatically created hard disk.  If you want, remove the automatically-created vmdk files inside your VM.  You’ll have to close the virtual machine from inside of VMware in order to remove some lock files.  To remove the rest of the files, you can use the Finder or the Terminal or however else you want to remove files.  Virtual Machines are just directories but they look special to the Finder.  The VM will appear as a bundle (just like applications) probably inside of your Documents/Virtual Machines directory.  You can right-click on the virtual machine icon inside of the Finder and “Show Package Contents” to get to the contents of the virtual machine, or else you can just use the Terminal.  Everything ending with .vmdk inside of this new virtual machine directory is safe to remove at this point.

This is a good time to disable snapshots and suspending from this VM.  To do this, edit the .vmx file in the first level of the VM bundle.  Add the lines:

suspend.disabled = “TRUE”
snapshot.disabled = “TRUE”

and save the file.

Open VMware Fusion again, and get to the Hard Disks section of System Settings of your new virtual machine.  Add a new disk.  Choose an existing disk, and navigate to the place where you saved the .vmdk and -pt.vmdk file.  Select the .vmdk file.  Copy it or move it, depending on which you’d rather do.  Apply, and then start the virtual machine.

A common error at this point is “Missing Operating System” or VMware Fusion complains that there are no bootable devices.  This can be a problem with the raw disk files.  To fix this, remove the hard drive, rerun vmware-rawdiskcreator after rebooting into the OS through rEFIt and rebooting back into OS X in order to get the boot records correct.

At this point you can install VMware Tools.

To set up the VM for the other OS, reboot into it through rEFIt, and then back into OS X.  You could also use fdisk -e like we did earlier, but this way lets you verify that the other OSes are still bootable.  Once you’re back in VMware, create the disk for the other OS through rawdisk-creator and follow the same process we did for the first OS.

Now, you might notice an issue with the share partition.  The share isn’t accessible from more than one OS at the same time.  This is for filesystem consistency reasons.  If more than one OS were editing the drive at the same time, corruption would likely occur.

How do we fix this?

We can use file sharing through something like Samba or NFS, or Shared Folders through VMware for the VMs.  Windows and Ubuntu will still be able to access the share when they’re booted natively, and through Shared Folders when they’re in a VM.

At this point, you should have a triple-boot Mac OS X / Windows / Ubuntu Linux system.   From inside OS X, you can open the Linux and Windows partitions as VMs.  Each OS can see the share partition when natively booted, and the virtual machines can see the shares if you set up Shared Folders.

Why did I choose this partitioning layout?

It was a lot like a logic puzzle, with little facts teased out after repeated installations.

  • Windows needs to be in the last spot in the MBR or you get install issues on an Intel Mac with OS X.
  • The EFI System partition is rumored to be used in firmware updates, so I left that in the first spot, where it puts itself.
  • Grub needs to be installed on a partition located in the MBR, but Linux doesn’t.
  • Windows can only see partitions in the MBR.
  • OS X can see all the partitions.
  • Linux can see all the partitions.
  • I want a share partition that can be read by all OSes without installing programs.  This means FAT32.  To be seen by all OSes it needs to be in the MBR.
  • OS X doesn’t need to be installed in the MBR.

Let me know how it works out for you guys.

Adding an audio jack to headphones

IMG_0126

A few months ago, my headphones broke. They were Sony MDR-NC6s, pretty cheap but the noise-canceling really helps cuts out the HVAC in a cube environment. They failed when I stood up and the cord was wrapped around my chair. The cord pulled out of the bottom of the headphones. I took them apart, and saw that there was probably enough room to fit a headphone jack in if I used a Dremel and some creativity. The cord now goes out the side of the headphones instead of the bottom, but the repair only took an hour and didn’t cost more than five dollars either. I now use an audio patch cable to connect my headphones to audio sources, and if I stand up with the cord tangled, the cord pops out of either end without damage.

I posted some pictures of the process on Flickr at the photoset Repairing my headphones.

The only real lesson learned is that I wouldn’t use the Radio Shack 3.5 mm stereo jack. It doesn’t grip the cable well. I will have to replace the jack next time I order from digikey.

Uncluttering with a Closet Workbench

(The photos are located at Adam’s Workbench)

I live in a pretty cool apartment building in south Minneapolis. While I’m lucky enough to have room to use a second bedroom as an office, the office doesn’t have enough space for a workbench. Even if it did have enough space for a workbench, I wouldn’t be able to keep work in progress on it without the room getting too cluttered. So I’m stuck hauling out a box labeled “Soldering Stuff” to the kitchen, constantly fearful of damaging the the kitchen table. I’m pretty handy with a soldering iron, but hey, fear is the mindkiller. I clean up after myself pretty well, but any stray lead solder bits that found themselves in a digestive tract would be dangerous.

When I lived in a 13×13 dorm room with a roommate, I used a lab bench I made from a melamine shelf on a pair of plastic saw horses. This was cheap, and fit under my bed when it wasn’t being used. The melamine held up well to solder burns and really added to the work surface. Once again, however, the bench needed to be put away or the room was unusable for anything else.

The time cost of setting up and breaking down an electronics workbench has been so high it had essentially eliminated my hardware tinkering–until now.

The office closet contained a half-height chrome wire shelfing unit, purchased at Target shortly after we moved in. It held around twenty clear plastic boxes, each about the size of a shoe box. These were labeled, and contained computer parts, electronics, and tools. Next to this was a toolbox, a tackle box I had already hacked to carry solderless breadboards and their projects, and a bunch of empty space.

Right half of closet with wire shelfing unit and plastic boxes

I figured I could put a table in this space, use some clip-on lights as task and background lighting, and have a pretty functional work area. As long as the table didn’t stick out, I figured I would be able to close the sliding doors. With this, I would finally be able to have a workspace I could keep a project in progress on.

After measuring the space (33″ x 25″ x 65″), my fianceé found the Galant table at Ikea. It came with a melamine finish and fit inside the space perfectly.

While at Ikea, I grabbed a power strip with an on-off switch. Another department store had reasonably priced whiteboards. After I hung the whiteboard on the back of the closet and ziptied the power strip to the side of the wire rack, the workspace was complete.

109_2852

Well, almost complete. I printed out a few pictures (1, 2) of everyone’s favorite DIY science heroes, Adam Savage and Jamie Hyneman of Mythbusters and stickytacked them to the wall.

I now have a pretty functional work bench that fits inside the closet of my apartment. I wouldn’t have thought such a tight fit would have produced the nice results they did, but I’m pretty impressed. I can just open the door, roll my office chair over, and work on my very own bench. When I’m done, I can close the door and put my chair back at my desk.

My favorite part that I didn’t anticipate? My soldering iron is much less of a hassle when the cord is out of the way, over my head, looped over the closet bar.

I’ve got more photos at the photoset Adam’s Workbench.

109_2858

What tools are my favorite? Oh, I replaced my Radio Shack iron with a $45 Weller, and am amazed at how much more I solder things when I don’t have to wait 10 minutes for it to heat up. I also got a Panavise Jr. for Christmas (Thanks, Mom!) and it’s a dream for soldering to boards, because its so easy to flip the board over.

Making a Doctor Who TARDIS Cake with Jelly Babies

Last Saturday marked the finale of the third season of the new Doctor Who. To celebrate, Amanda and I made a cake–a TARDIS cake, with Jelly Babies.

I had an easier time than I expected finding Jelly Babies. Our local grocery store had them in their tiny British section.

IMG_0674

We used Betty Crocker “Party Rainbow Chip” cake mix. This is the Betty Crocker version of Funfetti, the best cake in the universe.

IMG_0682.JPG

Amanda cut the Jelly Babies up in to little pieces, and we mixed them into the batter.

IMG_0698.JPG

When the cake finished, we flipped it over. The Jelly Babies had melted, coloring the cake. The melted Jelly Babies kept the cake moist, and when they crystallized, the cake was almost crispy. Yum!

IMG_0701.JPG

We cut the cake into the shape of the TARDIS, using some reference photos from the Internet.

IMG_0703.JPG

Because we haven’t had a lot of cake decorating experience, I wanted to minimize the amount of detail work we’d have to perform upon the cake itself. We melted almond bark to make the letters and the windows. For the letters, I put the almond bark in a plastic bag and pierced a corner to make a piping bag. For the windows, I lined the bottom of some tin tea canisters with aluminum foil, and filled them with almond bark.

IMG_0703.JPG

IMG_0706.JPG

Although we weren’t able to match the blue of the TARDIS perfectly, we did as well as we could with a lot of blue and a few drops of red food coloring.

IMG_0707.JPG

After spreading the frosting on the cake, we piped lines for the doors and other details.

IMG_0714.JPG

We then added the white windows made of almond bark, the letters, and outlined the panels.

IMG_0717.JPG

It was delicious. The melted Jelly Babies really added something special to the cake. A little Huon energy or something.

Next year, or perhaps for the Christmas special, we’ll have to do a few things differently. I’d like to match the blue better. I’d also like to use dark chocolate and white chocolate to make a proper “Police Box” panel. Maybe we’ll get fancy and use fondant and gum paste.

I have more pictures (and a slideshow) on Flickr.

Restoring to a single partition after installing Linux with Boot Camp

I recently installed Linux on my Macbook with Boot Camp. I recently decided to use a virtualization solution instead, and wanted to get that space back. When I ran Boot Camp, it wouldn’t combine my partitions together, possibly because I put Linux on there, instead of Windows, and it changed the partition type.

Using this article, I fixed it and successfully combined everything.

IF YOU FOLLOW THESE INSTRUCTIONS, AND YOU LOSE ALL YOUR DATA AND YOUR MOM MAKES YOU MOVE OUT, IT IS NOT MY FAULT.

If you decide to do what I did, please understand what you’re doing. Please read all applicable man pages, and have recent backups.

I booted into the Mac Install CD, and opened the Terminal. Using gpt show disk0 and diskutil list /dev/disk0 and diskutil info /dev/disk0, I confirmed a little bit about how my disk was being managed. My Linux partitions were /dev/disk0s3 and /dev/disk0s4, which corresponded to gtp indexes 3 and 4.

I used gpt remove -i 3 disk0 and gpt remove -i 4 disk0 to remove the gpt entries.

I ran diskutil resizeVolume disk0s2 limits to find out how large I could make the original partition. I then tried to run diskutil resizeVolume disk0s2 *maximum size* but diskutil told me it couldn’t resize it. I rebooted, back into the cd again, and ran the diskutil resizeVolume disk0s2 *maximum size*. It succeeded, I rebooted, and I had my space back.

Actiontastic Exporter v0.1

I love Actiontastic. I really do. After I realized how much I was relying on it, I read a few Django tutorials and started work on an open-source clone. The morning I was going to release my clone to the world, the author of Actiontastic announced he was going to open source Actiontastic. *sigh*

Anyway, he pledges a sweet XHTML/CSS export eventually, but until then, there is no export. However, the data is stored in an sqlite database, so I wrote a little bit of python to export actions to plain text.

Pretty pictures first!

Screenshots:

  • Actiontastic Exporter showing the help screen
  • Actiontastic Exporter sorting by project
  • Actiontastic Exporter sorting by project with more information
  • Actiontastic Exporter exporting to a file

This is a python script, intended for use in the terminal or in scripts. To use in the terminal, extract the zip, and open a terminal. Use cd to enter the directory of the zip, and run python actiontastic-exporter.py. For help and options, run python actiontastic-exporter.py -h.

If you extracted it to your desktop, it will be like this:

cd ~/Desktop/actiontastic-exporter-v0.1/
python actiontastic-exporter.py
python actiontastic-exporter.py -h

It can export the output to a file. It can read from an alternative input sqlite database. It can put a little more information in the exports if you want, and it will sort by project or context. It preserves action ordering within projects and contexts.

It ignores actions that are in the inbox, and ignores completed actions.

Actiontastic Exporter requires Python 2.4. This is newer than the python shipped with any Mac at this point, and I debated on writing my own sorted() so that people could use this without upgrading their python install. However, since Actiontastic Exporter requires pysqlite2, which doesn’t come stock, and the python install is *so* easy, I decided to use the builtin.

Requires:

It never writes to the database, but if it destroys your Actiontastic database, please don’t be angry at me. Please backup your ~/Library/Application Support/Actiontastic.sqlite.

Anyway, you can download Actiontastic Exporter at the main project page at http://feelslikeburning.com/projects/actiontastic-exporter/.

Any feature requests, questions, or comments, feel free to comment below or email me using the contact form.

Linux Driver to Print Index Cards on the Samsung ML-1740–Lazyweb Request

Dear internets,

I would like to be able to print index cards on my Samsung ML-1740 in Linux. It can be done in Windows. Have any of you ever gotten this to work? I’m not averse to writing code. I’ve written a CUPS backend in python. It wasn’t that bad.

I’ve looked far and wide for this, and have only found one lonely mailing list post of a solution that doesn’t really work right.

You’re my only hope!

Gregarius Sticky Post Exporter

The project page is located at http://feelslikeburning.com/projects/gregarius-sticky-exporter/.

I use Gregarius for my feed aggregator and reader. I’ve had a busy semester, and have marked close to 2000 items as “sticky” so I can read them when I have more time. I don’t like the built-in “Browse Sticky” functionality, and would prefer something basic, but flat. I want to be able to save it, and use it to check off what I’ve read in my massive backlog of items.

Gregarius Sticky Exporter

I waded through the PHP and SQL, and talked with Matthew a bit, and came up with a little .php file.

You can download it at http://files.feelslikeburning.com/gregarius-sticky-exporter/gregarius-sticky-exporter.zip.

To install, download the zip, extract it, put your database information in, and upload it to your webhost. Hit the page with your web browser, and it should generate the list. It doesn’t modify your database at all.

It doesn’t contain all the information in the tables, just the title, the url, and the body of each feed. The title is linked to the url, and the body is displayed after the titles. If you’re interested in something extra in your exports, feel free to contact me.

Reading Ebooks on the Nokia 770 with FBreader

The Nokia 770 is the best ebook reader I have ever used. I use FBreader, a very versatile reader for Linux. My favorite feature is screen rotate. This lets me use the 770 sideways in my hand, and I can use my finger change pages with the up-down rocker.
In these pictures, the scratches on the screen are on a cheap screen protector. The real screen doesn’t scratch that easily, but I’m paranoid anyway. The screen is a bajillion times brighter, but the flash washed it out.
Nokia 770 with FBreader in rotated orientation

Nokia 770 with FBreader in rotated orientation, showing the rocker page control

I’ve read ebooks on a Palm IIIxe, a Palm IIIc, a Treo 600, a Treo 650, and a Nokia 770, and the 770 is the best ebook reader I have ever used. I would have bought it simply as an ebook reader. I’ve used the Nokia 770 to surf to Baen Free Library, grab an ebook off there, and read it, all without an extra computer.

The 770 is better in almost all ways than a paperback. I never lose my spot. I don’t need to turn off the light to go to bed, as the screen is backlit. I can fit more paperbacks than I could fit in a house on a single memory card. The 770 fits in a single hand, and I can change pages with the same hand. The screen is beautiful.
The only real downsides are the fact that the Nokia 770 uses electricity and costs more than a single paperback.