Multibooting Linux, Windows, and Mac OS X on a Mac and using them as Virtual Machines

Recently, I set up my Macbook to boot OS X, Ubuntu, and Windows XP, with a large common partition that they all can natively read and write.  Then I decided it’d be nice to be able to interact with these other systems through OS X in VMware Fusion, so I set that up as well.  This took many tries, and a lot of experimentation.  In the spirit of science, after I had everything working, I documented it, and then started from scratch to verify my instructions.  I wrote up my instructions along with a dabble of theory.  They follow.

Windows XP, Ubuntu Linux, and Mac OS X

This is what shows up when I boot.

rEFIt showing Mac OS X, Ubuntu Linux, and Windows XP

If you try this, make sure to have a backup of everything.

To multiboot a Mac with OS X 10.5.5, Ubuntu 8.10, and Windows XP SP3 circa November 2008:

Ingredients:

  • Intel Macintosh
  • OS X 10.5 Media
  • Ubuntu 8.10
  • Windows XP SP2 or higher media
  • VMWare Fusion (If you want to be able to run Windows and/or Linux inside of OS X.)

There are many ways to get this working, but also many ways that do not.  One way that works is this:

  1. Boot the Mac from the OS X install media.  Instead of installing right away, go Disk Utility.  Use Disk Utility to partition the hard drive into four partitions, Linux, Share, Windows, and Mac.  Set the first three to be MS-DOS (FAT).  Set the Mac partition to be Mac OS Extended (Journaled).  Apply this.  Quit Disk Utility.
  2. Install OS X on the Mac partition.
  3. Reboot and set up your administrator user.  Do all software updates.  Reboot.  Check for software updates again.  Reboot and repeat until no updates are available.
  4. Install rEFIt.  rEFIt is an EFI boot menu.
  5. Set the windows partition to be active (bootable).  This can be done in OS X through the terminal.  Open the terminal (/Applications/Utilities/Terminal.app) and run the command: sudo fdisk -e /dev/rdisk0
  6. You will likely get the error, “fdisk: could not open MBR file /usr/standalone/i386/boot0: No such file or directory”.  That’s ok.  Type p to print. You should see four partitions.  If you look carefully, the first partition is a 200 megabyte one and what you thought was partition one is actually partition two.  OS X put an EFI System partition in the first spot when you formatted the disk, so you actually have five partitions on the disk.  Your last partition, the Mac one, shouldn’t be visible from this screen.  The disk is a GPT disk, which contains inside of it a Master Boot Record (MBR). This MBR is how older OSes find partitions and bootloaders.  With Apple, this MBR doesn’t support extended partitions, so you can only have your four primary partitions.  That’s what you’re seeing here.  You have to type f 4 to set the Windows partition as bootable.  After that, press q to quit and then y to save changes.
  7. Insert the Windows XP media and reboot.  Select the Windows CD when rEFIt asks how it should boot.
  8. After loading drivers, Windows will ask where it should install.  Once again, it’s going to see four partitions.  The 200 meg EFI partition, and the first three partitions you defined.  I’ve tried this step in many, many ways, and the only times I’ve been successful I’ve done the following:  Make sure that the install partition is labeled C: and is the last partition shown.
  9. Select it, and format the partition.  A FAT32 quick format and long format both work.  I haven’t tested NTFS.  If I didn’t format the partition, after Windows reboots and tries to continue the install, I have gotten “Missing Operating System” errors, “hal.dll not found” errors, as well as any other excuse Windows can find for why it cannot boot.  If a different partition is labeled C: than the Windows partition that we created at the beginning, quit the install, reboot into OS X, and go back to step 5.  See if there is a little asterisk next the partition we’re trying to install to.  That means active and bootable.  From my experiments, it looks like Windows sets C: to the the active partition if there is one, otherwise it is first FAT/NTFS partition.  Why don’t we make this easy and install Windows to the first FAT/NTFS partition?  If Windows isn’t installed on the last partition in the MBR, it fails to install properly.  I’ve tried mucking around in boot.ini and other files with no luck. Windows will need to reboot twice while installing.  Finish the install.
  10. Insert your OSX 10.5 disk.  This has all the drivers you need for your system.  If you don’t have it, or you have a developer edition that doesn’t have those drivers built in, you can extract the drivers from the Boot Camp Assistant application or find them on the internet.  Go through the “Boot Camp” driver auto-installer that autoruns, and then reboot.  When rEFIt shows up, select the Windows install on the hard drive, and do all the Windows updates. Reboot and continue to do updates until there are no further updates.
  11. At rEFIt, select the partitioning tool to sync the GPT and MBR.  If it doesn’t need syncing, it will tell you.  Then select OS X and verify it boots properly.  Insert your Ubuntu media and reboot into the disk using rEFIt.
  12. Install Ubuntu as you normally would, with two exceptions: partitioning, and the boot loader.  At partitioning, select the manual partition option.  It will scan your disks, and show you yet another view of the drive.  Check the format box on the Linux partition, and set it to mount at /.  This will be sda2 if you’re following the partitioning layout.  Then proceed.  The installer will complain about a lack of swap space.  If you need swap space, you can set it up as a swapfile after the installer.  Acknowledge the complaint and move on. At the last step before you let the install finish, there is a little button labeled “Advanced”.  Clicking that button opens a window that includes a dropdown box for setting the location where the GRUB bootloader is installed.  If you leave this as the default, you’ll have to go through GRUB for both Windows and Linux, which you probably don’t want.  Instead, set this to be the Linux partition (/dev/sda2).  You won’t have to tell Windows about this bootloader because rEFIt will use it directly.  After clicking next on this screen, Ubuntu will install and prompt you to remove the media and reboot the machine.
  13. Verify Ubuntu, Windows, and OS X all boot by choosing them at the rEFIt menu.  Boot into OS X last, and remember which non-OS X OS you booted into last before choosing OS X. (You shouldn’t need to, but if you have issues see if you need to sync your partition tables between GPT and the MBR using the partitioning tool in rEFIt.)

At this point, if you don’t want to be able to use those installs in VMWare while booted into OS X, you’re done!  Otherwise, trek on, my faithful reader.

In OS X, install VMWare Fusion.  There’s nothing special about VMWare Fusion over any of the other products, its just the one I already have.  Start VMWare Fusion, and ignore the “Boot Camp Partition” entry.  There’s a bug filed at VMWare to allow people to remove it.  There’s not a real reason why you need to ignore it, but I’m going to teach you how to do it manually. You’re going to have to do it manually at least once to get Linux working, and its pretty easy.

To create a VM that points to a partition on the disk, so we can use a partition to boot into as well as run in a VM, we need to do the following.

  1. Use vmware-rawdiskcreator to create a disk file that points to the partition.
  2. Create a new VM, and remove any autocreated harddrives and add our raw disk file.
  3. Edit the vmx preferences file to disable snapshots and suspend.  With these enabled, its too easy to corrupt your partition.

vmware-rawdiskcreator is a command-line program that allows VMware VMs to directly address partitions instead of using a file as a hard drive.  It can only address the MBR, so there are the same restrictions on which partitions can be pointed to as with Windows.  At this point, I’m going to copy and paste the help.

tredegarh:~ wolf$ /Library/Application Support/VMware Fusion/vmware-rawdiskCreator –help
Usage:

/Library/Application Support/VMware Fusion/vmware-rawdiskCreator print <diskDev>
Print the partition table of a physical disk.

/Library/Application Support/VMware Fusion/vmware-rawdiskCreator hasBootCamp <diskDev>
Check if a physical disk has an Apple Boot Camp partition (exit
code is 0) or not (exit code is non-0).

/Library/Application Support/VMware Fusion/vmware-rawdiskCreator create <diskDev> <partNum> <virtDiskPath> <adapterType>
Create a VMware virtual disk backed by a partition of a physical
disk.

<diskDev> is a disk device, e.g. “/dev/disk0”.

<partNum> is the partition number as printed by the “print”
command, e.g. 3. 0 is special and means “the Apple Boot Camp
partition”.

<virtDiskPath> is the path of the virtual disk description file to
create, e.g. “~/Virtual Machines/My VM/My Raw Disk”. Two files
will be created: “<virtDiskPath>.vmdk” and
“<virtDiskPath>-pt.vmdk”.

<adapterType> is the virtual disk type. It must be one of “ide”,
“buslogic”, or “lsilogic”.

<virtDiskPath>.vmdk is a human-readable parameters file.  It identifies the vmdk as a raw disk and points to the device representing the partition.  When we’re done with this, the booted VM is going to see this raw partition as the only partition on a virtual device.  What’s going to happen when the VM wants to read the MBR and partition tables of the virtual hard drive?  Those aren’t stored in the partition we’re pointing to.  This is where the second file comes in–the <virtDiskPath>-pt.vmdk.  According to my experiments, this file is a copy of the drive’s (<diskDev>) MBR and some modified partition tables.  This is created when vmware-rawdiskCreator is run.  It isn’t dynamic.  This is why we have to remember which non-OS X partition we booted last.  rEFIt marks a partition active/bootable when we select it from rEFIt’s menu.  If we make a raw disk file pointing to a non-bootable partition, the bootloader won’t start right when loaded by the virtual machine.

Back to the doing!

Assuming we’re following my partitioning layout, and assuming we booted Linux last, we run
/Library/Application Support/VMware Fusion/vmware-rawdiskCreator create /dev/disk0 2 linux ide

Now we need to create a VM.  Open VMware Fusion, and create a new VM.  When it asks about install media, skip with “Continue without disk,” then “Create a custom virtual machine.”  Pick the closest OS you can, then finish the wizard.  Then VMware opens up the System Settings.  In the Hard Disks section, remove the automatically created hard disk.  If you want, remove the automatically-created vmdk files inside your VM.  You’ll have to close the virtual machine from inside of VMware in order to remove some lock files.  To remove the rest of the files, you can use the Finder or the Terminal or however else you want to remove files.  Virtual Machines are just directories but they look special to the Finder.  The VM will appear as a bundle (just like applications) probably inside of your Documents/Virtual Machines directory.  You can right-click on the virtual machine icon inside of the Finder and “Show Package Contents” to get to the contents of the virtual machine, or else you can just use the Terminal.  Everything ending with .vmdk inside of this new virtual machine directory is safe to remove at this point.

This is a good time to disable snapshots and suspending from this VM.  To do this, edit the .vmx file in the first level of the VM bundle.  Add the lines:

suspend.disabled = “TRUE”
snapshot.disabled = “TRUE”

and save the file.

Open VMware Fusion again, and get to the Hard Disks section of System Settings of your new virtual machine.  Add a new disk.  Choose an existing disk, and navigate to the place where you saved the .vmdk and -pt.vmdk file.  Select the .vmdk file.  Copy it or move it, depending on which you’d rather do.  Apply, and then start the virtual machine.

A common error at this point is “Missing Operating System” or VMware Fusion complains that there are no bootable devices.  This can be a problem with the raw disk files.  To fix this, remove the hard drive, rerun vmware-rawdiskcreator after rebooting into the OS through rEFIt and rebooting back into OS X in order to get the boot records correct.

At this point you can install VMware Tools.

To set up the VM for the other OS, reboot into it through rEFIt, and then back into OS X.  You could also use fdisk -e like we did earlier, but this way lets you verify that the other OSes are still bootable.  Once you’re back in VMware, create the disk for the other OS through rawdisk-creator and follow the same process we did for the first OS.

Now, you might notice an issue with the share partition.  The share isn’t accessible from more than one OS at the same time.  This is for filesystem consistency reasons.  If more than one OS were editing the drive at the same time, corruption would likely occur.

How do we fix this?

We can use file sharing through something like Samba or NFS, or Shared Folders through VMware for the VMs.  Windows and Ubuntu will still be able to access the share when they’re booted natively, and through Shared Folders when they’re in a VM.

At this point, you should have a triple-boot Mac OS X / Windows / Ubuntu Linux system.   From inside OS X, you can open the Linux and Windows partitions as VMs.  Each OS can see the share partition when natively booted, and the virtual machines can see the shares if you set up Shared Folders.

Why did I choose this partitioning layout?

It was a lot like a logic puzzle, with little facts teased out after repeated installations.

  • Windows needs to be in the last spot in the MBR or you get install issues on an Intel Mac with OS X.
  • The EFI System partition is rumored to be used in firmware updates, so I left that in the first spot, where it puts itself.
  • Grub needs to be installed on a partition located in the MBR, but Linux doesn’t.
  • Windows can only see partitions in the MBR.
  • OS X can see all the partitions.
  • Linux can see all the partitions.
  • I want a share partition that can be read by all OSes without installing programs.  This means FAT32.  To be seen by all OSes it needs to be in the MBR.
  • OS X doesn’t need to be installed in the MBR.

Let me know how it works out for you guys.

18 thoughts on “Multibooting Linux, Windows, and Mac OS X on a Mac and using them as Virtual Machines”

  1. I found this using Google and, although my situation is a little bit different, it was helpful at a couple points in my own VMware Fusion + multi-booting odyssey. So thanks!

    Since I don’t know what blog software this is, I’ll just paste in the URL where I give part of my story: http://episteme.arstechnica.com/eve/forums/a/tpc/f/8300945231/m/274002236931

    I don’t recommend trying to read all of that unless you’re really bored, so hopefully you can get the gist from the first few paragraphs. Post here or there if you want to know more. I probably won’t have the time/energy to write a tool in C to do that for you, so, uh, feel free. 🙂

    Also, word to my fellow EE-type.

  2. Thanks for the detailed instructions on this not so easy task. I’m not interested in Linux but do want to triple boot OSX, XP and Vista Ultimate so that I can native boot into all three (a la Boot Camp) and also use VMW Fusion when native booted in OSX. Any ideas about necessary modifications to what you’ve posted here given what I want to do?

  3. willmo, thanks for the comment. I’m glad to see your progress. I might expand this post later when I try to throw opensolaris into the mess–it uses (used) a forked Grub that isn’t quite the same so it can boot from ZFS.

    Valdeane, the virtualization half of this post might be the same. I’m not sure on things like Windows oddities for installation orders, but I would start with any guide online on getting triplebooting for OS X, XP, and Vista on your machine. After that, use the raw disk creator tool to get them to work in Fusion. Make sure to post your results somewhere!

  4. Oh I am sorry … seems like I mixed it up again. So yeah now I figured out I have to run vmcreate after I did the reboot to sort out the boot selection. Now I get grub but I can not boot …

  5. Wow, great info! Got here from willmo’s post with the exact need.
    It would be sweet if this kind of info (of this level of detail) was available from Apple & VMware, but I think it even sweeter that the community provides so much (value) with so little (info) for nothing in return (+KARMA+).

  6. Great info but I got stuck! I got as far as this point:

    “Open VMware Fusion again, and get to the Hard Disks section of System Settings of your new virtual machine. Add a new disk. Choose an existing disk, and navigate to the place where you saved the .vmdk and -pt.vmdk file. Select the .vmdk file. Copy it or move it, depending on which you’d rather do. Apply, and then start the virtual machine.”

    The problem is that I only have the option to copy, and when I do it asks me for my system password. I enter it in correctly and it says “Insufficient permissions to access the file”. I’ve tried the root password and changing the .vmdk’s permissions to 666 unsuccessfully. Any thoughts?

  7. Edit: Problem solved. I copied and pasted the lines of code to disable snapshots/suspending. Turns out the quotation marks were the problem! Anyway, I’m past that point and I’m at the GRUB prompt. I’m researching how to boot into the OS now. Thank you!

  8. Sorry for spamming the comments! It seems that I can’t mount the ext3 partition (hda0,2). From the GRUB loader, I tried to simply view a file on the partition but I get an error “Error 17: Cannot mount selected partition”.

  9. I seem to have the same problem as Roach:
    I run into “Insufficient permissions to access the file”.
    The vmdk files are in my home directory and have 755 permissions.

    This is the output of vmware-rawdiskcreator:
    Nr Start Size Type Id Sytem
    — ———- ———- —- — ————————
    1 1 409639 BIOS EE Unknown
    2 409640 146538496 BIOS AF HFS+
    3 147210280 54265816 BIOS 83 Linux
    4 201738240 32702464 BIOS C Win95 FAT32 (LBA)

    And I also did a strings on my partition-table file:
    $ strings linux-pt.vmdk
    Missing operating system
    Operating system loading error
    EFI PART
    BSD 4.4
    <?EFI FAT32
    Non-system disk
    Press any key to reboot
    RRaA
    rrAa<
    BSD 4.4
    <?EFI FAT32
    Non-system disk
    Press any key to reboot
    RRaA
    rrAaA’

    Looks like something went wrong. I previously booted into my ubuntu and also saw in the partitioning-tool for refit, that my linux-partition is marked active with an asterisk.

    What could be wrong?

    1. What you’re seeing in that vmdk-pt file is part of the boot code. This has all the possible error strings in it–its not indicative that you messed it up.

      You need both permissions to read the vmdk-pt file, and *also* permission to unmount that partition (if its already mounted) and also to directly access the device. This requires administrator privileges (unless you make some permission changes.) That likely explains your permission error.

      My guess is that you didn’t have the partition marked active when you used rawdiskcreator. As I stated in the article, the partition tables don’t update in your vmdk pointer file if you update them on your disk. This means if you mark it active or unactivate it on disk, it won’t change your already created -pt.vmdk file.

      Activate the partition. From within OSX, verify that the partition is still marked bootable (refit will mark partitions bootable/active for you) and then use rawdiskcreator. I predict it will work fine.

  10. Hi Adam,

    Just looked it up in the Disk-Utility in OS X.
    It is as you stated: the partition is not marked active. However, when try to activate it I get an error. When I try to do a Volume check, I get:

    ** /dev/disk0s3
    Invalid sector size: 0

    I can boot into grub/ubuntu from rEFIt.
    What is Disk-Utility’s problem?

  11. OK. Now I used the CLI for disk-utility to get some more details about the trouble-maker partition:

    $ diskutil info disk0s3
    Device Identifier: disk0s3
    Device Node: /dev/disk0s3
    Part Of Whole: disk0
    Device / Media Name: Untitled

    Volume Name:
    Mount Point:
    File System: MS-DOS FAT32

    Partition Type: Microsoft Basic Data
    Bootable: Is bootable
    Media Type: Generic
    Protocol: SATA
    SMART Status: Verified

    Total Size: 25.9 Gi (27784097792 B) (54265816 512-byte blocks)
    Free Space: 0.0 B (0 B) (0 512-byte blocks)

    Read Only: No
    Ejectable: No
    Whole: No
    Internal: Yes

  12. I was wondering what sort of partitioning scheme would work for OS X 10.5, RHEL 5.3, and Windows 7 Beta, and wouldn’t you know, this one does. Thanks for posting!

    The only change I had to make was installing Windows last, and changing the active partition right before installing Windows. For some reason Anaconda insisted on marking the RHEL partition as active.

    — G

  13. So I got my triple boot system fixed after 10.5.6 update broke it. I fixed that now and I Finally have all 3 working just fine and dandy. thanx much. However im unable to get the linux to boot in the virtual machine. I am having the same problem Roach did. I can boot Ubuntu directly from bare metal just fine, but I can’t boot it from the VM. All i get is the grub menu and i get a cant access partition as well. I thought for the hell of it I would try using paragon to let me have read/write access to the partition, that didn’t do it.

    It seems to me, the vmware needs to be able to access and read/write the ext2 filesystem but it can’t. Any way on getting around this? or a fix? Any help you could provide would be much appreciated.

  14. Well i did it, i finally got it working. triple boot with bare metal and vmware fusion on the mac loading of the bare metal ubuntu and xp partition installation. All os’s in bare metal boot and vm access have access to a shared data partition that can read/write to the shared data partition. soooo nice to finally have it working, thank you all who documented steps on this. The issue i had was fixed simply by running a gpt sync from refit, amazing software that refit

Leave a Reply