{"id":764,"date":"2015-01-04T12:00:21","date_gmt":"2015-01-04T20:00:21","guid":{"rendered":"http:\/\/www.pagetable.com\/?p=764"},"modified":"2015-01-04T12:00:21","modified_gmt":"2015-01-04T20:00:21","slug":"using-the-os-x-10-10-hypervisor-framework-a-simple-dos-emulator","status":"publish","type":"post","link":"https:\/\/www.pagetable.com\/?p=764","title":{"rendered":"Using the OS X 10.10 Hypervisor Framework: A Simple DOS Emulator"},"content":{"rendered":"<p>Since Version 10.10 (Yosemite), OS X contains <a href=\"https:\/\/developer.apple.com\/library\/mac\/releasenotes\/MacOSX\/WhatsNewInOSX\/Articles\/MacOSX10_10.html\">Hypervisor.framework<\/a>, which provides a thin user mode abstraction of the Intel VT features. It enables apps to use virtualization without the need of a kernel extension (KEXT) &#8211; which makes them compatible with the <a href=\"https:\/\/developer.apple.com\/app-store\/review\/guidelines\/mac\/\">OS X App Store guidelines<\/a>.<\/p>\n<p>The idea is that the OS takes care of memory management (including nested paging) as well as scheduling virtual CPUs like normal threads. All we have to do is create a virtual CPU (or more!), set up all its state, assign it some memory, and run it&#8230; and then handle all &#8220;VM exits&#8221; &#8211; Intel lingo for hypervisor traps.<\/p>\n<p>There is no real documentation, but the headers contain a decent amount of information. Here are some declarations from <tt>Hypervisor\/hv.h<\/tt>:<\/p>\n<style type=\"text\/css\">\n\tp.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008400}\n\tp.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo}\n\tp.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px}\n\tp.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #004c14}\n\tspan.s1 {font-variant-ligatures: no-common-ligatures}\n\tspan.s2 {font-variant-ligatures: no-common-ligatures; color: #004c14}\n\tspan.s3 {font-variant-ligatures: no-common-ligatures; color: #bb2ca2}\n\tspan.s4 {font-variant-ligatures: no-common-ligatures; color: #008400}\n\tspan.Apple-tab-span {white-space:pre}\n<\/style>\n<p class=\"p1\"><span class=\"s1\">\/*!<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@function<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 <\/span>hv_vm_create<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@abstract<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 <\/span>Creates a VM instance for the current task<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@param<\/span><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 <\/span>flags<span class=\"Apple-converted-space\">\u00a0 <\/span>RESERVED<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@result<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 \u00a0 <\/span>0 on success or error code<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>*\/<\/span><\/p>\n<p class=\"p2\"><span class=\"s3\">extern<\/span><span class=\"s1\"> hv_return_t hv_vm_create(hv_vm_options_t flags) __HV_10_10;<\/span><\/p>\n<p class=\"p3\"><span class=\"s1\"><\/span><\/p>\n<p class=\"p1\"><span class=\"s1\">\/*!<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@function<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 <\/span>hv_vm_map<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@abstract<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 <\/span>Maps a region in the virtual address space of the current task<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 <\/span>into the guest physical address space of the VM<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@param<\/span><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 <\/span>uva<span class=\"Apple-converted-space\">\u00a0 \u00a0 <\/span>Page aligned virtual address in the current task<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@param<\/span><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 <\/span>gpa<span class=\"Apple-converted-space\">\u00a0 \u00a0 <\/span>Page aligned address in the guest physical address space<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@param<\/span><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 <\/span>size <span class=\"Apple-converted-space\">\u00a0 <\/span>Size in bytes of the region to be mapped<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@param<\/span><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 <\/span>flags<span class=\"Apple-converted-space\">\u00a0 <\/span>READ, WRITE and EXECUTE permissions of the region<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@result<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 \u00a0 <\/span>0 on success or error code<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>*\/<\/span><\/p>\n<p class=\"p2\"><span class=\"s3\">extern<\/span><span class=\"s1\"> hv_return_t hv_vm_map(hv_uvaddr_t uva, hv_gpaddr_t gpa, size_t size,<\/span><\/p>\n<p class=\"p2\"><span class=\"s1\"><span class=\"Apple-tab-span\">\t<\/span>hv_memory_flags_t flags) __HV_10_10;<\/span><\/p>\n<p class=\"p3\"><span class=\"s1\"><\/span><\/p>\n<p class=\"p1\"><span class=\"s1\">\/*!<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@function<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 <\/span>hv_vcpu_create<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@abstract<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 <\/span>Creates a vCPU instance for the current thread<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@param<\/span><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 <\/span>vcpu <span class=\"Apple-converted-space\">\u00a0 <\/span>Pointer to the vCPU ID (written on success)<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@param<\/span><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 <\/span>flags<span class=\"Apple-converted-space\">\u00a0 <\/span>RESERVED<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@result<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 \u00a0 <\/span>0 on success or error code<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>*\/<\/span><\/p>\n<p class=\"p2\"><span class=\"s3\">extern<\/span><span class=\"s1\"> hv_return_t hv_vcpu_create(hv_vcpuid_t *vcpu,<\/span><\/p>\n<p class=\"p2\"><span class=\"s1\"><span class=\"Apple-tab-span\">\t<\/span>hv_vcpu_options_t flags) __HV_10_10;<\/span><\/p>\n<p class=\"p3\"><span class=\"s1\"><\/span><\/p>\n<p class=\"p1\"><span class=\"s1\">\/*!<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@function<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 <\/span>hv_vcpu_run<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@abstract<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 <\/span>Executes a vCPU<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@param<\/span><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 <\/span>vcpu<span class=\"Apple-converted-space\">\u00a0 <\/span>vCPU ID<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s2\">@result<\/span><span class=\"s1\"> <span class=\"Apple-converted-space\">\u00a0 \u00a0 <\/span>0 on success or error code<\/span><\/p>\n<p class=\"p4\"><span class=\"s4\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <\/span><span class=\"s1\">@discussion<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 <\/span>Call blocks until the next VMEXIT of the vCPU<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>*<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>* <span class=\"Apple-converted-space\">\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 <\/span>Must be called by the owning thread<\/span><\/p>\n<p class=\"p1\"><span class=\"s1\"><span class=\"Apple-converted-space\">\u00a0<\/span>*\/<\/span><\/p>\n<p class=\"p2\"><span class=\"s3\">extern<\/span><span class=\"s1\"> hv_return_t hv_vcpu_run(hv_vcpuid_t vcpu) __HV_10_10;<\/span><\/p>\n<p>So let&#8217;s create a virtual machine that runs simple DOS applications in 16 bit real mode, and trap all &#8220;<tt>int<\/tt>&#8221; DOS system calls &#8211; similar to <a href=\"http:\/\/www.dosbox.com\">DOSBox<\/a>.<\/p>\n<p>First, we need to create a VM:<\/p>\n<pre>hv_vm_create(HV_VM_DEFAULT);<\/pre>\n<p>This creates a VM for the current Mach task (i.e. UNIX process). It&#8217;s implicit, so it doesn&#8217;t return anything. Then we allocate some memory and assign it to the VM:<\/p>\n<pre>#define VM_MEM_SIZE (1 * 1024 * 1024)\nvoid *vm_mem = valloc(VM_MEM_SIZE);\nhv_vm_map(vm_mem, 0, VM_MEM_SIZE, HV_MEMORY_READ |\n                                  HV_MEMORY_WRITE |\n                                  HV_MEMORY_EXEC);<\/pre>\n<p>And we need to create a virtual CPU:<\/p>\n<pre>hv_vcpuid_t vcpu;\nhv_vcpu_create(&amp;vcpu, HV_VCPU_DEFAULT);<\/pre>\n<p>Now comes the annoying part: Set up the CPU state. If the state is illegal or inconsistent, the CPU will refuse to run. You will need to refer to the <a href=\"http:\/\/www.intel.com\/content\/dam\/www\/public\/us\/en\/documents\/manuals\/64-ia-32-architectures-software-developer-vol-3c-part-3-manual.pdf\">Intel Manual 3C<\/a> for all the context. Luckily, most virtual machines start from 16 bit real mode, and mode changes will be done by the boot loader or operating system inside the VM, so you won&#8217;t have to worry about setting up any other state than real mode state. Real mode state setup looks something like this:<\/p>\n<pre>\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_CS_SELECTOR, 0);\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_CS_LIMIT, 0xffff);\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_CS_ACCESS_RIGHTS, 0x9b);\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_CS_BASE, 0);\n\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_DS_SELECTOR, 0);\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_DS_LIMIT, 0xffff);\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_DS_ACCESS_RIGHTS, 0x93);\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_DS_BASE, 0);\n\n[...]\n\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_CR0, 0x20);\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_CR3, 0x0);\nhv_vmx_vcpu_write_vmcs(vcpu, VMCS_GUEST_CR4, 0x2000);\n<\/pre>\n<p>After that, we should populate RAM with the code we want to execute:<\/p>\n<pre>\nFILE *f = fopen(argv[1], \"r\");\nfread((char *)vm_mem + 0x100, 1, 64 * 1024, f);\nfclose(f);\n<\/pre>\n<p>&#8230;and assign the GPRs the proper initial state &#8211; including the instruction pointer, which will point to the code:<\/p>\n<pre>\nhv_vcpu_write_register(vcpu, HV_X86_RIP, 0x100);\nhv_vcpu_write_register(vcpu, HV_X86_RFLAGS, 0x2);\nhv_vcpu_write_register(vcpu, HV_X86_RSP, 0x0);\n<\/pre>\n<p>The virtual CPU is fully set up, we can now run it!<\/p>\n<pre>\nhv_vcpu_run(vcpu);\n<\/pre>\n<p>This call runs the virtual CPU (while blocking the calling thread) until its time slice expires or a &#8220;VM exit&#8221; happens. A VM exit is a hypervisor-class exception, i.e. an event in the VM that the hypervisor wants to trap. We can trap events like exceptions, certain privileged instructions (<tt>CPUID<\/tt>, <tt>HLT<\/tt>, <tt>RDTSC<\/tt>, <tt>RDMSR<\/tt>, &#8230;) and control register (<a href=\"http:\/\/www.pagetable.com\/?p=364\">CR0, CR2, CR3, CR4, &#8230;<\/a>) accesses.<\/p>\n<p>After <tt>hv_vcpu_run()<\/tt> returns, we need to read the exit reason and act upon it, and run the virtual CPU again. Here is a minimal loop to handle VM exits:<\/p>\n<pre>\nfor (;;) {\n\thv_vcpu_run(vcpu);\n\n\tuint64_t exit_reason = hv_vmx_vcpu_read_vmcs(vcpu, VMCS_EXIT_REASON);\n\n\tswitch (exit_reason) {\n\t\tcase EXIT_REASON_EXCEPTION:\n\t\t\t[...]\n\t\t\tbreak;\n\t\tcase EXIT_REASON_EXT_INTR:\n\t\tcase EXIT_REASON_EPT_FAULT:\n\t\t\tbreak;\n\t\tdefault:\n\t\t\texit(1);\n\t}\n}\n<\/pre>\n<p><tt>EXIT_REASON_EXT_INTR<\/tt> is caused by host interrupts (usually it means that the time slice is up), so we will just ignore it. <tt>EXIT_REASON_EPT_FAULT<\/tt> happens every time the guest accesses a page for the first time, or when the guest accesses an unmapped page &#8211; this way we can emulate MMIO. In our case, we can also ignore those.<\/p>\n<p>For emulating DOS, we are catching <tt>EXIT_REASON_EXCEPTION<\/tt>, which is caused by the <tt>int<\/tt> instruction (if caught). We can get the number of the interrupt from the virtual CPU state without decoding instructions:<\/p>\n<pre>\nuint8_t interrupt_number = hv_vmx_vcpu_read_vmcs(vcpu, VMCS_IDT_VECTORING_INFO) &amp; 0xFF;\n<\/pre>\n<p>&#8230;and emulate the system call. We can read and write GPRs using the <tt>hv_vcpu_read_register()<\/tt> and <tt>hv_vcpu_write_register()<\/tt> calls.<\/p>\n<h2>hvdos &#8211; a simple DOS Emulator for OS X<\/h2>\n<p>The full source of <i>hvdos<\/i>, a simple DOS emulator using the OS X Hypervisor framework, is available at <a href=\"https:\/\/github.com\/mist64\/hvdos\">github.com\/mist64\/hvdos<\/a>.<\/p>\n<p>It contains an adapted version of the <a href=\"https:\/\/github.com\/libcpu\/libcpu\">libcpu<\/a> DOS system call library and manages to run (parts of) some <tt>.COM<\/tt> files. A good demo is the <a href=\"https:\/\/github.com\/libcpu\/libcpu\/blob\/2fa4a9574a3320bd3953d1b238c36f55090405fb\/test\/bin\/x86\/pkunzjr.com?raw=true\">pkunzjr.com<\/a> ZIP decompression tool.<\/p>\n<h2>Creating your own Hypervisor<\/h2>\n<p><i>hvdos<\/i> can serve as a template for your own Hypervisor.framework experiments. It contains wrapper functions for error handling, a header that defines all Intel VT constants (taken from FreeBSD), complete 16 bit real mode initialization, as well as a few helper functions to set up the fields <tt>VMCS_PIN_BASED_CTLS<\/tt>, <tt>VMCS_PRI_PROC_BASED_CTLS<\/tt>, <tt>VMCS_SEC_PROC_BASED_CTLS<\/tt> and <tt>VMCS_ENTRY_CTLS<\/tt> properly. These are needed to define, among other things, which events cause VM exits.<\/p>\n<p>You can easily add more CPUs by creating one POSIX thread per virtual CPU. For every thread, you create a virtual CPU and run a VM exit main loop.<\/p>\n<p>You can for example start writing an IBM PC emulator by running <a href=\"http:\/\/bochs.sourceforge.net\/doc\/docbook\/user\/rom-images.html\">Bochs BIOS<\/a> and trapping I\/O accesses, or running MS-DOS without BIOS by trapping BIOS <tt>int<\/tt> calls.<\/p>\n<p>Or you could bridge an existing open source solution (QEMU, QEMU+KVM, VirtualBox, DOSBox, &#8230;) to use Hypervisor.framework&#8230;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Since Version 10.10 (Yosemite), OS X contains Hypervisor.framework, which provides a thin user mode abstraction of the Intel VT features. It enables apps to use virtualization without the need of a kernel extension (KEXT) &#8211; which makes them compatible with the OS X App Store guidelines. The idea is that the OS takes care of &#8230; <a title=\"Using the OS X 10.10 Hypervisor Framework: A Simple DOS Emulator\" class=\"read-more\" href=\"https:\/\/www.pagetable.com\/?p=764\" aria-label=\"Read more about Using the OS X 10.10 Hypervisor Framework: A Simple DOS Emulator\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,35,38],"tags":[],"class_list":["post-764","post","type-post","status-publish","format-standard","hentry","category-dos","category-virtualization","category-x86"],"_links":{"self":[{"href":"https:\/\/www.pagetable.com\/index.php?rest_route=\/wp\/v2\/posts\/764","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pagetable.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pagetable.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pagetable.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pagetable.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=764"}],"version-history":[{"count":0,"href":"https:\/\/www.pagetable.com\/index.php?rest_route=\/wp\/v2\/posts\/764\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.pagetable.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=764"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pagetable.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=764"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pagetable.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=764"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}