Tool’s ‘Rosetta Stoned’ Live Performance to Include Apple I Schematic

December 12th, 2007 by Michael Steil

It probably takes a geek in the front row (which is likely for a concert in San Francisco) to notice that the computer schematics Tool showed on the screens during their “Rosetta Stoned” performance are those of an Apple 1 - but distorted:

It’s easy to read the mirrored text “MICROPROCESSOR”, and if you pay close attention, you can read “6502″ - a classic 8 bit CPU which has been featured in pop culture before.

The clipping shown during ‘Rosetta Stoned’ has been mirrored horizontally, and its width has been doubled. Here is the original schematic:

The schematic was shown in the background video in Los Angeles (10 Dec 2007) and San Francisco (11 Dec 2007), but IIRC not in Frankfurt (28 Aug 2007; I think I would have noticed), so it seems to be new in Tool’s Fall 2007 Tour.

But I forgot my pen…

64 bit is a lot!

November 18th, 2007 by Michael Steil

When people talk about porting their applications to 64 bit, I sometimes hear them wonder how long it will be until they have to port everything to 128 bit - after all, the swiches from 8 to 16 bit (e.g. CP/M to DOS), 16 to 32 bit (DOS/Windows 3 to Windows 95/NT) and 32 to 64 have all happened in the last 25 years.

But all these switches don’t, even after Moore-compensation, don’t push the limit in a linear, but in an exponential way: 64 bit extends addressable memory by a factor of 4 billion. A database holding 2^64 bytes can store a 1 Megapixel JPEG of every square meter of the earth’s surface.

AMD and Intel understood that no CPU in the next few decades would need as much RAM (!), and therefore decided to, completely opaque for user space, only implement 48 bit addressing for now, saving 2 extra levels of page tables, and thus keeping TLB complexity lower. That’s about the size of New Jersey.

Game Development Archeology: Zelda on Game Boy comes with source

June 29th, 2007 by Michael Steil

Imagine you’re writing a Game Boy game, and the resulting ROM with all the code and data is just a little over one megabyte in size. No big deal, just pad the game to two megabytes, and use a 2 MB ROM in the cartridge. Just tell the linker to allocate 2 MB or RAM, put the actual data at the beginning, and then write a 2 MB “.gb” image to disk, which will then be sent to the ROM chip factory.

Now imagine you’re doing this in MS-DOS. Your linker, probably written in C, calls malloc() of the runtime library of the C compiler. You already know where this is going?

While modern operating systems will always clear all malloc()ed memory, so that you cannot get to other processes’ data, this was uncommon in the single-user MS-DOS days. If you allocate 2 MB of RAM (the linker must have used a DOS extender or XMS), you’d get memory with random data in it: leftovers from whatever was in this memory before. (seppel tells me that this can also be caused by seek()ing over EOF in MS-DOS, in which case the previous data on the hard disk will be in the image.)

This is what happened with the 1998 Game Boy/Game Boy Color game “The Legend of Zelda - Link’s Awakening DX” (MD5: ee0424cf1523f67c5007566aed70696d). If you look at the image starting at 0×106000, you will find all kinds of interesting data, which will tell you a lot about the game’s development. Let’s call this “game development archeology”…

The ROM image includes big chunks of Borland’s Turbo C IDE (Turbo Vision interface) for DOS, as well as traces of the “QBasic” MS-DOS Editor. It is unclear which editor they used for what, but they might have used Turbo C to write DOS code to support building, as there is a complete copy of this C program in the ROM:

#include
#include

int main(void) {
	FILE *fp,*f1;
	int a=0xcd;
	int b=0xc6;
	int c=0×29;
	int ch;
	unsigned long i=0;

	if((fp=fopen(”zeldag.gb”,”rb”))==NULL) {
		printf(”can’t open the file”);
		return 0;
	}

	if((f1=fopen(”ttmp.asm”,”wt”))==NULL) {
		printf(”can’t new file ttmp.asm”);
		return 0;
	}

	while((ch=fgetc(fp))!=EOF) {
		if(a==ch) {
			i++;
			ch=fgetc(fp);
			if(b==ch) {
				i++;
				ch=fgetc(fp);
				if(c==ch)
					fprintf(f1,”%lXH, ” , i);
			}
		}
		i++;
	}

	fclose(fp);
	fclose(f1);
}

This writes the file offsets at which 0xcd,0xc6,0×29 was found in the ROM image into ttmp.asm. These bytes, interpreted as Z80 machine code, mean “CALL 0×29C6″. In the final ROM image, this sequence appears once, at 0×442B. If you have any idea why they look for this, please post it in the comments.

This is the list of files in their project directory at D:\GAMEBOY:
BANK37.ASM
CLEARKU.ASM
DAMA1.ASM
DAMA2.ASM
END.CPP
ENDEND.ASM
ENDEND.LST
ENDEND1.ASM
FEND1.ASM
FEND2.ASM
FEND3.ASM
FIND.ASM
FIND.CPP
IFCHAENL.ASM
INTWCHA.ASM
TEST.ASM
TTMP.ASM
TTMP.TXT
ZIPUTP.CPP

These filenames also appear in the ROM:
ADDPLAG.ASM
ADDPLAGF.ASM
CH64TBL.ASM
FEND.ASM
G.ASM
H.ASM
INSERTKU.ASM
INTWIN.ASM
KKKKKK.ASM
L.ASM
NOPLAY1.ASM
TAB.ASM
Y.ASM
ZXHPDM.ASM

And here comes the interesting part: There is actually some assembly source in the ROM; here is a small snippet:

JoyPort_1:
                 AND $02 ;LEFT
                 JR  NZ, JoyPort_2
                 CALL LEFTScroll
                 RET
JoyPort_2:
                 AND $04 ;UP
                 JR  NZ, JoyPort_3
                 CALL UPScroll
                 RET
JoyPort_3:
                 AND $08 ;DOWN
                 JR  NZ, JoyPort_4
                 CALL DOWNScroll

Well-documented, it seems. But there is also some assembly code that looks like this:

L_B000_28F7:
                LD A,$7F
                LD BC,$0800
                LD D,A
                LD HL,$9800
L_B000_2900:
                LD A,D
                LD (HLI),A
                DEC BC
                LD  A,B
                OR  A,C
                JR  NZ,L_B000_2900
                RET
L_B000_2914:
                LD  A,(HLI)
                LD  (DE),A
                INC DE
                DEC BC
                LD  A,B
                OR  A,C
                JR  NZ,L_B000_2914
                RET

The label names suggest that this code has been disassembled from existing Z80 machine code. Link’s Awakening DX is a color remake of an older Game Boy game, so it might very well be that they lost the original source, disassembled the old code and used it again for the remake. This could be easily proven by disassembling the original version and looking for this code.

If you want to do game development archeology yourself, you might want to look at titles like “X-Men - Wolverine’s Rage” (MD5: b1729716baaea01d4baa795db31800b0), which contains Windows 9x registry keys and INF files, “Mortal Kombat 4″ (MD5: 7311f937a542baadf113e9115158cde3), in which you can find some small source fragments, “Gift” (MD5: e6a51088c8fea7980649064bd3a9f9ff), which will tell you that the developers had some Game Boy emulators installed on their system, or the “BIT-MANAGERS” games “Spirou” (MD5:5aa012cf540a5267d6adea6659764441, Turbo C, MAP file, source) and “TinTin in Tibet” (Game Boy Color version, MD5: 8150a3978211939d367f48ffcd49f979), which, amongst other things, contains references to Nintendo’s Game Boy Advance (!) SDK (”C:\Cygnus\thumbelf-000512\H-i686-cygwin32\lib\gcc-lib\thumb-elf\2.9-arm-000512″, “/tantor/build/nintendo/arm-000512/i686-cygwin32/src/newlib/libc/stdio/stdio.c”).

If you find any more things like these, please post them (or links to your stories) as comments! Happy hacking!

Skype Reads Your BIOS and Motherboard Serial Number

February 6th, 2007 by myria

Users of Skype that run 64-bit versions of Windows like me probably have noticed that when starting Skype, the following dialog box appears:

The program or feature “\??\C:\Documents and Settings\Myria\Local Settings\Temp\12\1.com” cannot start or run due to incompatibility with 64-bit versions of Windows. Please contact the software vendor to ask if a 64-bit Windows compatible version is available.

Well, that’s weird. Skype’s trying to run a .com file, which won’t work on Win64 because there’s no NTVDM. Let’s try opening it in Hex Workshop. Access denied? OK, I’ll terminate Skype to read it. Still can’t?! This thing is really starting to annoy me. I’ll use WinDbg to terminate winlogon.exe to force a kernel panic. I reboot and NOW I can read the damn file.

An unreadable executable file coming from Skype sounds interesting, so I look at it. It’s 46 bytes long. For copyright reasons I can’t post the file or a complete disassembly. However, I can describe the program in terms of 16-bit DOS C:

int main(void)
{
fwrite((const void far*) 0xF0000000, 1, 0xFFFF, stdout);
fwrite((const void far*) 0xF000FFFF, 1, 1, stdout);
return 0;
}

It’s dumping your system BIOS, which usually includes your motherboard’s serial number, and pipes it to the Skype application. I have no idea what they’re using it for, or whether they send anything to their servers, but I bet whatever they’re doing is no good given their track record.

In 32-bit Windows NT, including Vista, the kernel permits NTVDM to make a read-only mapping of the BIOS at address 000F0000. This allows DOS programs running under NTVDM to make use of the BIOS. That’s how this 46-byte program is capable of sending the BIOS to the Skype application, and also explains why they use this mechanism to begin with.

If they hadn’t been ignorant of Win64’s lack of NTVDM, nobody would’ve noticed this happening.

Switching modes with Style

February 2nd, 2007 by Michael Steil
	pushl $(0xcb<<24)|0x08
	call .-1

What does this instruction sequence do? (This was a collaborative effort by Chuck Gray, Myria and Michael.)

(The solution has been added to the comments.)

How retiring segmentation in AMD64 long mode broke VMware

November 9th, 2006 by Michael Steil

UNIX, Windows NT, and all the operating systems in their class rely on virtual memory, or paging, in order to provide every process on the system a complete address space of its own. An easier way to protect processes from each other is segmentation: The 4 GB address space of a 32 bit CPU is divided into segments (consisting of a physical base address and a limit), one for each process, and every process may only access their own segment. This is what the 286 did.

The 386 then introduced virtual memory, but segmentation was still possible, either instead of, or on top of the paged virtual address space. Today, no modern operating system for the x86 uses segmentation any more, so for every process, the base for the code and data segments is set to 0, and the limit is set to 0xFFFFFFFF.

The AMD64 architecture, while still being fully compatible in 32 bit mode, retired a lot of legacy functionality in the new 64 bit long mode, including most of segmentation. The CS (code), DS (data 1), ES (data 2) and SS (stack) segment registers are practically gone, and the FS and GS segments still support a base (which can be used in tricks to quickly access data at a constant position, like the TCB), but the limit is no longer enforced. Now operating systems don’t have to save and restore most of these segment registers any more when switching contexts, making these switches faster.

But this broke VMware. While VMware could still virtualize 32 bit operating systems on AMD64 CPUs, they could not virtualize 64 bit operating systems, because they required segment limits.

In a nutshell, this is how VMware works: All user mode code of the guest runs in exactly the environment it expects; VMware makes sure the page mappings of the user mode address spaces are correct. All kernel mode code of the guest will be run in user mode, and again, VMware must layout memory as the guest kernel expects it to be. In both modes of operation, there can be exceptions, like system calls (by guest user mode code) or page table modifications (by guest kernel mode code). These have to be trapped by the virtual machine monitor, and the respective functionality has to be carried out in a modified way, so that they still seem to have the correct effect to the guest, but don’t interfere with the host operating system.

The virtual machine monitor’s trap handler must reside in the guest’s address space, because an exception cannot switch address spaces. So VMware’s trap handler sits at the very top of the every guest’s address space, which is unused by all major operating systems. According to Popek and Goldberg’s definition of virtualization, there must be no way for code inside a virtual machine to escape, and modify the host’s state in any way not directly controlled by the monitor. Therefore, it must be made sure that the guest code cannot write to the trap handler code. VMware does this using segment limits: The limits of all segment registers are set to something like 0xFFFFEFFF to protect the uppermost 4 KB of the address space where the trap handler resides.

With no segment limits any more for 64 bit code, this way to protect the trap handler was impossible. Unable to comply with Popek and Goldberg’s security requirement, VMware chose not to support 64 bit virtualization until AMD reintroduced (optional) segment limits on later models of their Opteron and Athlon 64 CPUs. Intel never implemented 64 bit segment limits on their EM64T/Intel64 CPUs, because their 64 bit processors soon implemented VT/Vanderpool, which also worked around the problem. So this is why VMware requires a certain model and stepping of the AMD CPU line or a VT-enabled Intel CPU in order to support 64 bit virtualization.

Now the question is: Why don’t they protect the uppermost page using the permission bits in the page table? This is how all operating systems protect themselves from user mode processes. If you have an answer on this, or otherwise have thoughts, please comment on this post. :-)

References: 1, 2, 3, 4, 5