June « 2009 « pagetable.com

Archive for June, 2009

Apple Copland Reference Documentation

Tuesday, June 30th, 2009

The Copland project was Apple’s ill-fated attempt in the mid 1990s to replace the aging classic Mac OS with a more modern operating system that had a microkernel, virtual memory and preemptive multitasking. Information on Copland is scarce, therefore I have compiled 20 hard to find Copland reference documents, as well as the 359 page book “Mac OS 8 Revealed”.

Note that Copland was supposed to be the next major OS release after System 7, so the while the first two beta releases D7E1 and D9 were called “Copland”, the final beta D11E4 was called “Mac OS 8″ everywhere. After the cancellation of the Copland project, Apple reused the term “Mac OS 8″ for a System 7 update.

Copland D9 (Copland Developer Release – Tools Edition)

November 1995

An Introduction to Copland. The Mac OS Foundation for the Next Generation of Personal Computers
Glossary of Copland Terminology. This Acrobat file contains the document âGlossary of Copland Terminology,â which defines many of the terms used throughout Copland developer documentation. Note, however, that the terms defined in this glossary are preliminary and subject to change.
The Copland Toolbox. This Acrobat file contains the two-chapter book The Copland Toolbox. The first chapter introduces the event-handling model and human interface elements supported by the Copland Toolbox, and it describes the programming concepts necessary to use the Toolbox in applications. This will help you plan frameworks and tools that take full advantage of Coplandâs Toolbox features. The second chapter compares the System 7 Toolbox with the equivalent Copland managers and capabilities, and this chapter includes information on backward compatibility with System 7 applications and requirements for Copland-savvy applications. This is especially useful for helping you adapt your existing System 7-based frameworks to work in Copland.
Software Extensibility. This Acrobat file contains the document âSoftware Extensibility and the System Object Model (SOM),â which describes where and why Copland uses SOM classes, dynamically linked libraries, and other mechanisms to support extension and modification of system software, and how you can use these to add extensibility to your own software.
Intro to Kernel and OS Services. This Acrobat file contains the document âIntroduction to Kernel and Operating System Services,â which provides conceptual information about how and when to use services provided by the microkernel and related portions of the Copland operating system. This document will help you plan frameworks and tools that take full advantage of Coplandâs operating-system features.
Microkernel White Paper. Containing the document âMicrokernel White Paper,â this file gives a conceptual overview of the Copland microkernel. This document discusses tasks, hardware and software interrupts, and privileged software execution; it describes address space management, including such topics as areas and memory reservations; and it describes messaging, such as how to use message objects, how to send and receive them, and how reply or cancel them. This version of the âMicrokernel White Paperâ differs only slightly from that delivered at the 1995 Worldwide Developerâs Conference.
Patch Manager. The document âPatch Managerâ contained in this Acrobat files describes the new Copland mechanism for patching the Mac OS. This will help you plan how to create patch-type system extensions for your development environment and to produce code that creates or uses extensions that patch Copland. The Patch Manager defines an application programming interface (API) that enables software to patch the system in a consistent, more easily supported manner. This is the only documentation provided for this release of Copland that describes an application programming interface. Note, however, that as with the rest of this release, all of the Patch Managerâs programming interfaces are subject to change.
I/O Architecture. This Acrobat file contains the document âAbout the Copland I/O Architecture,â which provides an overview of the architecture and features of Coplandâs new I/O system. This document will help you with any driver development that you might be planning for Copland, and will help you design or adapt software that writes to or otherwise directly manipulates devices. This document is identical to the chapter supplied in the Copland Technical Overview delivered at the 1995 Worldwide Developerâs Conference.
About the Copland file System. This Acrobat file contains the document âAbout the Copland file System,â which introduces the features of the new file system available with Copland. It describes the various components of the file system, including the file Manager, the Navigation Services API, and the file systems API. This document also describes how to get your program ready to use the new file Manager. finally, the Documentation for MPW folder contains information describing the Macintosh Programmerâs Workshop.
Copland Desktop. The Copland Desktop Picture is a very high quality version of the CD cover art. To use this as your desktop picture, place it on your Copland Drive. While running Copland, you can open the picture from the Appearance Control Panel.

Copland D11E4 (Mac OS 8 DDK 0.4)

June 1996 (WWDC)

This Inside Macintosh for Mac OS 8 Read Me provides a road map to the chapters within this folder, provides information on viewing the Acrobat files contained within this folder, and lists the methods you can use to give us your feedback on the Inside Macintosh documentation.
For an overview of new features supported by the Toolbox, such as human interface objects, panels, window groups, imaging objects, and themes, see the chapter “Introduction to the Mac OS 8 Toolbox” in Human Interface Toolbox. For reference information on human interface objects, see the chapter “HIObject Class Reference” in Human Interface Toolbox.
For an overview of how Apple events are used pervasively throughout Mac OS 8 by system services as well as other programs, see the chapters “Introduction to the Mac OS 8 Event Model” and “Event Model Reference” in Apple Events in Mac OS 8. For information on how Apple events are processed by the Toolbox, see “Toolbox Event Routing” and “Toolbox Events Reference” in Human Interface Toolbox.
For information on the Mac OS 8 File Manager, Navigation Services, and other file services, as well as reference information on the Mac OS 8 File Manager, see File Navigation and Access.
For information on the extensive support for text handling that enables you to develop your application for a worldwide market, such as support for Unicode, locales, and the use of text objects, see “Introduction to Text Handling and Internationalization” in Text Handling and Internationalization.
For information on the use of the System Object Model (SOM) in Mac OS 8, see Software Extensibility.
For information on creating resources that are new in Mac OS 8, see ResEdit 3.0 User’s Guide.
For an overview of the Mac OS 8 microkernel, including information on tasks, processes, scheduling, preemption, multithreading, allocation of memory, and other services, see the chapter “About Mac OS 8″ in Microkernel and Core System Services. See subsequent chapters in Microkernel and Core System Services for reference information on many of these topics.
For information on the Mac OS 8 I/O architecture, including information on families, plug-ins, and I/O services, see “About the I/O Architecture” in Modular I/O. For information on families for specific hardware devices, see subsequent chapters in Modular I/O.
For updated information on Open Transport, see Open Transport.
Please note that all presentations in this folder were created using Aldus Persuasion; an application we have not included on the DDK. (Displays Presentation, Keyboard Family Presentation, Mac OS 8 I/O Overview, OT General Presentation, OT Serial Presentation)

Tony Francis: Mac OS 8 Revealed

July 1996

With the next release of the Macintosh operating system, Apple will provide a state-of=the-art platform for developers to create new software products. At the core of Mac as 8 is a redesigned operating system foundation based on microkernel technology that will dramatically increase user productivity.

Author Tony Francis worked closely with the Mac as 8 engineering team to provide the inside story of the design and development of this innovative technology. The book describes the technical geography of Mac as 8 and illustrates how the system’s user benefits can be implemented in software and hardware products.

Written for developers, system administrators, and information systems professionals, Mac OS 8 Revealed explains the key technologies of Mac as 8:

How Mac as 8 exploits preemptive multitasking to perform many operations concurrently;
How the new memory protection model insulates the microkernel and other operating system services to provide system stability;
How to provide automatic, intelligent assistance to users;
How to allow the user to customize and scale the human interface.

The accompanying CD-ROM contains an electronic version of the book with animated illustrations that demonstrate the capabilities of Mac as 8. Simply click on a screen shot marked with a film strip in the book and see how a new feature in the operating system works.

Tony Francis has been with Apple for over 10 years as a lead writer on the Inside Macintosh team.

ISBN 0-201-47955-9

I publish these files for educational purposes, especially for all computer archeology enthusiasts like me. This data is copyrighted, but since the project has been canceled ~13 years ago, nobody should still care about it. If you disagree, contact me. Thanks to all the people who have worked on Copland and written these interesting documents!

Posted in archeology, literature | 16 Comments »

Readable and Maintainable Bitfields in C

Tuesday, June 23rd, 2009

Bitfields are very common in low level C programming. You can use them for efficient storage of a data structure with lots of flags, or to pass a set of flags between functions. Let us look at the different ways of doing this.

The straightforward way to deal with bitfields is to do the Boolean logic by hand:

Boolean Magic

#define FLAG_USER   (1 << 0)
#define FLAG_ZERO   (1 << 1)
#define FLAG_FORCE  (1 << 2)
/* bits 3-30 reserved */
#define FLAG_COMPAT (1 << 31)

int
create_object(int flags)
{
        int is_compat = (flags & FLAG_COMPAT);

        if (is_compat)
                flags &= ~FLAGS_FORCE;

        if (flags & FLAG_FORCE) {
                [...]
        }
        [...]
}

int
create_object_zero(int flags)
{
	create_object(flags | FLAGS_ZERO);
}

void
caller()
{
        create_object(FLAG_FORCE | FLAG_COMPAT);
}

You can see code like this everywhere, so most C programmers can probably read and understand this quite easily. But unfortunately, this method is very error-prone. Mixing up "&" and "&&" and omitting the "~" when doing "&=" are common oversights, and since the compiler only sees "int" types, this also doesn't give you any kind of type-safety.

Bitfields

Let us look at the same code using bitfields instead:

typedef unsigned int boolean_t;
#define FALSE 0
#define TRUE !FALSE
typedef union {
        struct {
                boolean_t user:1;
                boolean_t zero:1;
                boolean_t force:1;
                int :28;                /* unused */
                boolean_t compat:1;     /* bit 31 */
        };
        int raw;
} flags_t;

int
create_object(flags_t flags)
{
        boolean_t is_compat = flags.compat;

        if (is_compat)
                flags.force = FALSE;

        if (flags.force) {
                [...]
        }
        [...]
}

int
create_object_zero(flags_t flags)
{
	flags.zero = TRUE;
	create_object(flags);
}

void
caller()
{
        create_object((flags_t) { { .force = TRUE, .compat = TRUE } });
}

Flags can just be used like any variables. The compiler abstracts the Boolean logic away. The only downside is that the code with the static initializer requires some advanced syntax.

Endianness

Bitfields in C always start at bit 0. While this is the least significant bit (LSB) on Little Endian (bit 0 has a weight of 2^0), most compilers on Big Endian systems inconveniently consider the most significant bit (MSB) bit 0 (bit 0 has a weight of 2^31, 2^63 etc. depending on the word size), so in case your bitfield needs to be binary-compatible across machines with different endianness, you will need to define two versions of the bitfield.

The Raw Bitfield

Did you notice the "int raw" in the union? It lets us conveniently (and type-safely) export the raw bit value without having to use a cast.

	printf("raw flags: 0x%x\n", flags.raw);

We can use this to reconstruct the FLAG_* constants in the original example, in case some code needs it:

#define FLAG_USER   (((flags_t) { { .user   = TRUE } }).raw)
#define FLAG_ZERO   (((flags_t) { { .zero   = TRUE } }).raw)
#define FLAG_FORCE  (((flags_t) { { .force  = TRUE } }).raw)
#define FLAG_COMPAT (((flags_t) { { .compat = TRUE } }).raw)

This code constructs a temporary instance of the bitfield, sets one bit, and converts it into a raw integer - all at compile time.

Bitfield Access from Assembly

With the same trick, you can also access your bitfield from assembly, for example if the bitfield is part of the Thread Control Block in your operating system kernel, and the low level context switch code needs to access one of the bits. The "int raw" can be used to statically convert a flag into the corresponding raw mask:

typedef unsigned int boolean_t;

typedef union {
	struct {
		boolean_t bit0:1;
		boolean_t bit1:1;
		int :19;
		boolean_t bit31:1;
	};
	int raw;
} bitfield_t;

int test()
{
	int param = -1;
	int result;

	__asm__ volatile (
		"test    %2, %1    \n"
		"xor     %0, %0    \n"
		"setcc   %0        \n"
		: "=r" (result)
		: "r" (param),
		  "i" (((bitfield_t) { { .bit31 = TRUE } }).raw)
	);
	return result;
}

The corresponding x86 assembly code looks like this:

	.text
	.align	4,0x90
	.globl	_test
_test:
	pushl	%ebp
	movl	%esp, %ebp
	movl	$-1, %eax
	## InlineAsm Start
	test	$0x80000000, %eax
	xor	%eax, %eax
	setcc	%eax
	## InlineAsm End
	popl	%ebp
	ret

	.subsections_via_symbols

This works fine with LLVM, but unfortunately GCC (4.2.1) has problems detecting that the raw value is available at compile time, so the "i" has to be replaced with an "r": GCC will then pre-assign a register with the raw value instead of being able to use an immediate with the "test" instruction.

How to Not Do It

I have seen C++ code doing this:

enum {
	FLAG_USER,
	FLAG_ZERO,
	FLAG_FORCE,
	FLAG_COMPAT = 31
}

int
create_object(bitfield_t flags)
{
        bool is_compat = flags.is_set(FLAG_COMPAT);

        if (is_compat)
                flags -= FLAGS_FORCE;

        if (flags.is_set(FLAG_FORCE)) {
                [...]
        }
        [...]
}

int
create_object_zero(int flags)
{
	create_object(flags + FLAGS_ZERO);
}

void
caller()
{
        create_object(((bitfield_t)FLAG_FORCE) + FLAG_COMPAT);
}

This all looks quite weird. The constants are bit index values, and they are added and subtracted. The reason is C++ operator overloading:

class bitmask_t
{
    word_t      maskvalue;

public:
    [...]
    inline bitmask_t operator -= (int n)
        {
            maskvalue = maskvalue & ~(1UL << n);
            return (bitmask_t) maskvalue;
        }
    [...]
}

This is horrible. The code that uses this class makes no sense unless you read and understand the implementation of the class. And you have to be very careful: While it is possible to "add" a flag to an existing bitfield, you cannot just add two flags - it would do the arithmetic and add the two values.

Mapping the setting and clearing of bits onto the addition and subtraction operators is clearly wrong in the first place: Flags in a bitfield are equivalent to elements in a set. Setting a flag is equivalent to the "union" operation, which even in Mathematics has its own symbol instead of overloading the "+" operator.

Question

If you compile code that does something like "((bitfield_t) { { .bit31 = TRUE } }).raw" with GCC in C++ mode, it fails. Why?

Posted in puzzle, tricks | 32 Comments »

A Lot of Security

Tuesday, June 16th, 2009

I happened to drive through Cupertino, CA, USA last Wednesday and ended up in this situation:

Oh-oh, they got me. But they were not after me, they escorted two vans onto some company’s campus.

Five police cars, two police motorcycles, and lots of people with suits and sunglasses. For some reason, this outfit doesn’t have the same effect on me any more since “The Matrix”.

The people in the vans went into the building through a side door:

Here are some details:

A blonde woman in white, a woman with a red dress, a man in a brown uniform with a suitcase, and lots of more men in suits. The vans had license places from Maryland.

The question of today’s security puzzle is: Who is the very important person?

Posted in puzzle, security | 8 Comments »

Stamp Vending Machine DoS

Tuesday, June 9th, 2009

It makes sense for a stamp vending machine to have some limits and not print any amount of stamps or stamps of any value. The vending machines from the Deutsche Post are a little weird though:

You can only buy stamps worth 100 EUR at a time. Makes sense.

The maximum face value for a stamp is 36.75 EUR. Hmmm…

You can buy up to 100 stamps at a time, and the minimum face value is 1 cent. The poor machine will print worthless stamps for five minutes. The obvious question now is: What is the minimum amount of money that I have to invest to make it run out of paper.