Creating Z80 dynamic shared libraries

Published on 6 July 2023 by Andrew Owen (4 minutes)

Anyone who has used the Windows operating system for any length of time has probably encountered the phrase “DLL hell”, even if they haven’t directly experienced it. The concept of shared libraries is common to many operating systems. But what made the original version of Windows different was that the entire operating system was composed of Dynamic-Link Libraries (DLLs) running on top of the disk operating system (MS-DOS). Typically, problems occur with incompatibilities between versions of DLLs.

Traditionally on microprocessors like the Z80, where code is compiled at a fixed address, libraries would be statically linked at compile time. Z80 code is not relocatable, so a linker would be used to create a list of addresses in the binary to be patched when building the binary executable. But dynamic libraries can be loaded at any address in memory space at load time or run time. There are obvious benefits to doing this in a multi-user multitasking operating system, where processes can share libraries and reduce the overall memory overhead. Dynamic libraries can be loaded and unloaded as needed.

In fact, there is a multitasking operating system for the Z80 series of microprocessors called SymbOS that does exactly this. Code is compiled at address $0000 and a patch file is provided to modify the hard coded memory addresses. However, the patch file contains only the most significant byte of the addresses, which means code must be loaded on a page boundary (256 bytes).

Having reverse engineered a disk operating system (UnoDOS 3) and created a BASIC interpreter (SE Basic IV) for the Chloe 280SE project, I’m now working on a higher-level operating system (SE/OS). This includes a set of APIs, the optional set of disk based resources (such as the online help) and so on. The whole firmware has to fit in less than 24 kilobytes of memory at run time. This means that some of the functions that I’d like to include in the API are simply too large to implement.

The main source of inspiration for the APIs has been the suite of tech demos that I developed to go along with the firmware. There are about a dozen of them now, and they have a lot of repeated code. Wherever possible, I’ve been replacing this code with API calls. The one piece of code that almost all of them use that can’t be reduced to an API call is the music tracker player. This seems like a prime candidate for a library.

Unlike most modern operating systems, SE/OS is monolithic, single-user and single-tasking. The aim is to keep things relatively simple. However, when you have a 64 kilobyte address space, there is still an advantage to being able to use dynamic libraries that can be loaded and unloaded without having to page memory. The question then becomes: what format should the file take. I looked at RIFF as a possibility, but eventually decided to go for a simpler solution as proposed by Chris Smith.

The file should have a header, consisting of a byte identifier, a version number, the start address and length of the binary, and the start address and length of the patch data. By explicitly defining these properties, it becomes possible to extend the header format in future without breaking compatibility. Because the patch data occurs after the binary, the binary can be loaded, and the patch data can be parsed as it is read in from the file without having to put the entire contents in a buffer. Anything after the patch data will be ignored. So an optional info block of plain text can be included.

The binary should be compiled at $0000 and a set of patch addresses generated. The SE/OS API should provide an API method to load the DLL, providing a start address. If the DLL needs to be given a data address, this should be handled by the DLL itself. The API call should return the length of the binary. Now all I have to do is persuade my friendly assembler developer to implement support for this format.

But then I got to thinking that if I’m going to include files with headers in SE/OS, I should use a uniform approach. And the most logical one seems to be Resource Interchange File Format (RIFF). This is a wrapper format devised by IBM and Microsoft in 1991. It’s essentially a little-endian version of the Interchange File Format (IFF) created by Electronic Arts for the Amiga in 1985. And that was inspired by the four ASCII character OSTypes used in the original Macintosh operating system from 1984.

RIFF has been used for everything from AVI files to WAV files, and most recently was used by Google for the WebP image format in 2010. There are some tools available for working with RIFF, and as a rule I prefer to avoid reinventing the wheel wherever possible.

Image: Original by Jonathan Borba.