by Richard F. Drushel (drushel@apk.net)
The last issue of TWWMCA (9709.30) mentioned in general terms some of the advantages to allowing the EOS operating system to move into expansion RAM (XRAM). I invited public comment on the idea; and in response, I received the following query from Bart "Zonker" Lynch:
>rich, could you put your "explanation hat" on and have a go at making this >a bit more clear? i want to make sure i grasp the concept of what you are >proposing before i make any comments. please remember to K.I.S.S! :) >-zonker
I will try to clarify some of the technical issues, implications, and limitations of such an EOS redesign. First some basic facts:
The Z80 microprocessor of the ADAM can access 64K of memory. This memory can be RAM or ROM. Additionally, the ADAM hardware is designed so that memory can be accessed in 32K segments or "banks", which correspond to the lower 32K (addresses 0000H-7FFFH) and the upper 32K (addresses 8000H-FFFFH) of the memory space. Different physical memories, ROM and RAM, are assigned to either the low 32K or upper 32K, and these banks may be swapped in and out, mixed and matched, to make different 64K memory maps. Only one RAM or ROM may be active (i.e., directly visible to the Z80 microprocessor) in each low/high 32K bank at any one time. Here is a summary diagram of the various banks of ROM and RAM and what place they occupy in the 64K address space:
low 32K high 32K
0000H 2000H 6000H 8000H FFFFH
<-------------intrinsic RAM------------><----------intrinsic RAM---------->
<-------------expansion RAM------------><----------expansion RAM---------->
<----------cartridge ROM---------->
<--OS-7 ROM--><-----intrinsic RAM------>
<------------SmartWriter ROM----------->
<----EOS ROM--->
<----------expansion ROM---------->
Memory configuration is selected by writing certain values to dedicated hardware I/O ports; the exact details need not concern us here. The key idea is that you have a memory smorgasbord, and can choose one from column A and one from column B for the current active memory map. Note that when the OS-7 ROM is selected, there is also 24K of RAM included to fill out the low 32K. (This is the ColecoVision memory map.) Also, a note about memory expanders bigger than 64K: they are arranged as parallel 64K banks, with a separate bank select port which must be written to. (This is because XRAM greater than 64K is aftermarket and there was no existing Coleco standard.)
There are *two* default memory maps, depending upon which reset switch you pull to start your ADAM! If you pull the game reset, the memory map is:
low 32K high 32K 0000H 2000H 6000H 8000H FFFFH <--OS-7 ROM--><-----intrinsic RAM------><----------cartridge ROM---------->
If you pull the computer reset switch, however, you get the following:
low 32K high 32K 0000H 2000H 6000H 8000H FFFFH <------------SmartWriter ROM-----------><----------intrinsic RAM---------->
In each case, program execution begins at address 0000H, in whatever RAM or ROM happens to be switched in.
For a game reset, the code looks to see if a game cartridge is plugged in. If the "magic" identifier byte sequences AAH, 55H ("game", uses COLECOVISION title screen) or 55H, AAH ("test", skips the title screen) are found at 8000H- 8001H, a cartridge is present and the game code is executed (using the pointer at 800AH to find the start of game code). If not, the "TURN GAME OFF BEFORE INSERTING CARTRIDGE OR EXPANSION MODULE" message is displayed and the system halts.
For a computer reset, however, the boot code is far more complex (and interesting :-) ). Here's the sequence of events:
Look for an expansion ROM (center slot; a "boot ROM" on one of the Micro Innovations I/O cards; *also* a "language card" French or German version of SmartWriter for European ADAMs). If found, jump to the program in it. There is a "magic" identifier byte sequence at 8000H-8001H of 66H, 99H which specifies a valid XROM; executable code begins at 8002H.
low 32K high 32K 0000H 2000H 6000H 8000H FFFFH <------------SmartWriter ROM-----------><----------expansion ROM---------->
If there is no XROM present, swap in the high 32K RAM bank:
low 32K high 32K 0000H 2000H 6000H 8000H FFFFH <------------SmartWriter ROM-----------><----------intrinsic RAM---------->
Then copy some code from the SmartWriter ROM to high 32K RAM and jump to it. This code then swaps in the EOS ROM and copies it to address E000H in RAM (where EOS usually resides).
low 32K high 32K
0000H 2000H 6000H 8000H FFFFH
<----EOS ROM---><----------intrinsic RAM---------->
The EOS loader code then issues the _EOS_START function call and tries to boot a program from disk or tape. Part of _EOS_START switches to an all-RAM memory map, which is the default environment for user programs:
low 32K high 32K 0000H 2000H 6000H 8000H FFFFH <-------------intrinsic RAM------------><----------intrinsic RAM---------->
If no disk or tape is found, _EOS_START bails out to SmartWriter by issuing the _GOTO_WP function call, which swaps back in the SmartWriter ROM and jumps to the first byte of SmartWriter code at 0100H:
low 32K high 32K 0000H 2000H 6000H 8000H FFFFH <------------SmartWriter ROM-----------><----------intrinsic RAM---------->
SmartWriter itself makes a number of bank switches (including to XRAM), but I will not describe them further at this time.
Bank-switching gives the ADAM its great flexibility. Indeed, that's how the ADAM is able to implement a complete ColecoVision as a subset of its own functionalities--swap in the OS-7 ROM plus 24K RAM, and the cartridge ROM, and run the game.
However, some care must be used. For instance, you'd better not swap out the bank of memory that your code is currently executing in! That is, if the current instruction is executing in the upper 32K of memory, you *cannot* swap out the upper 32K of memory--if you do, your code becomes invisible to the Z80 (you swapped it out), and execution proceeds at whatever the next address is in the new bank of memory. Since this is likely to be garbage, there's no telling what will happen; in any case, you'll have to reboot the computer to get control again. (The only exception is the carefully designed case in which two banks of memory contain exactly the same code in exactly the same place; in this case, while there is a switch from one copy to the other, the program itself stays in phase, and there is no loss of control.)
You must also keep track of what bank of memory the program stack is in. The stack is a special area of read/write memory which is used for temporary storage of data, typically as CPU registers (AF, BC, DE, HL, IX, IY, PC). Any machine code which does PUSHes or POPs is using the stack. The current entry on the stack is pointed to by the SP (Stack Pointer) register. If you swap out the memory bank which contains the stack, you must not try to POP (because there will be garbage at the current SP) or PUSH (because you'll overwrite whatever is in the new bank of RAM, possibly program code). Both errors have disastrous results. Trying to PUSH or otherwise write to memory that is occupied by ROM is futile, because ROM is read-only as far as the Z80 is concerned. If your code needs to use a stack, you may need to allocate a temporary stack in the same bank of RAM you're in, for use only during the bank switch; of course, you have to save the current position of the *old* SP and restore it when you're done bank-switching.
Interrupts are also a problem during bank-switching. If enabled, maskable interrupts cause an automatic jump to address 0038H; non-maskable interrupts, to address 0066H. Both of these locations are in the lower 32K of memory. For a typical RAM-based program which does not do any bank- switching, the programmer provides interrupt handler code which resides at those addresses in RAM (typically a jump to somewhere else). If the program swaps out the lower 32K of RAM, however, and interrupts are still enabled, there is a chance that an interrupt will occur while the RAM is swapped out. If there is not appropriate code at 0038H and/or 0066H in the *new* bank of memory to deal with the interrupt, whatever garbage is present at those addresses will be executed as if it were code, and control will be lost. A trickier case is provided by SmartWriter, which has hard-coded jumps to locations in EOS RAM (upper 32K). If for some reason the SmartWriter ROM is swapped in but a copy of EOS code isn't present in the upper 32K, or some other memory bank (like XROM or cartridge ROM) is swapped in, then the interrupt will cause a jump to garbage code.
Operating system code is like any other program code: it can be stored in RAM or ROM, and at any memory location which is convenient. The convention in both the EOS and CP/M operating systems is to put the operating system at the top of memory, leave the bottom 0000H-00FFH for interrupt vectors, and load user programs at address 0100H, extending upward in memory. Here's a diagram (turned on its side for convenience):
0000H 0100H FFFFH
<---cold start----><-------user---------> ... <-------operating--------->
RST,INT, NMI program system
vectors code code
It is obvious from the above diagram that the maximum possible size of a (contiguous, non-bank-switched or non-overlayed) user program depends upon how big the operating system code is. For example, if the operating system code is 8K, then the user program has less than 56K of space available to it. In the case of EOS-5 (the default ROM version in every ADAM), the operating system occupies over 11K (8K for EOS code, data, and ADAMnet device control blocks, 2K for user file I/O buffers, 1K for system file I/O buffer, and 105 bytes for file control blocks). This leaves less than 53K for user programs.
How does a user program know where the top of its available memory is? This is very important for a program which needs every bit of available space for user-created objects, such as ADAMcalc (spreadsheets), SmartBASIC (programs), or SmartWriter (text files). Some operating systems provide a dedicated function call to find out what the highest available free address is. This lets a user program tailor its memory needs dynamically, based upon how much free space is available. This also allows future revisions of the operating system to get bigger and grow downward in memory, because it can then be expected that every program will call this top-of-memory function and adjust its memory usage accordingly.
Alas, EOS-5 has no such top-of-memory function call. (EOS-8 does, for the reasons described above.) EOS-5 is loaded from ROM to RAM address E000H, and 3K of file I/O buffers and 105 bytes of file control blocks are allocated below that. This is the de facto upper limit for user program memory. Even the EOS-7 rewrite, which has a default of 3 user file buffers (instead of 2), squeezes in the third file buffer and file control block under the old EOS-5 top-of-memory limit. (This was accomplished by wholesale rewriting and optimization for size of the EOS code, freeing up over 1K compared to EOS-5.) Thus, the otherwise pathetic Disk Manager program launching shell (which loaded EOS-7 from disk) could successfully run any existing Coleco software which assumed that it could have all memory up to the EOS-5 top-of-memory point.
Given that all the Coleco software (and all of the SmartBASIC-based applications) is structured around this EOS-5 top-of-memory limit, and that reverse-engineering of binaries to source code for modification is a laborious task, it is thus an inescapable constraint that any future EOS revision which would run existing Coleco software *must* have an intrinsic RAM footprint no bigger than EOS-5. If EOS code and data overflow this limit, they will get clobbered by some application program which thinks it can use the space for itself.
Note carefully that I said "intrinsic RAM footprint", since that's where (with a few exceptions) all the Coleco software runs from--the base 64K of RAM. (Or in the case of SmartWriter, with the upper 32K of RAM, where EOS code is sitting.)
The EOS function calls are accessed via a jump table from FC30H-FD5EH in the upper 32K of intrinsic RAM. Thus, any program which makes EOS function calls must have the upper 32K of intrinsic RAM switched in at the time it makes the calls. When the function is completed, the memory map is the same as it was at the start of the call. What happens in between?
The answer is, *anything* can happen in between. Operating systems are supposed to be black boxes. There are defined entrances and defined exits, but what's inside the building itself is not defined, and indeed can be *anything* necessary to perform the requested functions. You can change the entire internal workings of the operating system, but if the defined entrances and exits are preserved, user programs shouldn't notice any difference. (This "black box" aspect of operating systems is what allows ADAMserve to substitute server disk drives and hard drive for genuine ADAM hardware--as long as _READ_BLOCK takes A=device number, BCDE=block number, HL=transfer address, and returns ZF=1 for okay, ZF=0 and A=error code for error, it doesn't matter where the block of data comes from, or what I have to do to get it.)
By extension, there is nothing that says that I have to stay with the current (entry) memory map while I'm performing an EOS function call. I should be able to switch to any memory configuration I want (subject to the Perils of Bank-Switching described above), as long as I put things back the way they were before I exit.
Thus, I should be able to put some of my operating system code into another memory space (e.g., XRAM), and with proper design and care, be able to transparently bank-switch in and out of that memory space, returning with the system memory map the same as it was at the start. I could fill an entire 64K of XRAM with EOS code, keeping only the jump table and the code necessary for bank-switching in intrinsic RAM. This way, user programs can keep their existing memory maps (i.e., they don't have to be reverse-engineered to source and patched to become dynamically-allocating), and EOS code can effectively be as big as it needs to be.
If some of EOS code is made resident in XRAM, then XRAM becomes an important shared resource. User programs are no longer free to appropriate XRAM for themselves without regard to what might be stored there already, and in use by the operating system. If the user program overwrites the EOS code or data, then the system is corrupted and will crash if the garbage code is executed, or garbage data is used.
How can you tell if XRAM is present? As noted above, cartridge ROM and XROM have "magic" identifier bytes at reserved locations which signify that a valid ROM is present. At system reset, XRAM is wiped and contains data of undefined value, so there are no "magic" identifiers. However, the presence or absence of XRAM can be determined experimentally, by simply writing some data to the XRAM memory space, then trying to read it back. If XRAM is present, you will be able to read back what you wrote; if XRAM is absent, you'll get random data when you try to read it back. Since the data are random, it's best to write to more than one memory location (on the odd chance that random noise might give you the byte you wrote), and with a definite pattern of bits in the bytes. Typically, the "magic" bytes AAH and 55H are used, because AAH = 10101010 binary and 55H = 01010101 binary (an alternating pattern of set and clear bits).
XRAM greater than 64K can also be detected using this method. After selecting an XRAM memory configuration, each parallel bank of 64K can be selected (using the appropriate bank select port) and tested. Typically, to avoid having to exhaustively test every possible bank (a 2 MB expander has 32 parallel banks of 64K each), advantage is taken of the fact that memory expanders exist only in a few sizes (64K, 128K, 256K, 512K, 1 MB, and 2 MB), and only the "critical" bank for each size need be tested to determine its presence. Starting from the bottom, look at banks 0, 1, 3, 7, 15, and 31; the highest one detected determines the size of the XRAM.
Note that the XRAM detect method described (i.e, writing "magic" data and trying to read it back) is *destructive*--if XRAM is present, whatever data was already there at the test addresses gets wiped out. It is a simple matter to make detection non-destructive, however, by just reading the test addresses first, then doing the test write/read, and finally writing back the original contents if the test was successful. Unfortunately, none of the XRAM detection routines I have seen in the software that I have disassembled (SmartWriter, ADAMcalc, SmartBASIC 2.0, File Manager, PowerPaint, Walters RAMdisk) use non-destructive XRAM testing. Sad to say, I didn't either in the special HARDDISK 2.3/3.9 with RAMdisk that I created in 1991 for Herman Mason and George Koczwara; and I don't think I altered Tony Morehen's code for the XRAM detect in ADAMserve File Manager, either (though the necessary changes are trivial given that I have the source code). For shame!
One thing that I *did* do, however, in the HARDDISK 2.3/3.9 with RAMdisk was make sure that my XRAM-resident RAMdisk code sat in places other than where the XRAM detect routine of File Manager was making its (destructive) test writes. The need for this was experimentally-determined, after I had laid out my XRAM memory map as seemed most straightforward to me, implemented the RAMdisk driver, and found it got clobbered by File Manager. I don't remember if I had any problems with SmartWriter or ADAMcalc; and the hard drive version of PowerPaint had been patched by me anyway to use the hard drive instead of XRAM, so the latter wasn't a problem.
In any case, I hope you clearly see the difficulty in trying to create a shared resource when there are existing programs which use it with abandon. The choices are to alter the programs (usually very hard) or avoid using part of the shared resource that the programs want to use for their own (ultimately very limiting).
There is a final "easy" way out for me as programmer, but not for you as user: Ignore the first 64K of XRAM completely, and put EOS into the second 64K of XRAM, utilizing the leftovers (and any higher banks of XRAM) for a RAMdisk. This avoids the Coleco software problem because the Coleco software doesn't know about any XRAM above the first 64K. This is a disingenous choice, however, because it requires at least 128K of XRAM. 64K expanders are common enough that I can assume everyone left as an active ADAMite either has one or can get one; not so for the larger expanders. It is easier to try to get people to have their SmartWriter ROMs upgraded than to expect them to get 128K and greater memory expanders.
Looking back to Zonker's injunction/plea to "K.I.S.S.", I'm not sure if he will perceive this article as "simple". I have tried to be more explicit in my reasoning, but there is some necessary complexity here, which I don't think has been treated as a coherent article within recent memory. (I'm sure there's some back issue of NIAD or the MTAG newsletter or Nibbles 'n' Bits somewhere which also covers this ground, maybe better or clearer than I have done here.) Putting my "K.I.S.S." hat on very tightly, here's a go at the TV Guide blurb for the last couple of issues of TWWMCA:
"I want to put more functionality into EOS. I can't fit it all into 8K of EOS RAM, and I *must* to avoid breaking existing programs. Technically, I can move some of EOS into XRAM, but there are potential conflicts with interrupts and other programs which already use XRAM. I can either make EOS work around the conflicts, or eliminate them by patching the programs. Both are hard problems. Since everybody uses SmartWriter at some point (if only because of an abort), and SmartWriter uses XRAM, SmartWriter must be patched in ROM to *not* use XRAM, if an XRAM- based EOS is to be stable."
As always, comments and questions are welcome.
Ron Mitchell's Ron'sWeek'n'ADAM for 5 October 1997 raises a number of interesting points about program design, style, and maintainability. I will devote next week's TWWMCA to a Point-Counterpoint discussion of his article.
See you next week!
*Rich*
Next Article
Previous Article
TWWMCA Archive Main Page