by Richard F. Drushel (drushel@apk.net)
Since ADAMcon 0Ch last month, much of my ADAM time has been spent in disassembly and study of the SmartFiler database program, looking for the code responsible for its crashing under ADAMserve, my serially-linked device driver system to let ADAMs use PC hard drives and floppy drives. Before I had gotten very far with the disassembly, I had a pretty good idea what the problem was: SmartFiler was doing disk I/O at a level lower than ADAMserve EOS was emulating it. However, rather than immediately moving to test that hypothesis, I instead concentrated on getting the disassembly completely resolved, so that it could be edited and reassembled at will. At the last ADAM chat held at 9:00 PM EDT on Wednesday, 16 August 2000 (see http://www.adamcon.org/ for details on how to join this chat), Guy Bona's patient inquiries prompted me to try to solve his immediate problem (i.e., the crashes under ADAMserve) first, and leave the finer points of disassembly resolution for later.
Because I had already done a global search/replace for EOS global symbols in the SmartFiler disassembly listing, it was easy to find the probable offending code: a single call to _START_RD_1_BLOCK. For those not familiar with ADAM's I/O functions, let me just state that, because the original tape drives were so slow, especially to rewind, the operating system was designed to allow background tape I/O. Specifically, you could begin a read/write with _START_RD_1_BLOCK or _START_WR_1_BLOCK, then go off and do something else (like playing one level of a Super Game), letting the tape spin as needed, but without the user sitting at a static screen. For every _START_xxx function, there is an _END_xxx function, which can be polled at a low frequency to complete the transfer and reset ADAMnet for the next transfer. Anyway, ADAMserve (for reasons of code space, not for lack of my own cleverness) does not emulate ADAMnet devices as low as the _START_xxx or _END_xxx levels; it redirects only high levels like _READ_BLOCK and _WRITE_BLOCK to the server. Thus, any software which uses low-level I/O functions will not work correctly under ADAMserve. Fortunately, most ADAM software does not use the low-level I/O functions, and thus works fine under ADAMserve.
The fix was to "remove" the offending call by patching it out with NOP (No Operation) instructions, 00h, in the SmartFiler binary. I used File Manager to do this; but any block editing tool will work. I posted the specific instructions for doing this on an ADAMserve system to the Coleco ADAM mailing list; I will repeat them here.
run ADAMserve File Manager
Edit Catalog to find the entry for SMARTFILER
on my system, it starts at block 552; yours might vary
go to Media Functions/Edit Blocks
select the *2nd* block of SMARTFILER (on my system, it's 553)
go to sector 7 (by hitting HOME+down arrow to page through the sectors)
at byte offset 1004 decimal, you should see the following 3 bytes in hex: CD A2 FC
change these 3 bytes to: 00 00 00
hit return and save the changes
Shift-Undo to return to HARDDISK
boot SmartFiler as usual from hard disk
After this patch, SmartFiler did indeed work correctly under ADAMserve: it did not hang when trying to update an existing database.
Two additional verifications of this patch were performed:
an actual 160K floppy of SmartFiler was patched and then booted from disk (no ADAMserve). It worked fine.
an image file was made from this patched disk and then booted under Marcel de Kogel's ADAMEM emulator. It worked fine.
So, the original _START_RD_1_BLOCK was not needed...why? It might have been important; why not change it to _READ_BLOCK? Normally, this latter change would have been the correct one. However, from reading the context of the original code, this _START_RD_1_BLOCK call was merely being used as a "tape pre-wind" command, to get the tape drive spinning. All data is actually transferred with _READ_BLOCK and _WRITE_BLOCK. So, in this case, changing the _START_RD_1_BLOCK to _READ_BLOCK would have just duplicated all block reads. Omitting it actually was the correct thing to do.
You can experiment with these changes yourself on your own *backup* copy of SmartFiler.
The patched 160K floppy disk created above still does not function as a boot disk under ADAMserve, if you try to boot from it using the Boot Software option under HARDDISK. It hangs ADAMserve...and in the PC status window, you see infinite attempts to read block 2. Why?
Disassembly of the boot block shows that it, too, uses _START_RD_1_BLOCK to load the program, assumed to occupy specific blocks on the disk! There is a directory which shows the location and size of all the needed files, but the boot code never reads the directory: it assumes that LOGOS is a 3-block file at blocks 2-4, and that SMART_FILER is a 30-block file at blocks 5-34! Treating a program as a sequence of blocks is fine for Super Games (that's how they are designed), but not a good idea for general user applications.
Why was it done this way? At the end of the boot block is the following "orphaned" message string:
DEFB " "
DEFB "BY SIERRA ON-LINE "
DEFB 3EH,41H,5DH,55H,59H,55H,41H,3EH
DEFB "@ PROGRAMMED BY DON MCGLAUFLIN@@"
DEFB "@ ",1DH,"1984 SIERRA ON-LINE, INC.@"
DEFB "@ PLEASE WAIT WHILE LOADING GAME@@"
Since it was written by a games company, it's not surprising that they wrote user applications in the same way; you do things they way you've always done them. (BTW, the hex codes in line 3 probably formed some graphic logo for Sierra On-Line; the 1DH is a (c) copyright symbol.)
If the boot loader is rewritten to use _READ_BLOCK instead of _START_RD_1_BLOCK (there's a little more to it than that, as the loader routine also peeks at ADAMnet device control blocks to determine when it's time to load the next block), then it boots correctly under ADAMserve *from the 160K disk*. (I did this and tested it. It also works fine under ADAMEM. But see an additional clarification below.) This boot loader doesn't work from an ADAMserve hard drive because the necessary files aren't stored at the absolute blocks that the loader is looking for. In fact, the hard drive (and ADAMserve) version of SmartFiler is just the 30K SMART_FILER file; there are no boot graphics.
The boot loader could be rewritten to access the files by file through the disk directory, instead of by hard-coded block. However, the SmartFiler application, once loaded and running, creates and maintains databases through absolute blocks! It assumes that there is one database per disk/tape, and that it takes the entire disk/tape. Furthermore, it copies some data stored within the SMART_FILER program itself from the boot drive to the database disk/tape *as absolute blocks*. Thus, the current hard drive (and ADAMserve) versions of SmartFiler are *incapable* of creating a new database; they can only read/write an existing SmartFiler database. Correcting this serious defect will require a detailed understanding of the program code, and a completely resolved disassembly; and it may not even be possible, if the ability to read/write old-format databases is to be retained.
In addition to the _START_RD_1_BLOCK calls in the program loader, there was one more which was rather puzzling. Here is the relevant section of the boot code, with necessary equates:
BLOCK_1_ADDR EQU 0000H
PROG_START EQU 0100H
ZERO EQU 0000H
BLOCK_1 EQU 0001H ;directory block
BLOCK_5 EQU 0005H ;SMART_FILER start block
;Load the rest of SmartFiler, assuming a certain start block and length.
LD HL,PROG_START ;load address for SMART_FILER
LD DE,BLOCK_5 ;start block of SMART_FILER
LD B,1EH ;length of SMART_FILER
CALL LOAD_BLOCKS ;load SMART_FILER
;
;This reads the directory block.
;How nice that it assumes it is only one block long...
LD A,(BOOT_DEVICE) ;drive to read
LD HL,BLOCK_1_ADDR ;load address
LD DE,BLOCK_1 ;block loword
LD BC,ZERO ;block hiword
CALL _START_RD_1_BLOCK ;read it
Why does it do this? It will overwrite the first 768 bytes of SMART_FILER! This is fatal, because the last executable statement in the boot loader is
JP PROG_START ;and begin SmartFiler
However, the program clearly works as written. What gives?
The answer is truly evil. A quirk of ADAMnet is that it takes *2* reads of a block device to actually cause the data to be transferred into Z80 RAM. The first call makes the device read the data, but it doesn't get into Z80 RAM until the second consecutive call. (Look at the code for _READ_BLOCK in the EOS-5 commented listing and you will see the 2 calls to _READ_1_BLOCK.) So, the *effect* of a single call to _START_RD_1_BLOCK for block 1 is to just rewind the tape (if you're booting from data pack)--a major time savings for later.
This is a major trap for ADAM emulators (or programs like ADAMserve, which emulate ADAMnet devices). If this read-twice behavior is not correctly emulated, programs which depend upon this behavior will malfunction. Thus, in my rewritten SmartFiler boot block, I had to remove the entire read of block 1. (Initially, I changed it to _READ_BLOCK, because I had not grasped the purpose of the code; and it crashed spectacularly.)
My commented disassembly source of the original SmartFiler boot block is available:
The ADAMEM emulator uses disk and tape image files in place of actual media, so reading from an emulated disk or tape means using MS-DOS function calls to read from a file. This is very fast...so fast that you just see a blur of scrambled screen graphics when you boot an image of SmartFiler under ADAMEM.
I didn't like this, so I made a final modification to the new SmartFiler boot block that I made. I inserted a software delay after each boot screen was displayed. The delay is a simple counter, whose length can be varied by an input parameter. Here's the code for the programmable delay:
;Pause routine. Delay is about 5 seconds with B=10.
WAIT:
LD B,10 ;change to vary the total delay
WAIT1:
PUSH BC ;save outer loop counter
LD HL,0 ;initialize inner loop counter
WAIT2:
DEC HL ;one less
LD A,H
OR L ;is HL=0?
JR NZ,WAIT2 ;no, keep going
POP BC ;yes, so restore outer loop counter
DJNZ WAIT1 ;keep going until B=0
;
RET ;all done
With the delays in place, you can see the two boot screens very nicely under ADAMEM. What's more, the added delay is not objectionable if you boot from a real ADAM, or from an external floppy under ADAMserve. You might notice it if booting from tape, but I haven't tried this.
The commented source of the modified SmartFiler boot block, and a binary, are available:
This modified boot block also has read error detection (which the original lacks), so if there is an error reading any of the blocks, the program will abort to SmartWriter (instead of hanging forever).
I might hope that all the "tricky" bits of the SmartFiler boot block were thoroughly documented in the original Sierra On-Line source code (now probably lost forever); but I wouldn't be surprised it it weren't. Which is too bad, because the code is poorly designed enough that it's not very easy to figure out without such comments. And what I have looked at of the SMART_FILER program per se also shows lots of poor, unobvious design. I have disassembled many other Coleco applications (EOS, SmartBASIC, SmartWriter, ADAMcalc, ADAMlink IVa), games (Pitfall), and other 3rd-party ADAM software (PowerPaint, File Manager, HARDDISK). SmartFiler has been one of the hardest to decipher.
The SmartFiler binary appears to be a debugger dump of RAM, with several modules loaded via serial port from a host computer as Intel HEX records. All the data areas contain non-SmartFiler program code, some of which is a debugger, based upon the text strings. Moreover, the debugger code has 2651 UART I/O routines, like the Coleco Graphics Processor cartridge, which could also import code and graphics in Intel HEX format. Many assemblers which generate output as Intel HEX do not initialize data areas with any starting value; instead, they skip over the data areas, so when the individual records are interpreted and loaded, RAM which corresponds to data areas gets left with whatever was in it previously. There are 2 such large areas of non- SmartFiler program code in the SmartFiler binary.
Ideally, code fixes made in a debugger are propagated back to the source code, and a new binary made from re-assembly of the corrected source. I hypothesize that this did not happen with SmartFiler. The version date (8/16/1984) is very late in the history of the ADAM, well after Coleco had abandoned it; so perhaps the final version was produced under sub-optimal conditions, or even by someone who did not have access to the complete source code, perhaps not the original programmer(s).
Be that as it may, it is still very frustrating to have to understand and maintain a program which is not clear and straightforward in its design and implementation, or which exploits unusual or undocumented side effects, or (my biggest pet peeve) uses self-modifying code. SmartFiler is not organized internally into related modules; parts are interleaved and scattered all over. The use of _START_RD_1_BLOCK to effect a tape rewind without actually reading a block into Z80 RAM has been castigated previously. There is a long and complex routine which decides what sound effect to make depending upon what key you press, that is absolutely unintelligible unless you map it out as a branched decision tree on paper; talk about spaghetti GOTOs in BASIC! And the code used to sort database records is heavily self-modifying; there are data tables which actually contain Z80 opcodes, which are poked into various routines to change sorting parameters, and to terminate sorts. SmartFiler has lots of RAM left over; it doesn't need to resort to these tricks to save precious space.
Whether you're writing in assembler or C++, it's best to write clearly and logically *first*, and then worry about optimizing it for size or speed *if* field testing shows that it's necessary. Trying to write highly- optimized code on the first pass almost never works, and it's that much harder to debug. One programmer's opinion, anyhow...your mileage, of course, may vary.
I don't know what I'll do for next time, but it will be something interesting. Probably more work on the SmartFiler disassembly; it's almost completely resolved now, and once I can prove that it is, then I can start adding comments and changing some non-EOS global symbols to have meaningful names. And also there are Recipe Filer and Address Book to patch for ADAMserve, in probably exactly the same way as I have done for SmartFiler.
See you next week!
*Rich*
Next Article
Previous Article
TWWMCA Archive Main Page