Difference between revisions of "NEX file format"

From SpecNext official Wiki
Jump to: navigation, search
(V1.3 extensions proposal - refined some details after testing NEXLOAD2 prototype)
m
Line 114: Line 114:
 
For Layer 2 banks 9,10,11,12,13 are used to display the loading screen.
 
For Layer 2 banks 9,10,11,12,13 are used to display the loading screen.
  
EDIT: I'm deeply tempted to drop the palette hack, and move the four bytes into header as separate fields... so far I heard no opinions about these proposals, and if it stays like this, I will go for new header fields, the palette hack feels just wrong, and there's still enough free space in the header for extra 4B.
+
EDIT: I'm deeply tempted to drop the palette hack, and move the four bytes into header as separate fields... so far I heard no opinions about these proposals, and if it stays like this, I will go for new header fields, the palette hack feels just wrong, and there's still enough free space in the header for extra 4B.
 
|-
 
|-
 
| 153 || 1 || 1 = Has copper code block, extra 2048B block after last screen data block, which will be set to Copper and the copper will be started with %01 control code (reset CPC to 0, and start). This can be used for example as "loading screen animation" feature (be aware the timing of load may vary greatly).
 
| 153 || 1 || 1 = Has copper code block, extra 2048B block after last screen data block, which will be set to Copper and the copper will be started with %01 control code (reset CPC to 0, and start). This can be used for example as "loading screen animation" feature (be aware the timing of load may vary greatly).

Revision as of 17:05, 3 January 2020

The NEX file format was designed as very simple and straightforward format to load self-contained applications into memory and start them. Through various extensions it did reach v1.3 at this moment, which allows even for more complex use cases.

The official distribution sports some goodies for NEX files:

Additional resources:


The basic structure of the file is:

block size optional description
512 "Next" string followed by file header, containing also map of memory banks stored in the file
512 * optional palette (for Layer2 or LoRes screen)
49152 * Layer2 loading screen
6912 * classic ULA loading screen
12288 * LoRes loading screen
12288 * Timex HiRes (512x192) loading screen
12288 * Timex HiCol (8x1) loading screen
n * 16384 * 16kiB raw memory bank data in predefined order: 5,2,0,1,3,4,6,7,8,9,10,...,111 (particular bank may be omitted completely)
m * Optional binary data appended after "regular" NEX file content - custom format

Structure of the header block:

offset size description
0 4 "Next" string
4 4 string with NEX file version, currently "V1.0", "V1.1", "V1.2" or "V1.3"
8 1 RAM required: 0 = 768k, 1 = 1792k
9 1 Number of 16k Banks to Load: 0-112 (see also the byte array at offset 18, which must yield this count)
10 1 Loading-screen blocks in file (bit-flags):

128 = no palette block, 16 = Hi-Colour, 8 = Hi-Res, 4 = Lo-Res, 2 = ULA, 1 = Layer2 The loader does use common banks to load the graphics into, and show it from, i.e. bank5 for all ULA related modes and banks 9,10 and 11 for Layer2 graphics (loading these banks afterwards as regular bank will thus replace the shown data on screen). Only Layer2 and Lo-Res screens expect the palette block (unless +128 flag set). While one can include multiple screen data in single file (setting up all relevant bits), the recommended/expected usage is to have only one type of screen in NEX file.

11 1 Border Colour: 0-7
12 2 Stack pointer
14 2 Program counter (0 = don't run, just load)
16 2 "Number of extra files" (currently not used by anything/anyone?)
18 112 byte flag (0/1) of 16k banks included in the file - this array is in regular order 0..111, i.e. bank5 in file will set 1 to header byte at offset 18+5 = 23, but the 16kiB of data for bank 5 are first in the file (order of bank data in file is: 5,2,0,1,3,4,6,7,8,9,10,...,111)
130 1 Layer2 "loading bar" 0 = OFF, 1 = ON (works only in combination with Layer2 screen data)
131 1 "Loading bar" colour (0..255)
132 1 Loading delay per bank (0..255 amount of frames), 0 = no delay
133 1 Start delay (0..255 amount of frames), 0 = no delay
134 1 Preserve current Next-Registers values (0 = reset, 1 = preserve)
135 3 Required core version, three bytes 0..15 "major", 0..15 "minor", 0..255 "subminor" version numbers. (core version is checked only when reported machine-ID is 10 = "Next", on other machine or emulator=8 the latest loaders will skip the check)
138 1 Timex HiRes 512x192 mode colour, encoded as for port 255 = bits 5-3. I.e. values 0, 8, 16, .., 56 (0..7 * 8)
139 1 Entry bank = bank to be mapped into slot 3 (0xC000..0xFFFF address space), the "Program Counter" (header offset +14) and "File handle address" (header offset +140) are used by NEX loader after the mapping is set (The default ZX128 has bank 0 mapped after reset, which makes zero value nice default).
140 2 File handle address: 0 = NEX file is closed by the loader, 1..0x3FFF values (1 recommended) = NEX loader keeps NEX file open and does pass the file handle in BC register, 0x4000..0xFFFF values (for 0xC000..0xFFFF see also "Entry bank") = NEX loader keeps NEX file open and the file handle is written into memory at the desired address.
142 1 (V1.3) 0 = disable Expansion Bus by setting top four bits of Expansion Bus Enable Register ($80) to 0, 1 = do nothing (does apply only to cores 3.0.5+)
143 369 reserved for future extensions, set to zero in older versions of file format.

The V1.3 is currently work-in-progress (so far only one new flag has been added at offset 142), and there may be more new features coming in following weeks.


Currently I (Ped7g) am considering to add these fields to the V1.3 header, feel free to comment about these on SpectrumNext discord server in #tools channel (I believe I will have prototype implementation in following days, and eventually based on the feedback this may become final V1.3 header in a week or so):

Proposed further changes of the header block for V1.3:

offset size description
143 1 1 = Has checksum value, checksum algorithm is CRC-32C (Castagnoli), value itself is at the very end of the header block
144 4 File offset of first bank data (when loader is parsing known version, it should know where the banks start without this value, but it may use it for extra check whether the parsing of optional blocks between header and first bank was done correctly)
148 2 CLI buffer address (after "Entry bank" is paged in), 0 = no buffer
150 2 CLI buffer size - when address and size are provided, the original command line passed to NEX loader will be copied to defined buffer (and truncated to "size", shorter string may be zero-terminated) and register DE is set to the buffer address. The maximum size is 2048 bytes (longer lines can be probably salvaged from Bank 5 memory if the NEX file is not loading that bank and will search for the original line on its own).
152 1 Loading screen flags 2 - when first flag has value 64 set (+128 no-palette is partially valid for new cases too, other bits should be not mixed with new modes):
1 = Layer 2 320x256x8bpp (old +128 no-palette does apply too) blocks: 512B palette + 81920B data
2 = Layer 2 640x256x4bpp (old +128 no-palette does apply too) blocks: 512B palette + 81920B data
 (HiRes color value is used as L2 palette offset - does apply to both new Layer 2 modes)
3 = Tilemode screen, block 512B: 508B palette (254 colors only) + 4B NextRegs $6B, $6C, $6E, $6F
 Tilemap data are stored in regular bank 5 - no specialized data block is used.

For Layer 2 banks 9,10,11,12,13 are used to display the loading screen.

EDIT: I'm deeply tempted to drop the palette hack, and move the four bytes into header as separate fields... so far I heard no opinions about these proposals, and if it stays like this, I will go for new header fields, the palette hack feels just wrong, and there's still enough free space in the header for extra 4B.

153 1 1 = Has copper code block, extra 2048B block after last screen data block, which will be set to Copper and the copper will be started with %01 control code (reset CPC to 0, and start). This can be used for example as "loading screen animation" feature (be aware the timing of load may vary greatly).
154 354 reserved for future extensions, set to zero in older versions of file format.
508 4 optional CRC-32C (Castagnoli) - is calculated by checksumming file content from offset 512 (after this value) till end of file (including the optional custom binary appended after regular NEX file), and then further checksumming first 508 bytes of the header (the only skipped four bytes are then this checksum value). This is intended for PC tools working with NEX files, or extra check tool running on Next, but not for regular loading of NEX files.

I tried to consider also the multi-handle of NEX file feature, but I was unable to figure out implementation details how to achieve desired outcome and what are the limits and API, so it's not included in my current V1.3 proposal.


Possible roadmap/ideas for future (this is just collection of ideas, no timeframe or guarantee that they will ever happen, feel free to comment on them):

  • probably compatible with current V1.2 file format
    • checking machine memory and use the RAMREQ to report low memory
    • having entry bank dynamically allocated by NextZXOS (entrybank = 255?)
    • changing working directory if path with directories was provided (adjust that also when passing command line in V1.3 to make file-only like)
  • V1.3+
    • passing cmd line args to the loaded code - after discussion with Allen A. => BC = file handle (or 255), DE = command line (is equal to the field buffer value in header), zero terminated string (if the zero terminator is missing, the line is probably truncated to buffer size) = two new words into V1.3 header [cliBuffer, cliSize].
    • preserving original NextZXOS stack SP value (stack pointer = 2?) (0 is valid SP starting from very "top" of RAM) (preserving is risky, as there is always possibility of ramtop being set in such unfortunate manner, that preserved sp will cause destruction of the loaded code, so the SP should be set up by the header every time, but maybe there can be extra flag to put old value into the new stack by loader, so the code can eventually get to the content of old stack - if the NEX file doesn't overwrite the banks 5/2/0)
    • checksums incorporated into the NEX file (maybe into header?) - mostly for file archival/transfers, not loader itself
    • custom palettes also for ULA/HiRes/HiCol screens (maybe +64 flag?) (or invert the +128 in V1.3 for those three modes, i.e. 0 = no pal / 128 = has pal)
    • tilemap screen
    • return multiple open file handles to the .nex file. Specify how many handles and an address in the header.
    • With a flag (bit 7 of number of handles?) to force close all existing open handles first.