Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overlaying a WLA-DX ROM into a board's RAM #597

Open
cr1901 opened this issue Jul 25, 2023 · 17 comments
Open

Overlaying a WLA-DX ROM into a board's RAM #597

cr1901 opened this issue Jul 25, 2023 · 17 comments

Comments

@cr1901
Copy link
Collaborator

cr1901 commented Jul 25, 2023

Possibly related to #56?

I have a rare case where ROM and RAM overlap slightly. Specifically, I overlay my "ROM" image into RAM which is shared w/ a debugger's variables (the debugger itself is in Flash ROM in the upper half of the address space).

There are a few variables that the debugger provides that the user is expected to fill in. Specifically, the user populates interrupt vectors; the debugger has vectors at the normal $fffa and friends and jumps to routines pointed to at $7fea and friends. The remaining variables the debugger uses from $7c00 should not be touched by my code. Unfortunately, since ROM sections are the only place where a user can "lay down" initialized bytes into an image, I opt to include the debugger RAM as part of my ROM bank.

memorymap/rombankmap (I allocate 8kB for my own RAM. Hardcoding a RAM split is probably unavoidable):

    .MEMORYMAP
    DEFAULTSLOT 1
    SLOT 0 START $0000 SIZE $2000
    SLOT 1 START $2000 SIZE $6000
    .ENDME

    .ROMBANKSIZE $6000
    .ROMBANKS 1

linkfile:

[objects]
hello.o

[libraries]
bank 0 slot 0 ../common/periph.lib
bank 0 slot 0 ../common/debugger.lib

[ramsections]
bank 0 slot 0 org 0 force "DEBUGGER_ZERO_PAGE"
bank 0 slot 0 org $d0 force "BREAKPOINT_ZERO_PAGE"
bank 0 slot 0 org $100 force "STACK_PAGE"
bank 0 slot 2 orga $7c00 force "DEBUGGER_MAIN_RAM"
bank 0 slot 2 orga $7f00 force "PERIPHERALS_PAGE"

Vectors overwriting debugger RAM:

; FIXME: WLA doesn't handle ROM/RAM collisions well...
; Bare .ORGA without a section also works, but at least this signifies intent.
; The use is _expected_ to overwrite these debugger vars.
.BANK 0 SLOT 1
.SECTION "Vectors" OVERWRITE ORGA $7efa
.dw unused
.dw entry
.dw unused
.ENDS

OVERWRITE at least signifies intent that I'm doing something weird. Of course the output ROM fills the "unused" RAM/ROM locations leading up to the Vectors section with 0. But I can use WLA's symbol file with -x (extra definitions) to figure out (SECTIONSTART_*, SECTIONEND_*) which offsets into the ROM file should actually be loaded into my board's RAM.

I Want WLA-DX To Detect Conflicts, Not Silently Misassemble

This works, but I unfortunately lose WLA DX's ability to determine conflicts between sections/slots (ROM/RAM conflicts are not supported). DEBUGGER_MAIN_RAM begins at $7c00; what recourse do I have to have the link fail if my ROM section crosses into debugger RAM?

A New Output Format?

In addition, instead of me parsing symbol files, could there be a "standardized" output format which consists of instructions of where to load each section of the output binary? I found WDC has a simple loading format, as follows (bank:address and length are little-endian):

Header: 0x5A
One or more of: [bank:address of section, 3 bytes][length of section, 3 bytes][actual section data to load]
Footer: 6 bytes of zeroes

Here's a simple example of the format (comments mine):

5A ; Header
00 20 00 23 00 00 78 D8 A2 FF 9A A9 FF 8D C2 7F A2 00 BD 00 30 8D C0 7F E8 D0 F7 68 40 08 48 A9 FF 68 28 40 01 00 00 00 0A ; Load user code at 0x2000, 0x23 bytes
FA 7E 00 06 00 00 17 20 00 20 15 20 ; Load user interrupt vectors at 0x7efa, 0x6 bytes
FA FF 00 06 00 00 17 20 00 20 15 20 00 00 00 00 00 00 ; Load actual interrupt vectors at 0xfffa, 0x6 bytes- this is ignored by our debugger.
00 00 00 00 00 00 ; Footer

How difficult would this be to add? Note that section data wouldn't have any zero padding, which would result in significant file size/transfer time savings for small pieces of code for my board. I started taking a look at wlalink myself. It looks like g_rom_usage is the variable that controls how much of a section is actually used.

@vhelin
Copy link
Owner

vhelin commented Aug 14, 2023

Sorry for taking so long with this; first I was totally out of energy and didn't completely understand what is going on, but after reading this four times I think I have a hunch now.

So this issue contains actually two issues, which are not really connected?

  1. In your case you'd need to specify a .MEMORYMAP where ROM and BANK banks share partially the same address space. Would something like this work?

    .MEMORYMAP
    DEFAULTSLOT 1
    SLOT 0 START $0000 SIZE $2000
    SLOT 1 START $2000 SIZE $6000
    RAMSLOT 0 START $7C00 SIZE 4096
    .ENDME

EDIT: No? As ROM and RAM data need to occupy the same memory area? So actually we'll need a way to insert .RAMSECTIONs into normal SLOTs where .SECTIONs are? Maybe add a new keyword to a .RAMSECTION?

  1. You want a new output format, just like in WDC, the specifications are here in this issue's description? Also I'm interested that which platforms support such output format, and how does one "load"/"use" such format...

@vhelin
Copy link
Owner

vhelin commented Aug 14, 2023

A question regarding having .RAMSECTIONs sharing the same SLOT with normal .SECTIONs: Why not mimick a .RAMSECTION with a .SECTION?

.RAMSECTION "Vars"
var1: db
var2: db
.ENDS

... is kinda the same as

.SECTION "Vars"
var1: .db 0
var2: .db 0
.ENDS

?

@cr1901
Copy link
Collaborator Author

cr1901 commented Aug 14, 2023

In your case you'd need to specify a .MEMORYMAP where ROM and RAM banks share partially the same address space.

Yes, that is correct. Right now, I'm just plain assuming conflicts can't occur. Basically, I want "initialized global RAM data", and I want WLA-DX to be aware of initialized global RAM data. A RAMSLOT keyword or a new RAMSECTION keyword might be the way to do this, but I'm not right now. Anyways, the bold is me trying to avoid an XY Problem :D.

As ROM and RAM data need to occupy the same memory area?

Yes, I consider vectors to be ROM, even though they're physically placed in RAM in this specific board. Debugger variables which I don't touch surround the vectors. I would want a .MEMORYMAP to not touch the surrounding debugger vars. E.g. skip them, like in:

.RAMSECTION "DEBUGGER_MAIN_RAM"
. DSB 768 ; Peripherals at $7f00
.ENDS

However, I need some way to "insert" my "Vectors" section into that .RAMSECTION at the appropriate offset:

.SECTION "Vectors" OVERWRITE ORGA $7efa
.dw unused
.dw entry
.dw unused
.ENDS

A question regarding having .RAMSECTIONs sharing the same SLOT with normal .SECTIONs: Why not mimick a .RAMSECTION with a .SECTION?

This is indeed what I'm doing:

; FIXME: WLA doesn't handle ROM/RAM collisions well...
; Bare .ORGA without a section also works, but at least this signifies intent.
; The use is _expected_ to overwrite these debugger vars.
.BANK 0 SLOT 1
.SECTION "Vectors" OVERWRITE ORGA $7efa
.dw unused
.dw entry
.dw unused
.ENDS

"So what's the problem", you ask? Well, having a section like this is going to zero-pad the ROM all the way to the vectors. If I upload the "raw" ROM file, the zero-padding will overwrite debugging variables. I am using heuristics in my uploader program to avoid the zero-pad problem.

So this issue contains actually two issues, which are not really connected?

Obviously, raw ROM file format will not be able to support "holes"/"initialized global data" in the address space like this, but I think maybe WLA should be aware of "initialized RAM"/special RAM locations. That's why I brought up the WDC format.

the specifications are here in this issue's description?

Correct.

Also I'm interested that which platforms support such output format, and how does one "load"/"use" such format...

Seems to be limited to WDC IDEs. I'm not married to the format, I just think it's a very simple "not-raw binary" format :).

@cr1901
Copy link
Collaborator Author

cr1901 commented Aug 14, 2023

A question regarding having .RAMSECTIONs sharing the same SLOT with normal .SECTIONs: Why not mimick a .RAMSECTION with a .SECTION?

I also tried another approach that works sometimes but not all the time. I have a magic cookie value that is padded after all the sections I define but before the "Vectors". This indicates the "end of .text. My uploader program looks for the cookie as a place to stop uploading the .text.

Unfortunately, I couldn't find a way to consistently "append a section to ensure it's second-to-last in the ROM", and I think it's a bit hacky anyway :).

@vhelin
Copy link
Owner

vhelin commented Aug 15, 2023

Ok, I just added (didn't push to master, yet) support for OCCUPYROM keyword in .RAMSECTION

.RAMSECTION "Vars" OCCUPYROM
var1: db
var2: db
.ENDS

that turns it into a corresponding .SECTION when sections are sorted and placed so this makes .SECTION/.RAMSECTION collisions possible. But what puzzles is that should I mark also the same "area" occupied that the .RAMSECTION takes in a RAM bank? Here it just takes space in a ROM bank...

@cr1901
Copy link
Collaborator Author

cr1901 commented Aug 15, 2023

OCCUPYROM is technically correct for my specific board. But note that it's possible to have initialized RAM due to a loader without the need for RAM/ROM collisions. Actually, that's kinda what the WDC format I mentioned above is doing; it's loading both RAM and "ROM" into its required positions on the board's RAM. Is there a way to also accommodate initialized RAM due to a loader w/o collisions?

should I mark also the same "area" occupied that the .RAMSECTION takes in a RAM bank?

I would personally have the assembler be aware that initialized and uninitialized RAM are in the same .RAMSECTION. E.g. if you don't skip over those locations with .DSB in the corresponding .RAMSECTION, error out.

@vhelin
Copy link
Owner

vhelin commented Aug 16, 2023

The more I think about this the less I think I understand what is going on. :) If you already have a loader and everything is in the same address space, why not just make everything a .SECTION of some sort? Why would you need a .RAMSECTION for anything there? You can initialize data in .SECTIONs, and embed a special keyword to the names of such .SECTIONs where you don't want to copy the data to output so you can skip them later. EDIT: Ah, .SECTION names are not available in the symbol file or in the output in any way. Maybe it could be useful if I modified the symbol file writer to add .SECTION information into the symbol file?

vhelin added a commit that referenced this issue Aug 16, 2023
…ions] and [ramsections]. Might help fixing GitHub issue #597.
@cr1901
Copy link
Collaborator Author

cr1901 commented Aug 16, 2023

Ah, .SECTION names are not available in the symbol file or in the output in any way.

Yes, also this. I didn't think to mention that after being the first thing I tried. I'm looking at your new commits now to see if I can modify my loader program to do what I want.

why not just make everything a .SECTION of some sort?

My original answer was: I still think "initialized RAM" and "ROM" should be distinguished in some way, in case other object formats besides raw get added to WLALINK. For example o65 format. But reading o65 format spec:

The data segment actually is like the text segment, only it is allocated writable.

This is what you're suggesting, except without a marker to deliminate "this is RAM". Maybe a WRITABLE keyword to SECTION in the future to indicate "this SECTION stored in RAM and is meant to be writable, for output formats that care". This also enables self-modifying code w/ a "default" set of insns as well.

@cr1901
Copy link
Collaborator Author

cr1901 commented Aug 17, 2023

If you already have a loader and everything is in the same address space, why not just make everything a .SECTION of some sort?

I remembered the answer to this. Because you can't place .SECTIONs into a SLOT with a different length than the ROMBANKSIZE. You'll get a size error:

wla-65c02 -DUSING_DEBUGGER=1 -I ../common -x -i -k -o hello.o hello.asm
hello.asm:17: DIRECTIVE_ERROR: SLOT 0's size 8192 < BANK 0's size 24576.
hello.asm:17: ERROR: Couldn't parse ".BANK".
make: *** [Makefile:31: hello.o] Error 1

So I can place initialized RAM vars into SLOT 1, but even though I have a loader, I can't place initialized RAM vars into SLOT 0 because WLA will error out.

Because I'm placing initialized RAM into only SLOT 1, I think the new symbol file format is workable for my immediate needs.

Other Remarks

A BANK:OFFSET uniquely encodes a particular ROM address, because a SLOT uniquely identifies a ROM bank. But I don't think that's true for .RAMSECTIONs? Consider the following (empty sections not discarded):

    .MEMORYMAP
    DEFAULTSLOT 1
    SLOT 0 START $0000 SIZE $2000
    SLOT 1 START $2000 SIZE $6000
    .ENDME
.RAMSECTION "HelloVars"
.ENDS

.RAMSECTION "Test" SLOT 1
test DB
.ENDS

This results in the following symbol file:

[ramsections]
...
00:0000 2000 00000001 Test
00:0000 0000 00000000 HelloVars

Test and HelloVars are in the same bank (there's no bank switching), same offset, but different slots. As per docs, "ROM and ROM and RAM don’t usually overlap", but on my board they do.

Should there be slot information as well to uniquely identify a specific RAMSECTION without me having to do math?

c30fa7e#diff-2168ac48f5c76aca9c113f05cddde4475d1365833d42c82dcba4588054aba516R118

Each line specifies a .RAMSECTION: hexadecimal bank, RAM bank offset, memory address, size and name. Use this information for example to see where a .RAMSECTION was placed.

RAM slot offset?

@vhelin
Copy link
Owner

vhelin commented Aug 17, 2023

I can add slot number there per .RAMSECTION later today after work. Just thinking, should I also write .MEMORYMAP (-> [memorymap]) and .ROMBANKMAP (-> [rombankmap]) into the symbol file?

RAM slot offset?

Isn't the RAM slot offset the same as RAM bank offset?

@cr1901
Copy link
Collaborator Author

cr1901 commented Aug 17, 2023

should I also write .MEMORYMAP (-> [memorymap]) and .ROMBANKMAP (-> [rombankmap]) into the symbol file?

I'd hold off on this personally for now.

Isn't the RAM slot offset the same as RAM bank offset?

No, see my example (unless this is not intended to work), where two different RAM vars have the same bank:offset, but are different addresses:

[ramsections]
...
00:0000 2000 00000001 Test
00:0000 0000 00000000 HelloVars

I remembered the answer to this. Because you can't place .SECTIONs into a SLOT with a different length than the ROMBANKSIZE. You'll get a size error:

Is there any possible way around this for initialized RAM?

@vhelin
Copy link
Owner

vhelin commented Aug 17, 2023

No, see my example (unless this is not intended to work), where two different RAM vars have the same bank:offset, but are different addresses

Yeah, the address is different, but the bank/slot offset is the same, AFAIK. Bank starts at the beginning of a slot.

With .RAMSECTIONs each slot has a set of 256 RAM banks, I think. Let me double check this. If true, this might be actually a flaw in WLA DX, because some systems might have multiple slots where the same RAM banks could be inserted into...

EDIT: This seems to be correct - each SLOT has its own set of 256 RAM banks. I think I need to add a switch that makes the slots to share the same RAM banks...

EDIT: I added support for a RAM bank sharing between multiple slots, but when I was writing tests I realized it wouldn't work - WLALINK has one address for a label, but when changing slots the address would be different. So if one wants to use a RAM bank in multiple slots he/she must per slot fill the RAM banks with identical data, only the label names must be different. This also means that ROM banks cannot be used in multiple slots. Unless we figure out how to fix this situation. I guess the code would have to be given hints like BANK x IS IN SLOT y and WLALINK would get this information somehow and then change the addresses on the fly. Sounds really complicated...

Is there any possible way around this for initialized RAM?

I have to admit that I don't understand what you are doing LOL :D

@vhelin
Copy link
Owner

vhelin commented Aug 17, 2023

This is what you're suggesting, except without a marker to deliminate "this is RAM". Maybe a WRITABLE keyword to SECTION in the future to indicate "this SECTION stored in RAM and is meant to be writable, for output formats that care". This also enables self-modifying code w/ a "default" set of insns as well.

What would WRITABLE keyword do, codewise, in WLA/WLALINK? I mean I can add a new variable to section struct, but it has to be used somehow...

@vhelin
Copy link
Owner

vhelin commented Aug 17, 2023

A BANK:OFFSET uniquely encodes a particular ROM address, because a SLOT uniquely identifies a ROM bank. But I don't think that's true for .RAMSECTIONs? Consider the following (empty sections not discarded):

SLOT's just a slot in the memory where you can put any ROM bank that has the same size as the SLOT so no, SLOT doesn't uniquely indentify a ROM bank AFAIK... ?

@cr1901
Copy link
Collaborator Author

cr1901 commented Aug 17, 2023

With .RAMSECTIONs each slot has a set of 256 RAM banks, I think.

This was the part I was missing. I forgot that each slot has 256 RAM banks. Contrast to ROM banks, where you have to assign each ROM bank defined to a slot manually. IIRC BANKSTOTAL in ROMBANKMAP ycan be greater than 255. So a single slot can have 256 ROM banks or more (w/ the valid ROM bank numbers for that slot not necessarily being limited to numbers 0 to 255).

because some systems might have multiple slots where the same RAM banks could be inserted into...

You mean mirroring? I figured that was out of scope.

Sounds really complicated...

Indeed, let's not go down this path for now.

I have to admit that I don't understand what you are doing LOL :D

I'm trying to initialize RAM, similar to .data sections in Unix object files. The Unix loader ensures the RAM is initialized before the program begins.

However, at some point, I think I lost the plot. So let me take some time to gather my thoughts.

What would WRITABLE keyword do, codewise, in WLA/WLALINK? I mean I can add a new variable to section struct, but it has to be used somehow...

If other object formats are added, like o65, WRITABLE would tell WLA-DX/WLALINK "put this section in a place where the object format knows it's writable, not read-only". It's not very useful right now.

SLOT's just a slot in the memory where you can put any ROM bank that has the same size as the SLOT so no, SLOT doesn't uniquely indentify a ROM bank AFAIK... ?

There can only be a single slot where ROM BANK 0 is placed, correct? Ditto for ROM BANK 1, ROM BANK 2, etc. In contrast, each SLOT has 256 RAM BANKS. So each slot will have a RAM BANK 0, RAM BANK 1, etc up to 255. In that sense, if you have the ROM BANK number in the symbol file, you can uniquely identify which SLOT it's in. But you cannot do the same for RAM BANKS. Is this correct?

@vhelin
Copy link
Owner

vhelin commented Aug 19, 2023

There can only be a single slot where ROM BANK 0 is placed, correct?

You can actually place a ROM bank into multiple slots, but when you define a .SECTION you give it a slot so references to the labels inside that .SECTION work only when the ROM bank is in that SLOT. So you could in theory have .SECTIONs with SLOT 0 covering 50% of ROM bank 1, and the rest filled with .SECTIONs with SLOT 1...

@cr1901
Copy link
Collaborator Author

cr1901 commented Aug 21, 2023

You can actually place a ROM bank into multiple slots, but when you define a .SECTION you give it a slot so references to the labels inside that .SECTION work only when the ROM bank is in that SLOT. So you could in theory have .SECTIONs with SLOT 0 covering 50% of ROM bank 1, and the rest filled with .SECTIONs with SLOT 1...

Ahhh, I see. And placing a ROM bank into multiple slots only works without warnings when SLOT 0 and SLOT 1 have the same size as the ROM bank; if the bank is smaller than the SLOT, WLA-DX accepts it with a warning. If the bank is larger than the SLOT, WLA-DX errors out.

I have some ideas to fix my code now...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants