Why is LoRaMac-Node not presented as a library?

[dmeehan1968]

It seems a little perplexing to me that the repository is not presented as a CMake package that provides a library. I appreciate that region support needs to be defined at compile time (which isn't ideal but how the code is currently structured), but the current structure of presenting sample applications makes sense as a starting point, but complicates things once an application is underway.

I'm currently working on converting an application started using 4.4.5 to 4.6.0 and there are sufficient API differences to make this a complex process, added to the need to create my own CMakeLists as the once included are unusable outside of the example applications.

I've currently created a CMake function that uses FetchContent to download the LoRaMac-node repo and then create its own targets for board (generic HAL layer), system, radio, peripheral and mac/region support, as static libraries. The application only then need supply implementations for the actual board (a custom board design in my case) via the <Perph>Mcu<Function> API provided by the LoRaMac-node HAL.

[mluis1]

Due to project history since 2013.

Since quite some time our goal is to transform the "mac" directory into a stand alone library. Potentially creating a stand alone repository for it and keep this repository as examples on how to use the library.

However we did not had the time or resources to do it.
There is some work to be done such as defining the APIs to abstract

• The hardware timers handling
• The system time handling
• The functionality currently provided by the utilities.c/h files
• etc...

In our backlog there are other changes/improvements that we would like to apply such:

• Refactor the regions management. The regions can be categorized as of 2 types. Dynamic (AS923, EU868, ...) or Fixed (AU915, US915) channel plans as such I think that we could simplify a lot the implementation as well as to remove some code duplicates.
• The MAC layer has evolved a lot since LoRaWAN 1.0.0 specification and the code has not been always refactored/cleaned up.
• Applying new coding conventions to be closer to the ones used by most other embedded projects. These coding conventions would be similar to the ones provided in loramac_radio.c under v5.0.0-branch
• etc...

One of the concerns that we have with all these changes is that if we apply them we will make a lot of people unhappy when they will try to upgrade their projects.
Therefore we need to figure out a smooth way to make the transition towards these goals as simple as possible.

As a conclusion, in the background we are trying to move towards what I explained before. However it will take a lot of time to get there.

[dmeehan1968]

Thanks, history creates baggage and that's understandable. Your quoted loramac_radio.c uses a 'pimpl' type idiom that I already use extensively in my own code. I'm curious to know if the change to snake case from camel has any particular meaning?

As for my current usage of LoRaMac-node, I have created a CMake function that loads the repo via FetchContent, and then creates static libraries for board, system, radio, peripheral and mac without reference to the included CMakeLists.txt files. The first four have external dependencies that are satisfied in the final link phase (such as the specific implementation of the board features), which I create as another static library. Our mac static library includes the mac and region directories, creating a target for each of the region variants we want to support (at the moment a single region per firmware target, but we might have multi-region support in the future if there are no hardware devices and we can accommodate configuring the active region somehow).

I tripped up in testing an upgrade to v4.6.0 in that the SOFT_SE define is needed in both the mac and peripheral libraries (and the application code), I only had it on the peripheral library and that lead to memory corruption as the interpretted size of the NVM structure was different.

The reason for creating targets was in part mimicking the existing repo's CMake setup, but also because we are creating regional variants of our device firmware due to hardware differences. We are preferring that over a multi-region build and the need to configure the active region somehow. I could just do a single target with all required sources but its a little less efficient from a build perspective, as the common modules are rebuilt for each device target.

We are working towards 3-4 products with 3-4 regions support for each. Debug and Release builds too. Fortunately the build time is quite low so the overhead isn't all that much of a worry, its more that I'm having to reinvent the CMake files and watch out for nuances and interdependencies between modules.

Regards API compatibility, there are some significant changes between 4.4.5 (our current version) and 4.6.0, and I don't know if those were intentional (nvm read/write params were completely different). NVM read/write params for example. And there were build differences (deprecated files and new ones, and include dependencies that cross the logical groupings).

[mluis1]

Your quoted loramac_radio.c uses a 'pimpl' type idiom that I already use extensively in my own code. I'm curious to know if the change to snake case from camel has any particular meaning?

The reason is mostly because Stackforce and Semtech other code sources use snake case. In addition almost all project that I have seen using this repository code also use snake case. Other than trying to standardize this code there is no special reason.

Regards API compatibility, there are some significant changes between 4.4.5 (our current version) and 4.6.0, and I don't know if those were intentional (nvm read/write params were completely different). NVM read/write params for example. And there were build differences (deprecated files and new ones, and include dependencies that cross the logical groupings).

Up until v4.6.0 we have used the minor field to introduce major changes as well as API changes.
Recently, we have decided to closely follow the Semantic Versioning recommendations.

--

I fully understand all the problems one may have when integrating this code on other projects.
Historically this project was meant to be used standalone. However over the years this has become less true.
This is the reason why we have the goal to make the MAC layer as independent as possible from any project architecture or MCU platform.
Unfortunately I do not know at the moment when we will be able to achieve this goal.

[dmeehan1968]

I think it would be fairly straightfoward to add an option to the root CMakeLists.txt that allows build of either the current structure, or to build library targets. For my own use case I'd like to be able to build one or more libraries for the base modules (board api, peripherals, radio, system), and then specific targets with regional support. This is in effect what I'm already doing within my own project.

It concerned me that the repo has canned copies of the STM32 CMSIS/HAL (my target platform) under boards/mcu, as I'd prefer to be able to establish my own versions on a project specific basis and keep control of them. I suspect it wouldn't be too hard to devise a way for the integrator to specify the platform specific locations or revert to the canned versions.

I think this would look something like:

project(my_project)

# When FALSE, the build follows the current 'src' subdirectory. 
# When TRUE, it follows a new subdirectory that creates library targets based on the src files,
# but not the src CMakeLists.txt files (which are designed for the examples).
set(LORAMAC_LIB TRUE)

# This pass will build the non-region specific components in a shared location
set(LORAMAC_MAC_BUILD OFF)
add_subdirectory(${loramac_SOURCE_DIR})

# This pass will build the mac with region specific support in a variant specific location
set(LORAMAC_MAC_BUILD ON)
set(LORAMAC_SUFFIX EU}
set(REGION_EU868 ON)
add_subdirectory(loramac_SOURCE_DIR, ${CMAKE_CURRENT_BINARY_DIR}/${LORAMAC_SUFFIX})

add_target(${PROJECT_NAME})
target_link_libraries(loramac-${LORAMAC_SUFFIX})
# etc...

If someone wants multiple region specific builds (like I do) they can call add_subdirectory multiple times with a different binary location and the loramac-node CMakeLists.txt would tack the specified suffix onto the mac target (e.g. loramac-eu).

You would get one principle target, named loramac-${LORAMAC_SUFFIX}, which would have its own dependencies on the internal targets, such as loramac-board, loramac-system etc. Those internal targets are still accessible if someone wants to compose their own dependency list.

I'd be happy to look at a PR for this. Would you prefer an issue to be created with a proposal (or can this discussion be moved to issues)?

PS: I normally pull down the LoRaMac repo using FetchContent, but using the FetchContent_Populate method rather than FetchContent_MakeAvailable (which assumes the content is a CMake compatible and automatically calls add_subdirectory). To be fair, its possible that the above could be compatible with _MakeAvailable with a little tweaking.