Lvgl simulation run PineTime screens on your PC

[NeroBurner]

Add simulator to have pinetime Screens on PC for development.

The first Screen can be run locally, using your mouse to play Paddle.h!

based on lvgl 8 port #734

lv_sim based on: https://github.com/lvgl/lv_sim_eclipse_sdl

---

2021-11-03 updated screens with PineTime fonts

---

2021-11-07 new Screen Metronome and the first simulated external: MotorController (shown by red dot)

---

2021-11-15 new Screen BatteryInfo.h with a very low Battery, according to the BatteryController.h stub

---

2021-11-28 rebased on develop branch getting the new v1.7.1 version of FlashLight.h

---

2021-11-30 All three PineTime Watch Faces are working now: WatchFaceDigital.h, WatchFaceAnalog.h and PineTimeStyle.h

---

2021-12-02 Can trigger a Notification, got the system time, and the first Settings-Screen QuickSettings.h

---

2021-12-03 Fix "score" alignment in Twos.h by upgrading lvgl submodule to the current master (v8.1.0-64-g628299676)

2022-01-16 Ported the simulator to use lvgl7 like current PineTime firmware! much less porting and maintenance burden! 🎉 And Music.h has shown up!

---

2022-01-21: SystemInfo.h's first Screen, others available, but no swipe yet

---

2022-01-25: Swiping now works and most Screens are implemented

---

2022-01-27: add SettingSetDate.h and SettingSetTime.h

---

2022-01-29: add Screen is OFF overlay when brightness is set to Off. Also add Notifications.h

---

2022-01-30: add the last screens FirmwareValidator.h, FirmwareUpdater.h and PassKey.h

---

2022-02-02: increase and decrease steps (s/S) and heartrate (h/H)

[Quantum-cross]

woah this draws lvgl screens straight from the lib without hardware?

[NeroBurner]

yes indeed, it uses SDL to handle the window and https://github.com/lvgl/lv_drivers to handle the display and touch (mouse input)

[NeroBurner]

To build change into the lv_sim subdirectory, initialize the lv_drivers submodule and run cmake

cd lv_sim
git submodule update --init lv_drivers
cmake -S . -B build
cmake --build build

[JF002]

Thanks for this PR, @NeroBurner ! InfiniTime definitely needs a simulator for the UI like this one! I have already try to build one but haven't had the time to maintain it!

Now, I'm wondering if the simulator should be included in the same project/repo as InfiniTime of if we should create another project (under the InfiniTime organization) to keep them separated? What's your opinion about this?

[NeroBurner]

I'd add them to the same repository. My intention is to use the same files as InfiniTime itself, just provide simulator classes for the hardware specific bits. Otherwise I fear the simulator will be harder to maintain. I'd also add a CI job to build the simulator to prevent breakage of the simulator

[JF002]

That's a good point! However, I'm not sure it'll be practical to maintain and manage on the long run (code, history, commits, PR,... will be mixed, making the work of maintainers and reviewer more difficult).

Wouldn't it be possible to organize the code of the simulator so that it includes InfiniTime (as a submodule) in a subfolder, for example? The CI of the simulator would check that the version of InfiniTime defined in the submodule still compiles correctly with the simulator.

[NeroBurner]

Yes, that would be no problem. The disadvantage of an extra repo is the decoupling. If a PR breaks some assumption the simulator is using, the breakage is checked after the merge, making the simulator harder to maintain (because I (or other simulator maintainer) would have to fix the build afterwards)
The greatest advantage for the simulator (when integrated) is the possibility of checking the simulator for each PR with the CI. I don't know of a way to do that with an extra simulator-repo

[JF002]

It might be possible to trig an action on the repo of the simulator when a change is done in the repo of infinitime.
But I agree with you, it complicates the maintenance...
Maybe someone else have an opinion on this? @Avamander @geekbozu ?

[NeroBurner]

playing some Twos.h, unfortunately without coloring, don't know why though, and the left align of the "Score x" label is ignored

[lectrician1]

Ayaya you beat me to it by 6 days. I was just trying to implement the emulator myself!

I was able to render screens and fonts successfully with https://github.com/lvgl/lv_platformio as a starting point for my own emulator work.

However, I realized that importing all of the fonts, symbols, and lvgl files to a new repo and then using Platform.io with it's own build system wasn't optimal and I needed to base the emulator around the actual repo, just like you did!

Now, is it not possible to create an emulator that simulates the entire Pinetime system? The entire UI is built on LVGL, so it should be possible...

Creating your own lv_conf.h/lvgl project and then requiring source files to be manually added seems really tedious and hard to maintain.

Couldn't we somehow create a build target that is in the main repo and it includes SDL, lv_drivers, and the source files? That would be the optimal solution.

Let me know if this is possible!

[NeroBurner]

Finally fixed the Twos coloring under LVGL8 and and the swiping works as expected handling swipe events issued by LVGL 🎉

[NeroBurner]

@lectrician1 The manual adding of just one screen is just for development progress to incrementally build the simulator parts. My goal is to use unmodified PineTime Screens, but for that I need to provide the hardware specific headers as stubs. And I want to implement them bit by bit, otherwise I'd need to create everything at once, which is not feasible for me

I'm not sure if it is easier to have a separate "lv_sim" project or a target in the main project. As I stub some of the hardware dependencies (maybe FreeRTOS and nRF5 in the future) and the simulator using a different compiler (desktop amd64/arm64 GCC vs gcc-arm-none-eabi-9-2020-q2-update) having the simulator target in the main project is maybe pretty hard to manage, not unsolvable, just error prone I think

[NeroBurner]

Time to be creative! InfiniPaint on your PC 🎉

I reworked the code to use LVGL core widget lv_canvas and listen for LVGL touch events

[lectrician1]

Yeah, you are right that they use different compilers. And then libraries on top of that? That could be very error prone.

But... if we do get it to a state where the entire project can build and run, I think we shouldn't run into any future errors. It's not like we're going to be adding many new libraries after that. The entire project will be based around the ones we will have by then. LVGL should work, whether it's running on the PineTime or desktop.

So, I guess we should keep pressing on and making sure all components of the current project are compatible.

I understand and like your current approach of testing each of the components. Testing the entire project immediately would be an insurmountable task.

[NeroBurner]

One step closer to the real deal! Found the fonts, and now the lv_sim screens finally use the InfiniTime fonts 🎉

[NeroBurner]

Next Screen Metronome.h. But what is this second window? And it got a dot? But only sometimes? What does it do and where does it go when it isn't on the screen? Great questions!

The second window is intended to show the simulated externals of the PineTime. Currently only the MotorController.h is simulated. And when the motor is active the red dot is drawn. The red dot is a placeholder, but I have just SDL to work with, and I have little experience with that. Most probably the red dot will be replaced in the future, but no promises ;)

[NeroBurner]

Found a screen, which I've never seen on my PineTime: Meter.h

It spins right round, right round (clockwise)

[NeroBurner]

My BatteryInfo.h says, that my Battery is low. Jokes on you! I don't have a battery on my PC! 🤓

[NeroBurner]

Found my simulated battery and charging cable! Now I finally can charge my simulated InfiniTime!

The simulated controllers are now changing according to keyboard-events:

• r ... enable ringing
• R ... disable ringing
• m ... let motor run for 100 ms
• M ... stop motor let motor run for max time (255 ms)
• v ... increase remaining battery by 10 %
• V ... decrease remaining battery by 10 %
• c ... battery is charging
• C ... battery is not charging

[Avamander]

Just wondering, could this be on a separate repository but with InfiniTime as the submodule?

[NeroBurner]

@Avamander Yes it is possible and @JF002 had the same question as well and my answer is here #743 (comment)

Yes, that would be no problem. The disadvantage of an extra repo is the decoupling. If a PR breaks some assumption the simulator is using, the breakage is checked after the merge, making the simulator harder to maintain (because I (or other simulator maintainer) would have to fix the build afterwards)
The greatest advantage for the simulator (when integrated) is the possibility of checking the simulator for each PR with the CI. I don't know of a way to do that with an extra simulator-repo

[NeroBurner]

There shall be FlashLight.h!

The four white dots on the top right of the status window show the backlight status (off, low, medium, high) and can be changed with l and L. Although the backlight has no effect on the simulator other than the status dots.

and with one click it is dark

[NeroBurner]

I got the first watchface working!!! 🎉 See WatchFaceDigital.h

[NeroBurner]

rebased the lvgl-8.0 branch (and this lv_simulation branch) on the current develop branch. In the process got the update for the FlashLight.hpp Screen:

[NeroBurner]

Now all three WatchFaces are working. Introducing WatchFaceAnalog.h and PineTimeStyle.h

[NeroBurner]

two screens added to the simulated screen list: SettingSetDate.h and SettingSetTime.h, and with reenabling some SystemTask.cpp code the date and time setting actually stick (as long as the simulator runs, on start the simulator time is initialized with the system-time)

[NeroBurner]

Added an overlay if the screen is turned off (brightness-level is Off). Would be confusing to have an enabled screen, but input is just ignored. So let the user know!

[NeroBurner]

We can now use SystemTask.cpp from src instead of the modified simulated copy 🎉

[NeroBurner]

Finally can read my simulated Notifications!

[NeroBurner]

The wait for the FirmwareUpdater.h Screen is over!!! Although waiting for the actual (simulated) update to happen... hmmm... still waiting

Accessible currently with the screen 7

[NeroBurner]

That must be one of the most sophisticated passkey ever existed... how do they know my super secure password! o.O

And with PassKey.h, we got all the screens in the simulator, that are listed/used in DisplayApp.h, AND we're using the main DisplayApp.h and cpp files (as well as the main SystemTask.h+cpp) 🎉

[NeroBurner]

I like it when the firmware is validated, and with it the simulator is validated, isn't it? 😁

[NeroBurner]

rebased on current develop branch with 24 hour SettingSetTime.h and Alarm.h

Renamed the resulting executable to pinesim. I hope you like the new name, the old main was a bit boring

And we now have a workflow for lv_sim, which also provides a working pinesim executable for linux

[NeroBurner]

you can now increase your simulated steps away from the immovable 0000. Increase steps by 500 by pressing s, and decrease with S (shift+s)

Now after SO many steps, I bet your heartrate is high, let it be as high as you want! Increase heartrate by 10 by pressing h and stop the heartrate with H (shift+h)

[JF002]

As I said previously : I'm astonished by the amount of work you put in this simulator : CMake, simulator code, reuse of InfiniTime code, stub of hardware depend code, everything is there! There have been previous attempt at a simulator for InfiniTime (I have started one myself), but this one is the most advanced and complete 💯 !

This simulator will be very helpful to anyone wishing to design new UIs and apps in InfiniTime, as it allows to test and debug directly on the computer, without needed to flash the PineTime for each test!

In addition, this simulator paves the way for porting InfiniTime to another platform : all the hardware dependent code have already been identified! Now, we "just" need to encapsulate this code correctly so we can easily run InfiniTime on other platform or MCU. One of my target being the BL60x on the PineDio Stack (future dev kit from Pine64).

One think I would like to improve before merging is the integration of the simulator in the project. For now, the simulator is a project inside another project, with no real glue between the two : you cannot build the simulator from the main CMake file. Also, the toolchain and dependencies of the simulator are very different from the one needed by InfiniTime (native vs ARM toolchain, SDL drivers for LVGL, hardware for InfiniTime,...
So I was wondering if we should keep it that way, and try to improve the integration of both projects to ensure the dependcies and toolchains are correctly managed by CMake.
Or maybe we should split both project : keep InfiniTime as it is right now, and create a whole new project for the simulator. This new project would include InfiniTime as a git submodule, for example.

This 2nd solution has several advantages:

• Better separation between the 2 projects, no confusion with dependencies and toolchains
• The git history are separated
• PR for the simulator are also separated from the ones for InfniTime, which will probably allow for faster and easier merges.
• The simulator can specify which version of InfiniTime it pulls (via git submodule) so that a new change in InfiniTime won't break the simulator. This will allow the devs of the simulator to include those new changes when they are ready to support them in the simulator

Of course, there are also a few disadvantages

• 2 projects to maintain instead of one
• Keeping both project in sync will require a bit more rigore

I talked about this with @NeroBurner and we agree that splitting looks like a better idea. We will probably create the new project in the InfiniTime organization very soon !

[JF002]

Here are a few more info about how both project will work together:

As I said, InfiniTime repo will only contain the InfiniTime code, the code that will be flashed into you PineTime. The new repo will contain the code specific to the simulator and will include the whole InfiniTime project as a git submodule.

As I said earlier, this will make the maintenance of both projects easier. As you've probably noticed, it takes quite a long time for PRs to be reviewed and merged. That's partly because changes in InfiniTime need extensive testing and reviews to ensure good code quality and that those changes will actually run on the watch (while avoiding crashes, memory issues,...). If the simulator was included in InfiniTime, the time an efforts needed to merge a PR would increase : we'll need to check that the simulator still work with those new changes, otherwise, the simulator would be broken for everyone on develop.

Splitting allows to reduce the coupling between those projects: there'll be maintainers more dedicated to InfiniTime and others dedicated to the simulator. The simulator will include a specific version (commit) of InfiniTime, so that breaking changes in Infinitime won't be applied to the simulator until developers had the time to adapt to those changes. PR in the simulator will also be easier to merge as there are less constraints in code quality and testing for the simulator.

As I have mentionned earlier, the simulator helps to see what exactly needs to be done in InfiniTime to better abstract the hardware. In the future, I would like to improve those abstractions in InfiniTime so the simulator does not need to stub that many modules from InfiniTime. This will also help porting to other MCUs and writiing unit tests for both projects.

Now, how will you use those projects? If you want to develop on InfiniTime, nothing changes : clone, checkout, edit, build, test, commit, push,...
If you want to use InfiniTime with the simulator, just checkout the simulator and init/update the git submodule. It'll download the lastest version of InfiniTime that is know to work with the simulator. Then build, run and enjoy.
If you want to run the simulator against another version of InfiniTime, just checkout that version in the subfolder that contains the submodule, build and run!
If you want to create new apps for InfiniTime in the simulator, you can edit InfiniTime inside the git submodule folder (create a branch first), or point the simulator to the folder that contains your working copy of InfiniTime (via a CMake command line argument).

Of course, we will also probably create new Github Actions to build the simulator. It might also be possible to trigger an action that builds the simulator each time a new commit is pushed on InfiniTime. This way, we'll directly know if this commit breaks the simulator.

Note that the separate project does not exist yet, but @NeroBurner accepted to join the InfiniTime organization to work on and maintain the simulator! Welcome to the team, @NeroBurner !

[NeroBurner]

Thank you for your kind words. I'm happy to be part of such an awesome project.

---

Successfully created https://github.com/InfiniTimeOrg/InfiniSim project 🎉

the checked in submodule points to this branch because of some fixes needed to build the simulator

---

On this side the CI now downloads the InfiniSim project to compile the infinisim binary for all PRs in the future, to ensure that the simulator stays working in the future

[NeroBurner]

A small overview of the related PRs important for this PR.

This PR contains the commits of other PRs fixing various things to make the Simulator work:

• SystemMonitor: implement FreeRtosMonitor only if trace facility is set #967
• SystemTask: remove unused ble includes #975
• Notifications: replace newlines in label-copy instead of const char* title directly #976
• Notifications: use motorController object instead of class function #977
• Provide reference to BrightnessController in DisplayApp #979
• SettingShakeThreshold: add missing SystemTask.h, relative include #995
• SystemTask: forward declare BatteryController to fix of cyclic dependency #996
• Remove unused and not compiling DropDownDemo #1000

Then the lv_sim: CI: add lv_sim workflow and upload 'infinisim' executable is the final commit, which is very important to help keep the InfiniSim project in sync with the main InfiniTime project

[JF002]

@NeroBurner I've just merged the last PR that was needed for the simulator. I think we can close this issue now that InfiniSim has its own project :)

[NeroBurner]

Not yet, this branch contains the commit to compile the InfiniSim simulator on each pull request, which would be great to have to be able to keep compatibility between InfiniTime and InfiniSim. I'll cleanup and rebase this branch right now