Customizations for running the miner as a child process in a desktop environment. #21
Conversation
| let mut builder = env_logger::builder(); | ||
| builder.filter_level(opt.log_level()).parse_default_env(); | ||
| if opt.altlogs { | ||
| builder.format(|buf, record| { | ||
| let timestamp = Local::now().format("%Y-%m-%d %H:%M:%S%.3f%:z"); | ||
| writeln!(buf, "{} [{:>5}] {}", timestamp, record.level(), record.args()) | ||
| }); | ||
| } | ||
| builder.init(); |
There was a problem hiding this comment.
As you can see in the screenshot below, the desktop application runs kaspad and the miner and sanitizes logs, removing timestamps, which are not needed on the desktop. It is easier for me to use the same filtering function (+ the native logs have variable prefix lengths, and I wanted fixed length not to do any additional processing). I did not want to change dependencies to match kaspad, so I added a custom formatting function.
There was a problem hiding this comment.
note where it says: launching 1 cpu miners, Found a block.. etc
Cargo.toml
Outdated
| [target.'cfg(target_os = "windows")'.dependencies] | ||
| keccak = "0.1" | ||
| # [target.'cfg(target_os = "windows")'.dependencies] | ||
| # keccak = "0.1" |
There was a problem hiding this comment.
The right thing here is better feature gating over x86_64 (and consider adding an aarch64 implementation)
There was a problem hiding this comment.
Do you want me to create special conditions for aarch64? The complexity of cfg() gates was a bit unclear, so I opted to go the path of "enable on all, unless". The crate, if not used, should be optimized away in --release mode. Otherwise Cargo.toml has to contain enable for windows and enable for macos where target arch == aarch64... I felt that such complexity is better applied directly at the ASM import gates in the rust code, where it just loads wasm if needed, leaving toml alone...
|
Master has been merged into this PR. |
|
The speed at which this has been handled is not acceptable. I will be forking the project. |
|
I will rebase + squash this and merge, as I don't want the merge commits here modifying the history |
|
|
||
| [features] | ||
| default = ["keccak?/asm"] | ||
| parking_lot = ["parking", "tokio/parking_lot"] | ||
| bench = [] | ||
| no-asm = ["keccak"] | ||
| no-asm = [] |
| if let Some(sleep_duration) = throttle { | ||
| sleep(sleep_duration) | ||
| } |
There was a problem hiding this comment.
This is wrong, it only sleeps after you find a block
The following set of changes is needed for the Kaspa desktop application stack we are currently developing (aiming for a beta during the
testnet-11relaunch).The changes are mainly cosmetic, isolated by the newly introduced flags:
--throttle <millis>introduces sleep between each mining round.--altlogsresults in log output in the same format as kaspad.In addition, the project was not building on
aarch64(M1/M2), so I have removedoptionalfrom the externalkeccakcrate dependency. Ifasmis used, the library's presence should be benign and optimized away, allowing platform and feature control directly from within the rust code. (I left originals as comments).The purpose of the above changes is to make the miner "passive" in the execution environment, with the least impact on the CPU while still being able to mine on the testnet. @elichai if you find that there is a better (more precise way) to throttle the miner, please let me know or implement it. Ultimately, for
testnet-11we need to be able to throttle it from low (1%) to 100% of the CPU usage (albeit in the desktop application, both kaspad and the miner are run side-by-side). Thread count and the throttle parameters will be controllable by the user (probably with some upper cap). The desktop app has settings controls, so when those are changed, the miner will be automatically restarted.Alternative logs are needed for easier sanitizing and user display (using the same format makes it easier to parse both kaspad and the miner logs). I really didn't want to touch the original code, so I just added a small formatting function triggered by the flag. If you want this to make a permanent change, we can remove the flag.
Performance is not relevant here. This is used for testing only. Currently running with
1thread and5 msecthrottling works well.Unfortunately, I am unable to do any kind of core affinity or thread priority control due to this needing to be uniform and compatible with all operating systems (Windows, MacOS, Linux). I might look at introducing this later at the parent process level. That said, the throttle approach makes the miner "fire in bursts"... but I don't think there is an easy way around that.
Please review and fast-track the PR for publishing on crates.io as we are currently integrating from my fork, which requires additional steps for any developer interested in building the desktop stack. Once these changes are on crates.io, it will simplify the process, removing the requirement for having 2 repos (kaspad and the miner).
Down the road, I aim to make another PR that will expose this crate as a lib, so that it can be usable for integration testing. Throttling will also help with this by ensuring that the miner acts passively and doesn't consume resources. (The integration testing is needed for powering up a temporary network and issuing transactions as well as monitoring node activity using a variety of RPC calls from WASM libraries).