Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize torrent/peers list reading and writing by breaking up the tracker.torrents field into smaller pieces #495

Closed
mickvandijke opened this issue Oct 25, 2023 · 3 comments
Closed

Optimize torrent/peers list reading and writing by breaking up the tracker.torrents field into smaller pieces #495

mickvandijke opened this issue Oct 25, 2023 · 3 comments
Assignees
@mickvandijke
Labels
Code Cleanup / Refactoring Tidying and Making Neat Optimization Make it Faster
Milestone
v3.0.0

Comments

@mickvandijke
Copy link
Member

mickvandijke commented Oct 25, 2023 •
edited
Loading

We were talking about refactoring the torrents map in the Tracker struct to reduce the amount of write locks needed on the entire torrents map. This would theoretically improve torrent announce performance when you have a lot of torrents being updated at the same time.

Refactor:

torrents: RwLock<std::collections::BTreeMap<InfoHash, torrent::Entry>>,

To:

torrents: RwLock<std::collections::BTreeMap<InfoHash, Arc<Mutex<torrent::Entry>>>>,
@mickvandijke mickvandijke added Code Cleanup / Refactoring Tidying and Making Neat Optimization Make it Faster labels Oct 25, 2023
@mickvandijke mickvandijke self-assigned this Oct 25, 2023
@josecelano
Copy link
Member

josecelano commented Oct 25, 2023 •
edited
Loading

Hi @WarmBeer I would suggest taking a look at @Power2All's implementation here and also at this discussion.

@mickvandijke
Copy link
Member Author

Relevant discussion: #504

@mickvandijke mickvandijke mentioned this issue Nov 24, 2023
Merged
josecelano added a commit that referenced this issue Dec 21, 2023
@josecelano
Merge #522: feat: added benchmarking binary for torrent repository st…
20b052d 
…ruct

ebb7d4c chore: make Containerfile use nightly rust (Warm Beer)
1735a7a chore: only run contract, deployment & testing jobs in nightly rust (Warm Beer)
6087e4f feat: added benchmarking binary for torrent repository (Warm Beer)

Pull request description:

  Moved the `Tracker.torrents` field to be its own `TorrentRepository` struct with different implementations.

  Added a new crate that benchmarks the different `TorrentRepository` implementations for speed.

  ### Run benchmarks
  ```shell
  cargo run --release -p torrust-torrent-repository-benchmarks -- --threads 4 --sleep 0 --compare true
  ```

  #### Example result
  ```shell
  tokio::sync::RwLock<std::collections::BTreeMap<InfoHash, Entry>>
  add_one_torrent: Avg/AdjAvg: (146ns, 0ns)
  update_one_torrent_in_parallel: Avg/AdjAvg: (5.965616ms, 5.816375ms)
  add_multiple_torrents_in_parallel: Avg/AdjAvg: (14.050554ms, 14.132902ms)
  update_multiple_torrents_in_parallel: Avg/AdjAvg: (7.201337ms, 7.120315ms)

  std::sync::RwLock<std::collections::BTreeMap<InfoHash, Entry>>
  add_one_torrent: Avg/AdjAvg: (82ns, 83ns)
  update_one_torrent_in_parallel: Avg/AdjAvg: (27.311075ms, 26.869114ms)
  add_multiple_torrents_in_parallel: Avg/AdjAvg: (28.967141ms, 28.967141ms)
  update_multiple_torrents_in_parallel: Avg/AdjAvg: (48.316966ms, 0ns)

  std::sync::RwLock<std::collections::BTreeMap<InfoHash, Arc<std::sync::Mutex<Entry>>>>
  add_one_torrent: Avg/AdjAvg: (137ns, 125ns)
  update_one_torrent_in_parallel: Avg/AdjAvg: (5.057729ms, 5.057729ms)
  add_multiple_torrents_in_parallel: Avg/AdjAvg: (48.833962ms, 48.833962ms)
  update_multiple_torrents_in_parallel: Avg/AdjAvg: (6.193212ms, 6.071166ms)

  tokio::sync::RwLock<std::collections::BTreeMap<InfoHash, Arc<std::sync::Mutex<Entry>>>>
  add_one_torrent: Avg/AdjAvg: (174ns, 166ns)
  update_one_torrent_in_parallel: Avg/AdjAvg: (5.56332ms, 5.56332ms)
  add_multiple_torrents_in_parallel: Avg/AdjAvg: (16.360504ms, 15.872786ms)
  update_multiple_torrents_in_parallel: Avg/AdjAvg: (6.800225ms, 6.890521ms)

  tokio::sync::RwLock<std::collections::BTreeMap<InfoHash, Arc<tokio::sync::Mutex<Entry>>>>
  add_one_torrent: Avg/AdjAvg: (192ns, 208ns)
  update_one_torrent_in_parallel: Avg/AdjAvg: (6.374304ms, 6.17711ms)
  add_multiple_torrents_in_parallel: Avg/AdjAvg: (17.313591ms, 17.089898ms)
  update_multiple_torrents_in_parallel: Avg/AdjAvg: (6.955371ms, 6.975057ms)

  ```

  ### Relevant issues
  #496
  #495

Top commit has no ACKs.

Tree-SHA512: 062d18c91b1d89ab95eda5ddc2cb7674d481b705a21c43e9cf3a11e0bf71e52bb6bccd5b5e4404fcf8ebdc442764dd60006c6962aac2a414c4b4cdecfa422a67
@cgbosse cgbosse moved this to Maintenance in Torrust Solution Jan 8, 2024
@cgbosse cgbosse moved this from Maintenance to In Progress in Torrust Solution Jan 12, 2024
@cgbosse cgbosse added this to the v3.0.0 milestone Jan 12, 2024
@mickvandijke
Copy link
Member Author

Superseded by #565

@github-project-automation github-project-automation bot moved this from In Progress to Done in Torrust Solution Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mickvandijke mickvandijke

Labels
Code Cleanup / Refactoring Tidying and Making Neat Optimization Make it Faster
Projects
Torrust Solution
Status: Done
Milestone
v3.0.0
Development

No branches or pull requests
3 participants
@josecelano @mickvandijke @cgbosse