Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support serenity simd_json #105

Merged
merged 21 commits into from
Jul 25, 2022
Merged

Support serenity simd_json #105

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
ae69e5e
Deps: Depend on Serenity's `next` branch on Songbird's `next` branch …
arqunis Nov 15, 2020
3784490
Deps: Bump streamcatcher version -> 1.0 (#93)
tktcorporation Sep 22, 2021
94bd290
Examples: Fix serenity-next cache accesses (#99)
FelixMcFelix Oct 10, 2021
2129422
Gateway: Add generics to `Call` methods. (#102)
FelixMcFelix Oct 19, 2021
7956792
Gateway: Remove lifetime from Serenity setup trait (#103)
FelixMcFelix Nov 5, 2021
89de7a1
Support serenity simd_json
vicky5124 Dec 6, 2021
c16c17c
Missed a place
vicky5124 Dec 6, 2021
45fb51b
fmt
vicky5124 Dec 6, 2021
ce95d4b
Fully replace everything to serenity::json
vicky5124 Dec 7, 2021
3c83316
Fully replace serde_json with serenity::json
vicky5124 Dec 7, 2021
ec7d029
fmt
vicky5124 Dec 7, 2021
ab64728
Updated serenity to next from path
vicky5124 Dec 7, 2021
c749f2a
Remove simdjson features
vicky5124 Dec 7, 2021
86feb99
document simdjson feature
vicky5124 Dec 7, 2021
889cbed
Add simdjson to tests
vicky5124 Dec 7, 2021
8ca77e1
Add serenity's simdjson to test
vicky5124 Dec 7, 2021
4b7e9e9
Fix SIMD compilation
vicky5124 Dec 10, 2021
91d4c18
Merge remote-tracking branch 'upstream/next' into simd_json
FelixMcFelix Jul 24, 2022
1fe3edf
Better rustflag handling, and use our own simdjson (not just serenity)
FelixMcFelix Jul 24, 2022
f28fc85
Migrate metadata handling, all simd_json uses to crate::json
FelixMcFelix Jul 25, 2022
2d85f38
Attempt 2 at CI.
FelixMcFelix Jul 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .cargo/config.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[build]
rustflags = ["-C", "target-cpu=haswell"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not native?

Can this also be moved to CI such that it's only invoked for the simd-json build task? I'm unsure if the spurious MacOS SIGILL originates from this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, unless you do CI with Arm, this target-cpu will work just fine.
Haswell was chosen over native as I've seen a few people mention it's less likely for it to fail CI due to "Illegal Instruction"; MacOS failed because there's still a chance it can fail from that, serenity has had this issue for quite a while now, and it's on Actions for it to be fixed.

Moving this to only the SIMD check would make it more likely to pass, and I'll do it when I have time for it, but it's hard to test CI when it's blocked behind it being accepted due to this being my first PR.

3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ jobs:
- name: legacy tokio
features: serenity-rustls-tokio-02 driver-tokio-02
dont-test: true
- name: simd json
features: simdjson serenity-rustls driver gateway serenity/simdjson
dont-test: true

steps:
- name: Checkout sources
Expand Down
10 changes: 7 additions & 3 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,21 +74,23 @@ version = "0.8"

[dependencies.serenity]
optional = true
version = "^0.10.2"
default-features = false
features = ["voice", "gateway"]
git = "https://github.com/serenity-rs/serenity"
branch = "next"

[dependencies.serenity-voice-model]
optional = true
version = "0.1"
git = "https://github.com/serenity-rs/serenity"
branch = "next"

[dependencies.spin_sleep]
optional = true
version = "1"

[dependencies.streamcatcher]
optional = true
version = "0.1"
version = "1"

[dependencies.tokio]
optional = true
Expand Down Expand Up @@ -190,6 +192,8 @@ zlib-simd = ["twilight-gateway/zlib-simd"]
zlib-stock = ["twilight-gateway/zlib-stock"]
serenity-deps = ["async-trait"]

simdjson = []

rustls-marker = []
native-marker = []

Expand Down
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ The library offers:
* And, by default, a fully featured voice system featuring events, queues, RT(C)P packet
handling, seeking on compatible streams, shared multithreaded audio stream caches,
and direct Opus data passthrough from DCA files.
* To be able to use `simd-json` from serenity, you will need to enable the `simdjson`
feature on both songbird and serenity.

## Intents
Songbird's gateway functionality requires you to specify the `GUILD_VOICE_STATES` intent.
Expand Down
3 changes: 2 additions & 1 deletion examples/serenity/voice/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ tracing-futures = "0.2"
path = "../../../"

[dependencies.serenity]
version = "0.10"
features = ["client", "standard_framework", "voice", "rustls_backend"]
git = "https://github.com/serenity-rs/serenity"
branch = "next"

[dependencies.tokio]
version = "1.0"
Expand Down
14 changes: 7 additions & 7 deletions examples/serenity/voice/src/main.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ async fn main() {
#[command]
#[only_in(guilds)]
async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand Down Expand Up @@ -107,7 +107,7 @@ async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn join(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let channel_id = guild
Expand All @@ -134,7 +134,7 @@ async fn join(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand All @@ -157,7 +157,7 @@ async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand Down Expand Up @@ -212,7 +212,7 @@ async fn play(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
return Ok(());
}

let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand Down Expand Up @@ -245,7 +245,7 @@ async fn play(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand All @@ -268,7 +268,7 @@ async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn unmute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand Down
3 changes: 2 additions & 1 deletion examples/serenity/voice_events_queue/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ features = ["builtin-queue"]
path = "../../../"

[dependencies.serenity]
version = "0.10"
features = ["cache", "standard_framework", "voice", "rustls_backend"]
git = "https://github.com/serenity-rs/serenity"
branch = "next"

[dependencies.tokio]
version = "1.0"
Expand Down
20 changes: 10 additions & 10 deletions examples/serenity/voice_events_queue/src/main.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ async fn main() {

#[command]
async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx)
Expand Down Expand Up @@ -126,7 +126,7 @@ async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn join(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let channel_id = guild
Expand Down Expand Up @@ -241,7 +241,7 @@ impl VoiceEventHandler for ChannelDurationNotifier {
#[command]
#[only_in(guilds)]
async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx)
Expand Down Expand Up @@ -270,7 +270,7 @@ async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx)
Expand Down Expand Up @@ -339,7 +339,7 @@ async fn play_fade(ctx: &Context, msg: &Message, mut args: Args) -> CommandResul
return Ok(());
}

let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx)
Expand Down Expand Up @@ -470,7 +470,7 @@ async fn queue(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
return Ok(());
}

let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx)
Expand Down Expand Up @@ -518,7 +518,7 @@ async fn queue(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn skip(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx)
Expand Down Expand Up @@ -553,7 +553,7 @@ async fn skip(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn stop(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx)
Expand Down Expand Up @@ -581,7 +581,7 @@ async fn stop(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx)
Expand Down Expand Up @@ -614,7 +614,7 @@ async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn unmute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx)
.await
Expand Down
3 changes: 2 additions & 1 deletion examples/serenity/voice_receive/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ tracing-futures = "0.2"
path = "../../../"

[dependencies.serenity]
version = "0.10"
features = ["client", "standard_framework", "voice", "rustls_backend"]
git = "https://github.com/serenity-rs/serenity"
branch = "next"

[dependencies.tokio]
version = "1.0"
Expand Down
4 changes: 2 additions & 2 deletions examples/serenity/voice_receive/src/main.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ async fn join(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
},
};

let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand Down Expand Up @@ -245,7 +245,7 @@ async fn join(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand Down
3 changes: 2 additions & 1 deletion examples/serenity/voice_storage/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ tracing-futures = "0.2"
path = "../../../"

[dependencies.serenity]
version = "0.10"
features = ["cache", "framework", "standard_framework", "voice", "http", "rustls_backend"]
git = "https://github.com/serenity-rs/serenity"
branch = "next"

[dependencies.tokio]
version = "1.0"
Expand Down
14 changes: 7 additions & 7 deletions examples/serenity/voice_storage/src/main.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ async fn main() {
#[command]
#[only_in(guilds)]
async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand Down Expand Up @@ -184,7 +184,7 @@ async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn join(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let channel_id = guild
Expand Down Expand Up @@ -262,7 +262,7 @@ impl VoiceEventHandler for LoopPlaySound {
#[command]
#[only_in(guilds)]
async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand All @@ -285,7 +285,7 @@ async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand Down Expand Up @@ -318,7 +318,7 @@ async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn ting(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand All @@ -344,7 +344,7 @@ async fn ting(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;

let manager = songbird::get(ctx).await
Expand All @@ -368,7 +368,7 @@ async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command]
#[only_in(guilds)]
async fn unmute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).await.unwrap();
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await
.expect("Songbird Voice client placed in at initialisation.").clone();
Expand Down
4 changes: 3 additions & 1 deletion src/driver/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,14 @@ pub use crypto::CryptoMode;
pub(crate) use crypto::CryptoState;
pub use decode_mode::DecodeMode;

#[cfg(feature = "builtin-queue")]
use crate::tracks;
#[cfg(feature = "builtin-queue")]
use crate::tracks::TrackQueue;
use crate::{
events::EventData,
input::Input,
tracks::{self, Track, TrackHandle},
tracks::{Track, TrackHandle},
Config,
ConnectionInfo,
Event,
Expand Down
Loading