Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add synchronization to start of audio recordings #1984

Merged
merged 34 commits into from
May 1, 2023

Conversation

OmLanke
Copy link
Contributor

@OmLanke OmLanke commented Mar 24, 2023

Summary

When a Voice Channel is being recorded, users' recordings start when they start speaking, and not when the recording was actually started. This PR adds silence at the start of the recordings of users who weren't speaking when the recording started, but started speaking later.

This however relies on the receipt timestamp of the first packet, which can sometimes cause slightly inaccurate synchronisation, based on the network conditions.

This PR solves #1980.

Information

  • This PR fixes an issue.
  • This PR adds something new (e.g. new method or parameters).
  • This PR is a breaking change (e.g. methods or parameters removed/renamed).
  • This PR is not a code change (e.g. documentation, README, typehinting,
    examples, ...).

Checklist

  • I have searched the open pull requests for duplicates.
  • If code changes were made then they have been tested.
    • I have updated the documentation to reflect the changes.
  • If type: ignore comments were used, a comment is also left explaining why.
  • I have updated the changelog to include these changes.

@VincentRPS VincentRPS linked an issue Mar 24, 2023 that may be closed by this pull request
@OmLanke
Copy link
Contributor Author

OmLanke commented Mar 24, 2023

I'll add an example for this too. For now, this is enough to test the merging of audio recordings-

import asyncio
import io

from pydub import AudioSegment # pip install pydub

import discord
from discord.ext import commands
from discord.sinks import MP3Sink


class MyCog(commands.Cog):
    def __init__(self, bot: commands.Bot):
        self.bot = bot

    async def finished_callback(
        self,
        sink: MP3Sink,
        channel: discord.TextChannel,
    ):
        mention_strs = []
        audio_segs: list[AudioSegment] = []
        files: list[discord.File] = []

        longest = AudioSegment.empty()

        for user_id, audio in sink.audio_data.items():

            mention_strs.append(f"<@{user_id}>")

            seg = AudioSegment.from_file(audio.file, format="mp3")

            # Determine the longest audio segment
            if len(seg) > len(longest):
                audio_segs.append(longest)
                longest = seg
            else:
                audio_segs.append(seg)

            audio.file.seek(0)
            files.append(discord.File(audio.file, filename=f"{user_id}.mp3"))

        for seg in audio_segs:
            longest = longest.overlay(seg)

        with io.BytesIO() as f:
            longest.export(f, format="mp3")
            await channel.send(
                f"Finished! Recorded audio for {', '.join(mention_strs)}.",
                files=files + [discord.File(f, filename="recording.mp3")],
            )

    @commands.command()
    async def join(self, ctx: commands.Context):
        await ctx.author.voice.channel.connect()

    @commands.command()
    async def record(self, ctx: commands.Context):
        vc: discord.VoiceClient = ctx.voice_client

        await ctx.send("Recording...")

        vc.start_recording(
            discord.sinks.MP3Sink(),
            self.finished_callback,
            ctx.channel,
            sync_start=True,
        )

        await asyncio.sleep(15) # edit to change recording duration
        vc.stop_recording()
        await ctx.send("Stopped recording.")


def setup(bot: commands.Bot):
    bot.add_cog(MyCog(bot))

@codecov
Copy link

codecov bot commented Mar 24, 2023

Codecov Report

Merging #1984 (ecd3177) into master (4675c6c) will decrease coverage by 0.02%.
The diff coverage is 10.52%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1984      +/-   ##
==========================================
- Coverage   33.27%   33.25%   -0.02%     
==========================================
  Files          97       97              
  Lines       19017    19027      +10     
==========================================
  Hits         6328     6328              
- Misses      12689    12699      +10     
Flag Coverage Δ
macos-latest-3.10 33.24% <10.52%> (-0.02%) ⬇️
macos-latest-3.11 33.24% <10.52%> (-0.02%) ⬇️
macos-latest-3.8 33.25% <10.52%> (-0.02%) ⬇️
macos-latest-3.9 33.25% <10.52%> (-0.02%) ⬇️
ubuntu-latest-3.10 33.24% <10.52%> (-0.02%) ⬇️
ubuntu-latest-3.11 33.24% <10.52%> (-0.02%) ⬇️
ubuntu-latest-3.8 33.25% <10.52%> (-0.02%) ⬇️
ubuntu-latest-3.9 33.25% <10.52%> (-0.02%) ⬇️
windows-latest-3.10 33.24% <10.52%> (-0.02%) ⬇️
windows-latest-3.11 33.24% <10.52%> (-0.02%) ⬇️
windows-latest-3.8 33.25% <10.52%> (-0.02%) ⬇️
windows-latest-3.9 33.25% <10.52%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
discord/sinks/core.py 40.00% <0.00%> (-0.43%) ⬇️
discord/voice_client.py 20.89% <11.11%> (-0.46%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4675c6c...ecd3177. Read the comment docs.

OmLanke and others added 4 commits March 24, 2023 20:06
@OmLanke OmLanke marked this pull request as ready for review March 24, 2023 15:42
@OmLanke OmLanke requested a review from a team as a code owner March 24, 2023 15:42
@OmLanke OmLanke changed the title feat: Add synchronisation of start of audio recordings feat: Add synchronisation to start of audio recordings Mar 24, 2023
OmLanke and others added 4 commits March 25, 2023 19:01
Signed-off-by: Om <92863779+Om1609@users.noreply.github.com>
Signed-off-by: Om <92863779+Om1609@users.noreply.github.com>
@JustaSqu1d JustaSqu1d added status: awaiting review Awaiting review from a maintainer feature Implements a feature undocumented discord feature This isn't documented, might not receive support for it! labels Apr 2, 2023
@JustaSqu1d JustaSqu1d requested a review from Lulalaby April 2, 2023 02:26
@Lulalaby
Copy link
Member

Lulalaby commented Apr 2, 2023

Sigh

@Lulalaby Lulalaby added the priority: medium Medium Priority label Apr 2, 2023
discord/voice_client.py Outdated Show resolved Hide resolved
examples/audio_recording_merged.py Outdated Show resolved Hide resolved
Co-authored-by: JustaSqu1d <overenchanted.gaming@gmail.com>
Signed-off-by: Om <92863779+Om1609@users.noreply.github.com>
@plun1331
Copy link
Member

@Lulalaby any updates?

@OmLanke
Copy link
Contributor Author

OmLanke commented Apr 20, 2023

@Lulalaby any updates?

Check this message by Lala in the discord server

@BobDotCom
Copy link
Member

Please fix merge conflicts

Lulalaby
Lulalaby previously approved these changes Apr 29, 2023
@Lulalaby Lulalaby requested review from JustaSqu1d and VincentRPS and removed request for VincentRPS and JustaSqu1d April 29, 2023 08:04
@Lulalaby
Copy link
Member

i rly had a stroke rn lol

@VincentRPS VincentRPS changed the title feat: Add synchronisation to start of audio recordings feat: Add synchronization to start of audio recordings Apr 29, 2023
@Lulalaby
Copy link
Member

Lulalaby commented May 1, 2023

@plun1331 ping pong

@Lulalaby Lulalaby enabled auto-merge (squash) May 1, 2023 17:24
@Lulalaby Lulalaby merged commit 7a6a42c into Pycord-Development:master May 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Implements a feature priority: medium Medium Priority status: awaiting review Awaiting review from a maintainer undocumented discord feature This isn't documented, might not receive support for it!
Projects
None yet
6 participants