Skip to content

Commit

Permalink
Reimplement DECPS using DirectSound in place of MIDI (#13471)
Browse files Browse the repository at this point in the history
## Summary of the Pull Request

The original `DECPS` implementation made use of the Windows MIDI APIs to
generate the sound, but that required a 3MB package dependency for the
GS wavetable DLS. This PR reimplements the `MidiAudio` class using
`DirectSound`, so we can avoid that dependency.

## References

The original `DECPS` implementation was added in PR #13208, but was
hidden behind a velocity flag in #13258.

## PR Checklist
* [x] Closes #13252
* [x] CLA signed.
* [ ] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [x] I've discussed this with core contributors already. Issue number
where discussion took place: #13252

## Detailed Description of the Pull Request / Additional comments

The way it works is by creating a sound buffer with a single triangle
wave that is played in a loop. We generate different notes simply by
adjusting the frequency at which that buffer is played.

When we need a note to end, we just set the volume to its minimum value
rather than stopping the buffer. If we don't do that, the repeated
starting and stopping tends to produce a lot of static in the output. We
also use two buffers, which we alternate between notes, as another way
to reduce that static.

One other thing worth mentioning is the handling of the buffer position.
At the end of each note we save the current position, and then use an
offset from that position when starting the following note. This helps
produce a clearer separation between tones when repeating sequences of
the same note.

In an ideal world, we should really have something like an attack-decay-
sustain-release envelope for each note, but the above hack seems to work
reasonably well, and keeps the implementation simple.

## Validation Steps Performed

I've manually tested both conhost and Terminal with the sample tunes
listed in issue #8687, as well as a couple of games that I have which
make use of `DECPS` sound effects.

(cherry picked from commit bc79867)
Service-Card-Id: 84270205
Service-Version: 1.15
  • Loading branch information
j4james authored and DHowett committed Jul 19, 2022
1 parent a4a12ef commit b8847a3
Show file tree
Hide file tree
Showing 7 changed files with 108 additions and 71 deletions.
14 changes: 14 additions & 0 deletions .github/actions/spelling/expect/expect.txt
Original file line number Diff line number Diff line change
Expand Up @@ -438,7 +438,9 @@ ctlseqs
Ctlv
ctor
CTRLEVENT
CTRLFREQUENCY
CTRLKEYSHORTCUTS
CTRLVOLUME
Ctx
Ctxt
ctype
Expand Down Expand Up @@ -661,7 +663,14 @@ dropdown
DROPDOWNLIST
DROPFILES
drv
DSBCAPS
DSBLOCK
DSBPLAY
DSBUFFERDESC
DSBVOLUME
dsm
dsound
DSSCL
dst
DSwap
DTest
Expand All @@ -682,6 +691,7 @@ dwrite
dwriteglyphrundescriptionclustermap
dxgi
dxgidwm
dxguid
dxinterop
dxp
dxsm
Expand Down Expand Up @@ -718,6 +728,7 @@ endptr
endregion
ENQ
enqueuing
ENTIREBUFFER
entrypoint
ENU
enum
Expand Down Expand Up @@ -934,6 +945,7 @@ gitfilters
github
gitlab
gle
GLOBALFOCUS
globals
GLYPHENTRY
gmail
Expand Down Expand Up @@ -1369,6 +1381,7 @@ lpv
LPVOID
LPW
LPWCH
lpwfx
LPWINDOWPOS
lpwpos
lpwstr
Expand Down Expand Up @@ -2678,6 +2691,7 @@ WANTARROWS
WANTTAB
wapproj
wav
WAVEFORMATEX
wbuilder
wch
wchar
Expand Down
131 changes: 79 additions & 52 deletions src/audio/midi/MidiAudio.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,63 +5,38 @@
#include "MidiAudio.hpp"
#include "../terminal/parser/stateMachine.hpp"

namespace
{
class MidiOut
{
public:
static constexpr auto NOTE_OFF = 0x80;
static constexpr auto NOTE_ON = 0x90;
static constexpr auto PROGRAM_CHANGE = 0xC0;
#include <dsound.h>

// We're using a square wave as an approximation of the sound that the
// original VT525 terminals might have produced. This is probably not
// quite right, but it works reasonably well.
static constexpr auto SQUARE_WAVE_SYNTH = 80;
#pragma comment(lib, "dxguid.lib")
#pragma comment(lib, "dsound.lib")

MidiOut() noexcept
{
if constexpr (Feature_DECPSViaMidiPlayer::IsEnabled())
{
midiOutOpen(&handle, MIDI_MAPPER, NULL, NULL, CALLBACK_NULL);
OutputMessage(PROGRAM_CHANGE, SQUARE_WAVE_SYNTH);
}
}
~MidiOut() noexcept
{
if constexpr (Feature_DECPSViaMidiPlayer::IsEnabled())
{
midiOutClose(handle);
}
}
void OutputMessage(const int b1, const int b2, const int b3 = 0, const int b4 = 0) noexcept
{
if constexpr (Feature_DECPSViaMidiPlayer::IsEnabled())
{
midiOutShortMsg(handle, MAKELONG(MAKEWORD(b1, b2), MAKEWORD(b3, b4)));
}
}
using Microsoft::WRL::ComPtr;
using namespace std::chrono_literals;

MidiOut(const MidiOut&) = delete;
MidiOut(MidiOut&&) = delete;
MidiOut& operator=(const MidiOut&) = delete;
MidiOut& operator=(MidiOut&&) = delete;
// The WAVE_DATA below is an 8-bit PCM encoding of a triangle wave form.
// We just play this on repeat at varying frequencies to produce our notes.
constexpr auto WAVE_SIZE = 16u;
constexpr auto WAVE_DATA = std::array<byte, WAVE_SIZE>{ 128, 159, 191, 223, 255, 223, 191, 159, 128, 96, 64, 32, 0, 32, 64, 96 };

private:
HMIDIOUT handle = nullptr;
};
MidiAudio::MidiAudio(HWND windowHandle)
{
if (SUCCEEDED(DirectSoundCreate8(nullptr, &_directSound, nullptr)))
{
if (SUCCEEDED(_directSound->SetCooperativeLevel(windowHandle, DSSCL_NORMAL)))
{
_createBuffers();
}
}
}

using namespace std::chrono_literals;

MidiAudio::~MidiAudio() noexcept
{
try
{
#pragma warning(suppress : 26447)
// We acquire the lock here so the class isn't destroyed while in use.
// If this throws, we'll catch it, so the C26447 warning is bogus.
_inUseMutex.lock();
const auto lock = std::unique_lock{ _inUseMutex };
}
catch (...)
{
Expand Down Expand Up @@ -103,23 +78,75 @@ void MidiAudio::Unlock()
void MidiAudio::PlayNote(const int noteNumber, const int velocity, const std::chrono::microseconds duration) noexcept
try
{
// The MidiOut is a local static because we can only have one instance,
// and we only want to construct it when it's actually needed.
static MidiOut midiOut;

if (velocity)
const auto& buffer = _buffers.at(_activeBufferIndex);
if (velocity && buffer)
{
midiOut.OutputMessage(MidiOut::NOTE_ON, noteNumber, velocity);
// The formula for frequency is 2^(n/12) * 440Hz, where n is zero for
// the A above middle C (A4). In MIDI terms, A4 is note number 69,
// which is why we subtract 69. We also need to multiply by the size
// of the wave form to determine the frequency that the sound buffer
// has to be played to achieve the equivalent note frequency.
const auto frequency = std::pow(2.0, (noteNumber - 69.0) / 12.0) * 440.0 * WAVE_SIZE;
buffer->SetFrequency(gsl::narrow_cast<DWORD>(frequency));
// For the volume, we're using the formula defined in the the General
// MIDI Level 2 specification: Gain in dB = 40 * log10(v/127). We need
// to multiply by 4000, though, because the SetVolume method expects
// the volume to be in hundredths of a decibel.
const auto volume = 4000.0 * std::log10(velocity / 127.0);
buffer->SetVolume(gsl::narrow_cast<LONG>(volume));
// Resetting the buffer to a position that is slightly off from the
// last position will help to produce a clearer separation between
// tones when repeating sequences of the same note.
buffer->SetCurrentPosition((_lastBufferPosition + 12) % WAVE_SIZE);
}

// By waiting on the shutdown future with the duration of the note, we'll
// either be paused for the appropriate amount of time, or we'll break out
// of the wait early if we've been shutdown.
_shutdownFuture.wait_for(duration);

if (velocity)
if (velocity && buffer)
{
midiOut.OutputMessage(MidiOut::NOTE_OFF, noteNumber, velocity);
// When the note ends, we just turn the volume down instead of stopping
// the sound buffer. This helps reduce unwanted static between notes.
buffer->SetVolume(DSBVOLUME_MIN);
buffer->GetCurrentPosition(&_lastBufferPosition, nullptr);
}

// Cycling between multiple buffers can also help reduce the static.
_activeBufferIndex = (_activeBufferIndex + 1) % _buffers.size();
}
CATCH_LOG()

void MidiAudio::_createBuffers() noexcept
{
auto waveFormat = WAVEFORMATEX{};
waveFormat.wFormatTag = WAVE_FORMAT_PCM;
waveFormat.nChannels = 1;
waveFormat.nSamplesPerSec = 8000;
waveFormat.wBitsPerSample = 8;
waveFormat.nBlockAlign = waveFormat.nChannels * waveFormat.wBitsPerSample / 8;
waveFormat.nAvgBytesPerSec = waveFormat.nSamplesPerSec * waveFormat.nBlockAlign;

auto bufferDescription = DSBUFFERDESC{};
bufferDescription.dwSize = sizeof(DSBUFFERDESC);
bufferDescription.dwFlags = DSBCAPS_CTRLVOLUME | DSBCAPS_CTRLFREQUENCY | DSBCAPS_GLOBALFOCUS;
bufferDescription.dwBufferBytes = WAVE_SIZE;
bufferDescription.lpwfxFormat = &waveFormat;

for (auto& buffer : _buffers)
{
if (SUCCEEDED(_directSound->CreateSoundBuffer(&bufferDescription, &buffer, nullptr)))
{
LPVOID bufferPtr;
DWORD bufferSize;
if (SUCCEEDED(buffer->Lock(0, 0, &bufferPtr, &bufferSize, nullptr, nullptr, DSBLOCK_ENTIREBUFFER)))
{
std::memcpy(bufferPtr, WAVE_DATA.data(), WAVE_DATA.size());
buffer->Unlock(bufferPtr, bufferSize, nullptr, 0);
}
buffer->SetVolume(DSBVOLUME_MIN);
buffer->Play(0, 0, DSBPLAY_LOOPING);
}
}
}
12 changes: 11 additions & 1 deletion src/audio/midi/MidiAudio.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,17 @@ Module Name:

#pragma once

#include <array>
#include <future>
#include <mutex>

struct IDirectSound8;
struct IDirectSoundBuffer;

class MidiAudio
{
public:
MidiAudio() = default;
MidiAudio(HWND windowHandle);
MidiAudio(const MidiAudio&) = delete;
MidiAudio(MidiAudio&&) = delete;
MidiAudio& operator=(const MidiAudio&) = delete;
Expand All @@ -30,6 +34,12 @@ class MidiAudio
void PlayNote(const int noteNumber, const int velocity, const std::chrono::microseconds duration) noexcept;

private:
void _createBuffers() noexcept;

Microsoft::WRL::ComPtr<IDirectSound8> _directSound;
std::array<Microsoft::WRL::ComPtr<IDirectSoundBuffer>, 2> _buffers;
size_t _activeBufferIndex = 0;
DWORD _lastBufferPosition = 0;
std::promise<void> _shutdownPromise;
std::future<void> _shutdownFuture;
std::mutex _inUseMutex;
Expand Down
3 changes: 2 additions & 1 deletion src/cascadia/TerminalControl/ControlCore.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1342,7 +1342,8 @@ namespace winrt::Microsoft::Terminal::Control::implementation
{
if (!_midiAudio)
{
_midiAudio = std::make_unique<MidiAudio>();
const auto windowHandle = reinterpret_cast<HWND>(_owningHwnd);
_midiAudio = std::make_unique<MidiAudio>(windowHandle);
_midiAudio->Initialize();
}
return *_midiAudio;
Expand Down
4 changes: 0 additions & 4 deletions src/cascadia/TerminalControl/TerminalControlLib.vcxproj
Original file line number Diff line number Diff line change
Expand Up @@ -147,10 +147,6 @@
<PRIResource Include="Resources\en-US\Resources.resw" />
<OCResourceDirectory Include="Resources" />
</ItemGroup>
<ItemGroup Condition="'$(WindowsTerminalBranding)'=='' or '$(WindowsTerminalBranding)'=='Dev' or '$(WindowsTerminalBranding)'=='Preview'">
<!-- GH#13252 Only vend this dependency for Dev and Preview builds. -->
<SDKReference Include="Microsoft.Midi.GmDls, Version=10.0.22000.0" />
</ItemGroup>
<!-- ========================= Project References ======================== -->
<ItemGroup>
<ProjectReference Include="..\..\types\lib\types.vcxproj" />
Expand Down
12 changes: 0 additions & 12 deletions src/features.xml
Original file line number Diff line number Diff line change
Expand Up @@ -117,18 +117,6 @@
</alwaysDisabledBrandingTokens>
</feature>

<feature>
<name>Feature_DECPSViaMidiPlayer</name>
<description>Enables playing sound via DECPS using the MIDI player.</description>
<stage>AlwaysDisabled</stage>
<!-- We're disabling this for WindowsInbox and Stable because it requires an additional
package dependency or library dependency. -->
<alwaysEnabledBrandingTokens>
<brandingToken>Dev</brandingToken>
<brandingToken>Preview</brandingToken>
</alwaysEnabledBrandingTokens>
</feature>

<feature>
<name>Feature_ScrollbarMarks</name>
<description>Enables the experimental scrollbar marks feature.</description>
Expand Down
3 changes: 2 additions & 1 deletion src/host/consoleInformation.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,8 @@ MidiAudio& CONSOLE_INFORMATION::GetMidiAudio()
{
if (!_midiAudio)
{
_midiAudio = std::make_unique<MidiAudio>();
const auto windowHandle = ServiceLocator::LocateConsoleWindow()->GetWindowHandle();
_midiAudio = std::make_unique<MidiAudio>(windowHandle);
_midiAudio->Initialize();
}
return *_midiAudio;
Expand Down

0 comments on commit b8847a3

Please sign in to comment.