Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ETW API pain points #937

Closed
7 tasks done
nico-abram opened this issue May 6, 2022 · 12 comments
Closed
7 tasks done

ETW API pain points #937

nico-abram opened this issue May 6, 2022 · 12 comments
Assignees
Labels
bug Something isn't working missing api Some documented API is missing from the metadata rust Critical for Rust adoption usability Touch-up to improve the user experience for a language projection

Comments

@nico-abram
Copy link

nico-abram commented May 6, 2022

I've been using the ETW API, and I found a few things that seem odd in the generated bindings. I'm not sure if these are windows-rs problems, or metadata problems. First:

  • windows::Win32::System::SystemServices::SE_SYSTEM_PROFILE_NAME and other strings next to it are &str's, but the APIs that it's used with require a null terminator, so I had to manually add it (For example LookupPrivilegeValueA)
  • Same thing for windows::Win32::System::Diagnostics::Etw::KERNEL_LOGGER_NAMEA

That seemed like the biggest problem and the only thing I encountered that clearly looks wrong. Then there's some things that surprised me and could maybe be improved but that are not necessarily wrong:

@nico-abram nico-abram added the enhancement New feature or request label May 6, 2022
@kennykerr
Copy link
Contributor

Thanks Nicolas, great feedback. Some of this is on the win32 metadata and some may be possible for the bindgen crate to improve. I'll go through and itemize when I get a moment, unless someone else beats me to it.

@kennykerr
Copy link
Contributor

Most of these issues stem from problems with the win32 metadata, so I'll transfer the issue to that repo for resolution.

The question about null terminated string literals is interesting. I'll open a separate issue for that here.

@kennykerr kennykerr transferred this issue from microsoft/windows-rs May 20, 2022
@kennykerr kennykerr added bug Something isn't working and removed enhancement New feature or request labels May 20, 2022
@mikebattista mikebattista added usability Touch-up to improve the user experience for a language projection missing api Some documented API is missing from the metadata labels May 26, 2022
@mikebattista mikebattista self-assigned this May 26, 2022
@idigdoug
Copy link

One additional more here. Etw unfortunately used TRACEHANDLE with two different apis that sound related but are not. StartTrace and ControlTrace use the same meaning of TRACEHANDLE, which is distinct from the handle of OpenTrace/ProcessTrace/CloseTrace. A projection would hopefully fix this.

@Robo210
Copy link
Contributor

Robo210 commented Jun 28, 2022

I was not able to find the ETW trace event type constants anywhere in windows-rs. For example, https://docs.microsoft.com/en-us/windows/win32/etw/stackwalk mentions EVENT_ENABLE_PROPERTY_STACK_TRACE should be 32.
I was not able to find the event structs anywhere, for example https://docs.microsoft.com/en-us/windows/win32/etw/stackwalk-event and https://docs.microsoft.com/en-us/windows/win32/etw/sampledprofile

These things are MOF definitions for events, and are provided on MSDN for illustrative purposes only. You aren't supposed to hard-code those structures or values anywhere; use TDH to decode your events or use the lower-level WBEM APIs to query for the definitions and then parse the output per the MOF spec (but don't actually do this - use TDH).

@mikebattista
Copy link
Collaborator

Windows.Win32.System.Diagnostics.Etw.Apis.StartTraceW : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.StartTraceA : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.StopTraceW : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.StopTraceA : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.QueryTraceW : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.QueryTraceA : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.UpdateTraceW : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.UpdateTraceA : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.FlushTraceW : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.FlushTraceA : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.ControlTraceW : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.ControlTraceA : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.QueryAllTracesW : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.QueryAllTracesA : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.EnableTrace : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.EnableTraceEx : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.EnableTraceEx2 : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.EnumerateTraceGuidsEx : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.TraceSetInformation : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.TraceQueryInformation : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.CreateTraceInstanceId : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.TraceEvent : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.EnumerateTraceGuids : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.ProcessTrace : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.CloseTrace : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.QueryTraceProcessingHandle : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.SetTraceCallback : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.RemoveTraceCallback : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.TraceMessage : return...UInt32 => WIN32_ERROR
Windows.Win32.System.Diagnostics.Etw.Apis.TraceMessageVa : return...UInt32 => WIN32_ERROR

mikebattista added a commit that referenced this issue Aug 4, 2022
@mikebattista
Copy link
Collaborator

Windows.Win32.System.Diagnostics.Etw.ITraceRelogger.AddLogfileTraceStream : TraceHandle...UInt64* => TRACEHANDLE*
Windows.Win32.System.Diagnostics.Etw.ITraceRelogger.AddRealtimeTraceStream : TraceHandle...UInt64* => TRACEHANDLE*
Windows.Win32.System.Diagnostics.Etw.ITraceRelogger.CreateEventInstance : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.TRACEHANDLE not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.StartTraceW : TraceHandle...UInt64* => TRACEHANDLE*
Windows.Win32.System.Diagnostics.Etw.Apis.StartTraceA : TraceHandle...UInt64* => TRACEHANDLE*
Windows.Win32.System.Diagnostics.Etw.Apis.StopTraceW : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.StopTraceA : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.QueryTraceW : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.QueryTraceA : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.UpdateTraceW : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.UpdateTraceA : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.FlushTraceW : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.FlushTraceA : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.ControlTraceW : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.ControlTraceA : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.EnableTrace : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.EnableTraceEx : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.EnableTraceEx2 : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.TraceSetInformation : SessionHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.TraceQueryInformation : SessionHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.TraceEvent : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.TraceEventInstance : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.RegisterTraceGuidsW : RegistrationHandle...UInt64* => TRACEHANDLE*
Windows.Win32.System.Diagnostics.Etw.Apis.RegisterTraceGuidsA : RegistrationHandle...UInt64* => TRACEHANDLE*
Windows.Win32.System.Diagnostics.Etw.Apis.UnregisterTraceGuids : RegistrationHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.GetTraceLoggerHandle : return...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.GetTraceEnableLevel : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.GetTraceEnableFlags : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.OpenTraceW : return...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.ProcessTrace : HandleArray...UInt64* => TRACEHANDLE*
Windows.Win32.System.Diagnostics.Etw.Apis.CloseTrace : TraceHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.QueryTraceProcessingHandle : ProcessingHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.OpenTraceA : return...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.TraceMessage : LoggerHandle...UInt64 => TRACEHANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.TraceMessageVa : LoggerHandle...UInt64 => TRACEHANDLE

mikebattista added a commit that referenced this issue Aug 5, 2022
@mikebattista
Copy link
Collaborator

Windows.Win32.System.Diagnostics.Etw.Apis.ALPCGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.DiskIoGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.FileIoGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.ImageLoadGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.PageFaultGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.PerfInfoGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.ProcessGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.RegistryGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.SplitIoGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.TcpIpGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.ThreadGuid not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.UdpIpGuid not found in 1st winmd

mikebattista added a commit that referenced this issue Aug 5, 2022
@mikebattista
Copy link
Collaborator

Windows.Win32.System.Diagnostics.Etw.TRACEHANDLE not found in 2nd winmd
Windows.Win32.System.Diagnostics.Etw.ITraceRelogger.AddLogfileTraceStream : TraceHandle...TRACEHANDLE* => RELOGSTREAM_HANDLE*
Windows.Win32.System.Diagnostics.Etw.ITraceRelogger.AddRealtimeTraceStream : TraceHandle...TRACEHANDLE* => RELOGSTREAM_HANDLE*
Windows.Win32.System.Diagnostics.Etw.ITraceRelogger.CreateEventInstance : TraceHandle...TRACEHANDLE => RELOGSTREAM_HANDLE
Windows.Win32.System.Diagnostics.Etw.PROCESSTRACE_HANDLE not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.CONTROLTRACE_HANDLE not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.RELOGSTREAM_HANDLE not found in 1st winmd
Windows.Win32.System.Diagnostics.Etw.Apis.StartTraceW : TraceHandle...TRACEHANDLE* => CONTROLTRACE_HANDLE*
Windows.Win32.System.Diagnostics.Etw.Apis.StartTraceA : TraceHandle...TRACEHANDLE* => CONTROLTRACE_HANDLE*
Windows.Win32.System.Diagnostics.Etw.Apis.StopTraceW : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.StopTraceA : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.QueryTraceW : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.QueryTraceA : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.UpdateTraceW : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.UpdateTraceA : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.FlushTraceW : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.FlushTraceA : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.ControlTraceW : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.ControlTraceA : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.EnableTrace : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.EnableTraceEx : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.EnableTraceEx2 : TraceHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.TraceSetInformation : SessionHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.TraceQueryInformation : SessionHandle...TRACEHANDLE => CONTROLTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.TraceEvent : TraceHandle...TRACEHANDLE => UInt64
Windows.Win32.System.Diagnostics.Etw.Apis.TraceEventInstance : TraceHandle...TRACEHANDLE => UInt64
Windows.Win32.System.Diagnostics.Etw.Apis.RegisterTraceGuidsW : RegistrationHandle...TRACEHANDLE* => UInt64*
Windows.Win32.System.Diagnostics.Etw.Apis.RegisterTraceGuidsA : RegistrationHandle...TRACEHANDLE* => UInt64*
Windows.Win32.System.Diagnostics.Etw.Apis.UnregisterTraceGuids : RegistrationHandle...TRACEHANDLE => UInt64
Windows.Win32.System.Diagnostics.Etw.Apis.GetTraceLoggerHandle : return...TRACEHANDLE => UInt64
Windows.Win32.System.Diagnostics.Etw.Apis.GetTraceEnableLevel : TraceHandle...TRACEHANDLE => UInt64
Windows.Win32.System.Diagnostics.Etw.Apis.GetTraceEnableFlags : TraceHandle...TRACEHANDLE => UInt64
Windows.Win32.System.Diagnostics.Etw.Apis.OpenTraceW : return...TRACEHANDLE => PROCESSTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.ProcessTrace : HandleArray...TRACEHANDLE* => PROCESSTRACE_HANDLE*
Windows.Win32.System.Diagnostics.Etw.Apis.CloseTrace : TraceHandle...TRACEHANDLE => PROCESSTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.QueryTraceProcessingHandle : ProcessingHandle...TRACEHANDLE => PROCESSTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.OpenTraceA : return...TRACEHANDLE => PROCESSTRACE_HANDLE
Windows.Win32.System.Diagnostics.Etw.Apis.TraceMessage : LoggerHandle...TRACEHANDLE => UInt64
Windows.Win32.System.Diagnostics.Etw.Apis.TraceMessageVa : LoggerHandle...TRACEHANDLE => UInt64

@mikebattista
Copy link
Collaborator

@idigdoug please review the above diff.

mikebattista added a commit that referenced this issue Aug 5, 2022
@idigdoug
Copy link

idigdoug commented Aug 5, 2022

Looks good.

@kennykerr
Copy link
Contributor

Note that the windows and windows-sys crates now provide the original encoding for string constants: microsoft/windows-rs#2101

Most or all of the remaining issues appear to be resolved and have made their way into the Rust project.

@kennykerr kennykerr added the rust Critical for Rust adoption label Oct 18, 2022
@mikebattista
Copy link
Collaborator

Thanks. Will close this out then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working missing api Some documented API is missing from the metadata rust Critical for Rust adoption usability Touch-up to improve the user experience for a language projection
Projects
None yet
Development

No branches or pull requests

5 participants