-
-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unmanaged EmitCalli and netstandard2.0 #13
Comments
Good news! I noticed the following interesting thing. Final logic from implemented managed EmitCalli() and its calling convention 'Standard' is much closely related to actual logic that's required for our unmanaged type :) Decompiled versions below was just for my convenience when debugging at this case. But the original code you can find in original coreclr repo or from my mod repo for .NET DllExport project. So! What I found:All code below has been trimmed (+added specific parts to this case) just to illustrate the main flow between different implementations. Unmanaged EmitCalli ~ ...
ModuleBuilder moduleBuilder = (ModuleBuilder)m_methodBuilder.Module;
...
SignatureHelper methodSigHelper = SignatureHelper.GetMethodSigHelper(moduleBuilder, unmanagedCallConv, returnType);
if (parameterTypes != null)
{
for (int i = 0; i < num2; i++)
{
methodSigHelper.AddArgument(parameterTypes[i]);
}
}
if (returnType != typeof(void))
{
num++;
}
if (parameterTypes != null)
{
num -= num2; // num2 = parameterTypes.Length;
}
num--;
UpdateStackSize(OpCodes.Calli, num);
EnsureCapacity(7);
Emit(OpCodes.Calli);
RecordTokenFixup();
PutInteger4(moduleBuilder.GetSignatureToken(methodSigHelper).Token); Managed EmitCalli when 'Standard' ~ ...
ModuleBuilder moduleBuilder = (ModuleBuilder)m_methodBuilder.Module;
SignatureHelper memberRefSignature = GetMemberRefSignature(callingConvention, returnType, parameterTypes, optionalParameterTypes);
EnsureCapacity(7);
Emit(OpCodes.Calli);
if (returnType != typeof(void))
{
num++;
}
if (parameterTypes != null)
{
num -= parameterTypes.Length;
}
...
num--;
UpdateStackSize(OpCodes.Calli, num);
RecordTokenFixup();
PutInteger4(moduleBuilder.GetSignatureToken(memberRefSignature).Token);
// >> GetMemberRefSignature
return GetMemberRefSignature(call, returnType, parameterTypes, optionalParameterTypes, 0);
// >> GetMemberRefSignature
return ((ModuleBuilder)m_methodBuilder.Module).GetMemberRefSignature(call, returnType, parameterTypes, optionalParameterTypes, cGenericParameters);
// >> GetMemberRefSignature
SignatureHelper methodSigHelper = SignatureHelper.GetMethodSigHelper(this, call, returnType, cGenericParameters);
if (parameterTypes != null)
{
foreach (Type clsArgument in parameterTypes)
{
methodSigHelper.AddArgument(clsArgument);
}
}
...
return methodSigHelper; Did you notice it? All minimal instructions are the same to unmanaged implementation as we can see here. Good! We only need to inspect method signature! Method signature and internal GetMethodSigHelperBoth implementations above uses internal MdSigCallingConvention: CallConvMask = 0xF,
Default = 0x0,
C = 0x1,
StdCall = 0x2,
ThisCall = 0x3,
FastCall = 0x4,
Vararg = 0x5,
Field = 0x6,
LocalSig = 0x7,
Property = 0x8,
Unmgd = 0x9,
GenericInst = 0xA,
Generic = 0x10,
HasThis = 0x20,
ExplicitThis = 0x40 After some different layers they have the final m_signature processing as follow: if (m_currSig + 4 > m_signature.Length)
{
m_signature = ExpandArray(m_signature);
}
if (data <= 127)
{
m_signature[m_currSig++] = (byte)(data & 0xFF);
return;
}
if (data <= 16383)
{
m_signature[m_currSig++] = (byte)((data >> 8) | 0x80);
m_signature[m_currSig++] = (byte)(data & 0xFF);
return;
}
if (data <= 536870911)
{
m_signature[m_currSig++] = (byte)((data >> 24) | 0xC0);
m_signature[m_currSig++] = (byte)((data >> 16) & 0xFF);
m_signature[m_currSig++] = (byte)((data >> 8) & 0xFF);
m_signature[m_currSig++] = (byte)(data & 0xFF);
return;
} And this is it, an added m_signature item above (means when MdSigCallingConvention data) will be between 1 - 4 (inclusive) for unmanaged, and 0 for managed ('Standard' should be processed as MdSigCallingConvention.Default). That is, for example, we can try to control it like:
I'll update PR soon because I already have a working draft :p |
* NEW: Starting support of the .NET Core. PR #12 Full support: netcoreapp2.1 and netstandard2.1 (SDK 3). * NEW: Hack of the unmanaged EmitCalli to support .NET Standard 2.0. Issue #13. * NEW: NativeData and BReader now supports CharPtr, WCharPtr, BSTR types. * NEW: Implemented isolation for PE Modules. Issue #15. Manage it through `IConfig.IsolateLoadingOfModule`. Optionaly it will isolate module for a real new loading even if it was already loaded somewhere else. Full details in #15 * NEW: Added `ConariX` as an compatible DLR version of ConariL implementation: ``` using(dynamic l = new ConariX("...")) { // just everything is yours ~ l.curl_easy_setopt(curl, 10002, "http://example.com"); } ``` * NEW: Implemented kernel32 WinApi via new ConariX: ``` dynamic kernel32 = new Kernel32(); kernel32.GetModuleHandleA<IntPtr>("libcurl-x64"); kernel32.GetModuleHandleW<IntPtr>((WCharPtr)ustr); ``` * NEW: Implemented user32 WinApi via new ConariX: ``` dynamic user32 = new User32(); user32.ShowWindow(0x000A0A28, 3); user32.MessageBoxA(0, "Conari in action", "Hello!", 0); ``` * NEW: Added netfx based target platforms: net472. * NEW: Added initialization of UnmanagedString from IntPtr. * FIXED: Fixed bug with typeof(void) for return type when DLR. * FIXED: Fixed bug with x64 for CharPtr, WCharPtr, BSTR types. * CHANGED: Performance: Internal ModuleBuilder now is unified for all our new generated dynamic types. * CHANGED: Added hMSBuild 2.2 + GetNuTool 1.7 in nupkg packages for related build processes: https://www.nuget.org/packages/Conari/ * tools\gnt.bat - https://github.com/3F/GetNuTool * tools\hMSBuild.bat - https://github.com/3F/hMSBuild * NOTE: Official Releases: * NuGet: https://www.nuget.org/packages/Conari/ * GitHub: https://github.com/3F/Conari/releases/latest * NOTE: Please note again, .NET Standard 2.0 support is possible only because of our hack! While other netcoreapp2.1 + netstandard2.1 target platforms will not contain this modifications at all. Means most known behavior. Do not hesitate to contact: https://github.com/3F/Conari/issues
Cross-link to a project that provides an independent Unmanaged EmitCalli implementation |
* NEW: New modern NativeString<T> replaces UnmanagedString. * A complete support of CharPtr, WCharPtr, TCharPtr; * Support optional buffers for receiving and updating values; * Safe and fast reuse of the allocated memory regions; * Non sealed, extendable for a new features; ```csharp using NativeString<WCharPtr> data = new("Hello {p}!"); ``` * NEW: New TCharPtr marshalable type which aggregates WCharPtr + CharPtr at runtime. * NEW: New modern NativeStruct<T> replaces UnmanagedStructure. Two modes! * NativeStruct - Fully automatic way of working with structures without declarations using NativeData chains; * NativeStruct<T> - Semi-automatic way of working with structures using CLR types declarations; ```csharp using var u = NativeStruct.Make.f<UIntPtr>("start", "end").Struct; ``` ```csharp using var u = new NativeStruct<MatchResult>(); ``` ```csharp using NativeStruct u = new(); u.Native .f<UIntPtr>("start", "end") .build(out dynamic mres); ``` * NEW: Direct use "" dotnet strings including ref strings (ByRef&) in DLR. ```csharp string data = "number = 888;"; bool found = l.replace<bool>(ref data, "+??;", "2034;"); ``` * NEW: New VPtr type - Variable long pointer. This type supports adding long numbers to IntPtr (IntPtr + long), a complete comparing >,<,>=,<=,==,!= between VPtr and int/long, and more. * NEW: Added support for optional arguments in DLR via IProviderDLR.TrailingArgs. Can be applied only if varargs. Useful in avoiding any optional values. Eg. (1, 2, 3, [0, 0]). ```cpp bool func(const TCHAR* input, flagcfg_t options = 0, MatchResult* result = nullptr); ``` ```csharp l.func<bool>(input); -> input, [+ 0, + nullptr] ``` * NEW: Added .build() and .build(out dynamic result) to build native chains in a Raw and NativeData objects. ```csharp Accessor.Native() .t<DWORD>("VirtualAddress") .t<DWORD>("Size") .build(out dynamic idd); ``` * NEW: PE32/PE32+ Memory and Streams implementations. Configure it through IConfig.PeImplementation or use manually. * NEW: IMarshalableGeneric for marshaling generic types. Use or implement this to avoid possible: - "Exception has been thrown by the target of an invocation" - "System.Runtime.InteropServices.MarshalDirectiveException : Cannot marshal 'parameter ...': Generic types cannot be marshaled." - (IntPtr)u -> u etc. ```csharp using NativeString<CharPtr> data = new("Hello {p}!"); l.replace<bool>(data, ...) ``` * NEW: Easy concatenation for an unmanaged c-strings. ``` using var str = input + " " + "world"; ``` * NEW: New lightweight way of generating Dynamic MethodInfo. As part of optimization, * https://twitter.com/github3F/status/1396491762315808771 * https://twitter.com/github3F/status/1396492298972717057 This is the default way for DLR starting with 1.5. Use IProviderDLR.SignaturesViaTypeBuilder to configure it. * NEW: Native chains. Added support 8/16 bit characters via configurable .readChar() and writeChar(). * NEW: Native chains. Added conditional .eq(), .or() that verifies input T is equal to T. It can be checked via .check(), ifTrue(Action)/ifFalse(Action), and failed(bool when). ```csharp l.Memory .@goto(l.PE.Addresses.IMAGE_NT_HEADERS) .eq('P', 'E', '\0', '\0') .check() ``` * NEW: Added NativeArray<T> to work unmanaged arrays. ```csharp using NativeArray<short> nr = new(pointer); // points to ~0x2674F89EDF0 nr[0] = 1; nr[1] = 2; ``` ```csharp using NativeArray nr = new(2, -4, 6); Memory memory = nr; ``` * NEW: New modern Memory VPtr based implementation in native chains (serializable) to work with RAM related processes. * NEW: New modern NativeStream VPtr based implementation in native chains (serializable) to work with any supported stream related processes. * NEW: New modern LocalContent VPtr based implementation in native chains (serializable) to work with any local allocated data. * NEW: Added generic IConari<TCharIn> for a specific char type by default in used instance. * NEW: Added Allocator for allocating any data in unmanaged memory. `Allocator` based on `NativeArray<byte>`. ```csharp using Allocator alloc = new(0x70, 0xDE, 0x01, 0xAC, 0x04, 0xCB, 0x70, 0xDE, 0x93, 0x12, 0x74, 0x94); alloc.Memory // access via native chains Memory implementation ``` * NEW: The fast way to update an unmanaged strings without allocating new in memory through modern NativeString<T> and BufferedString<T>. * NEW: New BufferedString<T> a specialized native strings for future possible changing with additional buffer. `BufferedString<T>` is a type alias for `NativeString<T>` with the default buffer using factor 2.5f relative to the string; * NEW: Native chains. New working with a different positions Zone (Initial/Region/Current) and aliases (D/U/V) +rewind(Zone) Rewind the chain to a specific region. +@goto(VPtr) Go to a new address using absolute position. +back<T>(int count = 1); - Move back on T type size from the current position. + T1, T2, T3 aliases. * NEW: New `achar` and `wchar` type markers for a native chains. * NEW: Added extension methods: +NativeSize() for objects - Get the size in bytes of the selected managed type to be treated as an unmanaged. +IAccessor Access() - Accessing data via Memory or LocalContent implementation. * NEW: New for NativeData: * +assign<T>(string[]) - Assigns new fields for each presented name. +f<T>(string[]) - alias * +renew(SeekPosition) - An additional way to start the chain with specific SeekPosition +renew(out VPtr, SeekPosition) * +region() - Mark new region in the chain. +region(out int) +region(out VPtr) * +ofs<T>(int, string[]) - Use offset to the data. * +extend(byte[] bytes) - Extends local data using additional bytes. * +mode(ChainMode) - to control chain and its new fields via ChainMode. * NEW: PeImplType.Disabled to disable all related to PE features such as mangling, list of exported proc, etc. * NEW: Added IProviderDLR.TryEvaluateContext. It can help better understand what are you trying to call without specified casting. true value by default. * NEW: Added IProviderDLR.ManageNativeStrings. Collect information about all input INativeString and delegate control. true value by default. * NEW: Added IProviderDLR.BoxingControl. Control of boxing for input data. BoxingType.UnboxingAndBoxing value by default. * NEW: IProvider.addr() address of the specific item such streams std::cin etc. Part of Issue #17. ``` l.addr("get_GPtrVal") ``` * NEW: ConariL/X Make() wrapper to use both runtime dynamic and compile type objects. * NEW: Added manager NativeStringManager for a new modern NativeString<T> strings. Helps to manage c-strings easily. * NEW: Added the following static Members: +Is64bit +EmptyArray<T> for netfx 4 +SizeOf<T>() +SizeOf<T>(int) * NEW: Added IProviderDLR.RefModifiableStringBuffer. Allows configure additional buffer to process ref strings (ByRef&). * NEW: Adds IModuleIsolationRecipe for configurable isolation handlers. * NEW: New IConfig option. LoaderSyncLimit. Limit in milliseconds for how long to wait signals when synchronization of threads (processes). * NEW: INativeAccessor and IDlrAccessor unifies ConariL, ConariX objects, Native chains and various raw Memory accessing. * NEW: New IConfig option. CancelIfCantIsolate. Cancel processing in loader if module can't be isolated. * NEW: New IConfig option. Cts. Signals to cancel active operations as soon as possible. * NEW: New IConfig option. PeImplementation. Use specific implementation around PE processing. Affects loader, mangling, and other related core features. * NEW: New for IPE interface: + Addresses - Known addresses of the tables. + Characteristics + Magic + Machine + Export reference to (addresses + names + ordinals) * NEW: IConari now extends IStringMaker to provide direct `_T()` accessing in implemented ConariX / ConariL objects. ```csharp l.match<bool>(l._T("test")) ``` * NEW: IConari now specifies INativeStringManager Strings { get; } property for a quick accessing to used string manager in implemented objects. * FIXED: Fixed possible incorrect loading and resolving module in Loader. * FIXED: Fixed CalculateHashCode() affects comparing lot of Conari's types etc. * FIXED: Fixed clarifying the context when DLR. * FIXED: Fixed possible "Parameter count mismatch" when using caching and different signatures to the same pointer. For 1.4 it can be avoided by resetting cache. * FIXED: Improved multiprocessing of requests to the same module. Part of Issue #15. * CHANGED: Hack for .NET Standard now is under UnmanagedEmitCalli 1.1 * https://github.com/3F/UnmanagedEmitCalli * See #13 * CHANGED: Adds aggressive finalization for ConariX/L objects to avoid possible abandoned pointers. * CHANGED: New ArchitectureMismatchException will help clarify the Issue #4. * CHANGED: Improved Native chains and Unmanaged Memory accessing: IAccessor + Reader/Writter/Updater ++ as a common interface for a modern native chains implementations such as Memory, NativeStream, LocalContent, etc. All this implements also IPointer (VPtr based). ```csharp memory.Native() .f<WORD>("Machine", "NumberOfSections") // IMAGE_FILE_HEADER (0xF4) .align<DWORD>(3) .t<WORD>("SizeOfOptionalHeader") .t<WORD>("Characteristics") .region() .t<WORD>("Magic") // IMAGE_OPTIONAL_HEADER (0x108) .build(out dynamic ifh); ``` ```csharp acs .write(-4) .update(6) .rewind(Zone.Region) .read<byte>(2, out byte[] r) .next<short>(ref v) ``` ... * CHANGED: Loader now uses PE Memory implementation by default. * CHANGED: IDynamic & Dynamic changes: +Dynamic.Hash() - Hash the following types using specified algorithm. +IDynamic.Options - to configure generation globally. +IDynamic.getMethodInfo(DynamicOptions, ...) * CHANGED: Updated a bit Config as a class with top namespace +IConfig.Cache. * CHANGED: Fully updated WCharPtr + CharPtr types: * +ISerializable * Complete overloading an Equals logic between types and related conversion * Added Null values and StrLength property in addition to Length. * CHANGED: Removed old `_` method aliases from NativeData due to modern IDlrAccessor. Please use other available options (.ctor, Native extension, INativeAccessor, etc.) * CHANGED: Updated Microsoft.CSharp 4.7.0 * CHANGED: Updated System.Reflection.Emit 4.7.0 * CHANGED: Updated System.Reflection.Emit.Lightweight 4.7.0 * OBSOLETE: UnmanagedStructure marked as obsolete. * OBSOLETE: Marked as Obsolete: !IDynamic.UseCache !IDynamic.getKeyTypes(MethodInfo) !Dynamic.GetMethodInfo(bool cache, ...) * OBSOLETE: IPE.ExportedProcNamesArray marked as Obsolete. * OBSOLETE: BSTR marked as obsolete. * OBSOLETE: PtrSize marked as obsolete in WCharPtr + CharPtr types. * OBSOLETE: UnmanagedString marked as obsolete.
In #12 I already mentioned about the main problem for netstandard2.0. And this is what I'm talking about:
An unmanaged EmitCalli is available only with netcoreapp2.1+ https://github.com/dotnet/corefx/issues/9800
But
System.Reflection.Emit.ILGeneration
is just metadata whilemscorlib/src/System/Reflection/Emit/DynamicILGenerator
implements this since dotnet/coreclr#16546See System.Private.CoreLib.dll
More probably we can try inject this logic at runtime o_O Or does exist something related? Cecil is extra cost for just single EmitCalli <_<
The text was updated successfully, but these errors were encountered: