Downgrade dependency on Microsoft.WindowsAzure.ServiceRuntime to 2.4 #197
Conversation
…5 to 2.4 to be consistent with the "-1" rule for Azure SDK dependencies.
|
Hmm, do we have ETA when we get packages with the 2.5 SDK (yes, I just upgraded my side project to 2.5 and I'm wondering now if I should start with custom builds for the interim period :))? Without too much thinking, I would feel the issues at #174 should be resolved too when publishing packages with SDK 2.5, and perhaps with SDK 2.4 too as going to Azure SDK 2.5 seem to be blocker to some for other reasons. Not having binding redirects is a bit of blocker, as otherwise one is forced to go with the versions referenced by Orleans. |
|
@veikkoeeva We thought that requiring a current-1 version, 2.4 at the moment, will make life easy for both people that want to build against 2.4 and 2.5. 2.4 is the minimal required version and this shouldn't in any way prevent upgrading Azure NuGets to 2.5 if you want. Is the binding redirect not working the blocker here that prevents such an easy upgrade? And if we fix it, then you won't need to rebuild Orleans against 2.5 because the upgrade will work? |
|
@sergeybykov Azure SDK 2.5 has breaking changes relative to SDK 2.4, especially relative to diagnostics. I don't know if this is much a problem to people using Orleans or even to me (though I'd need to give up using VS 2015 for Orleans for a moment as it seems to support SDK 2.5 only). To me it's more important to be able to update Nuget packages when new versions of them come out, so that makes the binding redirect issue more important. I would guess it would make the development and adoption story smoother. That being written, I believe the redirection bug is much more important for me. It could be that I try to work on it around the end of next week if there's still a need. The bigger picture in my mind is that what would you guys do when the adoption surges? There will good reasons to do non-backwards compatible changes and then how to version changes and do about with fixes and features. And as Orleans is depedent on other libraries, there will be package versioning clashes on people's development environments. For the latter there is discussion going on Support for Internalizing/Merging external Dependencies [XRE] and Proposal: Static Linking [Roslyn] (remains me of my 10+ years of C++ :)) and in absence something backed in, ILMerge or somesuch may be a good choice. Some of the discussion concerns the fact that internalizing dependencies is currently extra work and error prone. Both of those contain a wealth of good argumentation surrounding servicing application, see especially the proposal on static linking. For the former, the process of selecting the tools to do something in an "ordinary enterprise setting", it may become important to show that there is a documented process to support releases with fixes, especially security fixes. Even if there isn't a formal policy to actively apply (security) fixes (I believe it's clear in FOSS that the one using the libraries bears some responsibility as part of the end product contract). For this, it may well be enough to just show there are release branches and one can port fixes between them, as well as new features if needed. On case-by-case basis this requirements extendes to also subcontractor and startups doing B2B (which is a majority over here). Defence sector, finance and some infrastructure projects have even more stringent requirements (albeit might benefit tremendously from a framework such as Orleans). Regarding taking dependency on supporting libraries, it may be good tactic now during the transition. Albeit as written, I think there ought to be something more formal. Orleans head = Current dependencies - 1 after some time means, I believe, just that people choose either Orleans head or Orleans head - n with what supports their needs the best but it will not include newer libraries (which people may want to take a dependency on as changing it may be problematic due to, say, corporate politics) until a new Orleans head is produced and it may take time. I suppose it would be beneficial to think contributor guidelines and the workflow guidelines (as it seem to become in GitHub, with showing the flow from cloning to rebasing to doing a PR and merging). I would think git-flow is a bit heavyhanded, see here for more authoritative pondering. Then something like GitHub Flow feels a bit light on release management as Orleans binaries can't be updated like web pages. I quite like Mike Flow II after practising it a short while at work (we found it before reading the book). It gives the benefits of git-flow with the easiness of GitHub Flow, I feel. :) Unfortunately it's in the book, though currently there's a 40% discount to be had (see bottom of the page). In any event, then there's always the possibility of engaging early adopters that use more of the bleeding edge libraries and are willing to risk debugging problems and so forth which may benefit other people. But I'm afraid I've written too long already, to paraphrase famous people. If I came across as ranting, I apologise. It's late and I've had too little sleep lately... :-) <edit: As an addition to my late night writing, a powerful message is that Orleans is designed to recover from failures, not simply being fault-tolerant. It factors directly into pricing, say, when ran in on-premises Orleans cluster and having storage tiering (yeah, bundling with another MS technology) underneath, the life-cycle costs may seem quite favourable, especially considering SMB sector (which is plentiful and unserved). One doesn't necessarily need to start with super fault-tolerant SAN setup, expensive databases, a handful of rockstart programmers and long sales cycles. I feel almost like a salesman, but this is how I see one part of the larger picture. :) |
|
@veikkoeeva These are great observations! Not a rant at all, very constructive, don't worry. :-) Thank you! I opened #200 as an umbrella to continue this discussion, so that it doesn't stop with this PR closed and other related issues getting resolved. We are just finishing upgrading a major internal customer of Orleans to the new NuGets, and are feeling some of the versioning pain. Looks like the binding redirect is the most immediate issue to investigate and fix. I'll try to look into it early next week. |
Downgrade dependency on Microsoft.WindowsAzure.ServiceRuntime to 2.4
|
@sergeybykov I can confirm that Orleans versions after #197 stop working in applications that use Azure SDK 2.5. |
|
@ca-ta I'm looking at this and am seeing a different picture. Help me understand what I'm missing. I'm testing this behavior locally within SDK\LocalSilo with OrleansHost.exe as the silo host. Orleans is built from GitHub sources against Microsoft.WindowsAzure.Storage 4.2. I have grain assembly built against Microsoft.WindowsAzure.Storage 4.3 which I deploy (copy) along with Microsoft.WindowsAzure.Storage.dll and other dependencies to LocalSilo\Applications\Grains. When I invoke a grain, it gets activated and Microsoft.WindowsAzure.Storage 4.3 gets loaded from LocalSilo\Applications\Grains in parallel with 4.2 already loaded from LocalSilo. This is what we expect from .NET for assembly resolution, right, so that apps can load different versions of the same assemblies from a subfolder? If I add a bindingRedirect oldVersion="0.0.0.0-4.3.0.0" newVersion="4.2.0.0" for Microsoft.WindowsAzure.Storage to OrleansHost.exe.config, 4.3 doesn't get loaded, and my grain assembly gets redirected to the 4.2 from LocalSilo. Again, that's what we would expect from .NET. Adding the same binding redirect to grains.dll.config LocalSilo\Applications\Grains instead of OrleansHost.exe.config has no impact. I'm testing a fix for reflection-only type discovery problem where we are failing to load types from assemblies with dependencies that aren't already loaded as reflection-only even if they are present in the folder. But the above simple test works without this fix. @ReubenBond You also mentioned an issue with assembly redirects. How was it different from what I'm trying here? |
|
I never investigated the assembly redirects issue deeply - I had a quick look, but then reverted the changes and continued on using the version of Orleans which references |
|
@sergeybykov I got an error saying 4.2.0.0 was not found, which was the same @ReubenBond got, if I remember correctly. As noted, I'd like to use newest, not just 4.2.0.0 or in general the one Orleans likes to load. By the way, @ReubenBond, the new 4.2.0.0 series work on my computer, if you don't try to reference or load newer ones. About the other thing, it may be that @ca-ta needs to rearrange his diagnostics code. Some of the issues people have encountered are documented quite well here and here. I remember during the pre-GitHub days I just updated the references to Azure SDK 2.5 and everything stopped working. I didn't investigate futher back then. I'm putting words to other's mouths, though. |
|
4.2.0.0 works for me if I don't reference anything newer, but it's not easy to arbitrarily hold dependencies down - especially in a team project. The fact that the trouble is realised only at runtime makes this issue extra nefarious. -----Original Message----- @sergeybykov I got an error saying 4.2.0.0 was not found, which was the same @ReubenBond got, if I remember correctly. As noted, I'd like to use newest, not just 4.2.0.0 or in general the one Orleans likes to load. By the way, @ReubenBond, the new 4.2.0.0 series work on my computer, if you don't try to reference or load newer ones. |
|
@sergeybykov Sorry for the delayed response - I've received a lot of github mail and missed your question (so now I have better message rules in place as a result :-) ) Here's a (I believe fairly common) set-up that surfaces the issue: I my case, Orleans is the only outlier and I am very reluctant to introduce workarounds in the projects' packaging/deployment process because of it, as, in my experience, workarounds of this sort tend to be brittle and cause headaches in the long run. I understand that there may be workarounds to this (as in coming up with a folder scheme similar to the one used by localSilo, or maybe using subfolders and assembly manifests - but I don't remember how that works off the top of my head. I do remember the pain of using it a few years back though. From my perusing of the issue list and corresponding discussions I inferred (and I may be off the mark as I did not look into this in any detail) that #174 is caused, at least in part, to Orleans' automatic grain and provider detection process, which requires Orleans to load and inspect all the assemblies in the local folder. In this matter I am partial to @yevhen's opinion that a configurable list of assemblies is preferable to magic. I am thinking that using configuration as the only option would be more than acceptable (as I suppose the list of grain/provider assemblies does not change that often). Additionally, this approach may lower complexity and increase silo start-up performance (in my particular case, Orleans would only need to inspect 1 assembly, rather than 28). Then again, I am reaching - let me know if my understanding is off. @veikkoeeva thank you for the pointers - yup, I know those links well :-) . I have (or I should rather say had) diagnostics running fine on 2.5 (projects were not migrated, but rather used 2.5 from the get go). I am saying "had" because apparently something went amiss in the latest Azure OS update and for some services and roles the diagnostics extension stopped working (should one try to enable it from VS, it reports that it's all OK but fails to enable). I also "love" how every time one enables the extension a new certificate ends up being added to the role's certificate list - all things considered, I find using Azure diagnostics to be a pretty frustrating experience. But I digress, and this is already turning into a novella :-)... |
|
@ca-ta I would go on the side of configurable list. It feels this could be as easy as a few preconfigured lambda functions that, if not otherwise indicated, would be used and would the usual thing of reflection all assemblies and applying policies (or optionally not applying). I would think this would allow for further security hardening to the silos, e.g. assemblies aren't just blindly loaded in. It was touched on at Codeplex (and in #174, if I recollect correctly). l looked at just yesterday abot the long issue #42 where Yevhen gives some examples too. I don't see why this couldn't be as easy as starting off an Owin web server and off one would go. Now, obviously I'm a bit of a hang-around (currently, at least) trotting ideas. :) Though, I believe, this easiness would go well with the intention of Orleans. I also believe we would need an official branching model to do this and perhaps official contributor guidelines as in #209. I plan to come back to this, I've just been very busy with all sorts of things lately. Then I'd need to make progress with my Orleans side projects to have an actual, real-world opinion and not just a bunch of ideas. Maybe a chat (Gitter?) would be nice. I think many would engage with little questions or have less than polished ideas that people would like to poke with leisurely, without "making them official by writing". |
|
We should probably merge this discussion into #200. |
Downgrade dependency on Microsoft.WindowsAzure.ServiceRuntime from 2.5 to 2.4 to be consistent with the "-1" rule for Azure SDK dependencies. We recently ran into issues upgrading Orleans-based solutions because of this version inconsistency.