[BUG] Storage container CreateIfNotExistsAsync -> null exception in Azure.Core if container already exists #9758
Comments
triage-new-issues
bot
added
the
needs-triage
pakrym
added
Client
triage-new-issues
bot
removed
the
needs-triage
|
After looking a the stack a little bit seems that the exception is happening in the application code. Can you provide a snippet of code that calls the container client? |
|
Nothing special: var containerClient = this.Client.GetBlobContainerClient(
blobContainerName: containerName);
return await containerClient.CreateIfNotExistsAsync(
publicAccessType: PublicAccessType.Blob,
cancellationToken: cancellationToken); |
|
I think one scenario where this occurs is:
Can you try to repro this and make it handle this better or throw a more reasonable Exception? Thanks! |
|
I have the same issue - but we've not deleted anything. We're trying to use CreateIfNotExists() on a container that has existed for years. Exists() also crashes with a NullReferenceException. Here's our code: We've tried with both c.CreateIfNotExists, and the below approach - same result. "container" is the name of our container, which exists, and _connectionString is a correct and valid connection string for the storage account. Edit: Seems like Exists() works - could it be the async API that is broken? |
|
issue #109 has existed for 8 years (!) and seems to be the exact same thing. As mentioned by @hallgeirl all sorts of methods that should not throw are throwing exceptions. |
Agreed. Our Azure logs have 3,000 + entires of 409's because of this. |
|
Same, thousands of errors, sure the Azure team should want to fix this? Looks like it needs to use the List Containers first to check if it exists: |
|
Hi,
Also I'm still trying to recreate your original scenario but I'm unable to recreate it. Could you enable logging and maybe we can see what the request is returning and maybe we can get to the bottom on why Azure.Core is returning the exception it is returning. https://docs.microsoft.com/en-us/dotnet/azure/sdk/logging
https://docs.microsoft.com/en-us/rest/api/storageservices/operations-on-containers It is as intended and expected that a 409 will be thrown and be in your logs, if you call this method and the Container does not exist. Even if an Exists REST API does exist, it would be not ideal for there to be a call to check existence and then a call to create the container. It would require two calls to storage instead of one call. |
|
Thanks Amanda. It's possible that we are getting ContainerBeingDeleted in the specific case where we are trying to delete something that is being deleted. My recollection is however that we were getting a non specific Exception. But maybe that applies only to other instances where we still see this error outside of this very narrow case. We still experience this problem when we are NOT deleting anything. We do use logging in Azure Functions through Application Insights. Is that enough to capture everything you need? |
|
Hi @zmarty , What I'm looking for in the logs is the request that the SDK is sending (when calling CreateIfNotExists on a container) and the response it is getting back by sending this request. You can also include the log for the scenario ContainerBeingDeleted, if you see an error aside from that one. I want to investigate the response we are getting back to cause an exception like this (or request we are sending). Something like this: If the logs from Azure Functions show this, that would be helpful, that would suffice. |
|
We are also still getting this periodically in the logs. @amnguye I looked in our logs when this happens and I see that we call CreateIfNotExistsAsync, which results in the null exception. Internally what happens is that there is an HTTP PUT request to https://SOMETHING.blob.core.windows.net/some-name-2020-05-22?restype=container and the storage API returns HTTP 409 Conflict Hopefully that gives you a clue as to what is going on. I will also send you an internal e-mail with more screenshots. |
|
Could you help me out with this? Some customers are seeing an NullReferenceException being thrown in Azure.Core. This happens more frequently for the customers. I am not able to reproduce it even with multiple retries to see if it happens intermittently. I confirmed with the customer was able to confirm that the service gives them a 409 - ContainerAlreadyExists response. However the NullReferenceException happens before the CreateIfNotExists is able get the ErrorCode. It might be happening when it's attempting to create the response. |
|
A quick skim here makes me think it's happening because CreateIfNotExists returns null for ContainerAlreadyExists and then await container.CreateIfNotExistsAsync(...); // Make sure it already exists
BlobContainerInfo containerInfo = await container.CreateIfNotExistsAsync(...);should fail here with a nullref when the await container.CreateIfNotExistsAsync(...); // Make sure it already exists
Response<BlobContainerInfo> response = await container.CreateIfNotExistsAsync(...);
if (response != null)
{
BlobContainerInfo containerInfo = response.Value;
// ...
}@amnguye - Can you try that out? If that's what's wrong, we can brainstorm whether there's something better to return here. |
|
@tg-msft Just tried that out, yeah I was able to get the NullReferenceException, I'm not sure what would we would replace default with though. I'm open to suggestions. I might have to make the fix to all the CreateIfNotExists/DeleteIfExists/Exists methods too. |
|
A fix for this issue would require a breaking change, which is worse for customers overall. See PR #12406 for my attempt to resolve the issue. We also discussed other solutions however it would be a breaking change elsewhere. In order to sidestep the issue you would have to avoid unwrapping the This would not throw an exception afterwards you would have to go and check the if This would also side step the issue if you wrote a method that returns the Apologies for the inconvenience. |
|
@amnguye Thank you, we are giving this a try |
|
Yeah, it looks like ALL CreateIfNotExists functions right now, whether applied at the container or blob level, are not working as intended which led me to waste a couple of hours with me being convinced that I was doing something wrong. When attempting to create containers and Append Blobs(haven't confirmed with blocks yet), using the CreateIfNotExists() functionality has resulted in NullReference exceptions. I got around it by just doing what @amnguye said and not using the implicit BlobContainerInfo return and using Response. Code example below: |
|
Do I understand this correctly that you cannot provide a fix for that due to breaking changes and the recommended way to work around this is to try/catch the NullRefEx? |
|
I have the same problem but I get an AggregateException instead of a NullReferenceException. Should I open a new issue for this or is the underlying issue the same as with the NullReferenceException? |
|
The solution you proposed above @amnguye doesn't work for me. It still breaks with the same NullReferenceException. Can you please help provide a better solution that actually works? |
|
Hi @zmarty, we deeply appreciate your input into this project. Regrettably, this issue has remained inactive for over 2 years, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support. |
Describe the bug
Getting null exception from Azure.Core if trying to create a blob container that already exists, but I am using CreateIfNotExistsAsync.
Expected behavior
CreateIfNotExistsAsync succeeds
Actual behavior (include Exception or Stack Trace)
CreateIfNotExistsAsync succeeds if container does NOT already exist, but throws null exception if container DOES already exist.
To Reproduce
Steps to reproduce the behavior (include a code snippet, screenshot, or any additional information that might help us reproduce the issue)
If the container ALREADY EXISTS, it results in this Exception:
System.NullReferenceException
HResult=0x80004003
Message=Object reference not set to an instance of an object.
Source=Azure.Core
StackTrace:
at Azure.Response
1.op_Implicit(Response1 response)at FastPathCommon.AzureStorage.BlobStorageV12.d__5.MoveNext() in C:\git\FastPath\src\FastPath\FastPathCommon\AzureStorage\BlobStorageV12.cs:line 38
Environment:
Full Stack Trace
The text was updated successfully, but these errors were encountered: