Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BlobTrigger uses Event Grid subscription #2438

Closed
wants to merge 2 commits into from

Conversation

alrod
Copy link
Member

@alrod alrod commented Feb 25, 2020

Using:

  1. EventGrid subscription with HTTP endpoint should be pre-created as we do not have a tool for the EventGrid subscription creation yet.
    https://[function app uri]/runtime/webhooks/blobs?functionId=[function id in format namespace.class.method]&code=[blob_extension key]
    Also you can specify a filter to get the blobs only from specified container/path.
  2. Blob trigger definition:
    public static async Task Run([BlobTrigger("test/{name2}", UseEventGrid = true, Connection = "Storage1")]string myBlob, ILogger log)

How it works:
Blob uploaded/created -> EventGrid sends a HTTP request to function app extension endpoint -> Function runtime receives the HTTP request and adds the message to internal queue -> internal queue listener processes new messages (poison queue supported)

Copy link
Member

@fabiocav fabiocav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some small comments, mostly questions. But this is looking good. A couple of spots that would be good to get @mathewc to chime in as well.

<PackageReference Include="StyleCop.Analyzers" Version="1.1.0-beta004">
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\Microsoft.Azure.WebJobs.Host\WebJobs.Host.csproj" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this an intentional change, or something you were doing while developing? We should be able to continue depending on the package only here, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@@ -87,7 +83,7 @@ public Task FlushAsync(CancellationToken cancellationToken = default(Cancellatio
private CloudQueue Convert(string queueMoniker)
{
// $$$ Review
var account = _storageAccountProvider.Get(ConnectionStringNames.Dashboard);
var account = _storageAccountProvider.Get(ConnectionStringNames.Storage);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this a bug?

Copy link
Member Author

@alrod alrod Mar 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Actually we did not use StorageLoadBalancerQueue class before but the class was checked in in the repository.

/// <summary>
/// Hadles HttpRequests that comes to extension http endpoint.
/// </summary>
internal class HttpEndpointManager : IHttpEndpointManager
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a new class? and interface?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we use HttpEndpointManager to register http enpoint for an extension and handle to http requests for the extension.

Interface is introduced for DI as implementation is used some internal stuff.


foreach (JObject jo in events)
{
// TODO ALROD - Azure Queues do not suport batch sending. Sending in parallel? ServiceBus?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would indeed be good to send them in parallel.

What is the desired behavior if enqueuing one of the messages fails? Right now, we'd end up in a partial number of messages enqueued and an exception bubbling up, correct?

Copy link
Member Author

@alrod alrod Mar 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added sending in parallel.

Right now, we'd end up in a partial number of messages enqueued and an exception bubbling up, correct?

Correct.

@@ -14,7 +14,7 @@ namespace Microsoft.Azure.WebJobs.Host.Dispatch
/// Defines the contract for executing a triggered function using a dispatch queue.
/// <see cref="ListenerFactoryContext.GetDispatchQueue(IMessageHandler)"/>
/// </summary>
internal interface IMessageHandler
public interface IMessageHandler
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the visibility of these interfaces changing for to allow the extension to consume them?

@mathewc can you please comment here as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we did not use the interface before (there was only E2E test). IMessageHandler must be public in order to use in an extension.

@alrod
Copy link
Member Author

alrod commented Mar 16, 2022

the change was done in track2

@alrod alrod closed this Mar 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants