Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial implementation for workflow log tracing #1176

Merged
merged 8 commits into from
Nov 14, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions src/Dapr.Workflow/WorkflowLoggingService.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
// ------------------------------------------------------------------------
// Copyright 2022 The Dapr Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ------------------------------------------------------------------------

namespace Dapr.Workflow
{
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Configuration;

/// <summary>
/// Defines runtime options for workflows.
/// </summary>
internal sealed class WorkflowLoggingService : IHostedService
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if hosted service is the right way to go here. I was thinking more that this would be injected into the various workflow runtime mechanics, like the context.

{
private readonly ILogger<WorkflowLoggingService> logger;
private static readonly HashSet<string> registeredWorkflows = new();
private static readonly HashSet<string> registeredActivities = new();

public WorkflowLoggingService(ILogger<WorkflowLoggingService> logger, IConfiguration configuration)
{
this.logger = logger;

}
public Task StartAsync(CancellationToken cancellationToken)
{
this.logger.Log(LogLevel.Information, "WorkflowLoggingService started");

this.logger.Log(LogLevel.Information, "List of registered workflows");
foreach (string item in registeredWorkflows)
{
this.logger.Log(LogLevel.Information, item);
}

this.logger.Log(LogLevel.Information, "List of registered activities:");
foreach (string item in registeredActivities)
{
this.logger.Log(LogLevel.Information, item);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Indentation is weird here.


return Task.CompletedTask;
}

public Task StopAsync(CancellationToken cancellationToken)
{
this.logger.Log(LogLevel.Information, "WorkflowLoggingService stopped");

return Task.CompletedTask;
}

public static void LogWorkflowName(string workflowName)
{
registeredWorkflows.Add(workflowName);
}

public static void LogActivityName(string activityName)
{
registeredActivities.Add(activityName);
}

}
}
4 changes: 4 additions & 0 deletions src/Dapr.Workflow/WorkflowRuntimeOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ public void RegisterWorkflow<TInput, TOutput>(string name, Func<WorkflowContext,
WorkflowContext workflowContext = new DaprWorkflowContext(innerContext);
return implementation(workflowContext, input);
});
WorkflowLoggingService.LogWorkflowName(name);
});
}

Expand All @@ -73,6 +74,7 @@ public void RegisterWorkflow<TInput, TOutput>(string name, Func<WorkflowContext,
TWorkflow workflow = Activator.CreateInstance<TWorkflow>();
return new OrchestratorWrapper(workflow);
});
WorkflowLoggingService.LogWorkflowName(name);
});
}

Expand All @@ -91,6 +93,7 @@ public void RegisterActivity<TInput, TOutput>(string name, Func<WorkflowActivity
WorkflowActivityContext activityContext = new(innerContext);
return implementation(activityContext, input);
});
WorkflowLoggingService.LogActivityName(name);
});
}

Expand All @@ -111,6 +114,7 @@ public void RegisterActivity<TActivity>() where TActivity : class, IWorkflowActi
TActivity activity = ActivatorUtilities.CreateInstance<TActivity>(serviceProvider);
return new ActivityWrapper(activity);
});
WorkflowLoggingService.LogActivityName(name);
});
}

Expand Down
2 changes: 1 addition & 1 deletion src/Dapr.Workflow/WorkflowServiceCollectionExtensions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ public static IServiceCollection AddDaprWorkflow(
#pragma warning disable CS0618 // Type or member is obsolete - keeping around temporarily - replaced by DaprWorkflowClient
serviceCollection.TryAddSingleton<WorkflowEngineClient>();
#pragma warning restore CS0618 // Type or member is obsolete

serviceCollection.AddHostedService<WorkflowLoggingService>();
serviceCollection.TryAddSingleton<DaprWorkflowClient>();
serviceCollection.AddDaprClient();
serviceCollection.AddDaprWorkflowClient();
Expand Down
7 changes: 7 additions & 0 deletions test/Dapr.E2E.Test.App/Dapr.E2E.Test.App.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,11 @@
<ProjectReference Include="..\Dapr.E2E.Test.Actors\Dapr.E2E.Test.Actors.csproj" />
<ProjectReference Include="..\..\src\Dapr.Workflow\Dapr.Workflow.csproj" />
</ItemGroup>

<ItemGroup>
<PackageReference Include="Serilog" Version="3.0.1" />
<PackageReference Include="Serilog.AspNetCore" Version="7.0.0" />
<PackageReference Include="Serilog.Sinks.Console" Version="4.1.0" />
<PackageReference Include="Serilog.Sinks.File" Version="5.0.0" />
</ItemGroup>
</Project>
27 changes: 14 additions & 13 deletions test/Dapr.E2E.Test.App/Program.cs
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
// ------------------------------------------------------------------------
// Copyright 2021 The Dapr Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ------------------------------------------------------------------------
// ------------------------------------------------------------------------
// Copyright 2021 The Dapr Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ------------------------------------------------------------------------
namespace Dapr.E2E.Test
{
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Hosting;
using Serilog;
public class Program
{
public static void Main(string[] args)
Expand All @@ -26,6 +27,6 @@ public static IHostBuilder CreateHostBuilder(string[] args) =>
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
}).UseSerilog();
}
}
14 changes: 14 additions & 0 deletions test/Dapr.E2E.Test.App/Startup.cs
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ namespace Dapr.E2E.Test
using Microsoft.Extensions.Hosting;
using System.Threading.Tasks;
using System;
using Microsoft.Extensions.Logging;
using Serilog;

/// <summary>
/// Startup class.
Expand Down Expand Up @@ -61,6 +63,10 @@ public void ConfigureServices(IServiceCollection services)
services.AddAuthentication().AddDapr();
services.AddAuthorization(o => o.AddDapr());
services.AddControllers().AddDapr();
services.AddLogging(builder =>
{
builder.AddConsole();
});
// Register a workflow and associated activity
services.AddDaprWorkflow(options =>
{
Expand Down Expand Up @@ -108,11 +114,19 @@ public void ConfigureServices(IServiceCollection services)
/// <param name="env">Webhost environment.</param>
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
var logger = new LoggerConfiguration().WriteTo.File("log.txt").CreateLogger();

var loggerFactory = LoggerFactory.Create(builder =>
{
builder.AddSerilog(logger);
});
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}

app.UseSerilogRequestLogging();

app.UseRouting();

app.UseAuthentication();
Expand Down
59 changes: 55 additions & 4 deletions test/Dapr.E2E.Test/Workflows/WorkflowTest.cs
Original file line number Diff line number Diff line change
Expand Up @@ -11,25 +11,76 @@
// limitations under the License.
// ------------------------------------------------------------------------
using System;
using System.IO;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Dapr.Client;
using FluentAssertions;
using Xunit;
using System.Linq;
using System.Diagnostics;

namespace Dapr.E2E.Test
{
[Obsolete]
public partial class E2ETests
{
[Fact]
public async Task TestWorkflowLogging()
{
// This test starts the daprclient and searches through the logfile to ensure the
// workflow logger is correctly logging the registered workflow(s) and activity(s)

Dictionary<string, bool> logStrings = new Dictionary<string, bool>();
logStrings["PlaceOrder"] = false;
logStrings["ShipProduct"] = false;
var logFilePath = "../../../../../test/Dapr.E2E.Test.App/log.txt";
var allLogsFound = false;
var timeout = 30; // 30s
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(timeout));
using var daprClient = new DaprClientBuilder().UseGrpcEndpoint(this.GrpcEndpoint).UseHttpEndpoint(this.HttpEndpoint).Build();
var health = await daprClient.CheckHealthAsync();
health.Should().Be(true, "DaprClient is not healthy");

var searchTask = Task.Run(() =>
{
using (StreamReader reader = new StreamReader(logFilePath))
{
string line;

while ((line = reader.ReadLine()) != null)
{
foreach (var entry in logStrings)
{
if (line.Contains(entry.Key))
{
logStrings[entry.Key] = true;
}
}
allLogsFound = logStrings.All(k => k.Value);
if (allLogsFound)
{
break;
}
}
}
}, cts.Token);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, this CancellationToken is only used if the Task has not yet already started, which it most likely has, so it's possible that the task never ends (in the case of not all expected logs being found).

You could use the async method Stream.ReadLineAsync(CancellationToken) to provide a means of abandoning the search when time expires. That would also allow you to avoid calling Task.WaitAll(), which is a synchronously-blocking method.


if (!Task.WaitAll(new Task[] {searchTask}, timeout))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: maybe hold off testing the result until after the log file has been deleted (rather than two deletion paths).

{
File.Delete(logFilePath);
Assert.True(false, "Expected logs weren't found within timeout period");
}
File.Delete(logFilePath);
}
[Fact]
public async Task TestWorkflows()
{
string instanceId = "testInstanceId";
string instanceId2 = "EventRaiseId";
string workflowComponent = "dapr";
string workflowName = "PlaceOrder";
var instanceId = "testInstanceId";
var instanceId2 = "EventRaiseId";
var workflowComponent = "dapr";
var workflowName = "PlaceOrder";
object input = "paperclips";
Dictionary<string, string> workflowOptions = new Dictionary<string, string>();
workflowOptions.Add("task_queue", "testQueue");
Expand Down
Loading