-
-
Notifications
You must be signed in to change notification settings - Fork 92
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Improve Promitor's metric configuration validation (#893)
* Initial implementation of new validation approach I've altered the way that the deserializers work and are defined so that they are simpler to create, but also so that we can record validation failures during the deserialization process. This allows us to provide much better validation messages than we can if we take the approach of validating based on the deserialized objects. In this initial change I haven't gone ahead and updated all of the deserializers because I wanted the change to be kept small. I've also not actually hooked up the validation messages in any way - instead I've just provided a stub ErrorReporter object that doesn't actually do anything yet. Part of #592 * Converted AzureMetricConfigurationDeserializer Converted AzureMetricConfigurationDeserializer to the new way of defining a deserializer, and added support in the base Deserializer for using another deserializer to deserialize child properties. I had to add a new `IDeserializer` interface that isn't generic to allow us to deserialize a child property at runtime without knowing its type. Part of #592 * Converted MetricAggregationDeserializer Part of #592 * Converted MetricDefaultsDeserializer Part of #592 * Converted MetricDefinitionDeserializer Also: - Fixed the metrics definition docs to indicate that 'labels' is optional. Part of #592 * Converted ScrapingDeserializer Part of #592 * Converted SecretDeserializer Part of #592 * Converted all of the resource type deserializers In order to do this I had to alter the type parameter in `IDeserializer<TObject>` to be covariant so that I could define the resource deserializers as their specific type (for example `IDeserializer<ContainerInstanceResourceV1>`), but still allow them to be returned as an `IDeserializer<AzureResourceDefinitionV1>` from the resource deserializer factory. I also ended up altering a few of the newer deserializers that are using inheritance (for example, SqlServer / SqlDatabase deserializers) to inherit directly from `ResourceDeserializer<TResource>` again. The reason I did this is that the `Deserializer` class needs to know the specific type of object to construct, and I decided that the extra complication of adding more generics to get it to work wasn't worth the hassle to save one or two lines of code. Part of #592 * Hook up the new validation to Promitor This change implements the ErrorReporter, and connects it to Promitor's startup validation so that the errors are actually reported. In addition: - Report error when a deserializer cannot be found. - Ignore the 'resources' field on metrics because we are manually deserializing it. - Make `AzureMetricConfiguration.Dimension` optional. - Fix a test in `ServiceBusQueueMetricsDeclarationValidationStepTests` that was asserting that validation was successful, when the test was meant to test for a validation failure. Part of #592 * Don't reuse context when deserializing Because the deserializers are singletons, the set of FieldContext objects was being reused when deserializing multiple nodes of the same type. This lead to a bug where if we had multiple metric definitions, and the first one to be deserialized set a required field, an error would not be reported if that required field was missing from subsequent definitions. To solve this I've pulled the FieldContext class out of the Deserializer, and I've split it into two separate objects: - `FieldDeserializationInfo` - contains immutable information about the field to be deserialized, and can be reused between multiple deserializations. - `FieldDeserializationContext` - contains any information about a specific deserialization attempt, and must be created each time we deserialize a node using a deserializer. In addition, I've also altered the way the errors / warnings are output to avoid issues I was seeing where the messages weren't always output in the correct order because of the way the logger works. Part of #592 * Add validation improvements to changelog Part of #592 * Refactor the validation error tests I've added some new methods to the `YamlAssert` class to make it easier to test that errors are reported when required properties are not supplied. I've also removed some TODOs for functionality that can be added later after this first set of validation changes are merged. Part of #592 * Fixing code-quality issues * fixup! Fixing code-quality issues
- Loading branch information
1 parent
804e699
commit a2f4569
Showing
78 changed files
with
1,927 additions
and
629 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
61 changes: 61 additions & 0 deletions
61
src/Promitor.Core.Scraping/Configuration/Serialization/DeserializationContext.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
using System.Collections.Generic; | ||
using System.Linq; | ||
|
||
namespace Promitor.Core.Scraping.Configuration.Serialization | ||
{ | ||
/// <summary> | ||
/// Contains the information needed to deserialize a node. | ||
/// </summary> | ||
public class DeserializationContext<TObject> | ||
{ | ||
private readonly ISet<string> _ignoredFields; | ||
private readonly Dictionary<string, FieldDeserializationContext<TObject>> _fields; | ||
|
||
/// <summary> | ||
/// Initializes a new instance of the <see cref="DeserializationContext{TObject}"/> type. | ||
/// </summary> | ||
/// <param name="ignoredFields">The fields that should be ignored.</param> | ||
/// <param name="fields">The fields that can be deserialized.</param> | ||
public DeserializationContext(ISet<string> ignoredFields, IReadOnlyCollection<FieldDeserializationInfo> fields) | ||
{ | ||
_ignoredFields = ignoredFields; | ||
_fields = fields.ToDictionary( | ||
fieldInfo => fieldInfo.YamlFieldName, fieldInfo => new FieldDeserializationContext<TObject>(fieldInfo)); | ||
} | ||
|
||
/// <summary> | ||
/// The fields that have not been set during deserialization. | ||
/// </summary> | ||
public IReadOnlyCollection<FieldDeserializationContext<TObject>> UnsetFields | ||
{ | ||
get | ||
{ | ||
return _fields | ||
.Where(f => !f.Value.HasBeenSet) | ||
.Select(f => f.Value) | ||
.ToList(); | ||
} | ||
} | ||
|
||
/// <summary> | ||
/// Returns whether the specified field is ignored. | ||
/// </summary> | ||
/// <param name="fieldName">The field name.</param> | ||
/// <returns>true if the field is ignored, false otherwise.</returns> | ||
public bool IsIgnored(string fieldName) | ||
{ | ||
return _ignoredFields.Contains(fieldName); | ||
} | ||
|
||
/// <summary> | ||
/// Tries to find the specified field. | ||
/// </summary> | ||
/// <param name="fieldName">The field name.</param> | ||
/// <param name="field">Set to the field if found, otherwise null.</param> | ||
/// <returns>true if the field was found, false otherwise.</returns> | ||
public bool TryGetField(string fieldName, out FieldDeserializationContext<TObject> field) | ||
{ | ||
return _fields.TryGetValue(fieldName, out field); | ||
} | ||
} | ||
} |
48 changes: 48 additions & 0 deletions
48
src/Promitor.Core.Scraping/Configuration/Serialization/DeserializationMessage.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
using YamlDotNet.RepresentationModel; | ||
|
||
namespace Promitor.Core.Scraping.Configuration.Serialization | ||
{ | ||
/// <summary> | ||
/// Represents a message reported during the deserialization process. | ||
/// </summary> | ||
public class DeserializationMessage | ||
{ | ||
/// <summary> | ||
/// Initializes a new instance of the <see cref="DeserializationMessage"/> type. | ||
/// </summary> | ||
/// <param name="messageType">The type of message reported.</param> | ||
/// <param name="node">The node the message is associated with.</param> | ||
/// <param name="message">The message.</param> | ||
public DeserializationMessage(MessageType messageType, YamlNode node, string message) | ||
{ | ||
MessageType = messageType; | ||
Node = node; | ||
Message = message; | ||
} | ||
|
||
/// <summary> | ||
/// Gets the type of message. | ||
/// </summary> | ||
public MessageType MessageType { get; } | ||
|
||
/// <summary> | ||
/// Gets the node the message has been reported against. | ||
/// </summary> | ||
public YamlNode Node { get; } | ||
|
||
/// <summary> | ||
/// Gets the message. | ||
/// </summary> | ||
public string Message { get; } | ||
|
||
/// <summary> | ||
/// Gets the message formatted for output to users. | ||
/// </summary> | ||
public string FormattedMessage => $"{MessageType} {Node.Start.Line}:{Node.Start.Column}: {Message}"; | ||
|
||
public override string ToString() | ||
{ | ||
return FormattedMessage; | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.