New Report Class - use case of user-defined data

[Gerritmerecs]

Thanks to the youtube video linked in the wiki, i could quickly implement a customized report.
As a user i have a few custom results, i wish to save to a specific test.

And here is my confusion. Is the user-defined variant, the place to save my custom results? In the rest of the frame work it was initially handled as a key value pair. So the same way as the properties variant. Because of this, i needed to modify the way the Process.vi handled Assertions for that.

I feel insecure about if i should store these custom results in the user-defined property or further modify the class to store my custom stuff. What is the use case of the user-defined property?

Many thanks in advance. I am happy to give further info about my use case if necessary.

[francois-normandin]

Hi @Gerritmerecs ,

I'm not certain why you would need to modify the Process.vi of the Test Manager, but I hope the explanation below helps to avoid going that route to solve your problem. The process VI is a private process to the test manager and any changes to it would leave you in a situation where you can never upgrade the Caraya kernel in the future.

In the "Test Result.lvclass:Assert Event Data" type definition (in your screenshot), the user-defined variant dictionary is meant for your custom Test Report class to use as you see fit. It is there primarily to preserve backward-compatibility with the test results before the addition of properties and measurements attributes. Although I do believe you should use properties to store your extensions as key-value pairs.

What I believe I understand from your question (do let me know if this is a wrong assertion!) can be addressed by extending the Test Report class as you mention, and by using property methods in your tests to populate the result`s properties variant. A conjunction of these would eliminate the need to modify the kernel.

If this is the basic structure of your test suite:

Then those two example VIs in the project would reflect the proposed way to use Key-Value Pair properties.

For each Test Node or each Assertion Node, you can attach a key-value pair with the property class method as shown below:

The custom key-value pairs and measurements are attached and reported:

And finally, if you want to use the user-defined attributes, you can do it within your Custom TestReport class.
One could take the properties and measurements, and compute derived attributes that would then be stored in the user-defined field.

To sum it up:

[Gerritmerecs]

Thank you very much for clarifying this for me. You got me 100%.

[..]A conjunction of these would eliminate the need to modify the kernel.[..]
I think with the same argument, you would argue putting my customized report class not into the lvlib/PPL? Am i right?

I dont know how fast i can rework this. I'll post some update here :)

Many thanks

[francois-normandin]

Glad that I could be of help.

Yes, you would create your own library to host your code and this library would have a dependency on the installed version of Caraya of your choice: either the one installed in vi.lib using VIPM, or a ppl version of Caraya that you would keep in a location relative to your own library.

[Gerritmerecs]

I could simplify my code very much.
That is my current process working with Jinja templates. I am making progress.

One question, i still have, is about the user-defined variant.
How would i write the Test Suite Results back from my test-report's export.vi? At least that's how i understand your last two screenshots.

Do i have access to the returned instance from destroy test suite? As i see it, it is send from the private process.vi and would be impossible to change that event's data, because that event has been sent before the Export.vi is executed.

[francois-normandin]

You cannot write back modifications after the report is generated.
The Test Manager core assembles the test results and delivers it when the TestSuite completes. At that moment, the TestSuite is destroyed and there is no writing back.

The export method only transforms the TestResult into a user-defined output (file, event, etc.). It cannot be used to modify the result and store it back into the Test Suite. If we allowed that, then two reports exported at different moments could yield different results. That would be a leaky abstraction.

Think of the Export method as the same as if it happened after the TestSuite is destroyed, except that Caraya calls a method to help the developer automate the processing of Test Result class without having to formally write a TestSuite.

[francois-normandin]

In a nutshell, the results cannot be changed, they can only be processed differently by the user, according to instructions in the Report class overrides.