[Feature]: Record Actions and Generate Test Engine Script with Simulated Data

[Grant-Archibald-MS]

Is your feature request related to a problem? Please describe.

The key problem today is the need to export from Test Studio, which is limited to Power Apps Canvas Apps. We want to record actions in Canvas Apps, Custom Pages, and Model Driven Apps. Many applications make use of Dataverse and connectors. By performing integration tests, we need to control and maintain the state

By recording these interactions, testers can create more accurate and reliable test scenarios. This will help in identifying issues early in the development process and ensure that the application behaves as expected under various conditions.

Describe the solution you'd like

Simulation of Data

Simulation of data involves recording calls to connectors and Dataverse, and then using these recordings to simulate calls during testing. This feature leverages issue #381 to enhance login debugging by including additional log information. By simulating calls, we can create isolated tests that accurately reflect real-world scenarios.

Importance

This feature is crucial because it allows for comprehensive testing of applications that rely on Dataverse and connectors. Without the ability to simulate calls, tests are limited to the "happy path" and cannot account for edge cases or exceptions. By simulating calls, we can ensure that the application behaves correctly under various conditions, leading to more robust and reliable software.

Impact

The impact of this feature is significant. It enables testers to create isolated tests that can simulate different scenarios, including edge cases and exceptions. This leads to higher quality applications, as potential issues can be identified and addressed early in the development process. Additionally, it reduces the time and effort required for manual testing, as automated tests can cover a wider range of scenarios.

Examples

• Simulate a Dataverse Patch Action:

  Experimental.SimulateDataverse({
       Action: "Patch",
      Entity: "Processes",
      When: {Status: 1},
      Then: {} // Return value
  });
  
  

• Simulate a Dataverse Query Action

Experimental.SimulateDataverse({
     Action: "Query",
    Entity: "Workflow",
    When: Table({Status: "Active", CreatedOn: "> 2023-01-01"}),
    Then: Table({Name: "Test", Owner: "John Doe"}) // Return Value
});

• Similute a Workflow

Experimental.SimulateWorkflow({
    Name: "SetupWizard>GetTenantID",
    Then: {Id:"a1234567-1111-2222-3333-4444555666"}
});

• Simulate a Connector:

Experimental.SimulateConnector({
   Name: "Office365Groups",
   When: {Action: "ListOwnedGroupsV2"},
   Then: Table({Name: "Test"})
});

User Interaction

User interaction recording involves capturing clicks and keyboard entries on the page and mapping these interactions to control test steps. For example, a click on a button can be mapped to Select(Button1), and text entry into an input box can be mapped to SetProperty(TextInput1.Text, "New Value").

Importance

Recording user interactions is essential for creating comprehensive test scripts. It ensures that all user actions are accurately captured and can be replayed during testing. This is particularly important for applications with complex user interfaces, where manual testing can be time-consuming and error-prone.

Impact

By expanding the recording of user interactions, we can automate the creation of test scripts. This reduces the manual effort required to write test scripts and ensures that all user interactions are accurately captured. It also allows for more thorough testing, as automated tests can cover a wider range of user actions and scenarios.

Examples

• Map a Click on a Button:

Select(Button1);

• Map Text Entry into an Input Box:

SetProperty(TextInput1.Text, "New Value");

Generation of Test Scripts and Test Data

Generating test scripts and test data involves creating a Power Fx test script based on recorded interactions and saving the generated test plan file in the TestOutput folder along with recorded input data. This provides a standardized way to create test cases and ensures consistency and reliability in testing.

Importance

This feature is important because it streamlines the testing process. By generating test scripts automatically, we can ensure that all test scenarios are covered and reduce the time and effort required to create test scripts manually. It also provides a standardized format for test scripts, making it easier to maintain and update tests as the application evolves.

Impact

The impact of this feature is substantial. It improves the efficiency and effectiveness of the testing process by providing a standardized test script that can be used to generate test cases. This leads to more reliable and consistent testing, as all test scenarios are covered and potential issues can be identified and addressed early in the development process. Additionally, it reduces the manual effort required for testing, allowing testers to focus on more complex and critical tasks.

Examples

1. Generated Test Plan File:

connector_yyyyMMdd_HHmmss.csv

1. Power Fx Test Script:

    testSteps:
    Select(Button1);
    SetProperty(TextInput1.Text, "New Value");
    Experimental.SimulateDataverse({
         Action: "Patch",
        Entity: "Processes",
        When: {Status: 1},
        Then: {} // Return value
    });
    Experimental.SimulateDataverse({
         Action: "Query",
        Entity: "Workflow",
        When: Table({Status: "Active", CreatedOn: "> 2023-01-01"}),
        Then: Table({Name: "Test", Owner: "John Doe"}) // Return Value
    });
    Experimental.SimulateWorkflow({
        Name: "SetupWizard>GetTenantID",
        Then: {Id:"a1234567-1111-2222-3333-4444555666"}
    });
    Experimental.SimulateConnector({
       Name: "Office365Groups",
       When: {Action: "ListOwnedGroupsV2"},
       Then: Table({Name: "Test"})
    });

Describe alternatives you've considered

No response

Additional context?

No response

[Grant-Archibald-MS]

Work in progress implementation now part of https://github.com/microsoft/PowerApps-TestEngine/tree/integration>

Will leave feature open to ensure compatibility with multiple provider types for canvas, powerapps.portal and mda