Frontend Testing Tracking Issue

[dawoodkhan82]

Tracking Issue for Frontend tests (E2E, Unit tests, & Visual tests)

Goals

• Have several fully fleshed out E2E tests (using Playwright) that follow how an actual user would use a demo
• Unit tests for every frontend component that tests util functions, internal logic, and the correct passing of props.
• Visual tests using Chromatic. Helpful for UI and visual regressions.

TODOs

End to End Tests
Run tests by:

• pnpm test:browser:full builds frontend and runs tests
• pnpm test:browser:verbose see all errors/warnings
• pnpm test:browser:debug opens chromium browser and allows you to step through each line of the test

Use kitchen_sink.spec.ts as reference.

• chatbot_multimodal demo test #4659
• gr.Interface test (gr.load()) maybe for demo/gpt_j
• Add more tests to outbreak_forecast.spec.ts #4746
• streaming use cases
• Event cancelling
• Making sure filename extensions and names are preserved (model3D can be good for this cause of .obj extension)
• .then .success events are triggered correctly #4687
• Clear button unit tests #4748
• Brainstorm more user storyboards for E2E testing

Unit Tests
Run tests by:
pnpm test:run

Use Textbox.test.ts as reference. List based on most used components

• Slider Unit Tests #4679
• Radio #4711
• Chatbot component unit tests #4660
• File - single vs multiple vs directory
• UploadButton unit tests #4701
• Dropdown
• Checkbox/CheckboxGroup
• Audio front end unit tests #4648
• Video Unit tests #4707
• js client unit tests #4658
• Label

Visual Testing

• dark vs light mode text looks good
• Loading custom themes looks good
• blocks_layout demo
• Gallery styling https://github.com/gradio-app/gradio/pull/4576/files#diff-78d9b29bfe32556861b3814d264f2093e1513af3b943aec92a501a3d2997c7f5
• Annotated image on small text size Fix AnnotatedImage styling + add demo #4628

[pngwn]

Some testing resources.

Playwright docs are excellent (and extensive). The API docs will be the most helpful as we write tests. The page class in particularly will probably be the mots commonly used.
Page API: https://playwright.dev/docs/api/class-page

When you actually get an element that you want to interact with it will return an instance of Locator which has lots of useful methods:
Locator API: https://playwright.dev/docs/api/class-locator

Playwright also have a nice document about best practices with some examples:
Best Practices: https://playwright.dev/docs/best-practices

They also have other useful guides (check the sidebar and scroll down). Everything i know about playwright is from trial + error and the official docs.

For vitest (untit and integration tests that run in node) we generally use two things.

@testing-library/svelte for getting elements and performing interactions and vitest's expect module

Testing library DOM functions: https://testing-library.com/docs/queries/about
expect API: https://vitest.dev/api/expect.html

Have a look around other parts of these sites too, testing tools like this often have some nice examples and good best practices that can be useful.

[pngwn]

Doing a bit of an inventory of our test coverage. Probably won't be complete but it is a start.

Component	value	visible	elem_id	elem_classes	container	label	show_label
AnnotatedImage	❌	❌	❌	❌	❌	❌	❌
Audio	❌	❌	❌	❌	❌	❌	❌
BarPlot	❌	❌	❌	❌	❌	❌	❌
Button	❌	❌	❌	❌	❌	❌	❌
Chatbot	❌	❌	❌	❌	❌	❌	❌
Checkbox	❌	❌	❌	❌	❌	❌	❌
CheckboxGroup	❌	❌	❌	❌	❌	❌	❌
ClearButton	❌	❌	❌	❌	❌	❌	❌
Code	❌	❌	❌	❌	❌	❌	❌
ColorPicker	❌	❌	❌	❌	❌	❌	❌
Dataframe	❌	❌	❌	❌	❌	❌	❌
Dataset	❌	❌	❌	❌	❌	❌	❌
Dropdown	❌	❌	❌	❌	❌	❌	❌
File	❌	❌	❌	❌	❌	❌	❌
Gallery	❌	❌	❌	❌	❌	❌	❌
HTML	❌	❌	❌	❌	❌	❌	❌
HighlightedText	❌	❌	❌	❌	❌	❌	❌
Image	❌	❌	❌	❌	❌	❌	❌
Interpretation	❌	❌	❌	❌	❌	❌	❌
JSON	❌	❌	❌	❌	❌	❌	❌
Label	❌	❌	❌	❌	❌	❌	❌
Lineplot	❌	❌	❌	❌	❌	❌	❌
Markdown	❌	❌	❌	❌	❌	❌	❌
Model3D	❌	❌	❌	❌	❌	❌	❌
Number	❌	❌	❌	❌	❌	❌	❌
Plot	❌	❌	❌	❌	❌	❌	❌
Radio	❌	❌	❌	❌	❌	❌	❌
ScatterPlot	❌	❌	❌	❌	❌	❌	❌
Slider	❌	❌	❌	❌	❌	❌	❌
State	❌	❌	❌	❌	❌	❌	❌
Textbox	❌	❌	❌	❌	❌	❌	❌
Timeseries	❌	❌	❌	❌	❌	❌	❌
UploadButton	❌	❌	❌	❌	❌	❌	❌
Video	❌	❌	❌	❌	❌	❌	❌

[pngwn]

Pretty happy that we are on our way with this now. I don't think we need to track it any more, we have pretty much everything in place.