[BUG] Discover 2.0 - Auto re-sizing of columns happens inappropriately (and frequently) when using saved queries

[gsmith-sas]

Describe the bug

In the new Discover interface in OpenSearch Dashboards 2.11, the columns are automatically resized too frequently: e.g. when refreshing the data, when clicking on the icon to "Inspect document details". This only seems to happen when working with a saved query. It does not happen if you start fresh and add columns to the table. It does not happen if you modify a saved query and add a new column. This behavior was not there in OpenSearch Dashboards 2.10.

To Reproduce
Steps to reproduce the behavior:

1. Go to the OpenSearch Dashboards Playground (https://playground.opensearch.org/app/home)
2. Click on 'Discover', click on Open and select the '[eCommerce] Orders' saved query
3. Resize one or more of the columns in some very noticeable way (e.g. shrink the "category" column down to very narrow, make "total quantity" very wide, etc.)
4. Click on the icon at the far left to "Inspect document details".
5. Notice that the columns have resized back to their original widths. You may need to close the Document Details window to see this.
6. Resize the columns again and then click the REFRESH button (upper left).
7. Notice the same resizing has happened.

Expected behavior
This is very annoying. After carefully resizing the columns to make it easier to understand what's being presented, all of my changes are lost as soon as I attempt to interact with the screen. My expectation (and I assume the expectation of any reasonable user) would be that my column resizings would remain in-place until I chose to change them. Or, perhaps, until I left the screen and came back. Realistically, I suspect everyone would expect the column widths to be persisted with the saved query.

OpenSearch Version
OpenSearch 2.11.0

Dashboards Version
OpenSearch 2.11.0

Screenshots
After resizing columns

After clicking on "Inspect Document Details"

Host/Environment (please complete the following information):

• Browser and version [e.g. 22]: Firefox 119.0.1 and Chrome 118.6045.160

Additional context

We use saved queries extensively to make it easier for our users. This bug makes the Discover interface unusable for those users. We did find a partial mitigation: adding a column to the saved query and then removing it (without resaving the query) disables the problematic auto-sizing. However, that is not an acceptable user experience.

[ananzh]

@gsmith-sas thanks for opening this. I will take a look.

[SebastianThorn]

This would be a great addition, have the same issue, crafting good views, but when loading them there is too much white-space for some fields.

[gsmith-sas]

@ananzh Is there any news to report on this bug? We found it makes OpenSearch Dashboards 2.11.0 unusable. We hope to move to OpenSearch 2.12.0 when it comes out in January but only if this issue has been fixed.

[ananzh]

Sry @gsmith-sas. I haven't got any capacities working on this. Might need to take a bit of time. I will check with the team.

[adamdepollo]

My team is having this same issue and it is a major impediment to usability of the discover feature. We have a lot of saved searches we use regularly.

[gsmith-sas]

@ananzh I see that you added the v2.12.0 label to this issue and assigned @abbyhu2000 to work on it a few weeks back. Those are very encouraging developments. Thank you.
As I mentioned earlier, this is show-stopper for us that is preventing us from updating OpenSearch Dashboards. And I suspect the general security concerns about running an older version will become an increasing challenge for us.
How are things going? Have you been able to isolate the problem and find a solution? I'm hoping the delay in the 2.12.0 delivery date will give you enough time to resolve this.

[abbyhu2000]

@gsmith-sas Hi, thanks for your patience! I was assigned to this bug along with a couple of discover bugs. I was able to reproduce this bug and i am still looking into the solution. This bug fix are still tracked in the 2.12 release. We will let you know as soon as we have any update.

[ananzh]

@gsmith-sas sorry for the late respond. I checked this issue and it seems it only happens in playground. Main has no such issue. Both myself and @MadaniKK have verified this behavior on main. Seems to me that when use main, column width in local storage but playground will not read local storage.

@AMoo-Miki Would like to confirm that playground picks up the latest main, right?

[gsmith-sas]

@ananzh I have confirmed that this problem occurs in OpenSearch Dashboards 2.11.1 (current release) as well as in OpenSearch Dashboards 2.11.0. The playground is currently running OpenSearch Dashboards 2.11.0. I would expect the playground to show the current "production" version which would be OpenSearch Dashboards 2.11.1.

@abbyhu2000 have you pushed a fix? Is that why @ananzh is reporting that she doesn't see the issue in the "main" branch? Does this mean the problem will be fixed in the 2.12.0 release? Thanks!

[ananzh]

@gsmith-sas we haven't try any fixes on this bug but apparently it is resolved in some fixes in Discover for 2.12. We will soon release 2.12 which should have no such bug. I tried to find the specific PR but given we only selected some fixes into 2.11 while working on 2.12 fixes, it would require extra efforts to locate which PR resolve it. If you don't mind, I would like to resolve it for now. I will test it once we cut 2.12 branch. If you see repeated bug in 2.12, we could re-open this issue. What do you think?

[gsmith-sas]

@ananzh You mentioned that you "will test it once we cut 2.12 branch." Thank you, that will be helpful. We're planning our work for the next month or two and I'm hoping the problem is resolved so we can move to the 2.12 release. I'd prefer to keep this issue open until you're able to complete that testing since it will impact our ability to upgrade. Since I'm consuming OpenSearch Dashboards via the Helm chart, I won't be able to validate that the bug has been fixed until 2.12 is actually released which is ~2 weeks after the release candidate branch is available based on the release schedule.

[ananzh]

@gsmith-sas sure. Let's keep it open and close it after 2.12 test. Thanks a lot for opening it.

[abbyhu2000]

@ashwin-pc @ananzh @gsmith-sas I just tested against main branch and 2.12 branch, and for both branch it seems like the bug does not exist anymore.

Screen.Recording.2024-02-15.at.2.11.10.PM.mov

[gsmith-sas]

@abbyhu2000 Excellent! Thank you for your efforts.

I noticed in your screen capture video that you were not using a saved query. I've taken down my 2.11.1 environment so I can't confirm this but my recollection is that the bug was mostly an issue when working with a saved query and may not have occurred when working with an ad-hoc (unsaved) query. Did you also test the saved query use-case? If your test environment doesn't have any saved queries, a quick test would be to create a view (adding fields, adjusting column widths, etc.) and then save the query (via the Save menu item). Then, after making more changes (or, perhaps, even logging off and then logging back in), reloading the saved query (via the Open menu item) and seeing if the columns retain their saved widths a) upon loading and b) even after refreshing the data and/or displaying the document details of one row.

[abbyhu2000]

@gsmith-sas Thanks for the response! After now i fully understand your question, this is not a bug but rather a feature request.

Currently we do not have the functionalities of persisting the user adjusted column size in discover page. After refreshing the page, the previously adjusted column sizes will be reset to default; same thing after saving a saved search object(or saved query in your response above), the column sizes will not be saved, and always show the default column sizes upon loading.

I do recognize that this can cause inconvenience, should we add this feature of persisting user adjested column width in our to do list for discover? @ashwin-pc @wbeckler @dagneyb

[urfin73]

I do recognize that this can cause inconvenience, should we add this feature of persisting user adjested column width in our to do list for discover?

Yes, sure. I find this to be a very convenient functionality.

[Alexxhn]

I confirm that the problem remains in version 2.12.0. For example, we adjusted the width of the columns and saved the search. We opened it, but the width of the columns was reset and the columns became the same width. We adjusted the width of the columns again and, for example, added a value to the filter. Immediately the column width was reset again. It is very uncomfortable.

For some it may be 'uncomfortable'. For people like me - it's hell of a pain, because eveything should be on it's places. And re-adjusting it every time page refreshes... please, fix this asap.

[ashwin-pc]

@Alexxhn in 2.12 we reimplemented the older table in Discover while we iron out these UX issues. Does that mitigate your problem at-least temporarily since its the same behaviour as 2.9?

[Alexxhn]

@Alexxhn in 2.12 we reimplemented the older table in Discover while we iron out these UX issues. Does that mitigate your problem at-least temporarily since its the same behaviour as 2.9?

Yeah, I'm currently using older version of Discover to be able to live with this somehow. Thank you.

[marcohald]

I do recognize that this can cause inconvenience, should we add this feature of persisting user adjested column width in our to do list for discover?

It would be a quite useful Feature to save the Column width within the saved search.
Will this tracked in this Issue or is there a new Issue for this Feature Request?

[ananzh]

@marcohald I have raised a PR but has some performance issue. I will finish it in 2.15.

[AMoo-Miki]

2.14's Discover has an enhancement to allow auto-sizing of columns and disallow frequent changes while scrolling.

[ashwin-pc]

@AMoo-Miki Can we close this issue then?

[AMoo-Miki]

I would love to hear from @gsmith-sas and others who have commented on this issue.

Also, this is not answered:

It would be a quite useful Feature to save the Column width within the saved search.
Will this tracked in this Issue or is there a new Issue for this Feature Request?

[ashwin-pc]

I think we should open a new issue for that. Reading the title i would never think to close this even if we fixed it. But dealers choice :) At the minimum lets update the title to accurately reflect what the issue is about now

[marcohald]

makes sense I opened #6794

[alperki]

In Discover mode, every new column added (by - toggle column in table) resets all settings made on the other columns. For example, after setting a sufficiently large width for a text field and a smaller width for a numeric field, adding a new column to the table for review causes all columns to resize. This feature makes it almost impossible to review logs, as it requires manually resizing the columns to follow their contents after each change. When tested through the Playground, the issue persists in the new versions (2.14). Developing a solution only for saved searches will not be sufficient. Even if it works for saved searches, adding a new column will still cause the previously set size adjustments for all columns to be lost.

Could you please consider this in the development as well? I wanted to reiterate to ensure there is no misunderstanding. The fix being worked on would apply to Discover mode as well, correct? @ananzh

[marcohald]

@alperki I'm not a developer you may intended to mention someone else?

[ananzh]

@alperki thanks for the feedback. Let me take a research on this in 2.16.

[urfin73]

Hi. Why was the option to switch to the old discovery removed? Column scaling is very difficult to work with in the new one.

[Alexxhn]

We're using v2.16.0
Losing all width settings while refreshing the page or opening saved query\filter.
With old UI option removed from existance - we're back in hell again.

[gsmith-sas]

I have only played around with v2.16.0 a little bit and only in the OpenSearch Playground Demo environment (accessible under the Platform link on the OpenSearch.org webpage). In that environment, it is possible to return to the original Discover UI by clicking on the Gear icon and selecting Enable legacy Discover (see screenshot).

It looks like the enabling the legacy Discover interface setting in the Discover page also impacts how saved searches appear in Dashboards. So, if you have enabled the legacy interface, it is used in the dashboards as well.

But I absolutely agree that the new interface should persist column widths and save them when saving queries. And it is disappointing that this hasn't been fixed in the 9 months (and 5 releases) since this issue was opened. This Discover interface is critical functionality for many (most) OpenSearch Dashboards users. The legacy interface was far from perfect...but the newer interface is not yet an adequate replacement.

[Alexxhn]

I have only played around with v2.16.0 a little bit and only in the OpenSearch Playground Demo environment (accessible under the Platform link on the OpenSearch.org webpage). In that environment, it is possible to return to the original Discover UI by clicking on the Gear icon and selecting Enable legacy Discover (see screenshot).

Thank you for 'Enable legacy Discover', wouldn't have found it otherwise.

[alperki]

@ananzh In which version are you planning to release this development, and could you provide more details on the timeline