add display name replacement and make search page router editable #49838

289Adam289 · 2024-09-27T10:39:00Z

Details

follow up from #49462 (comment)
makes search page input editable
allows for using filters to and from with emails taxRate with names and cardID with bank name

Fixed Issues

$#49121
PROPOSAL:

Tests

Open Search Router on any page
Insert query with filters from, to, taxRate e.g. from:name.name@email.com
Verify that search works correctly

On wide screen:

Use search page input
Verify that it is editable and that search works
Verify that no errors appear in the JS console

Offline tests

QA Steps

Open Search Router on any page
Insert query with filters from, to, taxRate e.g. from:name.name@email.com
Verify that search works correctly

On wide screen:

Use search page input
Verify that it is editable and that search works
Verify that no errors appear in the JS console

PR Author Checklist

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

Screen.Recording.2024-10-04.at.10.33.33.mp4

iOS: mWeb Safari

Screen.Recording.2024-10-04.at.10.37.13.mp4

MacOS: Chrome / Safari

Screen.Recording.2024-10-04.at.10.26.37.mp4

MacOS: Desktop

Screen.Recording.2024-10-04.at.10.40.58.mp4

289Adam289 · 2024-09-27T12:31:36Z

While working on this issue, I've stumbled upon two problems with the current Search implementation of filters: from, to, in, cardID, taxRate:

Search Router Input: We can replace emails, names, and bank names only with IDs that are stored in Onyx, hence the user is currently unable to search for expenses and messages that are not in Onyx. (At present, the API only supports IDs)
Saved Search: When a user saves a complex query and opens the app on another device, Onyx may not have all the necessary data to replace IDs with display names, leaving blank spaces in the SearchHeader.

Changing the API to support more than just IDs may be unavoidable, especially considering the first problem. It would also simplify frontend logic substantially by removing functions that replace IDs with names and vice-versa.
cc @Kicu @luacmartins

luacmartins · 2024-09-27T13:20:28Z

Agreed. I created an issue for it here and I'll work on it next week.

289Adam289 · 2024-10-02T16:57:09Z

@luacmartins The current implementations of advanced filters from, to, taxRate, cardID, and in all rely on IDs which enable the creation of more precise queries. Because of this, I doubt we can substitute them with names or emails. Consequently, the replacement of names in user-written queries becomes a necessity. Is that approach correct or should I look for solutions that are not using IDs?

How should we handle the case of the in filter? For instance, the #announce room is often repeated across different workspaces. Should we swap room names with IDs based on the currently selected workspace?

Moreover, how do we approach the situation where no workspace is selected through the workspace switcher?

luacmartins · 2024-10-02T20:24:20Z

I think this issue will be solved with autocomplete. I was working on the backend PR to allow emails/names but I ran into the same issue you described. I think allowing from/to to use emails is fine because those are unique. taxRate would also be ok because we'd just search for anything with for example 5% rate, but cardID and in are trickier because as you said, those are not unique, so I think we need to rely on autocomplete for those.

I think for now, we'll update from, to and taxRate to take in emails/value and leave cardID and in as is, until autocomplete is available.

Kicu · 2024-10-03T10:38:03Z

I think implementing from, to and taxRate first is a good idea 👍

@luacmartins can you expand a bit how would autocomplete help the rest of the cases? I understand that we can save the specific id somewhere in memory when a user picks it from autocomplete, but I don't think that solves everything.

Would we not allow user to type in just a string like in:#wave by hand? will we block it if he/she doesnt choose if there are multiple #wave rooms?

what about saved searches? we would have to save id in the search and not the text (like: #admins, #wave)
what about deeplinks that people might send to each other? it means we have to store id in the search query in URL

I think we should start thinking about these, because not having a single unique identifier (that is not id) for these fields is a bigger problem.

One interesting thing we found out yesterday while testing app.

You can only type in room mentions in the Composer if you are in a specific workspace (either selected by ws switcher or a workspace report). If you're in "All" room and normal chat - room mentions don't work.
I have no idea how related this might be to our case, but it's worth investigating a bit at least. Perhaps one can only search for a room in the context of policyID?

rec-room-mention.mp4

Kicu

Left some generic comments, I think you could spent a bit of time to simplify the component a bit, since Search is growing quite fast 😅

src/components/Search/SearchPageHeader.tsx

src/libs/SearchUtils.ts

melvin-bot · 2024-10-04T08:47:53Z

@rayane-djouah Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

luacmartins · 2024-10-04T14:29:48Z

@rayane-djouah let's prioritize this PR next since I think we should have this functionality in place before we enable the feature to all users.

rayane-djouah · 2024-10-04T14:53:19Z

Reviewer Checklist

I have verified the author checklist is complete (all boxes are checked off).
I verified the correct issue is linked in the ### Fixed Issues section above
I verified testing steps are clear and they cover the changes made in this PR
- I verified the steps for local testing are in the Tests section
- I verified the steps for Staging and/or Production testing are in the QA steps section
- I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
- I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
I checked that screenshots or videos are included for tests on all platforms
I included screenshots or videos for tests on all platforms
I verified tests pass on all platforms & I tested again on:
- Android: Native
- Android: mWeb Chrome
- iOS: Native
- iOS: mWeb Safari
- MacOS: Chrome / Safari
- MacOS: Desktop
If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
I verified proper code patterns were followed (see Reviewing the code)
- I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
- I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
- I verified that comments were added to code that is not self explanatory
- I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
- I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
- I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
- I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
- I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
- I verified the JSDocs style guidelines (in STYLE.md) were followed
If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
I verified that this PR follows the guidelines as stated in the Review Guidelines
I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
If a new component is created I verified that:
- A similar component doesn't exist in the codebase
- All props are defined accurately and each prop has a /** comment above it */
- The file is named correctly
- The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
- The only data being stored in the state is data necessary for rendering and nothing else
- For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
- Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
- All JSX used for rendering exists in the render method
- The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions
If any new file was added I verified that:
- The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory
If a new CSS style is added I verified that:
- A similar style doesn't already exist
- The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)
If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:
- I verified that all the inputs inside a form are aligned with each other.
- I added Design label and/or tagged @Expensify/design so the design team can review the changes.
If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

luacmartins

BUG: when changing status the header is updated with the search query

Perform a custom search
Change the status
Note that the header changes away from the Search bar and shows the query instead

Expected behavior: keep the search bar with the new status

Screen.Recording.2024-10-04.at.9.53.16.AM.mov

luacmartins · 2024-10-04T16:01:25Z

Also, I'm not sure about this one, but we're changing the user input to either add things like type:expense status:all or rearrange the order of filters, in my example below test amount>200 to amount>200 test. It feels unexpected. I think we should keep user input as is and just normalize the query under the hood

Screen.Recording.2024-10-04.at.9.59.39.AM.mov

289Adam289 · 2024-10-04T16:14:47Z

When user enters the query we navigate to SearchPage with a different root. Then new input is generated using url making it the only source of truth. This is the reason why user input changes after submit. I can make following changes:

Remove `type:{type} status:{status}" from input.
I can try to keep the order of filters but it may require changing the grammar e.g. currently all keywords are filtered from the query and an extra tree with OR operators is created

luacmartins · 2024-10-04T16:18:16Z

@289Adam289 I don't think we need to tackle that in this PR. I realize that it'd involve bigger changes and we need to align on the expected behavior. Let's not block this PR on that or make changes now.

289Adam289 · 2024-10-04T16:20:34Z

Ok I will fix the first bug. Should I leave the type and status in the input?

luacmartins · 2024-10-04T16:21:58Z

Yea, I think we can do that for now as we think of a more holistic approach to not change the user input

289Adam289 · 2024-10-04T16:43:37Z

@luacmartins Currently on main when user changes status via status Bar Search Page resets to canned Query. Is it an expected behavior or do we leave the custom query and replace status with status selected from statusBar?

src/libs/SearchUtils.ts

src/components/Search/SearchPageHeader.tsx

src/libs/SearchUtils.ts

rayane-djouah

BUG: App crashes when searching for a non-existent type and then clicking on the "Filters" button

Perform a custom search with a non-existent type, for example, type:exp status:all date>2024-10-01
Click on the "Filters" button.
Notice that the app crashes.

Screen.Recording.2024-10-07.at.12.33.47.AM.mov

rayane-djouah

BUG: App crashes when searching for a non-existent expense type and then selecting an expense type from the filters page

Perform a custom search with a non-existent expense type, for example, type:expense status:all expenseType:abcd
Click on the "Filters" button.
Click on the expense type menu
Notice that the app crashes.

Screen.Recording.2024-10-07.at.12.55.26.AM.mov

rayane-djouah

Problem: The app does not validate search filters before saving them in Onyx. When certain malformed or incorrect filter values are used (e.g., type:expense status:all date<abcd date>abcd amount>abcd amount<abcd expenseType:abc,abcd currency:abcd from:abc,abcd to:abc,abcd category:abc,abcd tag:abc,abcdd taxRate:abc,abc), it leads to inconsistent behavior:

Date Filter: Entering invalid dates results in NaN values displayed in the calendar and console warnings.

Currency Filter: Invalid currencies are shown in the filter page title but not included as selected options on the actual currency filter page

Screen.Recording.2024-10-07.at.1.07.11.AM.mov

Total filter: Total input fields are populated with the string NaN when invalid data is entered.

Screen.Recording.2024-10-07.at.1.09.02.AM.mov

Category and Tag Filters: Non-existent categories and tags are shown in the filter page title and are included as selected options in their respective filter pages. (I think that this is a correct behaviour)

Screen.Recording.2024-10-07.at.1.10.42.AM.mov

From and To filters: Non-existent From and To login values are displayed as empty strings, separated by commas

Screen.Recording.2024-10-07.at.1.12.31.AM.mov

Tax Rate Filter: Tax rate names that do not exist are ignored

Screen.Recording.2024-10-07.at.1.14.10.AM.mov

rayane-djouah · 2024-10-07T00:39:13Z

In Filter: When entering the input value type:chat status:all in:aaabbbccc with an invalid in filter value, the in value is cleared automatically, and an empty option appears on the in filter page.

Screen.Recording.2024-10-07.at.1.35.01.AM.mov

289Adam289 · 2024-10-07T08:28:50Z

I think correct behavior would be to ignore incorrect values in advanced filters. It maybe confusing for a user to see an option and to be unable to select it later. Also in case of from, to and in filters we don't have display name and icon to show. Similar behavior can be observed on github and slack. cc @luacmartins for thoughts

rayane-djouah

BUG: The status bar incorrectly displays a loading skeleton every time the input query is modified and the enter key is pressed, even if the type has not been changed

Perform a custom search, for example, type:expense status:all test
Modify the search input without changing the type, for example, change to type:expense status:all test2
Observe that the status bar displays a loading skeleton.

Expected behavior: The status bar should only display a loading skeleton if the type in the query is changed

Screen.Recording.2024-10-07.at.2.18.58.PM.mov

rayane-djouah · 2024-10-07T13:27:10Z

BUG: A parsing error occurs when special characters, such as ä, are used.

rayane-djouah · 2024-10-07T13:43:11Z

src/components/Search/SearchPageHeader.tsx

                    <SearchRouterInput
-                        value={text}
-                        setValue={() => {}}
+                        value={value}
+                        setValue={setValue}
+                        onSubmit={onSubmit}
                        updateSearch={() => {}}
-                        disabled
                        isFullWidth
                        wrapperStyle={[styles.searchRouterInputResults, styles.br2]}
                        wrapperFocusedStyle={styles.searchRouterInputResultsFocused}


We need to disable autoFocus for the TextInput within the SearchRouterInput when it is used in the SearchPageHeader

Screen.Recording.2024-10-07.at.2.38.40.PM.mov

289Adam289 · 2024-10-07T14:05:02Z

I think to allow for parsing special characters like in #49838 (comment) fundamental changes to grammar are inevitable. In previous problems with special character we could just wrap user input in quotation marks. Right now it is impossible without initial parsing. Maybe we should handle it as a follow up.

rayane-djouah · 2024-10-07T14:09:40Z

Addressing #49838 (comment) in a follow up sounds good to me 👍
cc @luacmartins

289Adam289 · 2024-10-07T15:39:15Z

I added last search type to SearchContext so that search bar skeleton visibility depends on last type and not on ref in Search that resets on every navigation. @rayane-djouah could you take a look?
I will fix remaining filters tomorrow.

add display name replacement

2dd0e87

luacmartins mentioned this pull request Sep 27, 2024

[Search v2.4] Update from, to, in, cardID, taxRate filters to accept names in addition to IDs #49846

Open

Kicu mentioned this pull request Sep 30, 2024

Search-New SearchButton has not been added #49785

Closed

6 tasks

289Adam289 added 2 commits October 2, 2024 08:35

Merge branch 'main' into 289Adam289/49121-add-ids-search-router-input

2a61b10

make input editable and allow names

c2a1ddd

Kicu reviewed Oct 3, 2024

View reviewed changes

src/components/Search/SearchPageHeader.tsx Outdated Show resolved Hide resolved

src/components/Search/SearchPageHeader.tsx Outdated Show resolved Hide resolved

src/libs/SearchUtils.ts Outdated Show resolved Hide resolved

289Adam289 added 5 commits October 3, 2024 13:39

improve variable and function names

08e045d

improve HeaderWrapper

f23f637

Merge branch 'main' into 289Adam289/49121-add-ids-search-router-input

0a134c3

fix merge

0ef31fa

improve input for keywords

b8c7e0b

289Adam289 marked this pull request as ready for review October 4, 2024 08:47

289Adam289 requested a review from a team as a code owner October 4, 2024 08:47

melvin-bot bot requested review from rayane-djouah and removed request for a team October 4, 2024 08:47

luacmartins self-requested a review October 4, 2024 14:26

luacmartins requested changes Oct 4, 2024

View reviewed changes

rayane-djouah suggested changes Oct 4, 2024

View reviewed changes

289Adam289 added 2 commits October 4, 2024 19:32

fix status bar for qustom queries

9ecb681

review

1b0e259

rayane-djouah suggested changes Oct 6, 2024

View reviewed changes

rayane-djouah suggested changes Oct 7, 2024

View reviewed changes

fix type, expensetype, amount, and date

35e32ba

rayane-djouah suggested changes Oct 7, 2024

View reviewed changes

fix search bar loading

7ca5af1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add display name replacement and make search page router editable #49838

add display name replacement and make search page router editable #49838

289Adam289 commented Sep 27, 2024 •

edited

Loading

289Adam289 commented Sep 27, 2024

luacmartins commented Sep 27, 2024

289Adam289 commented Oct 2, 2024

luacmartins commented Oct 2, 2024

Kicu commented Oct 3, 2024 •

edited

Loading

Kicu left a comment

melvin-bot bot commented Oct 4, 2024

luacmartins commented Oct 4, 2024

rayane-djouah commented Oct 4, 2024

luacmartins left a comment

luacmartins commented Oct 4, 2024 •

edited

Loading

289Adam289 commented Oct 4, 2024

luacmartins commented Oct 4, 2024 •

edited

Loading

289Adam289 commented Oct 4, 2024 •

edited

Loading

luacmartins commented Oct 4, 2024

289Adam289 commented Oct 4, 2024 •

edited

Loading

rayane-djouah left a comment

rayane-djouah left a comment

rayane-djouah left a comment

rayane-djouah commented Oct 7, 2024

289Adam289 commented Oct 7, 2024

rayane-djouah left a comment

rayane-djouah commented Oct 7, 2024

rayane-djouah Oct 7, 2024

289Adam289 commented Oct 7, 2024 •

edited

Loading

rayane-djouah commented Oct 7, 2024

289Adam289 commented Oct 7, 2024

add display name replacement and make search page router editable #49838

Are you sure you want to change the base?

add display name replacement and make search page router editable #49838

Conversation

289Adam289 commented Sep 27, 2024 • edited Loading

Details

Fixed Issues

Tests

Offline tests

QA Steps

PR Author Checklist

Screenshots/Videos

289Adam289 commented Sep 27, 2024

luacmartins commented Sep 27, 2024

289Adam289 commented Oct 2, 2024

luacmartins commented Oct 2, 2024

Kicu commented Oct 3, 2024 • edited Loading

Kicu left a comment

Choose a reason for hiding this comment

melvin-bot bot commented Oct 4, 2024

luacmartins commented Oct 4, 2024

rayane-djouah commented Oct 4, 2024

Reviewer Checklist

Screenshots/Videos

luacmartins left a comment

Choose a reason for hiding this comment

luacmartins commented Oct 4, 2024 • edited Loading

289Adam289 commented Oct 4, 2024

luacmartins commented Oct 4, 2024 • edited Loading

289Adam289 commented Oct 4, 2024 • edited Loading

luacmartins commented Oct 4, 2024

289Adam289 commented Oct 4, 2024 • edited Loading

rayane-djouah left a comment

Choose a reason for hiding this comment

rayane-djouah left a comment

Choose a reason for hiding this comment

rayane-djouah left a comment

Choose a reason for hiding this comment

rayane-djouah commented Oct 7, 2024

289Adam289 commented Oct 7, 2024

rayane-djouah left a comment

Choose a reason for hiding this comment

rayane-djouah commented Oct 7, 2024

rayane-djouah Oct 7, 2024

Choose a reason for hiding this comment

289Adam289 commented Oct 7, 2024 • edited Loading

rayane-djouah commented Oct 7, 2024

289Adam289 commented Oct 7, 2024

289Adam289 commented Sep 27, 2024 •

edited

Loading

Kicu commented Oct 3, 2024 •

edited

Loading

luacmartins commented Oct 4, 2024 •

edited

Loading

luacmartins commented Oct 4, 2024 •

edited

Loading

289Adam289 commented Oct 4, 2024 •

edited

Loading

289Adam289 commented Oct 4, 2024 •

edited

Loading

289Adam289 commented Oct 7, 2024 •

edited

Loading