Make return button read search in browsers

[andrewtavis]

Terms

• I have searched open and closed feature requests
• I agree to follow Scribe-iOS' Code of Conduct

Description

Currently Scribe keyboards don't change the return button to something like go or search when in a browser. It's possible that not many users would find Scribe helpful when searching the web, but as of now it functions well in that commands can be called, and after confirming the choice the search is updated. This is something that is standard in system keyboards, and should be included in Scribe if possible.

Initial inspections of this indicate that textFieldShouldReturn might be a trigger that could be used in these cases.

Possibly helpful SO questions are here, here, here and here.

[andrewtavis]

Whether the button should change color in these cases should also be discussed. If it is, changing it to the blue that the system keyboard uses for light and dark mode would be best. This would make it explicit to the user that when the button is Scribe blue and the normal arrow that means they're entering a command, and a more vibrant blue with search is for web searching.

[andrewtavis]

It appears that checking for if an input is UIReturnKeyType.Done could be the needed conditional to change the return key over.

[gitpushOmnik]

Hey @andrewtavis, I would love to work on this issue as well. Can you assign it to me please?

[andrewtavis]

Sure thing, @gitpushOmnik! Assigning you now. Let me know if you want me to research this a bit, and happy to direct you to where you need to make the change 😊

[gitpushOmnik]

Yes please. That would be helpful to get me started. I am researching on it myself as well

[andrewtavis]

Here are some things that came up recently, @gitpushOmnik:

• As stated above we can check if the return key type is done, but then I'm not sure if this is being transmitted or not 🤔
  • I found this in this SO question
• From this SO question I found isSearching
  • This is iOS 15.0+, whereas Scribe is 13.0+, so it would be good to avoid this
• We could also check for UISearchBarDelegate

Let me know what you think of the above and we can go from there 😊

[gitpushOmnik]

Hello @andrewtavis. I did research upon it a bit. I agree about not using isSearching since it might cause compatibility issues.

[gitpushOmnik]

One very trivial question I had is, the return button should be names as "search" or "go" only when the user is using the keyboard in browser right? Or should it read as "search" and "go" even when the user is normally using the keyboard in any application?

[andrewtavis]

Thanks for the question, @gitpushOmnik :)

One very trivial question I had is, the return button should be names as "search" or "go" only when the user is using the keyboard in browser right?

This is exactly what we're looking for :) Only in the browser, and we can check what the system keyboard for the given language switches to for what it should say 😊

[gitpushOmnik]

One solution I thought of initially is to detect when a user starts typing in a searchbar and using the delegate method we would know to change the title of the button. But I am not sure it is not possible for an app to detect if the user is typing in a search bar in a different application, due to privacy and security restrictions.

Apps are sandboxed on iOS, which means they are isolated from other apps and cannot access data or events from other apps unless they use system-provided APIs

[andrewtavis]

This is a good point, @gitpushOmnik :) There is a serious chance that this issue can't be done 🤔 I guess it makes sense that it's a system keyboard feature because those keyboards have greater access that custom keyboard extensions, but then Scribe does not request system access at all, and wouldn't for something as minor as this (the goal is to never need to ask).

Let me know if you have some final thoughts on this :) Appreciate your input!

[gitpushOmnik]

Maybe a more better approach would be to check if a user is currently running a browser or not. This can be checked using UIApplication.shared.windows.first?.windowScene?.userActivity?.webpageURL. This basically checks if there is a url present in the searchbar or not. If it is present, then it determines it as a browser. The only problem is that I am not sure if it will fail on cases where there is an empty searchbar (i.e when a user has just opened a browser and there is not url present in the searchbar yet)

[gitpushOmnik]

I am trying this approach now. Will let you know soon enough if it works out

[andrewtavis]

Great, thank you @gitpushOmnik! Looking forward to hearing the result, and appreciate your efforts on this 🙏

[gitpushOmnik]

Okay so here is my final analysis of the issue after scratching my head off the whole day and performing some deep research in Apple Documentation and SO. These are the ways I thought of implementing the feature and their drawbacks.

1. One of the simple and naive approaches was to check whether the searchbar/textfield that the user is currently typing on, belongs to a web browser. On iOS, apps are placed in a sandboxed environment, which essentially isolates them from other applications on the device. As a result, they are unable to interact with data or events from other apps unless they make use of system-provided APIs. So getting information about searcher is not possible in this scenario.

2. a) The second option was checking whether the app that the user is currently using is one of the web browsers( For instance(Safari, Chrome, Firefox). This could’ve been achieved by using

Bundle.main.object(forInfoDictionaryKey: "CFBundleName")

		OR

Bundle.main.object(forInfoDictionaryKey: "CFBundleDisplayName")

But it turns out that Apple removed the support for these two statement and these would no longer be supported in later OS versions. (Also Apple thought that this was contradicting their purpose of sandboxing since another plugin/extension should not be able to access the information about a different application.

2 b) The use of UIApplication.shared singleton to check if the device is currently running a browser also fails in this scenario. The idea is to use UIApplication.shared.windows.first?.windowScene?.userActivity?.webpageURL to detect whether the searchbar belongs to a web or not. But this fails when the user first opens the web browser application where there is an empty search bar or when there is no http OR https link present in the searchbar of the browser(Since the above statement searches for the same)

3.) Checking bundle identifier of the application that the user is currently on. This can be achieved using bundleIdentifier = UIApplication.shared.activeScene?.session.role.rawValue and checking the string value of bundleIdentifier if it contains any of the browser bundles(For instance, the bundle identifier for Safari might be “com.apple.mobilesafari”) But this does not work as well.

4.) Conforming to the UIWebViewDelegate method. This one seemed irrelevant and unnecessary

5.) Using UITextDocumentProxy protocol to check placeholder of the text to search for the words “Search” and “Web” since many of the web browsers contains these key words as the placeholder text. But this method again is of no use since Apple does not provide any methods to obtain the placeholder text of a search bar if it is present. It does allow copying and editing the typed text but not the placeholder text.

6.) Lastly, I went through the deep link hole and finally found one solution that might work. The protocol UITextDocumentProxy is inherited from UIKeyInput protocol which in turn again is inherited from UITextInputTraits. Now this protocol contains a variable known as keyboardType. The possible values for the type keyboardType are “default”, “asciiCapable”, “emailAddress” etc. One of the key value is “webSearch”. This value denotes a keyboard optimized for web searches. We can then use this value in KeyboardViewController as a check for whether the keyboard type of the proxy document that the user is typing in belongs to “webSearch”. If so, we change the value and symbol of the return button to say something like “go” or “search”. Else the return value symbol remains the default symbol. I am now implementing this change and will raise a pull request soon enough.

[andrewtavis]

Hey @gitpushOmnik! Let me say again how much I appreciate all this research you put in for this feature 🙏 Really is commendable. There are so many options here, and it really was great that you thought outside the box for instance like checking the prompt in the text field :) Really grateful to have all of this documented here as well 🙏🙏

Let me know on the question I asked in #267! Really hope the behavior works out so that all this hard work can be deployed!! 😊

[gitpushOmnik]

Of course @andrewtavis. Thank you so much for the appreciation. It's my pleasure.

[gitpushOmnik]

Regarding the search button in the keyboard to appear in the Google Forms as well, I did some study on it. So it turns out the default Apple's system keyboard says "go" only for textfields of Google Forms that accept single line inputs. If the input textfield of Google forms contains paragraphs as input(paragraphs as in multi-line inputs), then the return says "return" which in our case is denoted by the arrow symbol. So the only change that we can try to implement is finding out when a proxy object takes a single line input. If it does, then we can change the return button to say "go" or "search". One of the ways again is to play around the keyboardType attribute of proxy object. I tried that but couldn't figure it out but I am pretty sure that's one of the ways to implement the feature in Google forms

[andrewtavis]

@gitpushOmnik, f07f804 rolls back a lot of the changes from #267, but then your research and testing all of this made the final result possible. I think the final result is fairly non-intrusive in that we do get that blue color change that the user expects, but then as it's not happening for every search let's just leave it at the blue and not switch the key face. The return key changes enough as it is in Scribe 😅

I also added in a check to make sure that the color isn't changing when a command state is being used, so if for some reason the user wants to translate or conjugate something in the search bar, then the color change will not happen and we get the normal ScribeBlue for the key along with the play button :)

Happy with how this turned out 😊 Hope you are as well, and thanks for your help!

[gitpushOmnik]

Yeah that sounds good! Thank you