Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve search on website #686

Open
etiennebacher opened this issue Jan 9, 2024 · 13 comments
Open

Improve search on website #686

etiennebacher opened this issue Jan 9, 2024 · 13 comments
Labels
documentation Improvements or additions to documentation

Comments

@etiennebacher
Copy link
Collaborator

The search doesn't take into account the function names in the sidebar in "Reference", which makes the search experience a bit useless. For example, searching "sum_horizontal" doesn't point to pl$sum_horizontal() but only shows the output when we print all methods in one of the examples:

image

@etiennebacher etiennebacher added the documentation Improvements or additions to documentation label Jan 9, 2024
@etiennebacher
Copy link
Collaborator Author

etiennebacher commented Jan 11, 2024

The problem is that the search index is based on the md files, not the HTML ones: in the md file, pl_sum_horizontal exists, but not simply sum_horizontal. In addition to that, it can't search for chunks of words (for example searching for "ackage" doesn't suggest the word "package").

Searching for "pl_sum_horizontal" works fine (but is clearly not ideal because we'd need to prefix lazy functions with "LazyFrame_", etc.):

image

@etiennebacher
Copy link
Collaborator Author

etiennebacher commented Jan 11, 2024

That is something to fix in altdoc anyway because the search index is built when we run render_docs().

Edit: not sure of that actually, the problem is that the "Usage" section is originally really weird because of polars namespace

@etiennebacher
Copy link
Collaborator Author

This is annoying, the website loses half of its interest if search is impossible (or only with the trick of searching for e.g "DataFrame_"). To summarize the comments above, the problem is that the search index used by mkdocs is built using the markdown files (that are then converted to HTML by mkdocs). This is why we need to type "DataFrame_" to search in DataFrame methods: the "altdoc postprocessing" script that fixes the Usage section by going from "DataFrame_" to "<DataFrame>$" comes after the search index is built.

We may think that this can be fixed by running the "altdoc postprocessing" script earlier in the workflow, basically after every devtools::document(). However, this is not correct because R CMD check will error when it parses the ("fixed") Rd files (cf #701).

Therefore, one solution would be to do this step only in the altdoc workflow. This is what it would look like:

  • make a copy of the "man" folder (with the original Rd files) named "man_old"
  • run the script in build: add functions to update usage sections in Rd files #701 on the "man" folder. This fixes the formatting in the Usage section.
  • run the altdoc workflow (preprocessing script + render_docs()). Since it runs on the fixed "man" folder, the search index should be built correctly (typing "DataFrame$" in the search bar should point to DataFrame methods)
  • remove the "man" folder, and rename "man_old" to "man". Therefore, the "man" folder is now the same as what was generated by devtools::document()

About accessing docs inside R with ?: I think this cannot be solved. This uses the Rd files packaged with polars, and those have to pass R CMD check, which means they cannot be reformatted. This new workflow wouldn't change that.

Note that the solution probably shouldn't come from altdoc itself. polars is a very very special case, and I don't want to make big changes to altdoc just because of polars weird docs.

@eitsupi I'd like to have your opinion on this. Does this workflow make sense to you? Do you have other ideas?

@eitsupi
Copy link
Collaborator

eitsupi commented Jan 18, 2024

@etiennebacher Thanks for the detailed explanation. I think the solution is reasonable.

@eitsupi
Copy link
Collaborator

eitsupi commented Mar 4, 2024

@etiennebacher Any update on this?
It is quite troubling that the search is almost useless.
For example, when searching for "strptime".

@etiennebacher
Copy link
Collaborator Author

I had started working on this but I don't think the search was better in the end. My progress is here: https://github.com/pola-rs/r-polars/tree/rework-docs-workflow

I'll explore more, but we might have to wait for squidfunk/mkdocs-material#6307

@eitsupi
Copy link
Collaborator

eitsupi commented Mar 4, 2024

Thanks for working on that.

My other question was whether it might be worth moving from Mkdocs Material to Quarto, as Mkdocs Material is very powerful but has a lot of proprietary notations that would benefit this repository less.

@etiennebacher
Copy link
Collaborator Author

I'm not convinced it's worth moving to Quarto. I can try to make a small demo when I have time, but from what I can see on the marginaleffects website the search is also not great. Typing plot_pred to search for the man page of plot_predictions() returns tons of results, and it's not clear at all which page is the man page.

has a lot of proprietary notations that would benefit this repository less

Not sure I understand, what do you mean by proprietary notations?

@eitsupi
Copy link
Collaborator

eitsupi commented Mar 5, 2024

it's not clear at all which page is the man page.

There is no doubt about this.

Not sure I understand, what do you mean by proprietary notations?

If we can use these, the website will look much better.
https://squidfunk.github.io/mkdocs-material/reference/

@etiennebacher
Copy link
Collaborator Author

If we can use these, the website will look much better.
https://squidfunk.github.io/mkdocs-material/reference/

I don't see obvious features that would make the website better, but if you list some I can take care of including them

@squidfunk
Copy link

Author of Material for MkDocs here! We'd be happy to get you guys as a beta tester once we have the next search preview available. In the meantime, you can already test the research preview 2, which should improve accuracy. We really would love more feedback on this, because we finally want to build a search that doesn't suck ☺️ Additionally, for some more context see squidfunk/mkdocs-material#6307, where we explain the why and what we've planned.

@eitsupi
Copy link
Collaborator

eitsupi commented Mar 9, 2024

Sounds great, thanks for your comment @squidfunk!

I think it is really great that Material for Mkdocs provides a great local search feature out of the box.
(I have built Japanese documents with Docsaurus and had quite a bit of trouble setting up a local search feature for Japanese)

@etiennebacher
Copy link
Collaborator Author

@eitsupi FYI I'm quite happy with the recent improvements from #925 so I'm not actively looking into this issue anymore

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

3 participants