Faceting

[etj]

Overview

Implement an API for facet filtering

Motivation

Current faceting at the moment

• is fragmented into calls to multiple services
  • --> the proposal aims to have a single entrypoint to retrieve all the available facets
• the client must know in advance how to encode the filter for each facet
  • --> the new facet services should describe how the filter should be created
• current topics retrieval is slow and puts an heavy load on the DB
  • --> data handling should be improved
• current faceting does not pre-filter the results with the current filter

Proposal

API endpoints

• _GEONODE_VERSIONED_API_/facets
  Returns a list of available facets
  The returned payload is a dict

   {"facets": [facet,...]}

  • GET params:
    • include_topics (bool): include first n topics to every facet
    • filter: 0..N, string, if include_topics, this field is used to pre-filter resources (see pre-filtering)
    • lang: bool, opt, return localized labels if possible
    • add_links: bool, add links to facets endpoint

• _GEONODE_VERSIONED_API_/facets/<facet_name>
  Returns info for the requested facet.
  First page of topics is included.

  • GET params:
    • page/page_size: opt, (int,int) topics pagination params
    • filter: 0..N, string, used to pre-filter resources (see pre-filtering)
    • topic_contains: 0..1, string, used to pre-filter topics with a string
    • lang: 0..1, str (2 letters): return localized labels if possible
    • add_links: 0..1, bool: add links to prev/next pages

Pre-filtering

The filter parameters are parsed by the DREST library and are in the form filter{FIELD_EXPRESSION}=VALUE, for instance:

• filter{metadata_only}=false
• filter{date.gte}=2023-06-01T00:00:00

Payloads

• facet is a dict:

  key	type	opt	description
  name	str	no	Name of the facet (to be used in the /facet/<facet_name> entrypoint)
  key	str	no	string to be used as KVP param in the resources query
  label	str	no	Label for the facet (could be localized)
  is_localized	str	yes	If exists and it's true, tells that the label is already translated and requires no further localization actions by the client
  type	enum	no	Type of facet, may be used by the client for rendering. So far region, category, people, thesaurus
  hierarchical	bool	no	Tells if the facet has nested elements (don't know if this is really useful)
  order	int	opt	May be useful for telling the UI how to present the facets thesauri have their own internal ordering (where can it be configured?)
  link	url	opt	link to the facet - /facets: requested using the add_links param - /facets/<ID>: N/A
  prev	url	opt	link to previous paginated set of topics - /facets: N/A - /facets/<ID>: requested using the add_links param
  next	url	opt	link to next paginated set of topics - /facets: N/A - /facets/<ID>: requested using the add_links param
  topics	dict	opt (see descr)	Info on topic set. - /facets: only present if include_topics requested - /facets/<ID>: always present

• topics is a dict:

  key	type	description
  page	int	page number
  page_size	int	number of topics per page
  start	int	initial # of topic in current page
  total	int	total number of topics
  items	list	list of topic

• topic is a dict, having at least 3 mandatory elements:

  key	type	opt	description
  key	str		String to be used as KVP value in the query
  label	str		Label (may be localized)
  is_localized	str	yes	If exists and it's true, tells that the label is already translated and requires no further localization actions by the client
  count	int		Count of facet's item in current filter

The topics may also have other elements, according to the Facet handler;
i.e.:

• categories may have a link to an icon,
• regions may have a bbox

e.g.:

"items": [
   {
      "key": "Pianificazione",
      "label": "Pianificazione",
      "is_localized": true,
      "count": 156,
      "fa_class": "fa-home"
   },

Filtering

The facets payloads provide all the info to create a filter on the resources.
In order to create a filter on the topic you need to add in the KVP string FACET_KEY=ITEM_KEY.
For instance, if we have this payload:

{
   "name": "3-2-4-1-gemet-inspire-themes-rdf",
   "key": "filter{tkeyword}",
   "label": "Categorie tematiche INSPIRE",
   "is_localized": true,
   ...
   "topics": {
      ...
      "items": [
         {
            "key": 36,
            "label": "Zone a rischio naturale",
            "is_localized": true,
            "count": 152
         },
    ...

in order to filter by "Zone a rischio naturale" we need to add in the search query filter{tkeyword}=36

Configuration

The faceting framework is extensible and configurable:

• faceting is implemented by extending a base class FacetProvider
• the list of facets can be configured in the setting:

   FACET_PROVIDERS = (
       "geonode.facets.providers.category.CategoryFacetProvider",
       "geonode.facets.providers.users.OwnerFacetProvider",
       "geonode.facets.providers.thesaurus.ThesaurusFacetProvider",
   )

Backwards Compatibility

No backward compatibility issue, since the idea is to implement a brand new entrypoint

Future evolution

The idea is to implement filtering and faceting based on the current DB architecture.
In the future we way want to improve the filtering functionalities by integrating an indexing engine (Solr, ...), and leave this interface as much stable as possible.

[allyoucanmap]

@etj @giohappy I'm currently testing the facet for user and categories and I found some issues:

• type thesaurus has is_localized param while type user and categories have localized_label. We should use is_localized boolean for all the typ eof facets

• type category is returning a null category that seems to represent all the resources without categories. Probably we should remove this

• type category is providing this filter key filter{category__identifier} but it's not working, getting

• using the select we need a way to request specific topics information to populate the visible value with the correct label, is there a way to request /facets/owner?filter{key.in}=1000&filter{key.in}=10001 as we can do with processed uploads?

Screenshot shows the page reload with the selected admin (1000)

[etj]

Error: If a thesaurus keyword does not have a Label entry for the requested language, it will not be counted.

9e3c150 fixes this problem. Furthermore, it returns the alternate label as topic label, and its "is_localized" flag will be set to false.