Look into adjusting the naming / data structure of the agency data in the opportunity response

[chouinar]

Summary

Right now an opportunity returns the following relevant agency information in our API (fake data):

{
    "agency": "HHS-NIH11",
    "top_level_agency_name": "Department of Health and Human Services",
    "agency_name": "National Institutes of Health",
    "summary": {
      "agency_code": "USAID",
      "agency_contact_description": "google.com Contact Center\nHours of operation are 24 hours a day, 7 days a week.\nchristinarosales@example.org",
      "agency_email_address": "mark90@example.net",
      "agency_email_address_description": "Contact National Housing Administration via email",
      "agency_name": "National Housing Administration",
      "agency_phone_number": "123-456-0002",
    }
  }

The agency data is stored across two parts of the API response, two fields in the top-level, and the rest in the summary object. The agency code, and name appear twice as well, and are not consistent with one another.

This is almost certainly going to be confusing for users of the API, and we should correct this issue.

--

Context on why it is this way:

• The top-level agency field is actually the agency code of the opportunity
• The top-level agency_name is found by joining the agency field with the agency table
• The summary fields are fields attached to the opportunity summary itself in the DB, which is partially a duplicate of the top-level values and a static copied value of the actual agency info

From prior investigation, the agency code is almost never different between the two levels, but there are a few specific cases where they are (not intentional as far as I can tell - likely just an old bug). The top-level values are better supported/accurate and we shouldn't keep the agency code / agency name of the inner summary object.

--

Recommendation:

• Remove agency_code and agency_name from the summary object
• Rename the "agency" field in the top-level object to "agency_code" to be clearer.

As the only user of our API at this time is the front-end, we would just need to temporarily add another field at the top-level, and have them switch over.

This will make the request look like:

{
    "agency_code": "HHS-NIH11",
    "top_level_agency_name": "Department of Health and Human Services",
    "agency_name": "National Institutes of Health",
    "summary": {
      "agency_contact_description": "google.com Contact Center\nHours of operation are 24 hours a day, 7 days a week.\nchristinarosales@example.org",
      "agency_email_address": "mark90@example.net",
      "agency_email_address_description": "Contact National Housing Administration via email",
      "agency_phone_number": "123-456-0002",
    }
  }

--

Note that the DB will continue to contain a few extra fields for now, we can clean that up later.

Acceptance criteria

• Run this idea past product + front-end folks
• Make the change

[emilycnava]

Looking into this

[doug-s-nava]

a few notes on the current behavior

frontend UI

opportunity detail

• agency name taken from opportunity.agency_name

search UI

• result list item agency name taken from either opportunity.agency or
  • a lookup of the agency based on summary.agency_code that seems to be completely broken

filter behavior

• agency filter options are hardcoded on the frontend with label that is likely supposed to match opportunity.agency_name and a value that is likely supposed to match opportunity.agency.
• the hardcoded agency value is added to filters.agency on the search request body

Possible Todos

So, the todos from the frontend side seem to be:

• update search UI to look at agency_code instead of agency
• fix logic for looking up the agency name based on the top level agency code rather than summary agency code, if necessary. Would like @acouch's opinion on this one.\
• not sure how the backend handles the filters on the search request body but if it's working then probably no need to adjust anything there
• investigate the possibility of populating filter options based on live data rather than hardcoding it in the frontend?
• update local seed agency data to make more sense? I could be not understanding this, but the local data seems to be mapping DOC and DOC-EDA agency codes to the "Agency for International Development" agency name, which is a little confusing to me

[chouinar]

@doug-s-nava - just following up on a few of your notes:

• The filters have their own separate naming, I do think making those use agency_code as well would be good, but didn't want to break that at the moment (the API handles mapping those names for now).
• Happy to chat through the idea of populating the filters on the left-hand nav from the API. There's some complexities in that, but it ultimately makes more sense for the API to do it since the counts/possible values are just the facet counts. For agencies, we'll want to discuss how we do that because of the hierachy of the agencies which will make things a bit more complicated.
• I agree on the DOC/DOC-EDA name issue, in the PR I sent out I actually fixed that so newly generated data shouldn't be confusing

[acouch]

Overall, like the proposed change.

Happy to chat through the idea of populating the filters on the left-hand nav from the API. There's some complexities in that, but it ultimately makes more sense for the API to do it since the counts/possible values are just the facet counts. For agencies, we'll want to discuss how we do that because of the hierachy of the agencies which will make things a bit more complicated.

I would imagine we will have to do this at some point, as the facet / filter counts will have to come from the API. We haven't committed to including those yet.

From a user perspective it would be nice to keep the filters outside of a suspense boundary, so when you visit the search page, the filters aren't loading like the results are. Maybe there is a way to still get those from the API and partial pre-render them since they won't be changing very often.

[doug-s-nava]

@acouch Yeah, it's a good thought on what we would want to render while we pull down the list from the API. I think it would be safe enough to completely populate the easier ones optimisitically - status and funding instrument seem likely to be completely static, so we could pretty safely render those immediately, and just populate the others from the API? If we added a status or funding instrument, then we'd have to update a hardcoded list, but that seems like it'd be a rare occurance

[chouinar]

Technically if you wanted the left-hand filters, and we fully supported the hierarchy of agency data, you could always call search with no filters to get the full dataset and cache that to use on initial page load.

Effectively, the facet counts == the possible filters when you don't filter anything.

---

Agency is the only one that currently will change without a deliberate code change on the API side. The rest require adjusting an Enum in https://github.com/HHS/simpler-grants-gov/blob/main/api/src/constants/lookup_constants.py which affects both the API schemas and DB allowed values.