Purge topology origin servers when there is a duplicated server from Pelican

[haoming29]

Closes #1104

The core idea of this PR is as following:

• For all serverAd-s and their namespaces, we store them as-is, no matter they are from Topology or Pelican. We mark the ones from Topology with a new field FromTopology
• When director receives an object request, it will go over the serverAds and when doing match-making (related code is in getAdsForPath in director/cache_ads.go:
  • if origin candidate list is empty, set the matched origin server to the list, no matter from topology or not
  • if origin list is not empty, and the LAST item and the next matched origin ad are from the same source, simply add the next matched origin to the list
  • If they are different, then
    • If the next matched one is from topology, then ignore it (meaning existing candidates are all from Pelican)
    • If the next matched one is from Pelican, then remove the existing candidates and put the Pelican one into the list
  • For cache servers, as long as they matched the namespace, set them to the cache candidate list

What this PR hasn't covered yet, is the list of namespaces and their servers the director sends to the cache for generating scitokens/authfile. We should do the same check ands filter out the duplicated topology server. Not sure if I should include this aspect in this PR or make it a separate one.

This PR also make sure we stable-sort the serverAd first in getAdsForPath and always generate the stable result (the current code gives random result depending on serverAds ordering of its Items() which is random ordering).

[jhiemstrawisc]

I think I already know the answer to this question, but I just want to confirm -- if an old origin/namespace advertises to this new director with an old server/namespace ad that doesn't have FromTopology, will the director automatically unmarshal the response and set that value to false? If that's the case, I think we're okay to modify the struct without worrying about hurting backwards compatibility.

[haoming29]

I think I already know the answer to this question, but I just want to confirm -- if an old origin/namespace advertises to this new director with an old server/namespace ad that doesn't have FromTopology, will the director automatically unmarshal the response and set that value to false? If that's the case, I think we're okay to modify the struct without worrying about hurting backwards compatibility.

The server advertisement should be fine, as we manually set the value (or let it be default value for Pelican server: director/director.go line 612), for namespace ad, I'm pretty sure if the incoming namespaceAd does not have FromTopology, the field is initialized to false by encoding/json, so there shouldn't be a compatibility issue.

[jhiemstrawisc]

LGTM. Do we need/want to patch this one into 7.7?

[haoming29]

LGTM. Do we need/want to patch this one into 7.7?

What would be a good question for @bbockelm