| Reported on Slack | Dec-27-2022 |
| Mitigation | Dec-27-2022 |
| Solution Completed | Jan-04-2023 |
After updating all peers from Catalyst v5.1.10 to v5.1.12 the nodes stopped serving the base wearables (like the BaseFemale) on any available endpoint for this need. This led to an issue on the SDK preview where users could not use this feature at all and also affected the avatar preview and edition on both SDK and Explorer. That made it impossible for users to be able to edit or deploy a new version of their avatar.
The Catalyst nodes were updated to solve a case where the peers could not serve base wearables if they were requested using uppercases or mix cases, it was only working with lowercases parameters. After updating the peers to v5.1.12 the wearables were being returned as expected for both cases (upper and lower cases) on the first calls but after receiving a call to retrieve a wearable using mix cases, the cache ended up in an inconsistent state causing subsequent calls to fail even if they were being sent with only lowercase letters.
During the time of the incident (Dec 26th to Dec 27th) the users could not see, edit or deploy their avatars.
The entities pointers are saved in memory on the Catalysts to be able to link them with their respective ids for further retrievals. The solution is to normalize the key saved on this memory so the pointers urn:decentraland:off-chain:base-avatars:basefemale or urn:decentraland:off-chain:base-avatars:BaseFemale are mapped to the same entity id to avoid treating them as different entities. Therefore, the client could request any case variation of this wearable and the Catalyst will answer with the same entity every time.
Fix: https://github.com/decentraland/catalyst/commit/8e9e2abcd340a6d5c4753163a580c36d832114c0