3rd Community Review of BDX Allocator

[cryptoAmandaL]

First Review #20
Second Review #142

Latest Allocator Compliance Review https://compliance.allocator.tech/report/f03018484/1728866648/report.md

Given 2.5PiBs
DataCap was given to:
cryptoAmandaL/BDE-Allocator#22 (2PiB)
cryptoAmandaL/BDE-Allocator#8 (0.5PiB)

#22
After KYC and the first round allocation, we found the retrieval success rate to be poor.

Then we have communicated with clients on this issue and get a response from this client. We decided to give him a chance to wait for sps' improvement.

When cid bot showed that client use more than 75% datacap, I got a new report.

Compared with two results in reports, we've seen a significant increase in the success rate of sps' retrieval. So we decided to give them a new round of allocation.

#8

Compared with two results in reports, the retrieval success rate is decreasing. Client's sps are currently not capable of improving retrieval rates in a short time. So we closed this application and stop the work with this client.

[filecoin-watchdog]

@cryptoAmandaL
Allocator Application
Compliance Report
First Review
Second Review

1st Review score: 2.5PiB granted
2nd Review score: 2.5PiB granted

0.5 PiB granted to existing clients:

Existing Client Name	DC
NHGRI	0.5 PiB

2 PiB granted to new clients:

New Client Name	DC
ESGFand Pangeo	2 PiB

Example 1 - issue 8
The allocator has only allocated one allocation (512TiB) to this client since the last DC refresh, the client has not shown any improvement, so the allocator has closed the application.

I would like to point out, however, that the 3 previous CID reports were rather bad and the allocation of 512TiB seems to be unjustified.

Additionally, this dataset has already appeared multiple times in filecoin. The allocator should have checked and clarified these issues before granting the first allocation:

• https://github.com/search?q=repo%3Afilecoin-project%2Ffilecoin-plus-large-datasets+encode-public&type=issues
• [DataCap Application] <Socyber> - <Encode> Carohere/Caro-Allocator#14
• [DataCap Application] AWS-ENCODE luhong123/QC-Data-center#11

Example 2 - issue 22
The client requested 5 PiB and declared 7 data replicas of 1792TiB each. With this dataset size, a minimum of 12 PiB of data should be requested. It wasn’t raised by the allocator.

The dataset was stored multiple times:

• https://github.com/search?q=repo%3Afilecoin-project%2Ffilecoin-plus-large-datasets+cmip6-pds&type=issues
• [DataCap Application] Coupled Model Intercomparison Project 6 Carohere/Caro-Allocator#8
• [DataCap Application] <Coupled Model Intercomparison Project 6> Storify-LLC/Storify-Data-Pathway#14

SPs provided:
f02169597 Shenzhen
f02365890 Hongkong
f0509981 Guizhou
f01844118 Newyork
f02834511 Hongkong

SPs Updated in the issue:	1st CID	2nd CID
f03099988 HongKong	0.00	0
f01084941 HongKong	0.00	5.35
f03100001 Shenzhen	0.05	0.04
f02046115 USA (actually HK)	0.00	0
f03030649 China	-	65.38
f03156722 HongKong	-	98.88
f03100003 Shenzhen	0.04	0.07

Additional SPs used for deals:
f03100009 HK 0.00
f01975299 CN 0.00
f03100002 CN 0.05
f03100000 CN 0.07
f01084413 CN 90.91
f03214920 HK 0.00

The client updated the SP list after receiving the DC, so the SP list used for deals does not match the SP list provided in the form.

• The first CID report showed that all seven SPs have retrieval rates below 1% or do not support spark. The allocator commented that this should be fixed, but granted another tranche.

• The next report showed 6 additional SPs, about which the client did not inform the allocator via github issue. Only 1 SP has retrieval at a satisfactory level, while the rest are around 0.

• Compared with two results in reports, we've seen a significant increase in the success rate of sps' retrieval. So we decided to give them a new round of allocation.

After the second tranche, only 2 SPs improved their results significantly, but the allocator considered that the results improved and that cooperation could continue. Considering the allocator's comment and report analysis, I don't understand where the significant increase was observed.

• The last report shows that 5 out of 13 SPs have retrieval rates at a satisfactory level; 1 has retrieval around 30%, and the rest are 0%.
• The client declared 7 replicas, but according to the report, there are 16 of them already.
• The client declared that one of the SPs is from the US; according to the report, it is an SP from Hong Kong (f02046115). Suspected use of VPN.
• The allocator declared in its application that it will require 4 geopolitical regions and 5 physical locations, but in the case of this application, the SPs come only from Hong Kong or China.

[filecoin-watchdog]

@cryptoAmandaL Hi there. I wanted to remind you that the review was done.

[cryptoAmandaL]

Thank you @filecoin-watchdog
1
At first I allocate a small amount of datacap and look forward to client improvements. However, it was found that the client's report results were getting worse and it was decided not to allocate more datacap to this client.
I see that this dataset is updated daily, so I think this dataset can continue to be stored in filecoin.

2
I think the amount of datacap requested by this client may be related to their current capabilities and equipment available to them. I'll follow up with more attention to this.
Comparing the two reports from the client, I think the client's retrieval results have improved.

The client had asked if it was possible to continue to replace the sp, and I gave a positive answer.

Historical records seem to be consistently present in the cid report, including sps that the client is no longer working with.

We always expect our clients to get better and better. But if they still don't meet our criteria for cooperation after a period of time, we will choose to end the cooperation and not allocate any more datacaps.

[filecoin-watchdog]

At first I allocate a small amount of datacap and look forward to client improvements. However, it was found that the client's report results were getting worse and it was decided not to allocate more datacap to this client.

You gave this client 5 allocations; each report has been average or bad. Therefore, the conclusion that you decided not to allocate further based on the bad reports is incorrect. I understand giving one chance to prove better behaviour, but four seems too many. Especially when the last allocation is 10x bigger than the previous one. I don't get this strategy.

I see that this dataset is updated daily, so I think this dataset can continue to be stored in filecoin.

I highlighted this not because you might consider deleting this data from the network, but because you did not confront the client with the fact that the same dataset was being placed on the network before.

I think the amount of datacap requested by this client may be related to their current capabilities and equipment available to them. I'll follow up with more attention to this.

I understand that you have shown understanding towards the client, however, we operate on data and make decisions based on facts. Was the justification you are giving presented to you by the client? If so, I would like to see a transcript of that conversation.

[cryptoAmandaL]

Hello @filecoin-watchdog
From my understanding of spark retrieval, the sp needs to be actually packaged before it can be called by program to show the retrieval rate.

Regarding the case of storing the same dataset, I actually didn't seem to see this previously where it says not to do this.

Yes, I reconfirmed it with the client.

[filecoin-watchdog]

@cryptoAmandaL Thank you. I have no other questions or comments, so I'm leaving it to the gov team now.

[galen-mcandrew]

Given the evidence presented, this pathway is still not showing any developments towards a market-based or smart contract approach. This was flagged in the initial allocator review, and continues to be a gap.

Additionally, this is the third compliance review and similar problems are persisting. For example, almost 40% of a client's storage providers offering NO retrieval success? Compared to improvements we are seeing with other pathways, clients, and SPs, that is not a noticeable increase in retrieval strength. Perhaps there is some confusion from the report, but having 85% of the SPs scoring BELOW 75 on retrieval testing is bad. Additional issues were flagged by the watchdog, such as multiple tranches increasing in size despite poor client compliance.

Without the following areas being corrected, this allocator will be rejected:

• retrieval scores increasing above 75%
• dataset replicas being accurately reported and investigated
• SP distribution lists remaining accurate, with justification
• VPN usage being noted and tracked
• regional distribution increasing to match application requirements

Overall, the clients working with this pathway are not showing significant improvement, and this allocator is not showing evidence of developing or delivering on its initial claims of building market-based, smart contract data onboarding tooling. We will request a final 2.5PiB of DataCap, in the hopes that we can see any progress towards the design goals presented initially.