[FEA] Replace FAISS bfknn()
with a version using RAFT primitives
#803
Labels
feature request
New feature or request
bfknn()
with a version using RAFT primitives
#803
A large goal for RAFT is to remove the dependency on FAISS in order to remain as lightweight as possible. We have a version of IVF-Flat, which was added in 22.08. We are adding a version of IVF-PQ for 22.10 and have a KNN with a fused k-select which works well when k<64.
What is needed in order to become fully independent of FAISS is for the
bfknn()
call from FAISS to be ported to RAFT primitives. This call is mostly a tiled pairwise distance computation with a k-select performed over each tile. The port here should be fairly straightforward, but we will need to support all of the distances that FAISS supports and make sure there is no loss in performance along the way.In addition, FAISS' k-select has proven to be highly optimized and performant and we make heavy use of it throughout RAFT and cuML. We will need to make sure we have a suitable replacement in one of the k-select functions in RAFT. If it turns out we are better of continuing to use the k-select from FAISS, we are probably better off just copying the current version (it hasn't changed much at all over the years) directly into RAFT and providing the proper attribution / licensing info.
The text was updated successfully, but these errors were encountered: