Skip to content

Draft: Add single-query many-doc bulk scoring API to VectorUtilSupport (#16556) - #16640

Draft
CH-Abhinav wants to merge 2 commits into
apache:mainfrom
CH-Abhinav:perf/vectorutils-bulk-scoring
Draft

Draft: Add single-query many-doc bulk scoring API to VectorUtilSupport (#16556)#16640
CH-Abhinav wants to merge 2 commits into
apache:mainfrom
CH-Abhinav:perf/vectorutils-bulk-scoring

Conversation

@CH-Abhinav

Copy link
Copy Markdown
Contributor

Description

This is Draft PR addressing #16556 and subsequently progressing #15155

This PR is trying to move bulk scoring API down to VectorUtilSupport layer, allowing the PanamaVectorUtilSupport to see bulk operations.

Current progress (float32 dot product):

  1. Added zero-allocation void dotProductBulk(float[] query, float[][] docs, float[] results) to VectorUtilSupport.
  2. Added scalar fallback loop to DefaultVectorUtilSupport and pass-through to NativeVectorUtilSupport.
  3. Added 4 vector unrolling in PanamaVectorUtilSupport

Next steps:

  • Update RandomVectorScorer to consume the new VectorUtil.dotProductBulk() method to prove the end-to-end integration.
  • Add similar methods to in bulk format.

@CH-Abhinav

Copy link
Copy Markdown
Contributor Author

I went ahead and implemented off-heap path to PanamaVectorUtilSupport for dotProductBulk to make end to end connection.

while doing so I did following:-

  1. Left the base VectorUtilSupport strictly for on-heap arrays
  2. Added a public static SIMD batching method directly to PanamaVectorUtilSupport to handle off-heap vector unrolling
  3. Directed existing MemorySegmentBulkVectorOps to new method

If everyone is okay with this implementation pipeline then i will go ahead and implement the remaining similarity functions like cosine and squareDistance . So reviewers out there if you have some time please review this Draft once.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant