FeatureFuel
1

Add option to enable hardware acceleration for Adapter models

Source: supabase/supabase#39912 · opened by @kziovas
🧩 Feature Request: Add option to enable hardware acceleration for Adapter models Summary: Enable users to specify the hardware device (cuda, cpu, mps, npu, etc.) used for computation when initializing adapter-based models such as TextEmbedding. --- Current Behavior Currently, the TextEmbedding adapter in vecs.adapters.text_embedding initializes a SentenceTransformer model without any control over which device is used for inference: This defaults to automatic device detection inside sentence-transformers, but users cannot explicitly override it or ensure that hardware acceleration (e.g., GPU or Apple MPS) is utilized when available. --- Proposed Improvement Add a new optional argument to the TextEmbedding class: and use it in model initialization: This aligns with the SentenceTransformer constructor signature: --- Benefits • Improved performance: Enables GPU, MPS, or other accelerator usage for large-scale text embedding generation. • Better control: Allow…

No pledges yet. Be the first to back this.

Make a pledge

Pledge your monetary support if this feature is added.

$

Comments

No comments yet.

Replying to

Add a comment

What do you think about this feature request?


Similar requests