8 Commits

Author	SHA1	Message	Date
Nicolas Mowen	e8b2fde753	Support batch embeddings when reindexing (#14320) ... * Refactor onnx embeddings to handle multiple inputs by default * Process items in batches when reindexing	2024-10-13 12:33:27 -05:00
Nicolas Mowen	3a403392e7	Fixes for model downloading (#14305) ... * Use different requestor for downloaders * Handle case where lock is left over from failed partial download * close requestor * Formatting	2024-10-12 13:36:10 -05:00
Nicolas Mowen	6df541e1fd	Openvino models (#14283) ... * Enable model conversion cache for openvino * Use openvino directly for onnx embeddings if available * Don't fail if zmq is busy	2024-10-11 10:47:23 -06:00
Josh Hawkins	54eb03d2a1	Add config option to select fp16 or quantized jina vision model (#14270) ... * Add config option to select fp16 or quantized jina vision model * requires_fp16 for text and large models only * fix model type check * fix cpu * pass model size	2024-10-10 16:46:21 -06:00
Nicolas Mowen	dd6276e706	Embeddings fixes (#14269) ... * Add debugging logs for more info * Improve timeout handling * Fix event cleanup * Handle zmq error and empty data * Don't run download * Remove unneeded embeddings creations * Update timouts * Init models immediately * Fix order of init * Cleanup	2024-10-10 16:37:43 -05:00
Nicolas Mowen	a2ca18a714	Bug fixes (#14263) ... * Simplify loitering logic * Fix divide by zero * Add device config for semantic search * Add docs	2024-10-10 07:09:12 -06:00
Nicolas Mowen	bc3a06178b	Embedding gpu (#14253)	2024-10-09 19:46:31 -06:00
Josh Hawkins	d4925622f9	Use JinaAI models for embeddings (#14252) ... * add generic onnx model class and use jina ai clip models for all embeddings * fix merge confligt * add generic onnx model class and use jina ai clip models for all embeddings * fix merge confligt * preferred providers * fix paths * disable download progress bar * remove logging of path * drop and recreate tables on reindex * use cache paths * fix model name * use trust remote code per transformers docs * ensure tokenizer and feature extractor are correctly loaded * revert * manually download and cache feature extractor config * remove unneeded * remove old clip and minilm code * docs update	2024-10-09 15:31:54 -06:00