Ig~ddlmZmZmZmZmZmZmZmZddl m Z ddl m Z ddl mZddlmZmZddlmZddlZGdd eZy) )AnyDictIterableListOptionalSequenceTypeUnion) OnnxProvider)Bm25)Bm42)SparseEmbeddingSparseTextEmbeddingBase)SpladePPNceZdZUeeegZeee e d<e dee e effdZ dde dee deedeeeded eeed effd Z dd ee ee fd edeedeefdZdee ee fdeefdZxZS)SparseTextEmbeddingEMBEDDINGS_REGISTRYreturncjg}|jD]!}|j|j#|S)a Lists the supported models. Returns: List[Dict[str, Any]]: A list of dictionaries containing the model information. Example: ``` [ { "model": "prithvida/SPLADE_PP_en_v1", "vocab_size": 30522, "description": "Independent Implementation of SPLADE++ Model for English", "license": "apache-2.0", "size_in_GB": 0.532, "sources": { "hf": "qdrant/SPLADE_PP_en_v1", }, } ] ``` )rextendlist_supported_models)clsresult embeddings c/var/www/html/answerous/venv/lib/python3.12/site-packages/fastembed/sparse/sparse_text_embedding.pyrz)SparseTextEmbedding.list_supported_modelss8000 =I MM)99; < = model_name cache_dirthreads providerscuda device_ids lazy_loadc &t |||fi|dk(rtjdtdd|j D]=} | j } tfd| Ds(| |f|||||d||_ytdd ) Nzprithvida/Splade_PP_en_v1zvThe right spelling is prithivida/Splade_PP_en_v1. Support of this name will be removed soon, please fix the model_name) stacklevelzprithivida/Splade_PP_en_v1c3fK|](}j|djk(*yw)modelN)lower).0r(rs r z/SparseTextEmbedding.__init__..Es,^E:##%w)=)=)??^s.1)rr r!r"r#zModel z~ is not supported in SparseTextEmbedding.Please check the supported models using `SparseTextEmbedding.list_supported_models()`) super__init__warningswarnDeprecationWarningrranyr( ValueError) selfrrrr r!r"r#kwargsEMBEDDING_MODEL_TYPEsupported_models __class__s ` rr-zSparseTextEmbedding.__init__.s YB6B 4 4 MMW"   6J$($<$<  3IIK ^M]^^1 $')'    ZL!d d  r documents batch_sizeparallelc+^K|jj|||fi|Ed{y7w)a Encode a list of documents into list of embeddings. We use mean pooling with attention so that the model can handle variable-length inputs. Args: documents: Iterator of documents or single document to embed batch_size: Batch size for encoding -- higher values will use more memory, but be faster parallel: If > 1, data-parallel encoding will be used, recommended for offline encoding of large datasets. If 0, use all available cores. If None, don't use data-parallel processing, use default onnxruntime threading instead. Returns: List of embeddings, one per document N)r(embed)r3r8r9r:r4s rr<zSparseTextEmbedding.embedWs+,$4::##Iz8NvNNNs #-+-queryc+ZK|jj|fi|Ed{y7w)z Embeds queries Args: query (Union[str, Iterable[str]]): The query to embed, or an iterable e.g. list of queries. Returns: Iterable[SparseEmbedding]: The sparse embeddings. N)r( query_embed)r3r=r4s rr?zSparseTextEmbedding.query_embedos'*4::))%:6:::s !+)+)NNNFNF)N)__name__ __module__ __qualname__rr r rrr r__annotations__ classmethodrstrrrrintrr boolr-r rrr<r? __classcell__)r7s@rrrs6@H$PT?Ud#:;<Ud4S>&:>$(!%6:*.' ' C=' # ' H\23 '  ' T#Y'' ' X"& Ohsm+,OO3- O / " O0 ;sHSM'9!: ;RaIb ;rr)typingrrrrrrr r fastembed.commonr fastembed.sparse.bm25r fastembed.sparse.bm42r &fastembed.sparse.sparse_embedding_baserrfastembed.sparse.splade_pprr.rrrrQs3MMM)&&0k;1k;r