IgBdZddlmZddlmZmZmZmZddlZ ddl m Z m Z m Z ddlmZddlmZddlmZmZdd lmZdd lmZmZdd lmZd efd ZGddeeZGddeZGddeeZ GddeeZ!y)z@A chain for comparing the output of two models using embeddings.)Enum)AnyDictListOptionalN)AsyncCallbackManagerForChainRunCallbackManagerForChainRun Callbacks) Embeddings)pre_init) ConfigDictField)Chain)PairwiseStringEvaluatorStringEvaluatorRUN_KEYreturnc ddlm}|S#t$r) ddlm}n#t$r tdwxYwY|SwxYw)zaCreate an Embeddings object. Returns: Embeddings: The created Embeddings object. rOpenAIEmbeddingstCould not import OpenAIEmbeddings. Please install the OpenAIEmbeddings package using `pip install langchain-openai`.)langchain_openair ImportError%langchain_community.embeddings.openairs i/var/www/html/answerous/venv/lib/python3.12/site-packages/langchain/evaluation/embedding_distance/base.py_embedding_factoryrsa 5     N Q   O  s! A A5AAc$eZdZdZdZdZdZdZdZy)EmbeddingDistancea Embedding Distance Metric. Attributes: COSINE: Cosine distance metric. EUCLIDEAN: Euclidean distance metric. MANHATTAN: Manhattan distance metric. CHEBYSHEV: Chebyshev distance metric. HAMMING: Hamming distance metric. cosine euclidean manhattan chebyshevhammingN) __name__ __module__ __qualname____doc__COSINE EUCLIDEAN MANHATTAN CHEBYSHEVHAMMINGrrr*s"FIIIGr/rceZdZUdZeeZeed<ee jZ e ed<e de eefde eeffdZed Zedeefd Zd edefd Zde defdZedej6dej6dej6fdZedej6dej6dej:fdZedej6dej6dej:fdZedej6dej6dej:fdZ edej6dej6dej:fdZ!dej6de"fdZ#y)_EmbeddingDistanceChainMixina0Shared functionality for embedding distance evaluators. Attributes: embeddings (Embeddings): The embedding objects to vectorize the outputs. distance_metric (EmbeddingDistance): The distance metric to use for comparing the embeddings. )default_factory embeddings)defaultdistance_metricvaluesrcJ|jd}g} ddlm}|j| ddlm}|j||s t dt |t|r ddl}|S|S#t$rYNwxYw#t$rYEwxYw#t$r t dwxYw)zValidate that the TikTok library is installed. Args: values (Dict[str, Any]): The values to validate. Returns: Dict[str, Any]: The validated values. r3rrrNzThe tiktoken library is required to use the default OpenAI embeddings with embedding distance evaluators. Please either manually select a different Embeddings object or install tiktoken using `pip install tiktoken`.) getrrappendrr isinstancetupletiktoken)clsr6r3types_rr<s r_validate_tiktoken_installedz9_EmbeddingDistanceChainMixin._validate_tiktoken_installedHsZZ -   9 MM* +  N MM* +Q  j%- 0  v 5       !I s.A/A>'B / A;:A;> B  B  B"T)arbitrary_types_allowedcdgS)zgReturn the output keys of the chain. Returns: List[str]: The output keys. scorer.selfs r output_keysz(_EmbeddingDistanceChainMixin.output_keysxs yr/resultcDd|di}t|vr|t|t<|S)NrBr)rDrFparseds r_prepare_outputz,_EmbeddingDistanceChainMixin._prepare_outputs*6'?+ f $WoF7O r/metricc 8tj|jtj|jtj |j tj|jtj|ji}||vr||Std|)zGet the metric function for the given metric name. Args: metric (EmbeddingDistance): The metric name. Returns: Any: The metric function. zInvalid metric: ) rr)_cosine_distancer*_euclidean_distancer+_manhattan_distancer,_chebyshev_distancer-_hamming_distance ValueError)rDrJmetricss r _get_metricz(_EmbeddingDistanceChainMixin._get_metrics  $ $d&;&;  ' ')A)A  ' ')A)A  ' ')A)A  % %t'='=   W 6? "/x89 9r/abcX ddlm}d|||z S#t$r tdwxYw)zCompute the cosine distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.ndarray: The cosine distance. r)cosine_similarityzThe cosine_similarity function is required to compute cosine distance. Please install the langchain-community package using `pip install langchain-community`.g?)langchain_community.utils.mathrWr)rTrUrWs rrLz-_EmbeddingDistanceChainMixin._cosine_distancesB  H&q!,,,  6  s)cFtjj||z S)zCompute the Euclidean distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Euclidean distance. )nplinalgnormrTrUs rrMz0_EmbeddingDistanceChainMixin._euclidean_distancesyy~~a!e$$r/cXtjtj||z S)zCompute the Manhattan distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Manhattan distance. )rZsumabsr]s rrNz0_EmbeddingDistanceChainMixin._manhattan_distancevvbffQUm$$r/cXtjtj||z S)zCompute the Chebyshev distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Chebyshev distance. )rZmaxr`r]s rrOz0_EmbeddingDistanceChainMixin._chebyshev_distancerar/c2tj||k7S)zCompute the Hamming distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Hamming distance. )rZmeanr]s rrPz._EmbeddingDistanceChainMixin._hamming_distanceswwqAvr/vectorsc|j|j}||djdd|djddj}|S)zCompute the score based on the distance metric. Args: vectors (np.ndarray): The input vectors. Returns: float: The computed score. r)rSr5reshapeitem)rDrfrJrBs r_compute_scorez+_EmbeddingDistanceChainMixin._compute_scoresV!!$"6"67wqz))!R0'!*2D2DQ2KLQQS r/N)$r%r&r'r(rrr3r __annotations__rr)r5r rstrrr?r model_configpropertyrrEdictrIrS staticmethodrZndarrayrLfloatingrMrNrOrPfloatrlr.r/rr1r1<s#3EFJ F).7H7O7O)PO&P )$sCx.)T#s(^))V $LT#Ydt :"3::*-BJJ-2::-"**--( %rzz %bjj %R[[ % % %rzz %bjj %R[[ % % %rzz %bjj %R[[ % % RZZ BJJ 2;;   bjj U r/r1cleZdZdZedefdZedefdZede efdZ dde ee fde ede ee ffd Z dde ee fde ede ee ffd Zddddd d d ede edede e ede e ee fdede defdZddddd d d ede edede e ede e ee fdede defdZy)EmbeddingDistanceEvalChaina"Use embedding distances to score semantic difference between a prediction and reference. Examples: >>> chain = EmbeddingDistanceEvalChain() >>> result = chain.evaluate_strings(prediction="Hello", reference="Hi") >>> print(result) {'score': 0.5} rcy)zReturn whether the chain requires a reference. Returns: bool: True if a reference is required, False otherwise. Tr.rCs rrequires_referencez-EmbeddingDistanceEvalChain.requires_referencesr/c6d|jjdS)N embedding_ _distancer5valuerCs revaluation_namez*EmbeddingDistanceEvalChain.evaluation_namesD00667yAAr/c ddgS)eReturn the input keys of the chain. Returns: List[str]: The input keys. prediction referencer.rCs r input_keysz%EmbeddingDistanceEvalChain.input_keys sk**r/Ninputs run_managerctj|jj|d|dg}|j |}d|iS)a0Compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (Optional[CallbackManagerForChainRun], optional): The callback manager. Returns: Dict[str, Any]: The computed score. rrrBrZarrayr3embed_documentsrlrDrrrfrBs r_callz EmbeddingDistanceEvalChain._callsP(( OO + +VL-A6+CV,W X ##G,r/cK|jj|d|dgd{}tj|}|j |}d|iS7.w)a:Asynchronously compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. rrNrBr3aembed_documentsrZrrlrDrrembeddedrfrBs r_acallz!EmbeddingDistanceEvalChain._acall)sc99 L !6+#6 7  ((8$##G,  'AA/AF)r callbackstagsmetadatainclude_run_inforrrrrrkwargsc D|||d||||}|j|S)aEvaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The reference string (required) callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrrrrrrrI rDrrrrrrrrFs r_evaluate_stringsz,EmbeddingDistanceEvalChain._evaluate_strings?s52",9E-  ##F++r/crK|j||d||||d{}|j|S7w)aAsynchronously evaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrNacallrIrs r_aevaluate_stringsz-EmbeddingDistanceEvalChain._aevaluate_stringsasL2zz",9E- "  ##F++ 757N)r%r&r'r(rpboolryrnrrrrrrr rrrr rqrrr.r/rrwrwsDBBB+DI++=A S#X 89  c3h  0BF S#X =>  c3h  4$(#$(-1!& , ,C= ,  , tCy! ,4S>* , , ,  ,L$(#$(-1!& , ,C= ,  , tCy! ,4S>* , , ,  ,r/rwcFeZdZdZedeefdZedefdZ dde ee fde e de ee ffdZ dde ee fde ede ee ffd Zdddd d d ed edede eede e ee fdede defdZdddd d d ed edede eede e ee fdede defdZy)"PairwiseEmbeddingDistanceEvalChainaUse embedding distances to score semantic difference between two predictions. Examples: >>> chain = PairwiseEmbeddingDistanceEvalChain() >>> result = chain.evaluate_string_pairs(prediction="Hello", prediction_b="Hi") >>> print(result) {'score': 0.5} rc ddgS)rr prediction_br.rCs rrz-PairwiseEmbeddingDistanceEvalChain.input_keyssn--r/c6d|jjdS)Npairwise_embedding_r|r}rCs rrz2PairwiseEmbeddingDistanceEvalChain.evaluation_names$T%9%9%?%?$@ JJr/Nrrctj|jj|d|dg}|j |}d|iS)aCompute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (CallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. rrrBrrs rrz(PairwiseEmbeddingDistanceEvalChain._callsS(( OO + + %vn'=>   ##G,r/cK|jj|d|dgd{}tj|}|j |}d|iS7.w)a/Asynchronously compute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. rrNrBrrs rrz)PairwiseEmbeddingDistanceEvalChain._acallsc99 L !6.#9 :  ((8$##G,  rF)rrrrrrrrrrrc D|||d||||}|j|S)aEvaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrrr rDrrrrrrrrFs r_evaluate_string_pairsz9PairwiseEmbeddingDistanceEvalChain._evaluate_string_pairss54",lK-  ##F++r/crK|j||d||||d{}|j|S7w)aAsynchronously evaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to traces **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrNrrs r_aevaluate_string_pairsz:PairwiseEmbeddingDistanceEvalChain._aevaluate_string_pairssL8zz",lK- "  ##F++ rr)r%r&r'r(rprrnrrrrrr rrrr rrqrrr.r/rrrs.DI..KKK =A S#X 89  c3h  4BF S#X =>  c3h  6 $$(-1!&!,!, !,  !, tCy! !,4S>*!,!,!, !,P $$(-1!&#,#, #,  #, tCy! #,4S>*#,#,#, #,r/r)"r(enumrtypingrrrrnumpyrZ langchain_core.callbacks.managerrr r langchain_core.embeddingsr langchain_core.utilsr pydanticr rlangchain.chains.baserlangchain.evaluation.schemarrlangchain.schemarrrnrr1rwrr.r/rrsF,, 1)&'P$J*T$t5tnN,!=N,bL, "9L,r/