+#h dZddlZddlmZddlmZmZmZddlZddl Z ddl m Z  ddl m Z Gdd e j Zy#e$rZededZ[wwxYw) z%Clarifai as retriver to retrieve hitsN)ThreadPoolExecutor)ListOptionalUniondotdict)SearchzBClarifai is not installed. Install it using `pip install clarifai`c eZdZdZ d dededeedeffd ZdZ d de ee efdeed e jfd Z xZS) ClarifaiRMa Retrieval module uses clarifai to return the Top K relevant pasages for the given query. Assuming that you have ingested the source documents into clarifai App, where it is indexed and stored. Args: clarifai_user_id (str): Clarifai unique user_id. clarfiai_app_id (str): Clarifai App ID, where the documents are stored. clarifai_pat (str): Clarifai PAT key. k (int): Top K documents to retrieve. Examples: TODO clarifai_user_idclarfiai_app_id clarifai_patkc||_||_||ntjd|_||_t |j|j||j|_t|%|y)N CLARIFAI_PAT)user_idapp_idtop_kpat)r) rrosenvironrrr clarifai_searchsuper__init__)selfr r rr __class__s V/var/www/html/sandstorm/venv/lib/python3.12/site-packages/dspy/retrieve/clarifai_rm.pyrzClarifaiRM.__init__"sj& ' (4L"**^:T %LLA488  1cdd|ji}tj|jjj j |}|j|_|j }|S)N AuthorizationzKey )headers) rrequestsgetinputdatatexturlapparent_encodingencoding)rhitsheaderrequestrequested_texts r retrieve_hitszClarifaiRM.retrieve_hits4sX!T$((#45,,tzz3377H"44 rquery_or_queriesreturnc t|tr|gn|}g}|Dcgc]}|s| }}|D]}|jjddd|igi|}|D cgc]} | jD]} | } } } t d5} t | j|j| } ddd|jd D|Scc}wcc} } w#1swY0xYw)aUses clarifai-python SDK search function and retrieves top_k similar passages for given query, Args: query_or_queries : single query or list of queries k : Top K relevant documents to return Returns: passages in format of dotdict Examples: Below is a code snippet that shows how to use Marqo as the default retriver: ```python import clarifai llm = dspy.Clarifai(model=MODEL_URL, api_key="YOUR CLARIFAI_PAT") retriever_model = ClarifaiRM(clarifai_user_id="USER_ID", clarfiai_app_id="APP_ID", clarifai_pat="YOUR CLARIFAI_PAT") dspy.settings.configure(lm=llm, rm=retriever_model) ``` rankstext_raw ) max_workersNc36K|]}td|iyw) long_textNr).0ds r z%ClarifaiRM.forward..^sG!G[!$45Gs) isinstancestrrqueryr*rlistmapr.extend)rr/rkwargsqueriespassagesqr>search_responser%hitr*executorresultss rforwardzClarifaiRM.forward;s,*C0 !  %+1++ HE8d2288^U@S?T^W]^O%4IDtyyICICIDI#3 Gxx||D,>,>EF G OOGwG G H, J G GsCCC >&CC )N)N)__name__ __module__ __qualname____doc__r=rintrr.rrdspy PredictionrJ __classcell__)rs@rr r s} $'+ sm   $KO% %c49n 5%:B3-% %rr )rOrconcurrent.futuresrtypingrrrr"rQ dsp.utilsrclarifai.client.searchr ImportErrorerrRetriever r;rrr[s\+ 1(( -MM  L  sAA AA