+#hDddlZddlmZmZddlmZdZGddeZy)N)LiteralOptional)LMc i}|jD]?\}}|dk(r||d<|dk(r d|z |d<|dk(r||d<*|dk(r||d <5|d k(r;|||<A|S) Nnnum_return_sequencesfrequency_penaltyg?repetition_penaltypresence_penaltydiversity_penalty max_tokensmax_new_tokensmodel)items)kwargs hf_kwargskvs K/var/www/html/sandstorm/venv/lib/python3.12/site-packages/dsp/modules/hf.py openai_to_hfr sI  1 801I, - % %.1AgI* + $ $-.I) * , *+I& ' '\ IaL  cpeZdZddddifdedeedededd eed eef fd Zd Z d Z ddZ xZ S)HFModelNFautor checkpoint is_client hf_device_map)rbalancedbalanced_low_0 sequentialtoken model_kwargsct||d|_||_||_t |xst jjd}|j}|j} d|vr|jd| j||js ddl } ddl m} m} m} m}| j)| j*j-rd nd |_ | j.|fi|j0d d}d |vxsd |v|_d|vxsd|v|_|j2s|j4s Jd||j.||n|fi||_d|_|j2r| n| }|rjd|_|jr%|j.|fd|ji| |_n|j.|fi| j=|j(|_nb|jr%|j.|fd|ji| |_n1|j.|fi| j=|j(|_d|_g|_!yg|_!y#t$$r}t'd|d}~wwxYw#t@$rX| j.||n|fd|ji| |_d|_|j.|fi||_d|_Yg|_!ywxYw)aswrapper for Hugging Face models Args: model (str): HF model identifier to load and use checkpoint (str, optional): load specific checkpoints of the model. Defaults to None. is_client (bool, optional): whether to access models via client. Defaults to False. hf_device_map (str, optional): HF config strategy to load the model. Recommeded to use "auto", which will help loading large models using accelerate. Defaults to "auto". model_kwargs (dict, optional): additional kwargs to pass to the model constructor. Defaults to empty dict. hfHF_TOKEN)r! device_maprN) AutoConfigAutoModelForCausalLMAutoModelForSeq2SeqLM AutoTokenizerzGYou need to install Hugging Face transformers library to use HF models.cudacpu architecturesConditionalGenerationT5WithLMHeadModelCausalLMGPT2LMHeadModelz!Unknown HuggingFace model class: TF)"super__init__providerrr&dictosenvirongetcopypopupdatetorch transformersr'r(r)r* ImportErrorModuleNotFoundErrordevicer+ is_availablefrom_pretrained__dict__encoder_decoder_modeldecoder_only_model tokenizer rationalertodrop_prompt_from_output ValueErrorhistory)selfrrrrr!r"hf_autoconfig_kwargshf_autotokenizer_kwargshf_automodel_kwargsr<r'r(r)r*exc architectureAutoModelClass __class__s rr3zHFModel.__init__s=2  "'#%*M2::>>*3MN"6";";"=2779 < '   \ *""<0~~ oo  ,,1H1H1JvPUVDK? 49z99 * (? ,-. / /F.U.'<7*,6+E*mK\`lKl'..$2I2I?6ug>?I!>!>!>'/EZ"-" "&:>:T:T!6Zn&+DN %C^%C%C&&'+&2& &D^%C%C&&1&"T[[/ %C^%C%C!&'+&2& &D^%C%C!&1&"T[[/ 05, r K )] r 4A1AA'/EZ#* 04,!>!>!>"-"04,  4s,'I%*E+J% I?. I::I?AK#"K#c |}i|j|}|j|fi|}||||d}|jj||S)N)promptresponser raw_kwargs)r _generaterKappend)rLrUrrWrVrKs r basic_requestzHFModel.basic_requests] *DKK*6*!4>>&3F3 $   G$rc H|jrJitd i|jtd i|}t|tr |ddd}|j|dj|j}|jjd i||}|jr$|jj d}|dd|df}|jj#|d Dcgc]}d |i}}||d }|S#t t tf$rtdYwxYwcc}w) Nmessagesrcontentz,Failed to extract 'content' from the prompt.pt)return_tensorsT)skip_special_tokenstext)rUchoices)rrr isinstancer5KeyError IndexError TypeErrorprintrFrHr@rgeneraterI input_idsshape batch_decode) rLrUrinputsoutputs input_lengthc completionsrVs rrXzHFModel._generates1>>!!JL/4;;/J<3I&3IJ fd # F +A.y9t<?? L&$**%%99&9  ' '!++11!4La./G,0NN,G,Gei,G,jkq{k k" j)4 FDE Fls C:& D:DDc |sJd|dusJd|jdddkDs|jdddkDrd|d <|j|fi|}|d Dcgc]}|d  c}Scc}w) Nzfor nowFrr` temperaturegg?T do_samplercrb)r8request)rLrUonly_completed return_sortedrrVrqs r__call__zHFModel.__call__s(y(~%0y0% ::c1  !VZZ s%Cc%I"&F; 4<<1&1#+I#67a& 777s A-)TF) __name__ __module__ __qualname__strrboolrr5r3rZrXry __classcell__)rSs@rrrs%) #')ooSMo o   o}otnob 08rr)r6typingrrdsp.modules.lmrrrrdrrrs# $ &a8ba8r