#h+dZddlZddlmZddlmZmZddlmZeje Z dede d ee d ee d ee f d Z dd ee de d ee d ee deded ee fdZy)z% Utilities dedicated to URL sampling N)sample)ListOptional)UrlStoreurlstore samplesize exclude_min exclude_maxreturnc :g}|jD]}|j|Dcgc] }|jdvr|j"}}|r |t ||ks|/t ||kDr!t j d|t |t ||kDrtt||}n|}|j|Dcgc]}||z c}t jd|t |t |t |z |Scc}wcc}w)z4Iterate through the hosts in store and draw samples.)/Nzdiscarded (size): %s urls: %s)kz%s urls: %s prop.: %s) urldict _load_urlsurlpathpathlenLOGGERwarningsortedrextenddebug) rr r r output_urlsdomainpurlpathsmysamples M/var/www/html/sandstorm/venv/lib/python3.12/site-packages/courlan/sampling.py _make_sampler sK"" ((0 yy , FFH  &H +&H + NN=vs8} U  x=: %fX<=HH91FQJ9: '  M MCM )  - 8 7 (:s %D D input_urlsstrictverbosec|r$tjtjn#tjtjt dd||}|j |t||||S)zRSample a list of URLs by domain name, optionally using constraints on their numberTN) compressedlanguager"r#)rsetLevelloggingDEBUGERRORradd_urlsr )r!r r r r"r#rs r sample_urlsr,7sV & &4$vwWH j! *k; GG)NNFF)__doc__r(randomrtypingrrrr getLogger__name__rintstrr boolr,r-rr7s!   8 $$$$#$# $  #Y $T"&!% HS HH#H# H  H  H #YHr-