*#h0ddlZddlmZmZddlmZddlmZmZm Z m Z m Z m Z ddl ZddlZddlmZddlmZddlmZmZdd lmZdd lmZmZerd d lmZd \ZZ Z!eGddZ"y)N) dataclassfield)BytesIO) TYPE_CHECKINGAnyClassVarDictOptionalUnion)config)DownloadConfig)xopen xsplitext) array_cast)no_op_if_value_is_nullstring_to_dict) FeatureType)FFFc eZdZUdZdZeeed<dZe ed<dZ e ed<dZ ee ed<dZ ee ed <ej ej"ej$d Zeeed <edd d Ze ed<dZdee eefdefdZ ddedeee ee e dffdefdZdedee dfffdZdeej>ej@fdej@fdZ!dej@dej@fdZ"y)Audioa1Audio [`Feature`] to extract audio data from an audio file. Input: The Audio feature accepts as input: - A `str`: Absolute path to the audio file (i.e. random access is allowed). - A `dict` with the keys: - `path`: String with relative path of the audio file to the archive file. - `bytes`: Bytes content of the audio file. This is useful for archived files with sequential access. - A `dict` with the keys: - `path`: String with relative path of the audio file to the archive file. - `array`: Array containing the audio sample - `sampling_rate`: Integer corresponding to the sampling rate of the audio sample. This is useful for archived files with sequential access. Args: sampling_rate (`int`, *optional*): Target sampling rate. If `None`, the native sampling rate is used. mono (`bool`, defaults to `True`): Whether to convert the audio signal to mono by averaging samples across channels. decode (`bool`, defaults to `True`): Whether to decode the audio data. If `False`, returns the underlying dictionary in the format `{"path": audio_path, "bytes": audio_bytes}`. Example: ```py >>> from datasets import load_dataset, Audio >>> ds = load_dataset("PolyAI/minds14", name="en-US", split="train") >>> ds = ds.cast_column("audio", Audio(sampling_rate=16000)) >>> ds[0]["audio"] {'array': array([ 2.3443763e-05, 2.1729663e-04, 2.2145823e-04, ..., 3.8356509e-05, -7.3497440e-06, -2.1754686e-05], dtype=float32), 'path': '/root/.cache/huggingface/datasets/downloads/extracted/f14948e0e84be638dd7943ac36518a4cf3324e8b7aa331c5ab11541518e9368c/en-US~JOINT_ACCOUNT/602ba55abb1e6d0fbce92065.wav', 'sampling_rate': 16000} ``` N sampling_rateTmonodecodeiddictdtypebytespathpa_typeF)defaultinitrepr_typec|jSN)r!)selfs T/var/www/html/sandstorm/venv/lib/python3.12/site-packages/datasets/features/audio.py__call__zAudio.__call__Ms ||valuereturncR ddl}t|trd|dSt|tr|ddSd|vr8t }|j ||d|dd|jddS|jd :tjj|d r|d jd r|jd td |jd rItj|d tj j#tj$dz }n;tj&|d ddj#tj$dz }t t }|j |||dd|jddSd|jd dS|jd |jd #|jd |jd dSt)d|d#t$r}td|d}~wwxYw)zEncode example into a format for Arrow. Args: value (`str` or `dict`): Data passed as input to Audio feature. Returns: `dict` rNz;To support encoding audio data, please install 'soundfile'.rarrayrwav)formatr pcmzBTo use PCM files, please specify a 'sampling_rate' in Audio objectr)rihr)rmodezUAn audio sample should have one of 'path' or 'bytes' but they are missing or None in .) soundfile ImportError isinstancestrrrwritegetvaluegetosr isfileendswithKeyErrornp frombufferint16astypefloat32memmap ValueError)r(r,sferrbuffer bytes_values r)encode_examplezAudio.encode_examplePs f " eS !!51 1 u %"D1 1  YF HHVU7^U?-CEH R#__.= = YYv  *rww~~eFm/LV}%%e,99_-5"#ghh99W%"$--gbhh"O"V"VWYWaWa"bej"jK"$))E&M3"O"V"VWYWaWa"bej"jK )eO.DUS!'!2DAA!%uyy/@AA YYw  +uyy/@/L"YYw/69JK Kghmgnnop C f[\be e fsH H& H!!H&token_per_repo_idcR|js td|d|dt|dfn|ddf\}}||td|d ddl}ddl}|t|d d djnd}tjs|d k(r td tjs|d k(r td ||xsi}|jdd} | jtjrtjntj } t#| | d} || } t'| } t)|d| 5}|j+|\}}dddn|j+|\}}j,}|j.r|j1|}|j2r9|j2k7r*|j5|||j2}|j2}||dS#t $r}t d|d}~wwxYw#tt$f$rd} YwxYw#1swYxYw)aDecode example audio file into audio data. Args: value (`dict`): A dictionary with keys: - `path`: String with relative audio file path. - `bytes`: Bytes of the audio file. token_per_repo_id (`dict`, *optional*): To access and decode audio files from private repositories on the Hub, you can pass a dictionary repo_id (`str`) -> token (`bool` or `str`) Returns: `dict` zMDecoding is disabled for this feature. Please use Audio(decode=True) instead.rNr zJAn audio sample should have one of 'path' or 'bytes' but both are None in r6rzJTo support decoding audio files, please install 'librosa' and 'soundfile'.ropuszDecoding 'opus' files requires system library 'libsndfile'>=1.0.31, You can try to update `soundfile` python library: `pip install "soundfile>=0.12.1"`. mp3zDecoding 'mp3' files requires system library 'libsndfile'>=1.1.0, You can try to update `soundfile` python library: `pip install "soundfile>=0.12.1"`. z::repo_id)tokenrb)download_config)orig_sr target_sr)r r/r)r RuntimeErrorrrHlibrosar7r8rlowerr IS_OPUS_SUPPORTEDIS_MP3_SUPPORTEDsplit startswith HF_ENDPOINTHUB_DATASETS_URLHUB_DATASETS_HFFS_URLrrArrreadTrto_monorresample)r(r,rNr filerZrIrJ audio_format source_urlpatternrSrTrVfr/rs r)decode_examplezAudio.decode_examplesV&{{no oAFwA[eFmWU7^%<=bghnboquav d 9Iyq)!"-335t ''LF,Bh ((\U-Bh  < 1 7R D)"-J+5+@+@ASAS+T''Z`ZvZv  (W=iH)'2-59OtT?C 2q')wwqz$} 2 2$&774= E= 99OOE*E   $"4"4 "E$$UMTM_M_$`E ..Mu}MMS ujkqt t u0)   2 2s6G)HH) H2 G>>HHHH&rc^ddlm}|jr td|d|ddS)z[If in the decodable state, raise an error, otherwise flatten the feature into a dictionary.r)Valuez'Cannot flatten a decoded Audio feature.binarystringr)featuresrnrrH)r(rns r)flattenz Audio.flattens0# ;;FG G8_(O  r+storagectjj|jrltjdgt |ztj }tjj||gddg|j}nQtjj|jrltjdgt |ztj}tjj||gddg|j}ntjj|jri|jjdrNtj|jDcgc]}|tj!|nd!c}}n*tjj|jr|jj#ddk\r|j%d}n6tjdgt |ztj }|jj#ddk\r|j%d}n6tjdgt |ztj}tjj||gddg|j}t'||j(Scc}w)aCast an Arrow array to the Audio arrow storage type. The Arrow types that can be converted to the Audio pyarrow storage type are: - `pa.string()` - it must contain the "path" data - `pa.binary()` - it must contain the audio bytes - `pa.struct({"bytes": pa.binary()})` - `pa.struct({"path": pa.string()})` - `pa.struct({"bytes": pa.binary(), "path": pa.string()})` - order doesn't matter Args: storage (`Union[pa.StringArray, pa.StructArray]`): PyArrow array to cast. Returns: `pa.StructArray`: Array in the Audio arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})` Ntyperr maskr/r)patypes is_stringrvr/lenro StructArray from_arraysis_null is_binaryrp is_structget_all_field_indices to_pylistrrMget_field_indexrrr!)r(rs bytes_array path_arrayxs r) cast_storagezAudio.cast_storages$ 88  gll +((D6CL#8ryy{KKnn00+w1G'SYIZahapapar0sG XX   -4&3w<"7biikJJnn00':1FRXHY`g`o`o`q0rG XX   -',,2T2TU\2]hh^e^o^o^qrYZQ] 6 6q 9PT TrsG XX   -||++G49%mmG4  hhvG '<299;O ||++F3q8$]]62 XXtfs7|&;"))+N nn00+z1JWV\L]dkdsdsdu0vG'4<<00 ss$K=c dtd}tj|jDcgc]}||d ||dn|dndc}tj}tj|j djDcgc]%}|t jj|nd'c}tj}tjj||gddg|j}t||jScc}wcc}w)a8Embed audio files into the Arrow array. Args: storage (`pa.StructArray`): PyArrow array to embed. Returns: `pa.StructArray`: Array in the Audio arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. cjt|d5}|j}ddd|S#1swYSxYw)NrU)rrc)r rkbytes_s r) path_to_bytesz*Audio.embed_storage..path_to_bytess3tT" "a "M "Ms(2Nrr rurw)rryr/rrorr>r basenamerpr}r~rrr!)r(rsrrrr rs r) embed_storagezAudio.embed_storages     hh!**, UVTaQwZ-?qy)QwZgkk   XXNUmm\bNcNmNmNo pdt'7RWW  d #T A p ..,,k:-FRXHY`k`s`s`u,v'4<<00  qs D(*D-r')#__name__ __module__ __qualname____doc__rr int__annotations__rboolrrr:rrrystructrorpr!rrr%r*r rrrMr rlrr StringArrayr}rrr+r)rrs[)V$(M8C='D$FDB !E8C=!&RYYibiik'RSGXc]SwU?E3?/E#ud*:$;//d]aFNFN.6tCsDRVAW dataclassesrriortypingrrrr r r numpyrBpyarrowryr download.download_configr#download.streaming_download_managerrrtablerutils.py_utilsrrrqr_ffmpeg_warned_librosa_warned_audioread_warnedrrr+r)rs] (FF5BC%6I2!2 @1@1 @1r+