o ˷e1@s$dZddlmZddlZddlZddlZddlZddlZddlm Z ddlm Z ddlm Z ddlm Z ddl mZd Zd Zd Zd Zd ZdZeeZddZefddZefddZd.ddZddZddZddZddZd d!Z d"d#Z!d$d%Z"d&d'Z#d(d)Z$d*d+Z%Gd,d-d-e&Z'dS)/z4Shared utilities used by both downloads and uploads.)absolute_importN)parse_qs) urlencode)urlsplit) urlunsplit)commonrangez content-rangezCurrently using crcmod in pure python form. This is a slow implementation. Python 3 has a faster implementation, `google-crc32c`, which will be used if it is installed.zx-goog-generationz x-goog-hashzNo {checksum_type} checksum was returned from the service while downloading {} (which happens for composite objects), so client-side content integrity checking is not being performed.cCdS)zSimple default callback.Nr r r V/var/www/ideatree/venv/lib/python3.10/site-packages/google/resumable_media/_helpers.py do_nothing2r cCs,||}||vr|t|d|||S)aChecks that a specific header is in a headers dictionary. Args: response (object): An HTTP response object, expected to have a ``headers`` attribute that is a ``Mapping[str, str]``. name (str): The name of a required header. get_headers (Callable[Any, Mapping[str, str]]): Helper to get headers from an HTTP response. callback (Optional[Callable]): A callback that takes no arguments, to be executed when an exception is being raised. Returns: str: The desired header. Raises: ~google.resumable_media.common.InvalidResponse: If the header is missing. z$Response headers must contain header)rInvalidResponse)responsename get_headerscallbackheadersr r r header_required6srcCs<||}||vr|tjvr|tj|d|dg|R|S)a`Require a response has a status code among a list. Args: response (object): The HTTP response object. status_codes (tuple): The acceptable status codes. get_status_code (Callable[Any, int]): Helper to get a status code from a response. callback (Optional[Callable]): A callback that takes no arguments, to be executed when an exception is being raised. Returns: int: The status code. Raises: ~google.resumable_media.common.InvalidResponse: If the status code is not one of the values in ``status_codes``. zRequest failed with status codezExpected one of)r RETRYABLEr)r status_codesget_status_coder status_coder r r require_status_codeSs r@cCs0||}||kr |}tdd}||d|fS)aCalculate the amount of time to wait before a retry attempt. Wait time grows exponentially with the number of attempts, until ``max_sleep``. A random amount of jitter (between 0 and 1 seconds) is added to spread out retry attempts from different clients. Args: base_wait (float): The "base" wait time (i.e. without any jitter) that will be multiplied until it reaches the maximum sleep. max_sleep (float): Maximum value that a sleep time is allowed to be. multiplier (float): Multiplier to apply to the base wait. Returns: Tuple[float, float]: The new base wait time as well as the wait time to be applied (with a random amount of jitter between 0 and 1 seconds added). rigMbP?)randomrandint) base_wait max_sleep multiplier new_base_wait jitter_msr r r calculate_retry_waitss  r"c Cs`z ddl}|}W|Sty/zddl}|jd}tWY|Sty.tdww)zGet crc32c object Attempt to use the Google-CRC32c package. If it isn't available, try to use CRCMod. CRCMod might be using a 'slow' varietal. If so, warn... rNzcrc-32cz3Failed to import either `google-crc32c` or `crcmod`) google_crc32cChecksum ImportErrorcrcmod predefinedCrc_is_fast_crcmod)r#crc_objr&r r r _get_crc32c_objects     r+cCs:tdttdgd}t|dd}|stjttdd|S)Nz crcmod.crcmod_usingExtensionrF) stacklevel) __import__globalslocalsgetattrwarningswarn_SLOW_CRC32C_WARNINGRuntimeWarning) nested_crcmodfast_crcr r r r)s r)cCs|dkrdS|S)Nmd5md5Hashr  checksum_typer r r _get_metadata_keysr=cCst|}|dS)zConvert a checksum object into a digest encoded for an HTTP header. Args: bytes: A checksum digest bytestring. Returns: str: A base64 string representation of the input. zutf-8)base64 b64encodedecode)digest_bytestringencoded_digestr r r prepare_checksum_digests rCcCs|dvrtd|dvrF||}t|t||d}|dur3tj||d}t|t }||fS|dkr?t }||fSt }||fSd}t }||fS)aGet the expected checksum and checksum object for the download response. Args: response (~requests.Response): The HTTP response object. get_headers (callable: response->dict): returns response headers. media_url (str): The URL containing the media to be downloaded. checksum_type Optional(str): The checksum type to read from the headers, exactly as it will appear in the headers (case-sensitive). Must be "md5", "crc32c" or None. Returns: Tuple (Optional[str], object): The expected checksum of the response, if it can be detected from the ``X-Goog-Hash`` header, and the appropriate checksum object for the expected checksum. )r9crc32cN4checksum must be ``'md5'``, ``'crc32c'`` or ``None``)r9rD)checksum_labelNr;r9) ValueError_parse_checksum_headerget _HASH_HEADER_MISSING_CHECKSUMformatupper_LOGGERinfo_DoNothingHashhashlibr9r+)rr media_urlr<rexpected_checksummsgchecksum_objectr r r _get_expected_checksums,   rVcCs|durdSg}|dD]}|dd\}}||kr"||q t|dkr+dSt|dkr5|dSt|d|||)aDParses the checksum header from an ``X-Goog-Hash`` value. .. _header reference: https://cloud.google.com/storage/docs/ xml-api/reference-headers#xgooghash Expects ``header_value`` (if not :data:`None`) to be in one of the three following formats: * ``crc32c=n03x6A==`` * ``md5=Ojk9c3dhfxgoKVVHYwFbHQ==`` * ``crc32c=n03x6A==,md5=Ojk9c3dhfxgoKVVHYwFbHQ==`` See the `header reference`_ for more information. Args: header_value (Optional[str]): The ``X-Goog-Hash`` header from a download response. response (~requests.Response): The HTTP response object. checksum_label (str): The label of the header value to read, as in the examples above. Typically "md5" or "crc32c" Returns: Optional[str]: The expected checksum of the response, if it can be detected from the ``X-Goog-Hash`` header; otherwise, None. Raises: ~google.resumable_media.common.InvalidResponse: If there are multiple checksums of the requested type in ``header_value``. N,=rz.X-Goog-Hash header had multiple ``{}`` values.)splitlstripappendlenrrrL) header_valuerrFmatcheschecksumrvaluer r r rHs$    rHcCs2|dkrtS|dkrtS|durdStd)z~Respond with a checksum object for a supported type, if not None. Raises ValueError if checksum_type is unsupported. r9rDNrE)rQr9r+rGr;r r r _get_checksum_object(srbcCs(||}|td}|durdSt|S)aParses the generation header from an ``X-Goog-Generation`` value. Args: response (~requests.Response): The HTTP response object. get_headers (callable: response->dict): returns response headers. Returns: Optional[long]: The object generation from the response, if it can be detected from the ``X-Goog-Generation`` header; otherwise, None. N)rI_GENERATION_HEADERint)rrrobject_generationr r r _parse_generation_header7s  rfcCs>t|\}}}}}t|}|dd}|durdSt|dS)zRetrieve the object generation query param specified in the media url. Args: media_url (str): The URL containing the media to be downloaded. Returns: long: The object generation from the media url if exists; otherwise, None. generationNr)rrrIrd)rR_query query_paramsrer r r _get_generation_from_urlKs    rkc CsTt|dkr|St|\}}}}}t|}i||}t|dd}t|||||fS)aAdd query parameters to a base url. Args: media_url (str): The URL containing the media to be downloaded. query_params (dict): Names and values of the query parameters to add. Returns: str: URL with additional query strings appended. rT)doseq)r]rrrr) rRrjschemenetlocpathrifragparams new_paramsr r r add_query_parameters_s   rsc@seZdZdZddZdS)rPzDo-nothing hash object. Intended as a stand-in for ``hashlib.md5`` or a crc32c checksum implementation in cases where it isn't necessary to compute the hash. cCr )zDo-nothing ``update`` method. Intended to match the interface of ``hashlib.md5`` and other checksums. Args: unused_chunk (bytes): A chunk of data. Nr )self unused_chunkr r r update{r z_DoNothingHash.updateN)__name__ __module__ __qualname____doc__rvr r r r rPts rP)r)(rz __future__rr>rQloggingrr3 urllib.parserrrrgoogle.resumable_mediar RANGE_HEADERCONTENT_RANGE_HEADERr5rcrJrK getLoggerrwrNr rrr"r+r)r=rCrVrHrbrfrkrsobjectrPr r r r sF          *5