File manager - Edit - /opt/saltstack/salt/lib/python3.10/site-packages/libcloud/storage/drivers/__pycache__/oss.cpython-310.pyc
Back
o ;j�� � @ sp d dl Z d dlZd dlZd dlZd dlZd dlmZ d dlmZm Z m Z m Z m Z m Z d dlmZmZ d dlmZmZmZ d dlmZ d dlmZmZmZ d dlm Z m Z m Z d d l m Z m!Z!m"Z"m#Z#m$Z$m%Z% z d d l&m'Z'm(Z( W n e)y� d d l*m'Z'm(Z( Y nw g d �Z+d Z,d Z-dZ.dZ/G dd� de�Z0G dd� de0e�Z1G dd� de�Z2G dd� d�Z3G dd� de �Z4dS )� N)�sha1)�ET�b�httplib�tostring�urlquote� urlencode)�findtext�fixxpath)� RawResponse� XmlResponse�ConnectionUserAndKey)�read_in_chunks)� LibcloudError�InvalidCredsError�MalformedResponseError)�Object� Container� StorageDriver)�ContainerError�ObjectDoesNotExistError�ObjectHashMismatchError�ContainerIsNotEmptyError�InvalidContainerNameError�ContainerDoesNotExistError)�Element� SubElement)�OSSStorageDriver�OSSMultipartUpload�EXPIRATION_SECONDS� CHUNK_SIZE�MAX_UPLOADS_PER_RESPONSEz%a, %d %b %Y %H:%M:%S GMTi� i � i� c @ s8 e Zd ZdZejejejgZdd� Z dd� Z dd� Z dS )� OSSResponseNc C s, t | j�}d| kodkn p|| jv S )N�� i+ )�int�status�valid_response_codes)�self�i� r) �P/opt/saltstack/salt/lib/python3.10/site-packages/libcloud/storage/drivers/oss.py�successE s "zOSSResponse.successc C sf t | j�dkr | js | jS ztjdd�}tj| j�d�|d�}W |S ty2 td| j| j j d��w )z8 OSSResponse body is in utf-8 encoding. r �utf-8)�encoding)�parserzFailed to parse XML)�body�driver) �lenr/ Zparse_zero_length_bodyr Z XMLParserZXML�encode� Exceptionr � connectionr0 )r' r. r/ r) r) r* � parse_bodyI s � ��zOSSResponse.parse_bodyc C sl | j tjtjfv rt| j��| j tjkrtdtd��| j tj kr*td| j | j f ��td| j | jf td��)NzLThis bucket is located in a different region. Please use the correct driver.�r0 z7The method is not allowed. Status code: %d, headers: %sz(Unknown error. Status code: %d, body: %s) r% r � UNAUTHORIZED� FORBIDDENr r/ �MOVED_PERMANENTLYr r �METHOD_NOT_ALLOWED�headers�r' r) r) r* � parse_errorY s" � ���zOSSResponse.parse_error) �__name__� __module__� __qualname__� namespacer � NOT_FOUND�CONFLICT� BAD_REQUESTr& r+ r5 r= r) r) r) r* r" A s r" c @ s e Zd ZdS )�OSSRawResponseN)r> r? r@ r) r) r) r* rE l s rE c sn e Zd ZdZdZdZeZeZ e dd� �Z e dd� �Z dd � Z d d � Zd d � Z d� fdd� Z� ZS )� OSSConnectionzC Represents a single connection to the Aliyun OSS Endpoint z aliyuncs.comZossc C s" dddd�}i }t |�� �D ] \} } | �� } | |v r!| �� || < q| �|�r,| �� || < q|r5t|�|d< | g} t|�� �D ] \} } | �| � q>d�| �}g } t|�� �D ] \} } | �d� | | �� qUd�| �}g }|||fD ] } | rx|�| � qod�|�}t � t j t|�t|�td��� �}|S )z� Signature = base64(hmac-sha1(AccessKeySecret, VERB + " " + CONTENT-MD5 + " " + CONTENT-TYPE + " " + EXPIRES + " " + CanonicalizedOSSHeaders + CanonicalizedResource)) � )z content-md5� content-type�expiresrI � z{}:{})Z digestmod)�list�items�lower�strip� startswith�str�sorted�append�join�format�base64� b64encode�hmac�newr r �digest)�methodr; �paramsrI � secret_key�path� vendor_prefixZspecial_headersZvendor_headers�key�valueZ key_lower�buf�_Zstring_to_signZ header_stringZvalues_to_signZb64_hmacr) r) r* �_get_auth_signaturez s: � � �z!OSSConnection._get_auth_signaturec C sX d}d| v r | d }nd| v r| d }|r$zt |�W S ty# Y nw t t�� �t S )z> Get expires timeout seconds from parameters. NrI �Expires)r$ r3 �timer )r[ rI r) r) r* � _get_expires� s �zOSSConnection._get_expiresc C s( | � |�}t|�}| j|d< ||d< |S )NZOSSAccessKeyIdrd )rf rP Zuser_id)r' r[ Z expires_atrI r) r) r* �add_default_params� s z OSSConnection.add_default_paramsc C s t �tt �� �|d< |S )NZDate)re �strftime�GMT_TIME_FORMAT�gmtime)r' r; r) r) r* �add_default_headers� s z!OSSConnection.add_default_headersc C sP | j r d�| j j| j�}n| j}| j| j|||d | j|| jjd�|d< ||fS )Nz/{}{}rd )rZ r; r[ rI r\ r] r^ Z Signature) � _containerrT �name�actionrc rZ r_ r0 �http_vendor_prefix)r' r[ r; r] r) r) r* �pre_connect_hook� s � z OSSConnection.pre_connect_hookN�GETFc sr d� | j| j�| _|| _|r-|jr-d|jv r$d� |j|jd | j�| _n d� |j| j�| _t� j||||||d�S )Nz{}.{}�locationz{}.{}.{})rn r[ �datar; rZ �raw) rT �_default_location�_domain�hostrl rm �extra�super�request)r' rn r[ rs r; rZ rt � container�� __class__r) r* rz � s$ ��zOSSConnection.request)NNNrq FN)r> r? r@ �__doc__rv ru r" Z responseClsrE ZrawResponseCls� staticmethodrc rf rg rk rp rz � __classcell__r) r) r| r* rF p s( , �rF c @ s e Zd ZdZdd� Zdd� ZdS )r z; Class representing an Aliyun OSS multipart upload c C s || _ || _|| _dS )aT Class representing an Aliyun OSS multipart upload :param key: The object/key that was being uploaded :type key: ``str`` :param id: The upload id assigned by Aliyun :type id: ``str`` :param initiated: The date/time at which the upload was started :type created_at: ``str`` N)r_ �id� initiated)r' r_ r� r� r) r) r* �__init__� s zOSSMultipartUpload.__init__c C s d| j S )Nz <OSSMultipartUpload: key=%s>)r_ r< r) r) r* �__repr__ s zOSSMultipartUpload.__repr__N)r> r? r@ r~ r� r� r) r) r) r* r � s r c @ sV e Zd ZdZdZeZdZdZdZ dZ dZ dd � Z dAd d �Z d d � Zdd� ZdBdd�Zdd� Z dCdd�ZdBdd�Z dDdd�ZdAdd�Zd d � Zddefd d �ZdBd d!�Zd"d#� Z $ dEd%d&�Z $ dFd'd(�Z dGd)d*�Z dHd+d,�Z dBd-d.�Z dBd/d0�Z d1d2� Z!d3d4� Z"d5d6� Z#d7d8� Z$d9d:� Z%d;d<� Z&d=d>� Z'd?d@� Z(dS )Ir z Aliyun OSSz!http://www.aliyun.com/product/oss�md5FTNzx-oss-c C s> | j �d�}|jtjkr| j|jdd�}|S td|j | d��)N�/zBuckets/Bucket)�obj�xpath�Unexpected status code: %sr6 )r4 rz r% r �OK�_to_containers�objectr )r' �responseZ containersr) r) r* �iterate_containers s z#OSSStorageDriver.iterate_containersc c s� � | � ||�}i }|r||d< d}d}|s^|r||d< | jjd||d�}|jtjkr3td|j | d��| j|jd |d �}|j� t d | j d ��� � } | d k}d}|D ]} | j }| V qQ|rdS dS )a� Return a generator of objects for the given container. :param container: Container instance :type container: :class:`Container` :keyword prefix: Only return objects starting with prefix :type prefix: ``str`` :keyword ex_prefix: (Deprecated.) Only return objects starting with ex_prefix :type ex_prefix: ``str`` :return: A generator of Object instances. :rtype: ``generator`` of :class:`Object` �prefixNF�markerr� �r[ r{ r� r6 ZContents)r� r� r{ � IsTruncated�r� rA �false)Z_normalize_prefix_argumentr4 rz r% r r� r �_to_objsr� r r rA rM rm ) r' r{ r� Z ex_prefixr[ Zlast_keyZ exhaustedr� Zobjects� is_truncatedr� r) r) r* �iterate_container_objects s2 � �� �z*OSSStorageDriver.iterate_container_objectsc C s. | � � D ] }|j|kr| S qtd | |d��)N�r` r0 �container_name)r� rm r )r' r� r{ r) r) r* � get_containerO s �z OSSStorageDriver.get_containerc C sZ | j |d�}| �||�}| jj|d|d�}|jtjkr&| j|||jd�}|S t d | |d��)N)r� �HEAD�rZ r{ )� object_namer{ r; �r` r0 r� ) r� �_get_object_pathr4 rz r% r r� �_headers_to_objectr; r )r' r� r� r{ � object_pathr� r� r) r) r* � get_objectU s �zOSSStorageDriver.get_objectc C s� d}|rt d�}t|d�}||_t|�}d|i}nd}t||| d�}| jjd|d|d �}|jtj kr4|S |jtj krAt d || d ��|jtj krQt d |j || d ��td |j | d��)z� @inherits :class:`StorageDriver.create_container` :keyword ex_location: The desired location where to create container :type keyword: ``str`` NZCreateBucketConfigurationZLocationConstraintrr rG �rm rx r0 r� �PUT�rs rZ r{ zgContainer with this name already exists. The name must be unique among all the containers in the system�r` r� r0 z'Bad request when creating container: %sr� r6 )r r �textr r r4 rz r% r r� rC r rD r r/ r ) r' r� Z ex_locationrx �root�childrs r{ r� r) r) r* �create_containerb s2 � �z!OSSStorageDriver.create_containerc C s^ | j jdd|d�}|jtjkrdS |jtjkr td|j| d��|jtjkr-t d | |jd��dS ) Nr� �DELETEr� Tz1Container must be empty before it can be deleted.r� r� F) r4 rz r% r � NO_CONTENTrC r rm rB r )r' r{ r� r) r) r* �delete_container� s � z!OSSStorageDriver.delete_containerc C sL | � |j|j�}| jj|dd|jd�}| j|| j|||j|||d�tj d�S )Nrq T�rZ rt r{ )r� r� �destination_path�overwrite_existing�delete_on_failure�r� �callbackr� Zcallback_kwargsZsuccess_status_code) r� r{ rm r4 rz � _get_objectZ _save_objectr� r r� )r' r� r� r� r� �obj_pathr� r) r) r* �download_object� s ���z OSSStorageDriver.download_objectc C sD | � |j|j�}| jj|dd|jd�}| j|t||j|d�tj d�S )Nrq Tr� )�iterator� chunk_sizer� ) r� r{ rm r4 rz r� r r� r r� )r' r� r� r� r� r) r) r* �download_object_as_stream� s � �z*OSSStorageDriver.download_object_as_streamc C s | j |||||d�S )N)r{ r� rx � file_path� verify_hash)� _put_object)r' r� r{ r� rx r� r; r) r) r* � upload_object� s �z OSSStorageDriver.upload_objectc C s* d}d }| j r | j||||||d|d�S )Nr� F)r{ r� rx rZ � query_args�streamr� r; )�supports_multipart_uploadr� )r' r� r{ r� rx r; rZ r[ r) r) r* �upload_object_via_stream� s �z)OSSStorageDriver.upload_object_via_streamc C sT | � |j|j�}| jj|d|jd�}|jtjkrdS |jtjkr(t d | |jd��dS )Nr� r� Tr� F) r� r{ rm r4 rz r% r r� rB r )r' r� r� r� r) r) r* � delete_object� s z OSSStorageDriver.delete_objectc # s" � � j s td� d��d}d|i}|r||d< |r ||d< � fdd�} � jj|||d �}|jtjkr;td |j � d��|�� } | �t d � j d ��D ]} || d�} || d�} || d�} t | | | �V qI| � t d� j d ��}|� � dkrtdS | � t d� j d ��}| � t d� j d ��}||d< ||d< q#)a Extension method for listing all in-progress OSS multipart uploads. Each multipart upload which has not been committed or aborted is considered in-progress. :param container: The container holding the uploads :type container: :class:`Container` :keyword prefix: Print only uploads of objects with this prefix :type prefix: ``str`` :keyword delimiter: The object/key names are grouped based on being split by this delimiter :type delimiter: ``str`` :keyword max_uploads: The max upload items returned for one request :type max_uploads: ``int`` :return: A generator of OSSMultipartUpload instances. :rtype: ``generator`` of :class:`OSSMultipartUpload` zFeature not supportedr6 z /?uploadsz max-uploadsr� � delimiterc s | � t|� jd��S �Nr� )r r rA )�noder� r< r) r* �finder s z=OSSStorageDriver.ex_iterate_multipart_uploads.<locals>.finderTr� z.Error fetching multipart uploads. Got code: %sZUploadr� �Key�UploadIdZ Initiatedr� r� ZNextUploadIdMarkerZ NextKeyMarkerz key-markerzupload-id-markerN)r� r r4 rz r% r r� r5 �findallr rA r r rM )r' r{ r� r� Z max_uploads� request_pathr[ r� r� r/ r� r_ � upload_idr� r� Z upload_markerZ key_markerr) r< r* � ex_iterate_multipart_uploads� sB � � ��z-OSSStorageDriver.ex_iterate_multipart_uploadsc C s: | j ||dd�D ]}| �||j�}| j||j|d� qdS )a7 Extension method for removing all partially completed OSS multipart uploads. :param container: The container holding the uploads :type container: :class:`Container` :keyword prefix: Delete only uploads of objects with this prefix :type prefix: ``str`` N)r� �r{ )r� r� r_ �_abort_multipartr� )r' r{ r� Zuploadr� r) r) r* � ex_abort_all_multipart_uploads: s �z/OSSStorageDriver.ex_abort_all_multipart_uploadsc C s t |�}|S �N)r )r' rm r) r) r* �_clean_object_nameK s z#OSSStorageDriver._clean_object_namer� c C s |pi }|rt j�|�std| ��|dur$t|d�s$t|d�s$td��| j|||d�|d< |rG| jj||||d| d �} | � || � � �\} } n@t |d ��}| jj||||d| d �} W d � n1 scw Y t |d ��}| � || � � �\} } W d � n1 s�w Y | | | d �S ) z� Helper function for setting common request headers and calling the passed in callback which uploads an object. zFile %s does not existN�next�__next__z-iterator object must implement next() method.)r� z Content-TypeT)rZ rs r; rt r{ �rb)r� �bytes_transferred� data_hash) �osr] �exists�OSError�hasattr�AttributeErrorZ_determine_content_typer4 rz Z_hash_buffered_stream�_get_hash_function�open)r' r� � content_typer� �request_methodr; r� r� �chunked� multipartr{ r� Z stream_hashZ stream_lengthZ file_streamr) r) r* �_upload_objectO sN �� � � � � ��z OSSStorageDriver._upload_objectc C sV | pi } |pi }|� dd�} |� dd�} |� dd�} | r2t| �� �D ]\} }| jd| } || | < q"| rE| dvr>td| ��| | | jd < | �||�}|rTd �||f�}| j|| ||| |||d �}|d }|d }|j} | d � dd�}|r�|d � � |� � kr�t d� ||d �|| d��|j tjkr�t|||d| i| || d�}|S td|j | d��)zL Create an object and upload data using the given function. r� N� meta_data�aclzmeta-%s)z public-readZprivatezpublic-read-writezinvalid acl value: %sz object-acl�?)r� r� r� r� r; r� r� r{ r� r� �etag�"rG r� z&MD5 hash {} checksum does not match {})r` r� r0 �rm �size�hashrx r� r{ r0 z&Unexpected status code, status_code=%sr6 )�getrK rL ro r� r� rS r� r; �replace�upperr rT r% r r� r r )r' r{ r� rZ r� rx r� r� r� r; r� r� r� r_ r` r� Z result_dictr� r� � server_hashr� r) r) r* r� � sj � �� � �z OSSStorageDriver._put_objectc C s� | � ||�}|j�� |_|�� }|�td| jd��j} z| j ||| ||d�} | \} } } | j || | |d�}W n t yK } z | j || |d� |�d}~ww ||j d< d| | fS )a� Callback invoked for uploading data to OSS using Aliyun's multipart upload mechanism :param response: Response object from the initial POST request :type response: :class:`OSSRawResponse` :param data: Any data from the initial POST request :type data: ``str`` :param iterator: The generator for fetching the upload data :type iterator: ``generator`` :param container: The container owning the object to which data is being uploaded :type container: :class:`Container` :param object_name: The name of the object to which we are uploading :type object_name: ``str`` :keyword calculate_hash: Indicates if we must calculate the data hash :type calculate_hash: ``bool`` :return: A tuple of (status, checksum, bytes transferred) :rtype: ``tuple`` r� r� r� Nr� T)r� r� �readr/ r5 �findr rA r� �_upload_from_iterator�_commit_multipartr3 r� r; )r' r� rs r� r{ r� �calculate_hashr� r/ r� �result�chunksr� r� r� �er) r) r* �_upload_multipart� s" � �� z"OSSStorageDriver._upload_multipartc C s d}|r| � � }d}d}g } d|i} t|tddd�D ]Z} |t| �7 }|r)|�| � | � � } | �| � t�| �� ��d�} d| i} || d < d � |t | �f�}| j j |d | | |d �}|j tjkrdtd | d��|jd }| �||f� |d7 }q|r{|�� }| ||fS )a� Uploads data from an iterator in fixed sized chunks to OSS :param iterator: The generator for fetching the upload data :type iterator: ``generator`` :param object_path: The path of the object to which we are uploading :type object_name: ``str`` :param upload_id: The upload id allocated for this multipart upload :type upload_id: ``str`` :keyword calculate_hash: Indicates if we must calculate the data hash :type calculate_hash: ``bool`` :keyword container: the container object to upload object to :type container: :class:`Container` :return: A tuple of (chunk info, checksum, bytes transferred) :rtype: ``tuple`` Nr � �uploadIdT)r� Z fill_sizeZ yield_emptyr, z Content-MD5Z partNumberr� r� )rZ rs r; r{ zError uploading chunkr6 r� )r� r r r1 �updaterU rV rY �decoderS r r4 rz r% r r� r r; rR � hexdigest)r' r� r� r� r� r{ r� r� �countr� r[ rs Z chunk_hashr; r� �respr� r) r) r* r� sD � � z&OSSStorageDriver._upload_from_iteratorc C s� t d�}|D ] \}}t|d�}t|d�} t|�| _t|d�} t|�| _qt|�} d|i} d�|t| �f�} | jj| | d|d�}|j t j kr\|j }|j |d �\}}d �||�}t|| d ��|�� }|�td| jd ��j}|S ) a� Makes a final commit of the data. :param object_path: Server side object path. :type object_path: ``str`` :param upload_id: ID of the multipart upload. :type upload_id: ``str`` :param upload_id: A list of (chunk_number, chunk_hash) tuples. :type upload_id: ``list`` :keyword container: The container owning the object to which data is being uploaded :type container: :class:`Container` ZCompleteMultipartUploadZPartZ PartNumber�ETagr� r� �POSTr� )�elementz"Error in multipart commit: {} ({})r6 r� )r r rP r� r rS r r4 rz r% r r� r� Z_parse_error_detailsrT r r5 r� r rA )r' r� r� r� r{ r� r� r� �partZpart_noZetag_idrs r[ r� r� r �code�message�msgr/ r� r) r) r* r� b s* � z"OSSStorageDriver._commit_multipartc C sN d|i}d� |t|�f�}| jj|d|d�}|jtjkr%td|j | d��dS )a� Aborts an already initiated multipart upload :param object_path: Server side object path. :type object_path: ``str`` :param upload_id: ID of the multipart upload. :type upload_id: ``str`` :keyword container: The container owning the object to which data is being uploaded :type container: :class:`Container` r� r� r� r� z(Error in multipart abort. status_code=%dr6 N)rS r r4 rz r% r r� r )r' r� r� r{ r[ r� r� r) r) r* r� � s ��z!OSSStorageDriver._abort_multipartc c s, � |� t|| jd��D ]}| �|�V q d S r� )r� r rA � _to_container)r' r� r� r r) r) r* r� � s ��z OSSStorageDriver._to_containersc C s@ t |d| jd�t |d| jd�d�}tt |d| jd�|| d�}|S )NZ CreationDate�r r� rA ZLocation)Z creation_daterr �Namer� )r rA r )r' r rx r{ r) r) r* r � s ���z OSSStorageDriver._to_containerc s$ � �fdd�|� t|�jd��D �S )Nc s g | ]}�� |� ��qS r) )�_to_obj)�.0r �r{ r' r) r* � <listcomp>� s ��z-OSSStorageDriver._to_objs.<locals>.<listcomp>r� )r� r rA )r' r� r� r{ r) r r* r� � s �zOSSStorageDriver._to_objsc C s� t |d| jd�}t |d| jd�}d|| �|�d�i}t |d| jd�}d|i}| �t |d| jd��}t|tt |d | jd��t |d | jd��d d �|||| d �} | S )NzOwner/IDr zOwner/DisplayName�owner)r� Z display_nameZ LastModified� last_modifiedr� ZSizer� r� rG r� )r rA � _safe_decoder r$ r� ) r' r r{ Zowner_idZowner_display_namer� r rx rm r� r) r) r* r � s, ���� zOSSStorageDriver._to_objc C s< z|rt �|�\}}|�d�W S |W S ty | Y S w )ze Decode it as an escaped string and then treat the content as UTF-8 encoded. r, )�codecs� escape_decoder� r3 )r' �encodedZ unescapedZ_ignr) r) r* r � s �z OSSStorageDriver._safe_decodec C s d|j S )z� Return a container path :param container: Container instance :type container: :class:`Container` :return: A path for this container. :rtype: ``str`` �/%s)rm )r' r{ r) r) r* �_get_container_path� s z$OSSStorageDriver._get_container_pathc C s | � |�}d| }|S )ax Return an object's path. Aliyun OSS api puts the container name in the host, so ignore container here. :param container: Container instance :type container: :class:`Container` :param object_name: Object name :type object_name: :class:`str` :return: A path for this object. :rtype: ``str`` r )r� )r' r{ r� Zobject_name_cleanedr� r) r) r* r� � s z!OSSStorageDriver._get_object_pathc C s� |d � dd�}|d |d d�}i }d|v r |d |d< |�� D ] \}}|�� �| jd �s0q!|� | jd d�}|||< q!t|t|d �||||| d �} | S ) Nr� r� rG rH )r� r� z last-modifiedr zmeta-zcontent-lengthr� )r� rL rM rO ro r r$ ) r' r� r{ r; r� rx r� r_ r` r� r) r) r* r� s( � z#OSSStorageDriver._headers_to_object)NNr� )FT)NTN)r� NNNFFN)r� NNNNFN)T)TN))r> r? r@ rm ZwebsiterF Z connectionClsZ hash_typeZsupports_chunked_encodingr� rA ro r� r� r� r� r� r� r� r� r� r� r� r! r� r� r� r� r� r� r� r� r� r� r r� r r r r� r� r) r) r) r* r sx / ' � � � O �B �Q �: � L 0 r )5r� rW re rU r �hashlibr Zlibcloud.utils.py3r r r r r r Zlibcloud.utils.xmlr r Zlibcloud.common.baser r r Zlibcloud.utils.filesr Zlibcloud.common.typesr r r Zlibcloud.storage.baser r r Zlibcloud.storage.typesr r r r r r Z lxml.etreer r � ImportErrorZxml.etree.ElementTree�__all__ri r r r! r" rE rF r r r) r) r) r* �<module> s: �+
| ver. 1.4 |
Github
|
.
| PHP 8.2.30 | Generation time: 0 |
proxy
|
phpinfo
|
Settings