File manager - Edit - /opt/saltstack/salt/lib/python3.10/site-packages/libcloud/storage/__pycache__/base.cpython-310.pyc
Back
o ;j� � @ s� d Z ddlZddlZddlZddlZddlmZmZm Z m Z m Z ddlm Z ddlZddlmZmZ ddlmZmZmZ ddlmZ ddlmZ g d �Zd Z d Z G d d � d �Z G dd� d�Z G dd� de�Z dS )z0 Provides base classes for working with storage � N)�Dict�List�Type�Iterator�Optional)�join)�b�httplib)� BaseDriver� Connection�ConnectionUserAndKey)� LibcloudError)�ObjectDoesNotExistError)�Object� Container� StorageDriver� CHUNK_SIZE�DEFAULT_CONTENT_TYPEi� zapplication/octet-streamc @ sf e Zd ZdZdd� Zdd� Zdd� Zdd d �Zdd d�Z ddd�Z ddd�Z dd� Z dd� Z d S )r z& Represents an object (BLOB). c C s6 || _ || _|| _|| _|pi | _|pi | _|| _dS )au :param name: Object name (must be unique per container). :type name: ``str`` :param size: Object size in bytes. :type size: ``int`` :param hash: Object hash. :type hash: ``str`` :param container: Object container. :type container: :class:`libcloud.storage.base.Container` :param extra: Extra attributes. :type extra: ``dict`` :param meta_data: Optional object meta data. :type meta_data: ``dict`` :param driver: StorageDriver instance. :type driver: :class:`libcloud.storage.base.StorageDriver` N)�name�size�hash� container�extra� meta_data�driver)�selfr r r r r r r � r �I/opt/saltstack/salt/lib/python3.10/site-packages/libcloud/storage/base.py�__init__1 s ! zObject.__init__c C � | j j| d�S �N)�obj)r �get_object_cdn_url�r r r r � get_cdn_urlZ � zObject.get_cdn_urlc C r r )r �enable_object_cdnr# r r r � enable_cdn^ r% zObject.enable_cdnFTc C � | j j| |||d�S )N)r! �destination_path�overwrite_existing�delete_on_failure�r �download_object)r r) r* r+ r r r �downloadb � �zObject.downloadNc C s | j j| |d�S )N)r! � chunk_size�r �download_object_as_stream)r r0 r r r � as_streamk s zObject.as_streamc C s | j j| |||||d�S �N)r! r) � start_bytes� end_bytesr* r+ �r �download_object_range)r r) r5 r6 r* r+ r r r �download_rangeo s �zObject.download_rangec C r( �N)r! r5 r6 r0 �r � download_object_range_as_stream)r r5 r6 r0 r r r �range_as_stream� r/ zObject.range_as_streamc C � | j �| �S �N�r � delete_objectr# r r r �delete� � z Object.deletec C s d� | j| j| j| jj�S )Nz4<Object: name={}, size={}, hash={}, provider={} ...>)�formatr r r r r# r r r �__repr__� s �zObject.__repr__�FTr? �NFT�NN) �__name__� __module__� __qualname__�__doc__r r$ r' r. r3 r9 r= rB rE r r r r r , s ) � r c @ s� e Zd ZdZdd� Zd#dd�Zd#dd�Zd d � Zd d � Zd d� Z d$dd�Z d#dd�Z d%dd�Z d&dd�Z d'dd�Zd#dd �Zd d � Zd d � Zd!d"� ZdS )(r zJ Represents a container (bucket) which can hold multiple objects. c C s || _ |pi | _|| _dS )a :param name: Container name (must be unique). :type name: ``str`` :param extra: Extra attributes. :type extra: ``dict`` :param driver: StorageDriver instance. :type driver: :class:`libcloud.storage.base.StorageDriver` N)r r r )r r r r r r r r � s zContainer.__init__Nc C � | j j| ||d�S �N)r �prefix� ex_prefix)r �iterate_container_objects�r rO rP r r r �iterate_objects� � �zContainer.iterate_objectsc C rM rN )r �list_container_objectsrR r r r � list_objects� rT zContainer.list_objectsc C r �N)r )r �get_container_cdn_urlr# r r r r$ � r% zContainer.get_cdn_urlc C r rW )r �enable_container_cdnr# r r r r' � r% zContainer.enable_cdnc C s | j j| j|d�S )N)�container_name� object_name)r � get_objectr )r r[ r r r r\ � s zContainer.get_objectTc C s | j j|| ||||d�S )N)r � verify_hash�headers)r � upload_object)r � file_pathr[ r r] r^ r r r r_ � s �zContainer.upload_objectc C s | j j|| |||d�S )N)r r^ )r �upload_object_via_stream)r �iteratorr[ r r^ r r r ra � s �z"Container.upload_object_via_streamFc C � | j j||||d�S )N)r* r+ r, �r r! r) r* r+ r r r r- � s �zContainer.download_objectc C s | j �||�S r? r1 �r r! r0 r r r r2 � r% z#Container.download_object_as_streamc C s | j j||||||d�S r4 r7 �r r! r) r5 r6 r* r+ r r r r8 � s �z Container.download_object_rangec C rc r: r; �r r! r5 r6 r0 r r r r< � s �z)Container.download_object_range_as_streamc C s | j �|�S r? r@ �r r! r r r rA rC zContainer.delete_objectc C r> r? )r �delete_containerr# r r r rB rC zContainer.deletec C s d� | j| jj�S )Nz!<Container: name={}, provider={}>)rD r r r# r r r rE s zContainer.__repr__rH �NTNrF r? rG )rI rJ rK rL r rS rV r$ r' r\ r_ ra r- r2 r8 r< rA rB rE r r r r r � s* � � r c @ sL e Zd ZdZeZdZdZdZdZ dd� Z dd� Z dBd d �Z dBd d �Z d d� Zdd� Zdd� Zdd� Zdd� Zdd� Zdd� Zdd � Z dCd d �ZdDd d!�Z dEd"d#�ZdBd$d%�Z dFd&d'�ZdBd(d)�Zd*d+� Z d,d-� Z d.d/� Z dDd0d1�Z dGd2d3�Z 4 dHd5d6�Z!dDd7d8�Z"dId:d;�Z#d<d=� Z$dDd>d?�Z%dJd@dA�Z&dS )Kr z. A base StorageDriver to derive from. N�md5Fc C � t d��)z� Return a iterator of containers for the given account :return: A iterator of Container instances. :rtype: ``iterator`` of :class:`libcloud.storage.base.Container` z2iterate_containers not implemented for this driver��NotImplementedErrorr# r r r �iterate_containers s z StorageDriver.iterate_containersc C s t | �� �S )z� Return a list of containers. :return: A list of Container instances. :rtype: ``list`` of :class:`Container` )�listro r# r r r �list_containers& s z StorageDriver.list_containersc C rl )a� Return a iterator of objects for the given container. :param container: Container instance :type container: :class:`libcloud.storage.base.Container` :param prefix: Filter objects starting with a prefix. :type prefix: ``str`` :param ex_prefix: (Deprecated.) Filter objects starting with a prefix. :type ex_prefix: ``str`` :return: A iterator of Object instances. :rtype: ``iterator`` of :class:`libcloud.storage.base.Object` z9iterate_container_objects not implemented for this driverrm �r r rO rP r r r rQ 0 s z'StorageDriver.iterate_container_objectsc C s t | j|||d��S )a� Return a list of objects for the given container. :param container: Container instance. :type container: :class:`libcloud.storage.base.Container` :param prefix: Filter objects starting with a prefix. :type prefix: ``str`` :param ex_prefix: (Deprecated.) Filter objects starting with a prefix. :type ex_prefix: ``str`` :return: A list of Object instances. :rtype: ``list`` of :class:`libcloud.storage.base.Object` )rO rP )rp rQ rr r r r rU C s z$StorageDriver.list_container_objectsc C s |r t �dt� |S |S )NzOThe ``ex_prefix`` argument is deprecated - please update code to use ``prefix``)�warnings�warn�DeprecationWarningrR r r r �_normalize_prefix_argumentV s �z(StorageDriver._normalize_prefix_argumentc c sD � |d urt �d| jj � |D ]}|d u s |j�|�r |V qd S )NzgDriver %s does not implement native object filtering; falling back to filtering the full object stream.)rs rt � __class__rI r � startswith)r ZobjectsrO r! r r r � _filter_listed_container_objectsa s �����z.StorageDriver._filter_listed_container_objectsc C rl )z� Return a container instance. :param container_name: Container name. :type container_name: ``str`` :return: :class:`Container` instance. :rtype: :class:`libcloud.storage.base.Container` �*get_object not implemented for this driverrm �r rZ r r r � get_containerm � zStorageDriver.get_containerc C rl )z� Return a container CDN URL. :param container: Container instance :type container: :class:`libcloud.storage.base.Container` :return: A CDN URL for this container. :rtype: ``str`` z5get_container_cdn_url not implemented for this driverrm �r r r r r rX z r} z#StorageDriver.get_container_cdn_urlc C rl )a3 Return an object instance. :param container_name: Container name. :type container_name: ``str`` :param object_name: Object name. :type object_name: ``str`` :return: :class:`Object` instance. :rtype: :class:`libcloud.storage.base.Object` rz rm )r rZ r[ r r r r\ � s zStorageDriver.get_objectc C rl )z� Return an object CDN URL. :param obj: Object instance :type obj: :class:`libcloud.storage.base.Object` :return: A CDN URL for this object. :rtype: ``str`` z2get_object_cdn_url not implemented for this driverrm rh r r r r" � r} z StorageDriver.get_object_cdn_urlc C rl )z� Enable container CDN. :param container: Container instance :type container: :class:`libcloud.storage.base.Container` :rtype: ``bool`` z4enable_container_cdn not implemented for this driverrm r~ r r r rY � � z"StorageDriver.enable_container_cdnc C rl )z� Enable object CDN. :param obj: Object instance :type obj: :class:`libcloud.storage.base.Object` :rtype: ``bool`` z1enable_object_cdn not implemented for this driverrm rh r r r r& � r z StorageDriver.enable_object_cdnTc C rl )a� Download an object to the specified destination path. :param obj: Object instance. :type obj: :class:`libcloud.storage.base.Object` :param destination_path: Full path to a file or a directory where the incoming file will be saved. :type destination_path: ``str`` :param overwrite_existing: True to overwrite an existing file, defaults to False. :type overwrite_existing: ``bool`` :param delete_on_failure: True to delete a partially downloaded file if the download was not successful (hash mismatch / file size). :type delete_on_failure: ``bool`` :return: True if an object has been successfully downloaded, False otherwise. :rtype: ``bool`` z/download_object not implemented for this driverrm rd r r r r- � s z StorageDriver.download_objectc C rl )a$ Return a iterator which yields object data. :param obj: Object instance :type obj: :class:`libcloud.storage.base.Object` :param chunk_size: Optional chunk size (in bytes). :type chunk_size: ``int`` :rtype: ``iterator`` of ``bytes`` z9download_object_as_stream not implemented for this driverrm re r r r r2 � s z'StorageDriver.download_object_as_streamc C rl )a8 Download part of an object. :param obj: Object instance. :type obj: :class:`libcloud.storage.base.Object` :param destination_path: Full path to a file or a directory where the incoming file will be saved. :type destination_path: ``str`` :param start_bytes: Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is "0". :type start_bytes: ``int`` :param end_bytes: End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file. :type end_bytes: ``int`` :param overwrite_existing: True to overwrite an existing file, defaults to False. :type overwrite_existing: ``bool`` :param delete_on_failure: True to delete a partially downloaded file if the download was not successful (hash mismatch / file size). :type delete_on_failure: ``bool`` :return: True if an object has been successfully downloaded, False otherwise. :rtype: ``bool`` z5download_object_range not implemented for this driverrm rf r r r r8 � s ,z#StorageDriver.download_object_rangec C rl )a Return a iterator which yields range / part of the object data. :param obj: Object instance :type obj: :class:`libcloud.storage.base.Object` :param start_bytes: Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is "0". :type start_bytes: ``int`` :param end_bytes: End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file. :type end_bytes: ``int`` :param chunk_size: Optional chunk size (in bytes). :type chunk_size: ``int`` :rtype: ``iterator`` of ``bytes`` z?download_object_range_as_stream not implemented for this driverrm rg r r r r< s z-StorageDriver.download_object_range_as_streamc C rl )a Upload an object currently located on a disk. :param file_path: Path to the object on disk. :type file_path: ``str`` :param container: Destination container. :type container: :class:`libcloud.storage.base.Container` :param object_name: Object name. :type object_name: ``str`` :param verify_hash: Verify hash :type verify_hash: ``bool`` :param extra: Extra attributes (driver specific). (optional) :type extra: ``dict`` :param headers: (optional) Additional request headers, such as CORS headers. For example: headers = {'Access-Control-Allow-Origin': 'http://mozilla.com'} :type headers: ``dict`` :rtype: :class:`libcloud.storage.base.Object` z-upload_object not implemented for this driverrm )r r` r r[ r r] r^ r r r r_ / s #zStorageDriver.upload_objectc C rl )a Upload an object using an iterator. If a provider supports it, chunked transfer encoding is used and you don't need to know in advance the amount of data to be uploaded. Otherwise if a provider doesn't support it, iterator will be exhausted so a total size for data to be uploaded can be determined. Note: Exhausting the iterator means that the whole data must be buffered in memory which might result in memory exhausting when uploading a very large object. If a file is located on a disk you are advised to use upload_object function which uses fs.stat function to determine the file size and it doesn't need to buffer whole object in the memory. :param iterator: An object which implements the iterator interface. :type iterator: :class:`object` :param container: Destination container. :type container: :class:`libcloud.storage.base.Container` :param object_name: Object name. :type object_name: ``str`` :param extra: (optional) Extra attributes (driver specific). Note: This dictionary must contain a 'content_type' key which represents a content type of the stored object. :type extra: ``dict`` :param headers: (optional) Additional request headers, such as CORS headers. For example: headers = {'Access-Control-Allow-Origin': 'http://mozilla.com'} :type headers: ``dict`` :rtype: ``libcloud.storage.base.Object`` z8upload_object_via_stream not implemented for this driverrm )r rb r r[ r r^ r r r ra T s (z&StorageDriver.upload_object_via_streamc C rl )z� Delete an object. :param obj: Object instance. :type obj: :class:`libcloud.storage.base.Object` :return: ``bool`` True on success. :rtype: ``bool`` z-delete_object not implemented for this driverrm rh r r r rA ~ r} zStorageDriver.delete_objectc C rl )z� Create a new container. :param container_name: Container name. :type container_name: ``str`` :return: Container instance on success. :rtype: :class:`libcloud.storage.base.Container` z0create_container not implemented for this driverrm r{ r r r �create_container� r} z StorageDriver.create_containerc C rl )z� Delete a container. :param container: Container instance :type container: :class:`libcloud.storage.base.Container` :return: ``True`` on success, ``False`` otherwise. :rtype: ``bool`` z0delete_container not implemented for this driverrm r~ r r r ri � r} z StorageDriver.delete_containerc C sj |pt j}t|ttf�s|g}n|}|j|v r |di |��S |jt jkr,t|jd| d��t d|j | d��)a Call passed callback and start transfer of the object' :param obj: Object instance. :type obj: :class:`Object` :param callback: Function which is called with the passed callback_kwargs :type callback: :class:`function` :param callback_kwargs: Keyword arguments which are passed to the callback. :type callback_kwargs: ``dict`` :param response: Response instance. :type response: :class:`Response` :param success_status_code: Status code which represents a successful transfer (defaults to httplib.OK) :type success_status_code: ``int`` :return: ``True`` on success, ``False`` otherwise. :rtype: ``bool`` � )r[ �valuer zUnexpected status code: %s�r� r Nr ) r �OK� isinstancerp �tuple�status� NOT_FOUNDr r r )r r! �callbackZcallback_kwargs�responseZsuccess_status_codeZsuccess_status_codesr r r � _get_object� s zStorageDriver._get_objectc C s |pt }tj�|�}|stj�|�std| | d��|s#t||j�} n|} tj�| �r7|s7td| d | d��d} t| d�� } |j � |�D ]} | � t | �� | t | �7 } qEW d � n1 s_w Y |s�t|j�t| �kr�|r�zt�| � W dS ty� Y dS w dS d S ) a" Save object to the provided path. :param response: RawResponse instance. :type response: :class:`RawResponse` :param obj: Object instance. :type obj: :class:`Object` :param destination_path: Destination directory. :type destination_path: ``str`` :param delete_on_failure: True to delete partially downloaded object if the download fails. :type delete_on_failure: ``bool`` :param overwrite_existing: True to overwrite a local path if it already exists. :type overwrite_existing: ``bool`` :param chunk_size: Optional chunk size (defaults to ``libcloud.storage.base.CHUNK_SIZE``, 8kb) :type chunk_size: ``int`` :param partial_download: True if this is a range (partial) save, False otherwise. :type partial_download: ``bool`` :return: ``True`` on success, ``False`` otherwise. :rtype: ``bool`` zPath %s does not existr� z File %s already exists, but zoverwrite_existing=Falser �wbNFT)r �os�path�basename�existsr �pjoinr �openZ _responseZ iter_content�writer �len�intr �unlink� Exception) r r� r! r) r* r+ r0 Zpartial_download� base_namer` �bytes_transferredZ file_handle�chunkr r r � _save_object� s< * � � � ��zStorageDriver._save_object�PUTc C s |pi }|rt j�|�std| ��|dur$t|d�s$t|d�s$td��| j|||d�|d< |rF| jj||||dd �} | � || � � �\} } n?t |d ��} | jj||| |dd �} W d � n1 saw Y t |d ��} | � | | � � �\} } W d � n1 s�w Y | | | d �S ) z� Helper function for setting common request headers and calling the passed in callback which uploads an object. zFile %s does not existN�next�__next__z-iterator object must implement next() method.)r` z Content-TypeT)�method�datar^ �raw�rb)r� r� Z data_hash) r� r� r� �OSError�hasattr�AttributeError�_determine_content_type� connection�request�_hash_buffered_stream�_get_hash_functionr� )r r[ � content_typeZ request_pathZrequest_methodr^ r` �stream�chunked� multipartr� Z stream_hashZ stream_lengthZ file_streamr r r �_upload_object sJ �� � � � � ��z StorageDriver._upload_objectc C s@ |r|S |p|}t jj�|�\}}| jr |s td| ��|p tS )NzUFile content-type could not be guessed for "%s" and no content_type value is provided)�libcloud�utils�filesZguess_file_mime_type� strict_moder� r )r r� r[ r` r �_r r r r� X s ��z%StorageDriver._determine_content_type� c C sJ d}t |d�s t |d�rQt |d�r3z|�d� W n ty2 } z|jtjkr(|�W Y d }~nd }~ww tjjj|d�D ]}|� t |�� |t |�7 }q;|� � |fS t |d�sl|D ] }|� |� |t |� }qX|� � |fS |�) |� |�}t |�dkr�|t |� }|� |� |� |�}t |�dkszW d � n1 s�w Y |� � |fS )Nr r� r� �seek)rb �__exit__)r� r� r� �errnoZESPIPEr� r� r� Zread_in_chunks�updater r� � hexdigest�read) r r� Zhasher� blocksizeZ total_len�er� �s�bufr r r r� g s< ��� �� � z#StorageDriver._hash_buffered_streamc C s2 z t t| j�� }W |S ty td| j ��w )zh Return instantiated hash function for the hash type supported by the provider. z$Invalid or unsupported hash type: %s)�getattr�hashlib� hash_typer� � RuntimeError)r �funcr r r r� � s ��z StorageDriver._get_hash_functionc C s< |dk rt d��|dur ||krt d��||kr t d��dS )zo Method which validates that start_bytes and end_bytes arguments contain valid values. r z"start_bytes must be greater than 0Nz*start_bytes must be smaller than end_byteszGstart_bytes and end_bytes can't be the same. end_bytes is non-inclusiveT)� ValueError)r r5 r6 r r r � _validate_start_and_end_bytes� s �z+StorageDriver._validate_start_and_end_bytesc C s8 d| }|dur|r|t |�7 }|S |t |d �7 }|S )a" Return range string which is used as a Range header value for range requests for drivers which follow standard Range header notation This returns range string in the following format: bytes=<start_bytes>-<end bytes>. For example: bytes=1-10 bytes=0-2 bytes=5- bytes=100-5000 :param end_bytes_inclusive: True if "end_bytes" offset should be inclusive (aka opposite from the Python indexing behavior where the end index is not inclusive). z bytes=%s-N� )�str)r r5 r6 Zend_bytes_inclusiveZ range_strr r r �_get_standard_range_str� s �z%StorageDriver._get_standard_range_strrH rF r? rG rj )FTNF)r� NNNFF)r� )NF)'rI rJ rK rL r Z connectionClsr r� Zsupports_chunked_encodingr� ro rq rQ rU rv ry r| rX r\ r" rY r& r- r2 r8 r< r_ ra rA r� ri r� r� r� r� r� r� r� r� r r r r r sj � � . � %* , �V � ; + r )!rL r� r� Zos.pathr� rs �typingr r r r r r r� Zlibcloud.utils.filesr� Zlibcloud.utils.py3r r Zlibcloud.common.baser r r Zlibcloud.common.typesr Zlibcloud.storage.typesr �__all__r r r r r r r r r �<module> s$ kw
| ver. 1.4 |
Github
|
.
| PHP 8.2.30 | Generation time: 0 |
proxy
|
phpinfo
|
Settings