File manager - Edit - /opt/saltstack/salt/lib/python3.10/site-packages/libcloud/storage/drivers/__pycache__/local.cpython-310.pyc
Back
o ;j�Y � @ s d Z ddlZddlZddlZddlZddlZddlZddlmZ ddl m Z m Z ddl m Z ddlmZmZ ddlmZ ddlmZmZmZ dd lmZmZmZmZm Z m Z zddl Z W n e yg e d ��w d d gZ G d d� d�Z!G dd� d�Z"G dd� de�Z#dS )z; Provides storage driver for working with local filesystem � N)�sha256)�u�relpath)� Connection)�read_in_chunks�exhaust_iterator)� LibcloudError)�Object� Container� StorageDriver)� ObjectError�ObjectDoesNotExistError�ContainerIsNotEmptyError�InvalidContainerNameError�ContainerDoesNotExistError�ContainerAlreadyExistsErrorzQMissing fasteners dependency, you can install it using pip: pip install fastenersz.lockz.hashc @ s& e Zd Zd dd�Zdd� Zdd� ZdS ) �NoOpLockLocalStorage� c C s || _ || _d S �N)�path�lock_acquire_timeout��selfr �timeout� r �R/opt/saltstack/salt/lib/python3.10/site-packages/libcloud/storage/drivers/local.py�__init__6 s z NoOpLockLocalStorage.__init__c C s dS )NTr )r r r r � __enter__: � z NoOpLockLocalStorage.__enter__c C s |S r r �r �type�value� tracebackr r r �__exit__= r z NoOpLockLocalStorage.__exit__N�r )�__name__� __module__� __qualname__r r r# r r r r r 5 s r c @ s* e Zd ZdZd dd�Zdd� Zdd� Zd S ) �LockLocalStoragez� A class which locks a local path which is being updated. To correctly handle all the scenarios use a thread based and IPC based lock. r c C sN || _ || _tj �t�� dt|�d���� �| _ t � � | _ t �| j �| _d S )Nz%s.lockzutf-8)r r �os�join�tempfileZ gettempdirr �encode� hexdigest� ipc_lock_path� threading�Lock� thread_lock� fastenersZInterProcessLock�ipc_lockr r r r r G s � zLockLocalStorage.__init__c C s� | j }tt�� �}|| }tt�� �|k r'| jjdd�}|r ntt�� �|k s|s2td| j|f ��| jjd|d�}|sGtd| j| j|f ��d S )NF)�blockingz7Failed to acquire thread lock for path %s in %s secondsT)r4 r z9Failed to acquire IPC lock (%s) for path %s in %s seconds) r �int�timer1 �acquirer r r3 r. )r r Z start_timeZend_time�successr r r r V s, ��� ���zLockLocalStorage.__enter__c C s8 | j �� r | j �� | j�� r| j�� |d ur|�d S r )r1 �locked�releaser3 �existsr r r r r# q s �zLockLocalStorage.__exit__Nr$ )r% r& r' �__doc__r r r# r r r r r( A s r( c s e Zd ZdZeZdZdZdZ d8� fdd� Z d9d d �Z d d � Z d d� Z dd� Z dd� Zdd� Zd:dd�Zdd� Zd;dd�Zd d � Zd d � Zd d!� Zd"d#� Z d<d$d%�Zd=d&d'�Z d>d(d)�Zd:d*d+�Z d?d,d-�Zd:d.d/�Z d0d1� Z d2d3� Z d4d5� Z d;d6d7�Z � Z!S )@�LocalStorageDriverz� Implementation of local file-system based storage. This is helpful where the user would want to use the same code (using libcloud) and switch between cloud storage and local storage z Local Storagezhttp://example.com�md5NTc sX || _ tj�| j �std��|| _| jrt| _nt| _t � j d|||||d�|�� dS )aZ :param ex_use_locking: True if locking should be used when working with files. This value should almost always be left as-is and only changed to False when there is a specific need for it (e.g. for specific tests and micro benchmarks). z The base path is not a directory)�key�secret�secure�host�portNr ) � base_pathr) r �isdirr Z_ex_use_lockingr( � _lock_clsr �superr )r r? r@ rA rB rC Zex_use_locking�kwargs�� __class__r r r � s "zLocalStorageDriver.__init__c C sZ zt �|� W dS ty, } z|jtjkr|s!|�W Y d}~dS W Y d}~dS d}~ww )z@ Create a path by checking if it already exists N)r) �makedirs�OSError�errno�EEXIST)r r �ignore_existing�expr r r � _make_path� s ���z LocalStorageDriver._make_pathc C s" d|v sd|v rt d| |d��dS )z� Check if the container name is valid :param container_name: Container name :type container_name: ``str`` �/�\N�r! �driver�container_name)r �r rV r r r �_check_container_name� s �z(LocalStorageDriver._check_container_namec C s� | � |� tj�| j|�}zt�|�}tj�|�s td��W n ty, td| |d��w i }|j |d< |j |d< |j |d< t ||| d�S )z� Create a container instance :param container_name: Container name. :type container_name: ``str`` :return: Container instance. :rtype: :class:`Container` z Target path is not a directoryNrT � creation_time� access_time� modify_time)�name�extrarU ) rX r) r r* rD �statrE rL r �st_ctime�st_atime�st_mtimer )r rV � full_pathr^ r] r r r �_make_container� s � � z"LocalStorageDriver._make_containerc C s� t j�| j|j|�}t j�|�rtd| |d��zt �|�}W n ty, t d| |d��w | � � }|� t |j ��d�� |�� }i }|j|d< |j|d< |j |d< t||j|| ||dd�S )a Create an object instance :param container: Container. :type container: :class:`Container` :param object_name: Object name. :type object_name: ``str`` :return: Object instance. :rtype: :class:`Object` N)r! rU � object_name�asciirY rZ r[ )r\ �sizer] rU � container�hashZ meta_data)r) r r* rD r\ rE r r^ � Exceptionr Z_get_hash_function�updater ra r, r- r_ r` r �st_size)r rg rd rb r^ Z data_hashr] r r r � _make_object� s0 � �z LocalStorageDriver._make_objectc c sB � t �| j�D ]}t j�| j|�}t j�|�sq| �|�V qdS )z� Return a generator of containers. :return: A generator of Container instances. :rtype: ``generator`` of :class:`Container` N)r) �listdirrD r r* rE rc )r rV rb r r r �iterate_containers s � �z%LocalStorageDriver.iterate_containersc c s| � | j |dd�}tj|dd�D ],\}}}tD ] }||v r!|�|� q|D ]}tj�||�}t||d�} | �|| �V q$qdS )zY Recursively iterate through the file-system and return the object names T��check)�topdown)�startN) �get_container_cdn_urlr) �walk�IGNORE_FOLDERS�remover r* r rl ) r rg ZcpathZfolderZ subfolders�filesZsubfr\ rb rd r r r � _get_objects s � � ��z LocalStorageDriver._get_objectsc C s2 | � ||�}| �|�}t|dd� d�}| �||�S )a� Returns a generator of objects for the given container. :param container: Container instance :type container: :class:`Container` :param prefix: Filter objects starting with a prefix. :type prefix: ``str`` :param ex_prefix: (Deprecated.) Filter objects starting with a prefix. :type ex_prefix: ``str`` :return: A generator of Object instances. :rtype: ``generator`` of :class:`Object` c S s | j S r )r\ )�or r r �<lambda>< s z>LocalStorageDriver.iterate_container_objects.<locals>.<lambda>)r? )Z_normalize_prefix_argumentrx �sortedZ _filter_listed_container_objects)r rg �prefixZ ex_prefixZobjectsr r r �iterate_container_objects) s z,LocalStorageDriver.iterate_container_objectsc C s | � |�S )z� Return a container instance. :param container_name: Container name. :type container_name: ``str`` :return: :class:`Container` instance. :rtype: :class:`Container` )rc rW r r r � get_container@ s z LocalStorageDriver.get_containerFc C s6 t j�| j|j�}|rt j�|�std| |jd��|S )a7 Return a container CDN URL. :param container: Container instance :type container: :class:`Container` :param check: Indicates if the path's existence must be checked :type check: ``bool`` :return: A CDN URL for this container. :rtype: ``str`` NrT )r) r r* rD r\ rE r )r rg rp r r r r rs L s z(LocalStorageDriver.get_container_cdn_urlc C s | � |�}| �||�S )a Return an object instance. :param container_name: Container name. :type container_name: ``str`` :param object_name: Object name. :type object_name: ``str`` :return: :class:`Object` instance. :rtype: :class:`Object` )rc rl )r rV rd rg r r r � get_object` s z LocalStorageDriver.get_objectc C s t j�| j|jj|j�S )z� Return an object CDN URL. :param obj: Object instance :type obj: :class:`Object` :return: A CDN URL for this object. :rtype: ``str`` )r) r r* rD rg r\ )r �objr r r �get_object_cdn_urlp s z%LocalStorageDriver.get_object_cdn_urlc C sD | � |�}| �|�� | �|� W d � dS 1 sw Y dS )z� Enable container CDN. :param container: Container instance :type container: :class:`Container` :rtype: ``bool`` NT)rs rF rQ )r rg r r r r �enable_container_cdn| s ��z'LocalStorageDriver.enable_container_cdnc C s� | � |�}| �|��4 tj�|�r W d � dS z t|d�}|�� W n ty5 Y W d � dS w W d � dS 1 sAw Y dS )z� Enable object CDN. :param obj: Object instance :type obj: :class:`Object` :rtype: ``bool`` NF�wT)r� rF r) r r; �open�closeri )r r� r �obj_filer r r �enable_object_cdn� s" � �� � � z$LocalStorageDriver.enable_object_cdnc C sp | � |�}| j|||d�}z t�||� W dS ty7 |r4z t�|� W Y dS ty3 Y Y dS w Y dS w )a Download an object to the specified destination path. :param obj: Object instance. :type obj: :class:`Object` :param destination_path: Full path to a file or a directory where the incoming file will be saved. :type destination_path: ``str`` :param overwrite_existing: True to overwrite an existing file, defaults to False. :type overwrite_existing: ``bool`` :param delete_on_failure: True to delete a partially downloaded file if the download was not successful (hash mismatch / file size). :type delete_on_failure: ``bool`` :return: True if an object has been successfully downloaded, False otherwise. :rtype: ``bool`` �r� �destination_path�overwrite_existingFT)r� �_get_obj_file_path�shutil�copyrL r) �unlinkri )r r� r� r� �delete_on_failure�obj_path� file_pathr r r �download_object� s( � � ���z"LocalStorageDriver.download_objectc c sN � | � |�}t|d��}t||d�E dH W d � dS 1 s w Y dS )a4 Return a generator which yields object data. :param obj: Object instance :type obj: :class:`Object` :param chunk_size: Optional chunk size (in bytes). :type chunk_size: ``int`` :return: A stream of binary chunks of data. :rtype: ``object`` �rb)� chunk_sizeN)r� r� r )r r� r� r r� r r r �download_object_as_stream� s � "�z,LocalStorageDriver.download_object_as_streamc C sl | j ||d� | j|||d�}| j|||d�}t|d��} | �t|�� W d � dS 1 s/w Y dS )N�� start_bytes� end_bytesr� )r� r� r� �wbT)� _validate_start_and_end_bytesr� � download_object_range_as_streamr� �writer ) r r� r� r� r� r� r� r� �iterator�fpr r r �download_object_range� s �� ��z(LocalStorageDriver.download_object_rangec c s� � | j ||d� | �|�}t|d��5}t|�� �}|r#||kr#td��|d u r.|| d }n|| }|�|� |�|�} | V W d � d S 1 sJw Y d S )Nr� r� z"end_bytes is larger than file size� )r� r� r� �len�read� ValueError�seek) r r� r� r� r� r r� � file_size� read_bytes�datar r r r� � s � "�z2LocalStorageDriver.download_object_range_as_streamc C s� | j |dd�}tj�||�}tj�|�} | �| � | �|�� t�||� W d � n1 s/w Y t� |t dd�� | � ||�S )an Upload an object currently located on a disk. :param file_path: Path to the object on disk. :type file_path: ``str`` :param container: Destination container. :type container: :class:`Container` :param object_name: Object name. :type object_name: ``str`` :param verify_hash: Verify hast :type verify_hash: ``bool`` :param extra: (optional) Extra attributes (driver specific). :type extra: ``dict`` :param headers: (optional) Headers (driver specific). :type headers: ``dict`` :rtype: ``object`` Tro N�664� ) rs r) r r* �dirnamerQ rF r� r� �chmodr5 rl ) r r� rg rd r] Z verify_hash�headersr r� rD r r r � upload_object s ! � z LocalStorageDriver.upload_objectc C s� | j |dd�}tj�||�}tj�|�}| �|� | �|��' t|d��} |D ]} | �| � q'W d � n1 s9w Y W d � n1 sHw Y t� |t dd�� | � ||�S )a� Upload an object using an iterator. If a provider supports it, chunked transfer encoding is used and you don't need to know in advance the amount of data to be uploaded. Otherwise if a provider doesn't support it, iterator will be exhausted so a total size for data to be uploaded can be determined. Note: Exhausting the iterator means that the whole data must be buffered in memory which might result in memory exhausting when uploading a very large object. If a file is located on a disk you are advised to use upload_object function which uses fs.stat function to determine the file size and it doesn't need to buffer whole object in the memory. :type iterator: ``object`` :param iterator: An object which implements the iterator interface and yields binary chunks of data. :type container: :class:`Container` :param container: Destination container. :type object_name: ``str`` :param object_name: Object name. :type extra: ``dict`` :param extra: (optional) Extra attributes (driver specific). Note: This dictionary must contain a 'content_type' key which represents a content type of the stored object. :param headers: (optional) Headers (driver specific). :type headers: ``dict`` :rtype: ``object`` Tro r� Nr� r� ) rs r) r r* r� rQ rF r� r� r� r5 rl ) r r� rg rd r] r� r r� rD r� r� r r r �upload_object_via_stream< s & � �� z+LocalStorageDriver.upload_object_via_streamc C s� | � |�}| �|�� zt�|� W n ty" Y W d � dS w W d � n1 s-w Y tj�|�}|j�� }||krnzt� |� W n t yc } z|j t j kr]W Y d}~dS |�d}~ww tj�|�}||ksAdS )z� Delete an object. :type obj: :class:`Object` :param obj: Object instance. :return: ``bool`` True on success. :rtype: ``bool`` NFT) r� rF r) r� ri r r� rg Z get_cdn_url�rmdirrL rM Z ENOTEMPTY)r r� r Z container_urlrP r r r � delete_objectl s2 �� � ��� � z LocalStorageDriver.delete_objectc C s� | � |� tj�| j|�}z | j|dd� W n/ ty7 } z|jtjkr+t d|| d��t d| | d��d}~w t yE t d| | d��w | � |�S )z� Create a new container. :type container_name: ``str`` :param container_name: Container name. :return: :class:`Container` instance on success. :rtype: :class:`Container` F)rO zgContainer with this name already exists. The name must be unique among all the containers in the system�r! rV rU zError creating container %s)rU N) rX r) r r* rD rQ rL rM rN r r ri rc )r rV r rP r r r �create_container� s" �� � z#LocalStorageDriver.create_containerc C s� | � |�D ] }td|j| d��| j|dd�}| �|��! zt�|� W n ty3 Y W d � dS w W d � dS 1 s?w Y dS )z� Delete a container. :type container: :class:`Container` :param container: Container instance :return: True on success, False otherwise. :rtype: ``bool`` zContainer is not emptyr� Tro NF)rx r r\ rs rF r� �rmtreeri )r rg r� r r r r �delete_container� s& � �� ��z#LocalStorageDriver.delete_containerc C sn t j�|�}|st j�|�std| | d��|s!t j�||j�}n|}t j�|�r5|s5td| d | d��|S )NzPath %s does not exist)r! rU z File %s already exists, but zoverwrite_existing=False)r) r �basenamer; r r* r\ )r r� r� r� � base_namer� r r r r� � s �z%LocalStorageDriver._get_obj_file_path)NTNNT)T)NN)F)FTr )NFT)NTN)"r% r& r' r< r Z connectionClsr\ ZwebsiteZ hash_typer rQ rX rc rl rn rx r} r~ rs r r� r� r� r� r� r� r� r� r� r� r� r� r� � __classcell__r r rI r r= | sT � . � - � � .0$! r= )$r< r) r6 rM r� r+ r/ �hashlibr Zlibcloud.utils.py3r r Zlibcloud.common.baser Zlibcloud.utils.filesr r Zlibcloud.common.typesr Zlibcloud.storage.baser r r Zlibcloud.storage.typesr r r r r r r2 � ImportErrorru r r( r= r r r r �<module> s2 �� ;
| ver. 1.4 |
Github
|
.
| PHP 8.2.30 | Generation time: 0 |
proxy
|
phpinfo
|
Settings