File manager - Edit - /opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/__pycache__/azurefs.cpython-310.pyc
Back
o ;j�4 � @ s d Z ddlZddlZddlZddlZddlZddlZddlZddl Zddl Zddl Zddl Zddl mZ zddlZeejj�ed�k rGed��dZW n eefyW dZY nw dZe�� Zd d � Zd d d �Zdd� Zdd� Z dd� Z dd� Z dd� Z dd� Z dd� Z!d d � Z"d d � Z#dS )!a� The backend for serving files from the Azure blob storage service. .. versionadded:: 2015.8.0 To enable, add ``azurefs`` to the :conf_master:`fileserver_backend` option in the Master config file. .. code-block:: yaml fileserver_backend: - azurefs Starting in Salt 2018.3.0, this fileserver requires the standalone Azure Storage SDK for Python. Theoretically any version >= v0.20.0 should work, but it was developed against the v0.33.0 version. Each storage container will be mapped to an environment. By default, containers will be mapped to the ``base`` environment. You can override this behavior with the ``saltenv`` configuration option. You can have an unlimited number of storage containers, and can have a storage container serve multiple environments, or have multiple storage containers mapped to the same environment. Normal first-found rules apply, and storage containers are searched in the order they are defined. You must have either an account_key or a sas_token defined for each container, if it is private. If you use a sas_token, it must have READ and LIST permissions. .. code-block:: yaml azurefs: - account_name: my_storage account_key: 'fNH9cRp0+qVIVYZ+5rnZAhHc9ycOUcJnHtzpfOr0W0sxrtL2KVLuMe1xDfLwmfed+JJInZaEdWVCPHD4d/oqeA==' container_name: my_container - account_name: my_storage sas_token: 'ss=b&sp=&sv=2015-07-08&sig=cohxXabx8FQdXsSEHyUXMjsSfNH2tZ2OB97Ou44pkRE%3D&srt=co&se=2017-04-18T21%3A38%3A01Z' container_name: my_dev_container saltenv: dev - account_name: my_storage container_name: my_public_container .. note:: Do not include the leading ? for sas_token if generated from the web � N)�Versionz0.20.0z+azure.storage.__version__ must be >= 0.20.0TF�azurefsc C s2 t td vrdS ts dS dtvrdS t� sdS dS )zQ Only load if defined in fileserver_backend and azure.storage is present Zfileserver_backendFr T)�__virtualname__�__opts__� HAS_AZURE�_validate_config� r r �K/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/azurefs.py� __virtual__M s r �basec K s� ddd�}t �dg �D ]D}|�dd�|krq tj�t|�| �}tj�|�rOtj� t | �sO||d< | |d< zt t� |��|d< W | S t yN Y | S w q |S ) z6 Search the environment for the relative path � )�path�relr �saltenvr r r �stat) r �get�osr �join�_get_container_path�isfile�salt� fileserverZis_file_ignored�listr � Exception)r r �kwargs�fnd� container�fullr r r � find_file` s" ���r c C s4 g } t �dg �D ] }| �|�dd�� qtt| ��S )z_ Each container configuration can have an environment setting, or defaults to base r r r )r r �appendr �set)Zsaltenvsr r r r �envs� s r! c s ddd�}d}t � fdd�|D ��s t�dd�|�� ��� |S |d s&|S |d |d < � �d d �}tj�|d �}t j j � |d ��9}|� � d � |�td �}|r_t j j �|�s_|�t�}|ro|rot j j�||�}||d < ||d< W d � |S 1 s~w Y |S )z? Return a chunk from a file based on the data received r )�data�dest)r �locr c 3 � � | ]}|� v V qd S �Nr ��.0�x��loadr r � <genexpr>� � � z serve_file.<locals>.<genexpr>z<Not all of the required keys present in payload. Missing: %sz, r r r# �gzipN�rbr$ Zfile_buffer_sizer" )�all�log�debugr � differencer r r �normpathr �utils�files�fopen�seek�readr Z is_binary�decodeZ__salt_system_encoding__Z gzip_util�compress)r+ r �retZrequired_load_keysr. Zfpath�fp_r" r r* r � serve_file� s4 � � � r> c C s� t d D �]�} t| �}z tj�|�st�|� ntj�|�s(t�|� t�|� W n t y@ } z t � d� W Y d}~qd}~ww t | �}| d }z|� |�}W n t yg } z t � d� W Y d}~qd}~ww dd� |D �}t|�}tjj�|�D ]?\}} } | D ].} tj�|| �} tj�| |�} | |vr�tj�| d | � zt�| � W q� t y� Y q�w q�| s�| s�t�|� qz|D ]�}tj�||j�} d }tj�| �r�|jjj}t� tjj � | d �� d ��}||kr�d }nd }|�rZtj�tj� | ���st�tj� | �� | d }tj�|| � tjj!�"|d �� W d � n 1 �s w Y z |�#||j| � W n t �yG } z t � d� W Y d}~q�d}~ww zt�|� W q� t �yY Y q�w q�|d }|d }tj�||� tjj!�"|d �� W d � n 1 �s~w Y tjj!�"|d ��}tjj$�%||� W d � n 1 �s�w Y zt�|� W n t �y� Y nw ztj�t d dd�}t�|� W q t �y� t � d� Y qw dS )a" Update caches of the storage containers. Compares the md5 of the files on disk to the md5 of the blobs in the container, and only updates if necessary. Also processes deletions by walking the container caches and comparing with the list of blobs in the container r z3Error occurred creating cache directory for azurefsN�container_namez-Error occurred fetching blob list for azurefsc S s g | ]}|j �qS r )�name)r( �blobr r r � <listcomp>� s zupdate.<locals>.<listcomp>�.lkF�md5�hexT�wz)Error occurred fetching blob from azurefs�.list�cachedir�hashesz;Problem occurred trying to invalidate hash cach for azurefs)&r r r r �exists�makedirs�isdir�shutil�rmtreer r1 � exception�_get_container_serviceZ list_blobsr r r5 Zos_walkr �relpathr � wait_lock�unlinkr@ Z propertiesZcontent_settingsZ content_md5�base64� b64encode� hashutils�get_hashr: �dirnamer6 r7 Zget_blob_to_path�json�dump)r r �exc� blob_servicer@ Z blob_listZ blob_namesZblob_set�root�dirsr6 �f�fnamerQ rA �updateZ source_md5Z local_md5Zlk_fn�container_listr= � hash_cachedirr r r ra � s� � �� �� �� � �� � ���� � ����ra c sP t � fdd�dD ��s dS dtd i}|d }|d }tj�td d d �}tjj�|� d d �|td ��}tj�|�s�tj� tj� |��sOt� tj� |�� tjj � |td �|d < tjj�|d��}|�tjj�|d �� W d � |S 1 s{w Y |S tjj�|d��}tjj�|�� �|d < W d � |S 1 s�w Y |S )zL Return a file hash based on the hash type set in the master config c 3 r% r& r r' r* r r r, r- z file_hash.<locals>.<genexpr>)r r )r NZ hash_typer r rH r rI r z {}.hash.{}Zhsumzw+Nr/ )r0 r r r r r r5 �formatr rJ rX rK rV rW r6 r7 �writeZ stringutilsZto_strZ to_unicoder9 )r+ r r< rQ r rc Zhashdestr= r r* r � file_hash s6 � �� ��rf c C s� t � }zTtd D ]J}|�dd�| d krqt|�d }|d }tj�||d� tj� |�s.qtj j � |d��}|� t tj j�|��� W d � n1 sMw Y qW t|�S tyq } zt�d � W Y d}~t|�S d}~ww ) z? Return a list of all files in a specified environment r r r rG rC � �rNz�azurefs: an error ocurred retrieving file lists. It should be resolved next time the fileserver updates. Please do not manually modify the azurefs cache directory.)r r r r r r rR r r rJ r5 r6 r7 ra rY r+ r r1 �errorr )r+ r< r rb Zlkr= r[ r r r � file_list( s. �������rj c C sF t � }t| �}|D ]}|}|r tj�|�}|r |�|� |sq t|�S )zE Return a list of all directories in a specified environment )r rj r r rX �addr )r+ r< r6 r_ rX r r r �dir_listB s ��rl c C sF t j�td d�}d�| �dd�| �dd�| �dd��}t j�||�S ) z� Get the cache path for the container in question Cache paths are generate by combining the account name, container name, and saltenv, separated by underscores rH r z{}_{}_{}� account_namer r? r r )r r r r rd r )r r] Z container_dirr r r r Q s �r c C s` d| v rt jj| d | d d�}nd| v r"t jj| d | d d�}nt j�| d �}|�� }|S )z� Get the azure block blob service for the container in question Try account_key, sas_token, and no auth in that order � account_keyrm )rn � sas_token)ro )�azure�storageZCloudStorageAccountZcreate_block_blob_service)r Zaccountr\ r r r rP a s � �rP c C sl t td t�st�d� dS td D ]!} t | t�s"t�d| � dS d| vs*d| vr3t�d| � dS qdS ) zF Validate azurefs config, return False if it doesn't validate r z?azurefs configuration is not formed as a list, skipping azurefsFzdOne or more entries in the azurefs configuration list are not formed as a dict. Skipping azurefs: %srm r? z\An azurefs container configuration is missing either an account_name or a container_name: %sT)� isinstancer r r1 ri �dict)r r r r r u s$ ���r )r )$�__doc__rT �loggingr rM Zsalt.fileserverr Zsalt.utils.filesZsalt.utils.gzip_utilZsalt.utils.hashutilsZsalt.utils.jsonZsalt.utils.pathZsalt.utils.stringutilsZsalt.utils.versionsr Z azure.storagerp rq � __version__� ImportErrorr �AttributeErrorr � getLoggerr1 r r r! r> ra rf rj rl r rP r r r r r �<module> sD / � c
| ver. 1.4 |
Github
|
.
| PHP 8.2.30 | Generation time: 0 |
proxy
|
phpinfo
|
Settings