File manager - Edit - /opt/saltstack/salt/lib/python3.10/site-packages/salt/pillar/__pycache__/azureblob.cpython-310.pyc
Back
o ;j}7 � @ s� d Z ddlZddlZddlZddlZddlmZ ddlZddl Zddl m Z dZ z ddl mZ dZ W n ey; Y nw dZe�e�Zd d � Z d!d d�Zdd� Zdd� Zdd� Zdd� Z d"dd�Zdd� Zdd � Z d d � Z d d � Z dS )#a� Use Azure Blob as a Pillar source. .. versionadded:: 3001 :maintainer: <devops@eitr.tech> :maturity: new :depends: * `azure-storage-blob <https://pypi.org/project/azure-storage-blob/>`_ >= 12.0.0 The Azure Blob ext_pillar can be configured with the following parameters: .. code-block:: yaml ext_pillar: - azureblob: container: 'test_container' connection_string: 'connection_string' multiple_env: False environment: 'base' blob_cache_expire: 30 blob_sync_on_update: True :param container: The name of the target Azure Blob Container. :param connection_string: The connection string to use to access the specified Azure Blob Container. :param multiple_env: Specifies whether the pillar should interpret top level folders as pillar environments. Defaults to false. :param environment: Specifies which environment the container represents when in single environment mode. Defaults to 'base' and is ignored if multiple_env is set as True. :param blob_cache_expire: Specifies expiration time of the Azure Blob metadata cache file. Defaults to 30s. :param blob_sync_on_update: Specifies if the cache is synced on update. Defaults to True. � N)�deepcopy)�PillarF)�BlobServiceClientT� azureblobc C s t sdS tS )N)FznThe following dependency is required to use the Azure Blob ext_pillar: Microsoft Azure Storage Blob >= 12.0.0 )�HAS_LIBS�__virtualname__� r r �I/opt/saltstack/salt/lib/python3.10/site-packages/salt/pillar/azureblob.py� __virtual__C s r �base� c C s& t j�t j�t� ||��}td �|g �|gkri S t|||||�} t� d| � |rdt� d� | � � D ]+\} } t | �� � D ] \}} | D ]} t || | �}t� d|| | � t|| | || |� qCq=q3t� d� tt�}|rrt j�||�gn|g|d |< dd� |d D �|d< t|t| |�}|jd d �}|S ) a� Execute a command and read the output as YAML. :param container: The name of the target Azure Blob Container. :param connection_string: The connection string to use to access the specified Azure Blob Container. :param multiple_env: Specifies whether the pillar should interpret top level folders as pillar environments. Defaults to false. :param environment: Specifies which environment the container represents when in single environment mode. Defaults to 'base' and is ignored if multiple_env is set as True. :param blob_cache_expire: Specifies expiration time of the Azure Blob metadata cache file. Defaults to 30s. :param blob_sync_on_update: Specifies if the cache is synced on update. Defaults to True. Z pillar_rootszBlob metadata: %sz-Syncing local pillar cache from Azure Blob...z %s - %s : %sz2Sync local pillar cache from Azure Blob completed.c S s g | ]}d |vr|�qS )r r )�.0�xr r r � <listcomp>� s z ext_pillar.<locals>.<listcomp>� ext_pillarF)�ext)�os�path�normpath�join�_get_cache_dir�__opts__�get�_init�log�debug�info�items� _find_files�_get_cached_file_name�_get_file_from_blobr r Z __grains__Zcompile_pillar)Z minion_idZpillar� container�connection_string� multiple_env� environment�blob_cache_expireZblob_sync_on_updateZ pillar_dir�metadata�saltenv�env_meta�files� file_path�cached_file_path�optsZpilZcompiled_pillarr r r r N sH � � ���� � r c C s� t |�}t�� | }tj�|�rtj�|�}nd}||k}t�d||r%dnd|| |� |r7t| ||||�} nt |�} t�d| � | S )a .. versionadded:: 3001 Connect to Blob Storage and download the metadata for each file in all containers specified and cache the data to disk. :param connection_string: The connection string to use to access the specified Azure Blob Container. :param container: The name of the target Azure Blob Container. :param multiple_env: Specifies whether the pillar should interpret top level folders as pillar environments. Defaults to false. :param environment: Specifies which environment the container represents when in single environment mode. Defaults to 'base' and is ignored if multiple_env is set as True. :param blob_cache_expire: Specifies expiration time of the Azure Blob metadata cache file. Defaults to 30s. r zQBlob storage container cache file %s is %sexpired, mtime_diff=%ss, expiration=%ss� znot z#Blob container retrieved pillars %s) � _get_containers_cache_filename�timer r �isfile�getmtimer r � _refresh_containers_cache_file�_read_containers_cache_file) r" r! r# r$ r% � cache_file�expZcache_file_mtimeZexpiredZpillarsr r r r � s( � � r c C s6 t j�td d�} t j�| �st�d� t �| � | S )zf .. versionadded:: 3001 Get pillar cache directory. Initialize it if it does not exist. ZcachedirZpillar_azureblobz$Initializing Azure Blob Pillar Cache)r r r r �isdirr r �makedirs)� cache_dirr r r r � s r c C s> t j�t� || |�}t j�t j�|��s t �t j�|�� |S )a% .. versionadded:: 3001 Return the cached file name for a container path file. :param container: The name of the target Azure Blob Container. :param saltenv: Specifies which environment the container represents. :param path: The path of the file in the container. )r r r r �exists�dirnamer7 )r! r' r r* r r r r � s r c C s0 t � }tj�|�st�|� tj�|| � d��S )z� .. versionadded:: 3001 Return the filename of the cache for container contents. Create the path if it does not exist. :param container: The name of the target Azure Blob Container. z -files.cache)r r r r9 r7 r )r! r8 r r r r. � s r. c s� z t �| �}|����W n ty% } z t�d|� W Y d}~dS d}~ww i �d � ����fdd� � � |d� tj�|�rCt� |� t� d� t j j �|d ��}t��|� W d � �S 1 sbw Y �S ) a� .. versionadded:: 3001 Downloads the entire contents of an Azure storage container to the local filesystem. :param connection_string: The connection string to use to access the specified Azure Blob Container. :param container: The name of the target Azure Blob Container. :param cache_file: The path of where the file will be cached. :param multiple_env: Specifies whether the pillar should interpret top level folders as pillar environments. :param environment: Specifies which environment the container represents when in single environment mode. This is ignored if multiple_env is set as True. � Exception: %sNFr c s� �j |d�} zt|�}W n ty Y d S w t�d|� |j�d�r+� |jd� q�r;|r3|dkr5dn|d d� } | �vrCi �| < ��| vrOg �| �< �| � �|� q) N)Zname_starts_withTzRaw blob attributes: %s�/)�prefix�.r ���)Z walk_blobs�next� StopIterationr r �name�endswith�append)r' r= Z blob_listZblob�� _walk_blobsr! �container_clientr& r# r r rF * s& � �z3_refresh_containers_cache_file.<locals>._walk_blobs)r' z%Writing Azure blobs pillar cache file�wb)r N)r �from_connection_string�get_container_client� Exceptionr �errorr r r0 �remover �salt�utilsr) �fopen�pickle�dump)r" r! r4 r# r$ �blob_service_client�exc�fp_r rE r r2 s, � �� ��r2 c C sJ t �d� tjj�| d��}t�|�}W d � |S 1 s w Y |S )z� .. versionadded:: 3001 Return the contents of the containers cache file. :param cache_file: The path for where the file will be cached. z Reading containers cache file�rbN)r r rN rO r) rP rQ �load)r4 rU �datar r r r3 V s ��r3 c C sR i }| � � D ] \}}||vrg ||< dd� |D �}|| dd� |D �7 < q|S )z� .. versionadded:: 3001 Looks for all the files in the Azure Blob container cache metadata. :param metadata: The metadata for the container files. c S s g | ]}|d �qS )rB r �r �kr r r r w s z _find_files.<locals>.<listcomp>c S s g | ] }|� d �s|�qS )r< )rC rY r r r r y s )r )r& �retr! rX Z file_pathsr r r r g s r c C sZ || v r| | ni }||v r|| ni }|D ]}t |�}d|v r*|d |kr*| S qdS )an .. versionadded:: 3001 Looks for a file's metadata in the Azure Blob Container cache file. :param metadata: The metadata for the container files. :param container: The name of the target Azure Blob Container. :param saltenv: Specifies which environment the container represents. :param path: The path of the file in the container. rB N)�dict)r& r! r' r r( Zcontainer_metaZ item_metar r r �_find_file_meta~ s ��r] c C s t j�|�r4t||||�}|r d�tttj|d ���nd}t j j � |d�}t �d|||� ||kr4dS zt�| �} | �|�} | �|�} W n ty^ } z t �d| � W Y d} ~ dS d} ~ ww t j j�|d��} | �| �� �� � W d � dS 1 s|w Y dS ) a� .. versionadded:: 3001 Downloads the entire contents of an Azure storage container to the local filesystem. :param connection_string: The connection string to use to access the specified Azure Blob Container. :param metadata: The metadata for the container files. :param saltenv: Specifies which environment the container represents when in single environment mode. This is ignored if multiple_env is set as True. :param container: The name of the target Azure Blob Container. :param path: The path of the file in the container. :param cached_file_path: The path of where the file will be cached. r- ZetagN�md5z%Cached file: path=%s, md5=%s, etag=%sr; FrH )r r r0 r] r �list�filter�str�isalnumrN rO Z hashutilsZget_hashr r r rI rJ Zget_blob_clientrK rL r) rP �writeZ download_blob�readall)r" r& r' r! r r+ Z file_metaZfile_md5Z cached_md5rS rG Z blob_clientrT Zoutfiler r r r � s< ��� �� ��r )Fr r T)Fr ) �__doc__�loggingr rQ r/ �copyr Zsalt.utils.filesrN Zsalt.utils.hashutilsZ salt.pillarr r Zazure.storage.blobr � ImportErrorr � getLogger�__name__r r r r r r r. r2 r3 r r] r r r r r �<module> sD ' � �O5 �N
| ver. 1.4 |
Github
|
.
| PHP 8.2.30 | Generation time: 0 |
proxy
|
phpinfo
|
Settings