File manager - Edit - /opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/__pycache__/hgfs.cpython-310.pyc
Back
o ;j�{ � @ s� d Z ddlZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl Z ddl Z ddl Z ddlZ ddlZ ddlZ ddlZ ddlZ ddlmZ ddlmZ ddlmZ dZdZzddlZd Z W n e yq d Z Y nw e� e �Z d Z!d Z"d d� Z#dd� Z$dd� Z%dd� Z&dd� Z'dd� Z(dd� Z)dd � Z*d d � Z+d d � Z,d!d"� Z-d#d$� Z.d%d&� Z/dDd'd(�Z0dDd)d*�Z1d+d,� Z2d-d.� Z3dEd/d0�Z4dFd2d3�Z5d4d5� Z6d6d7� Z7d8d9� Z8d:d;� Z9d<d=� Z:d>d?� Z;d@dA� Z<dBdC� Z=dS )Ga� Mercurial Fileserver Backend To enable, add ``hgfs`` to the :conf_master:`fileserver_backend` option in the Master config file. .. code-block:: yaml fileserver_backend: - hgfs .. note:: ``hg`` also works here. Prior to the 2018.3.0 release, *only* ``hg`` would work. After enabling this backend, branches, bookmarks, and tags in a remote mercurial repository are exposed to salt as different environments. This feature is managed by the :conf_master:`fileserver_backend` option in the salt master config file. This fileserver has an additional option :conf_master:`hgfs_branch_method` that will set the desired branch method. Possible values are: ``branches``, ``bookmarks``, or ``mixed``. If using ``branches`` or ``mixed``, the ``default`` branch will be mapped to ``base``. .. versionchanged:: 2014.1.0 The :conf_master:`hgfs_base` master config parameter was added, allowing for a branch other than ``default`` to be used for the ``base`` environment, and allowing for a ``base`` environment to be specified when using an :conf_master:`hgfs_branch_method` of ``bookmarks``. :depends: - mercurial - python bindings for mercurial (``python-hglib``) � N)�datetime)�DEFAULT_HASH_TYPE��FileserverConfigError)�tagify)�branches� bookmarks�mixed)�base� branch_method� mountpoint�rootTF�hgfs)�hgc C sh t td vrdS tst�d� dS td tvr"t�dtd t� dS tjj� d�du r2t�d� dS t S ) z- Only load if mercurial is available Zfileserver_backendFzeMercurial fileserver backend is enabled in configuration but could not be loaded, is hglib installed?Zhgfs_branch_methodz6Invalid hgfs_branch_method '%s'. Valid methods are: %sr Nz2hgfs requested but hg executable is not available.) �__virtualname__�__opts__�HAS_HG�log�error�VALID_BRANCH_METHODS�salt�utils�path�which� r r �H/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/hgfs.py� __virtual__Q s$ � � r c C s dd� | � � D �}|S )z5 Returns all branches for the specified repo c S �6 g | ]}t jj�|d �|d t jj�|d �f�qS �r � � �r r � stringutils�to_str��.0�xr r r � <listcomp>q � (��z!_all_branches.<locals>.<listcomp>)r )�repor r r r � _all_branchesj s �r* c �2 z� fdd�t | �D �d W S ty Y dS w )z9 Find the requested branch in the specified repo c � g | ] }|d � kr|�qS �r r r$ ��namer r r' } � z _get_branch.<locals>.<listcomp>r F)r* � IndexError�r) r/ r r. r � _get_branchx � �r3 c C s dd� | � � d D �}|S )z6 Returns all bookmarks for the specified repo c S r r r! r$ r r r r' � r( z"_all_bookmarks.<locals>.<listcomp>r )r )r) r r r r �_all_bookmarks� s �r5 c r+ )z; Find the requested bookmark in the specified repo c r, r- r r$ r. r r r' � r0 z!_get_bookmark.<locals>.<listcomp>r F)r5 r1 r2 r r. r � _get_bookmark� r4 r6 c C s dd� | � � D �S )z1 Returns all tags for the specified repo c S sR g | ]%}t jj�|d �dkrt jj�|d �|d t jj�|d �|d f�qS )r �tipr r � r! r$ r r r r' � s ��z _all_tags.<locals>.<listcomp>)�tags)r) r r r � _all_tags� s �r: c r+ )z6 Find the requested tag in the specified repo c r, r- r r$ r. r r r' � r0 z _get_tag.<locals>.<listcomp>r F)r: r1 r2 r r. r �_get_tag� r4 r; c C s� |dkr| d }|| d ks|t � v rV| d dkr't| d |�p&t| d |�S | d dkr;t| d |�p:t| d |�S | d dkrVt| d |�pUt| d |�pUt| d |�S dS )z1 Return ref tuple if ref is in the repo. r r r r) r r F)�envsr3 r; r6 r2 r r r �_get_ref� s � �r= c C s dd� | j |d d�D �}|S )z Get manifest for ref c S sL g | ]"}t jj�|d �t jj�|d �|d |d t jj�|d �f�qS )r r r r8 � r! r$ r r r r' � s ���z!_get_manifest.<locals>.<listcomp>r )�rev)�manifest)r) �refr@ r r r � _get_manifest� s � rB c C s t d��)zB Fatal fileserver configuration issue, raise an exception z$Failed to load hg fileserver backendr r r r r � _failhard� s rC c C s t j�td d�} d}g }i }tD ] }ttd|� � �||< qtd D �]}}t�|�}t|t �r�t t |��}dd� t j j�|| ��� D �}|sQt�d|� t� |�d |d �} | tvrkt�d | |d �t�� t� d} d d � |D �D ]}t�d||d �t�� d} qt| r�t� |�|� n|}t|t�s�t�d|� t� z t j j�|d �|d< W n ty� Y nw ttt�dt ��} | |� d��� � } t j�| | �} t j� | �s�t � | � t �!| �s�t"�#| �}|�$� d}d}z�zt"�%| �}W n2 t"jj&�y t�d| |� t� Y n t'�y& } zt�d||� t� W Y d}~nd}~ww z|j(dd�}W n t"jj)�y= d}Y nw |�sut j�| dd�}t j j*�+|d��}|�,d � |�,t j j-�.d |� d ��� W d � n 1 �spw Y |�||| | t j�td d| � d ��d �� |�/|� W |�r�|�$� q#|�r�|�$� w w |�rt j�td d!�}zCt j j*�+|d��0}t0�1� �2d"�}|�,d#|� d �� |D ]}|�,t j j-�.d$�3|d% |d& ��� �q�W d � n 1 �s�w Y W n t4�y� Y |S w t�5d'|� |S )(zE Return a list of hglib objects for the various hgfs remotes �cachedirr FZhgfs_Z hgfs_remotesc S s i | ] \}}|t |��qS r )�str)r% �key�valr r r � <dictcomp>� s ��zinit.<locals>.<dictcomp>z�Invalid per-remote configuration for hgfs remote %s. If no per-remote parameters are being specified, there may be a trailing colon after the URL, which should be removed. Check the master configuration file.r zdInvalid branch_method '%s' for remote %s. Valid branch methods are: %s. This remote will be ignored.z, c s s � | ] }|t vr|V qd S �N)�PER_REMOTE_OVERRIDESr$ r r r � <genexpr> s � zinit.<locals>.<genexpr>z|Invalid configuration parameter '%s' for remote %s. Valid parameters are: %s. See the documentation for further information.TzZInvalid hgfs remote %s. Remotes must be strings, you may need to enclose the URL in quotesr � hash_typezutf-8Nz�Cache path %s (corresponding remote: %s) exists but is not a valid mercurial repository. You will need to manually delete this directory on the master to continue to use this hgfs remote.z<Exception '%s' encountered while initializing hgfs remote %ss paths)�namesz.hgZhgrc�w+z[paths] z default = � z .update.lk)r) �url�hashrD �lockfilezhgfs/remote_map.txtz%d %b %Y %H:%M:%S.%fz# hgfs_remote map as of z{} = {} rQ rP z Wrote new hgfs_remote map to %s)6�osr �joinr rJ rE �copy�deepcopy� isinstance�dict�next�iterr r �dataZrepack_dictlist�itemsr r rC �getr �updaterP Z strip_proto� TypeError�getattr�hashlibr �encode� hexdigest�isdir�makedirs�listdir�hglib�init�close�openZ ServerError� ExceptionZconfig� CommandError�files�fopen�writer" r# �appendr �now�strftime�format�OSError�info)�bp_Z new_remote�reposZper_remote_defaults�param�remoteZ repo_confZrepo_urlZper_remote_confr Zper_remote_errorsrL Z repo_hashZrp_�clientr) �exc�refsZ hgconfpathZhgconfigZ remote_map�fp_� timestampr r r rh � s ������ � � � � ���� � ���� �� ���� �� � rh c C s2 t j�td d�} zt �| �}W n ty g }Y nw t� }|D ]}z |�|d � W q! ty5 Y q!w g }|D ]}|dv rAq:t j�| |�}t j� |�rS|� |� q:g }|r�|D ].}zt � |� W n ty� } zt �d|| � |� |� W Y d} ~ qZd} ~ ww t �d|� qZ|D ]} |�| � q�t|�|fS )zC Remove cache directories for remotes no longer configured rD r rQ )rQ r| z0Unable to remove old hgfs remote cachedir %s: %sNz hgfs removed old cachedir %s)rS r rT r rf rt rh �remove� ValueErrorrd rp �shutil�rmtreer r �debug�bool) rv Z cachedir_lsrw r) � to_remove�itemr �failed�rdirr{ Zfdirr r r �_clear_old_remotes� sF � � ��� r� c C s� t j�td d�} t j�td d�}g }| |fD ]-}t j�|�rEzt�|� W q tyD } z|�d|� d|� �� W Y d}~qd}~ww q|S )z% Completely clear hgfs cache rD r �file_lists/hgfszUnable to delete z: N) rS r rT r �existsr� r� rt rp )Z fsb_cachedir� list_cachedir�errorsr� r{ r r r � clear_cache� s "���r� c C �� dd� }t | t�r || �S g }g }t� D ]S}zJ| rIzt�|d | �s+W W |d �� qW n tyH t�|d t| ��sFY W |d �� qY nw ||�\}}|�|� |�|� W |d �� q|d �� w ||fS )z� Clear update.lk ``remote`` can either be a dictionary containing repo configuration information, or a pattern. If the latter, then remotes for which the URL matches the pattern will be locked. c S s dd� }g }g }t j�| d �r�z t �| d � W nU tyn } zI|jtjkrIz t�| d � W n' tyH } z ||| |� W Y d }~n d }~ww ||| |� W Y d }~||fS W Y d }~||fS W Y d }~||fS d }~ww d� | d �}t � |� |� |� ||fS )Nc S s. d� |d |d |�}t�|� | �|� d S )Nz-Unable to remove update lock for {} ({}): {} rP rR )rs r r� rp )Zerrlistr) r{ �msgr r r � _add_error� s � z6clear_lock.<locals>._do_clear_lock.<locals>._add_errorrR zRemoved lock for {}rP ) rS r r� r rt �errnoZEISDIRr� r� rs r r� rp )r) r� �successr� r{ r� r r r �_do_clear_lock� s4 �� � ��� z"clear_lock.<locals>._do_clear_lockrP r) �rW rX rh �fnmatchri r_ rE �extend)ry r� Zclearedr� r) r� r� r r r � clear_lock� s0 � �� r� c C r� )z� Place an update.lk ``remote`` can either be a dictionary containing repo configuration information, or a pattern. If the latter, then remotes for which the URL matches the pattern will be locked. c S s� g }g }t j�| d �sdz tjj�| d d�� W d � n1 s"w Y W n* tyR } z d�| d | d |�}t � |� |� |� W Y d }~||fS d }~ww d�| d �}t � |� |� |� ||fS )NrR �wz*Unable to set update lock for {} ({}): {} rP zSet lock for {}) rS r r� r r rm rn rt rs r r� rp )r) r� r� r{ r� r r r �_do_lock s* ��� �� zlock.<locals>._do_lockrP r) r� )ry r� �lockedr� r) r� r� r r r �lock s0 � �� r� c C s� ddd�} t � \| d< }|D ]�}z�tj�|d �r,t�d|d |d � W |d �� qt|�\}}|rDt�d |d � W |d �� qt� d |d � |d � � |d � � }z|d � � W n t y~ } ztjd ||d tjd � W Y d }~nd }~ww |d � � }|d |d kr�d| d< W |d �� n|d �� w t|� qtj�td d�}| �dd�du s�tj�|�s�tj�|�} tj�| �s�t�| � tdd�} tjj�|d��} | � tj � | �� t� d|� W d � n1 s�w Y t�dd��r*tjj j!dtd tdd��} | �"| t#ddgdd�� W d � n 1 �s%w Y ztj$�%tj�td d �t&� W d S t'�yF Y d S w ) z0 Execute an hg pull on all of the repos Fr )�changedZbackendr� rR z�Update lockfile is present for hgfs remote %s, skipping. If this warning persists, it is possible that the update process was interrupted. Removing %s or running 'salt-run fileserver.clear_lock hgfs' will allow updates to continue for this remote.rP r) z7Unable to set update lock for hgfs remote %s, skipping.zhgfs is fetching from %sz1Exception %s caught while updating hgfs remote %s)Zexc_info_on_loglevelNr TrD � hgfs/envs.p)� ignore_cachezwb+zWrote env cache data to %sZfileserver_eventsZmasterZsock_dir)�opts�listenr^ � fileserver)�prefix� hgfs/hash)(r� rS r r� r �warningri r� r r� rj r7 Zpullrk �logging�DEBUGr� rT r r] �isfile�dirnamere r< r r rm rn ro �payload�dumpsZtrace�eventZ get_eventZ fire_eventr r� Zreap_fileserver_cache_dir� find_filert ) r[ rw r) �_r� Zcurtipr{ Znewtip� env_cacheZ env_cachedirZnew_envsr} r� r r r r^ 5 s� � ��� ��� � �� � ��r^ c C s t jjj| td td d�S )zc Check if an environment is exposed by comparing it against a whitelist and blacklist. Zhgfs_saltenv_whitelistZhgfs_saltenv_blacklist)Z whitelistZ blacklist)r r r" Zcheck_whitelist_blacklistr )�envr r r �_env_is_exposed� s �r� c C s | st j�td d�}tj�t|�}|dur|S t� }t� D ]e}z\|d � � |d dv rGt |d �D ]}|d }||d krAd}|� |� q3|d d v rgt |d �D ]}|d }||d krad}|� |� qS|� d d � t|d �D �� W |d �� q |d �� w d d � t|�D �S ) z@ Return a list of refs that can be used as environments rD r� Nr) r )r r r r )r r c S s g | ]}|d �qS r- r r$ r r r r' � s zenvs.<locals>.<listcomp>c S s g | ]}t |�r|�qS r )r� r$ r r r r' � s )rS r rT r r r� Zcheck_env_cache�setrh rj r* �addr5 r^ r: ri �sorted) r� r� � cache_match�retr) �branchZ branch_nameZbookmarkZ bookmark_namer r r r< � s0 r< r c K s: ddd�}t j�| �s|t� vr|S t j�td d|| �}t j�td d|| � d��}t j�td d|| � d��}t j�td d|| � d��}t j�|�}t j�|�} t j�|�stzt �|� W n t ys t � |� t �|� Y nw t j�| �s�zt �| � W n t y� t � | � t �| � Y nw t � D �]�} �zw| d r�| � | d t jj �s�W | d �� q�| t| d �d � �t jj �} | d r�t j�| d | �} | d �� t| |�} | s�| d �� W | d �� q�tj�||� t j�|��rDt j�|��rDtjj�|d ��3} | �� }|| d k�r4| |d< ||d< | d �� |W d � W | d �� S W d � n 1 �s?w Y z| d jtjj � d| � ��g| d |d� W n t j j �yt | d �� Y W | d �� q�w tjj�|d�� W d � n 1 �s�w Y t!�!|�D ]}zt � |� W �q� t"�y� Y �q�w tjj�|d��} | �#tjj �$| d �� W d � n 1 �s�w Y zt � |� W n t �y� Y nw | |d< ||d< z t%t �&|��|d< W n t"�y� Y n w W | d �� | S W | d �� | S | d �� w |S )z Find the first file to match the path and ref, read the file out of hg and send the path to the newly cached file � )r �relrD z hgfs/refsr� z.hash.*z.hash.blob_sha1z.lkr r) Nr �rr r� r zpath:)r? �outputr� rN �stat)'rS r �isabsr< rT r r� rd re rt r rh � startswith�sepri �len�lstriprj r= r r� Z wait_lockr� r rm rn �read�catr" �to_bytesrg r rl �globrk ro r# �listr� )r Ztgt_env�kwargs�fnd�destZ hashes_globZ blobshadestZlk_fnZdestdirZhashdirr) Z repo_pathrA r} Zsha�filenamer r r r� � s� �� � � �= � 3� �0� � � !� �� �� �� � �r� c s d� v r � � d� ddd�}t� fdd�dD ��s|S |d s!|S |d |d < � �d d �}tj�|d �}tjj� |d ��9}|� � d � |� t d �}|rZtjj� |�sZ|�t�}|rj|rjtjj�||�}||d < ||d< W d � |S 1 syw Y |S )z? Return a chunk from a file based on the data received r� r� )r[ r� c 3 � � | ]}|� v V qd S rI r r$ ��loadr r rK � � z serve_file.<locals>.<genexpr>)r �loc�saltenvr r� r� �gzipN�rbr� Zfile_buffer_sizer[ )�pop�allr] rS r �normpathr r rm rn �seekr� r Z is_binary�decodeZ__salt_system_encoding__Z gzip_util�compress)r� r� r� r� Zfpathr} r[ r r� r � serve_file s. � � r� c s" d� v r � � d� t� fdd�dD ��sdS dtd i}|d }|d }tj�td d � d d �|td ��}tj�|�sktj j � |td �|d < tj j � |d��}|�|d � W d � |S 1 sdw Y |S tj j � |d��}tj j�|�� �|d < W d � |S 1 s�w Y |S )zL Return a file hash, the hash type is set in the master config file r� c 3 r� rI r r$ r� r r rK ; r� z file_hash.<locals>.<genexpr>)r r� r� rL r� r rD r� r� z {}.hash.{}ZhsumrN Nr� )r� r� r rS r rT rs r� r r Z hashutilsZget_hashrm rn ro r" Z to_unicoder� )r� r� r� �relpathr Zhashdestr} r r� r � file_hash3 s6 � �� ��r� c C s� d| v r | � d� tj�td d�}tj�|�s1zt�|� W n ty0 t� d|� g Y S w tj�|d� | d ��}tj�|d� | d ��}t j � t|||�\}}}|dur[|S |r|i }t| �|d < t| �|d < |rvt j �t|||� |�|g �S g S ) zD Return a dict containing the file lists for files and dirs r� rD r� zUnable to make cachedir %sz{}.pr� z.{}.wNrm �dirs)r� rS r rT r rd re rt r �criticalrs r r� Zcheck_file_list_cache�_get_file_list� _get_dir_listZwrite_file_list_cacher] ) r� Zformr� Z list_cacheZw_lockr� Z refresh_cacheZ save_cacher� r r r � _file_listsQ s2 � � r� c C � t | d�S )zR Return a list of all files on the file server in a specified environment rm �r� r� r r r � file_listr � r� c C s� d| v r | � d� d| vs| d t� vrg S t� }t� D ]I}z@|d �� t|| d �}|rWt|d |d�}|D ] }tj� |d |d �}|� d�sV|� tj� |d |�� q8W |d � � q |d � � w t|�S ) zO Get a list of all files on the file server in a specified environment r� r� r) �rA r> r �../r )r� r< r� rh rj r= rB rS r r� r� r� rT ri r� )r� r� r) rA r@ �tupr� r r r r� y s$ � r� c C s g S )z> Return a list of all empty directories on the master r r� r r r �file_list_emptydirs� s r� c C r� )z8 Return a list of all directories on the master r� r� r� r r r �dir_list� r� r� c C s. d| v r | � d� d| vs| d t� vrg S t� }t� D ]k}zb|d �� t|| d �}|ryt|d |d�}|D ]@}|d }|�dd�}t|�dkrxt j � |d |d �}|d krj|� d �sj|� t j �|d |�� |d �dd�}t|�dksJq8W |d �� q |d �� w |d r�|� |d � t|�S ) z5 Get a list of all directories on the master r� r� r) r� r> �/r r r �.r� r )r� r< r� rh rj r= rB �rsplitr� rS r r� r� r� rT ri r� ) r� r� r) rA r@ r� �filepath�splitr� r r r r� � s4 �� r� rI )F)r )>�__doc__rU r� r� r� ra r� rS r� r Zsalt.fileserverr Zsalt.utils.dataZsalt.utils.filesZsalt.utils.gzip_utilZsalt.utils.hashutilsZsalt.utils.stringutilsZsalt.utils.urlZsalt.utils.versionsZ salt.configr Zsalt.exceptionsr Zsalt.utils.eventr r rJ rg r � ImportError� getLogger�__name__r r Z__virtual_aliases__r r* r3 r5 r6 r: r; r= rB rC rh r� r� r� r� r^ r� r< r� r� r� r� r� r� r� r� r� r r r r �<module> sx % � && >4M i !
| ver. 1.4 |
Github
|
.
| PHP 8.2.30 | Generation time: 0 |
proxy
|
phpinfo
|
Settings