File manager - Edit - /opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/__pycache__/__init__.cpython-310.pyc
Back
o ;j�w � @ s d Z ddlZddlZddlZddlZddlZddlZddlmZ ddl Z ddl Z ddl Z ddl Z ddlZ ddlmZ ddlmZ e�e�Zdd� Zdd � Zd!d d �Zd d � Zdd� Zdd� Z dd� Z dd� Z dd� Z dd� Z d"dd �Z!G d d � d �Z"G d d � d �Z#dS )#z= File server pluggable modules and generic backend functions � N)�Sequence)�get_function_argspec)�ensure_unicode_argsc C s� t j�| �sdS z t j�| �rt �| � W dS t j�| �r%t �| � W dS W dS tyA } zt� d| |� W Y d}~dS d}~ww )z) Unlock a FS file/dir based lock Nz Error removing lockfile %s: %s) �os�path�exists�isdir�rmdir�isfile�unlink�OSError�log�trace��w_lock�exc� r �L/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/__init__.py� _unlock_cache s � ��r c C sT zt �| � W n ty! } z|jtjkr� W Y d }~dS d }~ww t�d| � dS )NFzLockfile %s createdT)r �mkdirr �errnoZEEXISTr r r r r r � _lock_cache' s �� r c C s� t j�| �sdS t j�|�s t�d� t j�|�s t| � dS d}|r)t�� | }d}t �|�j} t�d� t j�| �s?dS t �|�j}||krX|d7 }|dkrWt| � dS n|}|rkt�� |krkt d� ||| ���q2)z� If the write lock is there, check to see if the file is actually being written. If there is no change in the file size after a short sleep, remove the lock and move forward. F� Nr T� z&Timeout({}s) for {} (lock: {}) elapsed) r r r �time�sleepr r �stat�st_size� ValueError�format)Zlk_fn�destZ wait_timeout�timeoutZs_countZs_size�sizer r r � wait_lock3 s@ � ���r# c C s� d}d}t ||d� tj�|�st|�rd}n�d}|dk r�z�tj�|�r*t ||d� tj�|�rTt�|�}tt�� �}t|j �} | |krOt � d||| � d} n || } n| � d d �d } | dk ret � d � d| krs| � d d �k r�n n3tjj�|d ��"} t � d| | � d d �|� tj�| �� |g �ddfW d � W S 1 s�w Y nt|�r�d}W nW n ty� t�d� |d 7 }Y qw |dk s |dkr�d}d}d||fS )�� Checks the cache file to see if there is a new enough file list cache, and returns the match (if found, along with booleans used by the fileserver backend to determine if the cache needs to be refreshed/written). FTi, r � i� zUCache file modified time is in the future, ignoring. file=%s mtime=%s current_time=%s���Zfileserver_list_cache_time� r z.The file list_cache was created in the future!�rbz7Returning file list from cache: age=%s cache_time=%s %sNg�������?� )r# r r r r r r �intr �st_mtimer �debug�get�warning�salt�utils�files�fopen�payload�load� Exceptionr ) �optsZform� list_cacher Z refresh_cacheZ save_cache�attemptZ cache_statZ current_timeZ file_mtimeZage�fp_r r r �check_file_list_cachea sd � �$�� ��0 r: c C s\ t jj�|d�� }|�t j�|�� t|� t� d|� W d � dS 1 s'w Y dS )r$ zw+bzLockfile %s removedN) r/ r0 r1 r2 �writer3 �dumpsr r r )r6 �datar7 r r9 r r r �write_file_list_cache� s "�r> c C sv t j�|�sdS z(tjj�|d��}t�d|� tj � |�W d � W S 1 s)w Y W dS t y: Y dS w )zG Returns cached env names, if present. Otherwise returns None. Nr( z Returning env cache data from %s) r r r r/ r0 r1 r2 r r r3 r4 r )r6 Z env_cacher9 r r r �check_env_cache� s $� ��r? c C s� i }|� � D ]B\}}|D ];}tjj�|�D ]1\}}}|D ])} ztj�|| �} t| | �r-W q tj�| �|| < W q t yE t � d| � Y q w qq q|S )z. Generate a dict of filename -> mtime z,Failed to get mtime on %s, dangling symlink?) �itemsr/ r0 r �os_walkr �join�is_file_ignored�getmtimer r �info) r6 Zpath_mapZfile_map�saltenv� path_listr � directory�_� filenames�item� file_pathr r r �generate_mtime_map� s( �����rM c C s< t | �t |�kr dS | �� D ] \}}|| |kr dS qdS )z> Is there a change to the mtime map? return a boolean TF)�sortedr@ )Zmap1Zmap2�filename�mtimer r r �diff_mtime_map� s �rQ c C s� t �| �D ]k}t j�| |�}tjj�|�D ]Z\}}}|s0|s0t�� t j�|� dkr/t � |� q|D ]<}t j�||�}t j� ||�} z | � dd�\} } } W n t y\ t �d|� Y q2w || |d�} | d dkrnt �|� q2qqdS ) z� Remove unused cache items assuming the cache directory follows a directory convention: cache_base -> saltenv -> relpath �< �.� zDFound invalid hash file [%s] when attempting to reap cache directory)rF r � N)r �listdirr rB r/ r0 rA r �getctimer �relpath�rsplitr r r. r )Z cache_baseZ find_funcrF �env_base�root�dirsr1 Zfile_rL Z file_rel_pathrO rI Z hash_type�retr r r �reap_fileserver_cache_dir� s4 �� ����r^ c C sl | d r| d D ]}t �||�rt�d|� dS q| d r4| d D ]}t�||�r3t�d|� dS q"dS )z� If file_ignore_regex or file_ignore_glob were given in config, compare the given file path against all of them and return True on the first match. Zfile_ignore_regexz-File matching file_ignore_regex. Skipping: %sTZfile_ignore_globz,File matching file_ignore_glob. Skipping: %sF)�re�searchr r, �fnmatch)r6 �fnameZregex�globr r r rC s � �rC �updatec C s: d|� d|� d�}|r|d|� �7 }t �|� | ||d�S )a� Function to allow non-fileserver functions to clear update locks clear_func A function reference. This function will be run (with the ``remote`` param as an argument) to clear the lock, and must return a 2-tuple of lists, one containing messages describing successfully cleared locks, and one containing messages describing errors encountered. role What type of lock is being cleared (gitfs, git_pillar, etc.). Used solely for logging purposes. remote Optional string which should be used in ``func`` to pattern match so that a subset of remotes can be targeted. lock_type : update Which type of lock to clear Returns the return data from ``clear_func``. z Clearing z lock for � remotes� matching )�remote� lock_type)r r, )Z clear_funcZrolerg rh �msgr r r � clear_lock% s rj c @ s� e Zd ZdZdd� Zd6dd�Zdd� Zd d � Zd d � Zd6d d�Z d7dd�Z d7dd�Z d6dd�Z d6dd�Z d8dd�Zd6dd�Zd6d d �Zd d � Zd d!� Zd6d"d#�Zd$d%� Zd&d'� Zd(d)� Zd*d+� Zd,d-� Zed.d/� �Zed0d1� �Zed2d3� �Z ed4d5� �Z dS )9� Fileserverz� Create a fileserver wrapper object that wraps the fileserver functions and iterates over them to execute the desired function within the scope of the desired fileserver backend. c C s || _ tj�||d �| _d S )N�fileserver_backend)r6 r/ �loaderZ fileserver�servers)�selfr6 r r r �__init__J s zFileserver.__init__Nc C s |s| j d }n t|t�s%z|�d�}W n ty$ t|��d�}Y nw t|t�r.t|�}g }t|t�s7|S | j�� }z t dd� |D ��}W n tyP Y n#w |rs| j d }|D ]}|dd� � d�|v rp|� |dd� � qZ|S |D ]}|� d�|v r�|� |� qu|S )z) Return the backend list rl �,c s s � | ]}|� d �V qdS )�-N�� startswith��.0�xr r r � <genexpr>k s � z&Fileserver.backends.<locals>.<genexpr>r N�.envs) r6 � isinstance�list�split�AttributeError�strr rn �keys�all�remove�append)ro �backr] Z server_funcsZ subtract_only�subr r r �backendsN s> � � � �zFileserver.backendsc C s | j S )z& Simplify master opts )r6 �ro r4 r r r � master_opts| s zFileserver.master_optsc C s< t | d�r| jd ur t | jd�r| j�� i | _d S d S d S )Nrn �destroy)�hasattrrn r� �ro r r r r� � � �zFileserver.destroyc C sJ | j �� D ] \}}zd|jv r|jd �| j� W q ty" Y qw d S )NZ__opts__)rn r@ � __globals__rd r6 r} )ro �name�funcr r r � update_opts� s � ��zFileserver.update_optsc C sr | � |�}g }g }|D ])}|� d�}|| jv r4t�d|� | j| � }|r+|�|� q |�d|� d�� q ||fS )z� Clear the cache of all of the fileserver backends that support the clear_cache function or the named backend(s) only. z .clear_cachez Clearing %s fileserver cachezThe z* fileserver cache was successfully cleared)r� rn r r, �extendr� )ro r� �cleared�errors�fsb�fstrZfailedr r r � clear_cache� s ��zFileserver.clear_cachec C s� | � |�}g }g }|D ]C}|� d�}|| jv rNd|� d�}|r5t|t�s.|�d|� d�� q |d|� �7 }t�|� | j| |d�\}} |�|� |�| � q ||fS )z� ``remote`` can either be a dictionary containing repo configuration information, or a pattern. If the latter, then remotes for which the URL matches the pattern will be locked. z.lockzSetting update lock for re z Badly formatted remote pattern '�'rf �rg )r� rn rz r~ r� r r, r� ) ro r� rg �lockedr� r� r� ri �good�badr r r �lock� s$ �zFileserver.lockc C sd | � |�}g }g }|D ]"}|� d�}|| jv r-t| j| ||d�\}}|�|� |�|� q ||fS )ai Clear the update lock for the enabled fileserver backends back Only clear the update lock for the specified backend(s). The default is to clear the lock for all enabled backends remote If specified, then any remotes which contain the passed string will have their lock cleared. z .clear_lockr� )r� rn rj r� ) ro r� rg r� r� r� r� r� r� r r r rj � s �zFileserver.clear_lockc K sL | � |�}|D ] }|� d�}|| jv r#t�d|� | j| di |�� qdS )zi Update all of the enabled fileserver backends which support the update function z.updatez Updating %s fileserver cacheNr )r� rn r r, )ro r� �kwargsr� r� r r r rd � s ��zFileserver.updatec C s@ | � |�}i }|D ]}|� d�}|| jv r | j| � ||< q |S )z� Return the update intervals for all of the enabled fileserver backends which support variable update intervals. z.update_intervals�r� rn )ro r� r] r� r� r r r �update_intervals� s �zFileserver.update_intervalsFc C s� | � |�}t� }|r i }|D ]:}|� d�}dt| j| �jv r*| jd dkr*ddini }|r;| j| di |��||< q|�| j| di |��� q|rM|S t|�S )zO Return the environments for the named backend or all backends ry Z ignore_cacheZ__roleZminionTNr )r� �set�_argspecrn �argsr6 rd r{ )ro r� �sourcesr] r� r� r� r r r �envs� s" �� zFileserver.envsc C s6 |du ri }|� dd� | j|�dd�|�dd�d�S )zS Return environments for all backends for requests from fileclient N�cmdr� r� )r� r� )�popr� r- r� r r r � file_envs s zFileserver.file_envsc C s8 | � |�}|D ]}|� d�}|| jv r| j| � qdS )zX Initialize the backend, only do so if the fs supports an init function z.initNr� )ro r� r� r� r r r �init s ��zFileserver.initc C �0 |� d�}|s ddd�S |� dd�}| �||�S )zL Convenience function for calls made using the RemoteClient r rU �r �relrF �base�r- � find_file�ro r4 r Ztgt_envr r r � _find_file � zFileserver._find_filec C r� )zK Convenience function for calls made using the LocalClient r rU r� rF r� r� r� r r r � file_find$ r� zFileserver.file_findc C sL t jj�|�}t jj�|�}| �|�}i }t jj�|�r$t jj�|�}n,d|v rP|�d�}|d }|d �d�}|D ]}d|vrAq:|�dd�}|d ||d < q:ddd�} t j � |�s_d|v ra| S d |v rj|� d � d |v rs|� d �}t |t�s|t|�}|D ]%} | � d �} | | jv r�| j| ||fi |��} | �d �r�| | d < | S q~| S )z{ Find the path and return the fnd structure, this structure is passed to other backend interfaces. �?r r �&�=rU r� z../�envrF z .find_filer r� )r/ r0 � stringutils� to_unicoder� �urlZ is_escapedZunescaper| r r �isabsr� rz r~ rn r- ) ro r rF r� r� Zhcomps�comps�compr� �fndr� r� r r r r� . sB �zFileserver.find_filec C s� ddd�}d|v r|� d� d|vsd|vsd|vr |S t|d t�s+t|d �|d< | �|d |d �}|�d�s<|S d�|d �}|| jv rP| j| ||�S |S ) z, Serve up a chunk of a file rU )r= r r� r �locrF r� z {}.serve_file)r� rz r~ r� r- r rn )ro r4 r] r� r� r r r � serve_file\ s zFileserver.serve_filec C s� d|v r |� d� d|vsd|vrdS t|d t�s"t|d �|d< | �tjj�|d �|d �}|�d�s8dS |�dd�}d� |d �}|| j v rT| j | ||�|fS dS ) z; Common code for hashing and stating files r� r rF �rU Nr� r Nz {}.file_hash) r� rz r~ r� r/ r0 r� r� r- r rn )ro r4 r� � stat_resultr� r r r Z__file_hash_and_stats s � z Fileserver.__file_hash_and_statc C s* z| � |�d W S ttfy Y dS w )z1 Return the hash of a given file r rU �� _Fileserver__file_hash_and_stat� IndexError� TypeErrorr� r r r � file_hash� s �zFileserver.file_hashc C s& z| � |�W S ttfy Y dS w )zA Return the hash and stat result of a given file r� r� r� r r r �file_hash_and_stat� s �z Fileserver.file_hash_and_statc C s� d|v r |� d� |�dg �}|durNt|t�s:z dd� |�d�D �}W n ty9 dd� t|��d�D �}Y nw t|�D ]\}}t|t�sMt|�||< q>i }| �|� dd��}t j � | j d d �}zt � |�}W n+ ty� } z | jtjkr�t�d � i W Y d} ~ S t�d | � W Y d} ~ nd} ~ ww |D ]�} z t � t j � || ��} W n ty� } z t�d | | � W Y d} ~ q�d} ~ ww | D ]p} z | �dd�\} }W n ty� Y q�w |dkr�q�| |vs�|dur�| |vr�t�d| | � q�z t �t j � || | �� W n# t�y! } z| jtjk�rt�d| j| j� W Y d} ~ q�d} ~ ww |�| g ��| � t�d| | � q�q�|D ] }|| �� �q6|S )z4 Deletes the file_lists cache files r� rF Nc S � g | ]}|� � �qS r ��stripru r r r � <listcomp>� � z4Fileserver.clear_file_list_cache.<locals>.<listcomp>rq c S r� r r� ru r r r r� � r� � fsbackendZcachedirZ file_listszNo file list caches foundzMFailed to get list of saltenvs for which the master has cached file lists: %sz4Failed to find file list caches for saltenv '%s': %srS r �pz,Skipping %s file list cache for saltenv '%s'zFailed to remove %s: %sz+Removed %s file list cache for saltenv '%s') r� r- rz r{ r| r} r~ � enumerater� r r rB r6 rV r r �ENOENTr r, �errorrY r r� rO �strerror� setdefaultr� �sort)ro r4 rF �idx�valr] r� Z list_cachedirZfile_list_backendsr r� Z cache_filesZ cache_fileZ cache_saltenv� extension�keyr r r �clear_file_list_cache� s� � � ��� � �� ������ z Fileserver.clear_file_list_cachec � d|v r |� d� t� }d|vrg S t|d t�s!t|d �|d< | �|� dd��D ]}|� d�}|| jv r@|�| j| |�� q*|�dd��d�� � dkrW� fd d �|D �}t |�S ) zF Return a list of files from the dominant environment r� rF r� Nz .file_list�prefixrU �/c � g | ] }|� � �r|�qS r rs �rv �f�r� r r r� � z(Fileserver.file_list.<locals>.<listcomp>� r� r� rz r~ r� rn rd r- r� rN �ro r4 r] r� r� r r� r � file_list� � �zFileserver.file_listc s� d|v r |� d� t� }d|vrg S t|d t�s!t|d �|d< | �d�D ]}|� d�}|| jv r<|�| j| |�� q&|�dd��d�� � dkrS� fdd �|D �}t |�S ) z= List all emptydirs in the given environment r� rF Nz.file_list_emptydirsr� rU r� c r� r rs r� r� r r r� r� z2Fileserver.file_list_emptydirs.<locals>.<listcomp>r� r� r r� r �file_list_emptydirs s �z Fileserver.file_list_emptydirsc r� ) z? List all directories in the given environment r� rF r� Nz .dir_listr� rU r� c r� r rs r� r� r r r� 6 r� z'Fileserver.dir_list.<locals>.<listcomp>r� r� r r� r �dir_list r� zFileserver.dir_listc s� d|v r |� d� i }d|vri S t|d t�s t|d �|d< | �|� dd��D ]}|� d�}|| jv r<| j| |�}q)|�dd��d�� � dkrU� fd d �|�� D �}|S ) z; Return a list of symlinked files and dirs r� rF r� Nz .symlink_listr� rU r� c s i | ] \}}|� � �r||�qS r rs )rv rw �yr� r r � <dictcomp>O s z+Fileserver.symlink_list.<locals>.<dictcomp>)r� rz r~ r� rn r- r� r@ )ro r4 r] r� Zsymlstrr r� r � symlink_list9 s �zFileserver.symlink_list)N)NN)NF) �__name__� __module__� __qualname__�__doc__rp r� r� r� r� r� r� rj rd r� r� r� r� r� r� r� r� r� r� r� r� r r� r� r� r� r r r r rk C s> . . Q rk c @ s, e Zd ZdZdd� Z d dd�Zdd � ZdS ) �FSChanz} A class that mimics the transport channels allowing for local access to to the fileserver class class structure c K sn || _ || _t| j �| _| j�� | j �dd�dkr*d| j vr)| j�� d| j d< n| j�� i i d�| _d S )NZ file_clientrg �localZ __fs_updateT)Z master_topsZ ext_nodes)r6 r� rk �fsr� r- rd �cmd_stub)ro r6 r� r r r rp Y s � zFSChan.__init__NFc C sf d|vr t �d|� i S |d �d�}|| jv r | j| S t| j|�s+t �d|� i S t| j|�|�S )zU Emulate the channel send method, the tries and timeout are not used r� z Malformed request, no cmd: %srI z"Malformed request, invalid cmd: %s)r r� �lstripr� r� r� �getattr)ro r4 Ztriesr! �rawr� r r r �sendf s z FSChan.sendc C s< t | d�r| jd ur t | jd�r| j�� d | _d S d S d S )Nr� r� )r� r� r� r� r r r �closew r� z FSChan.close)NNF)r� r� r� r� rp r� r� r r r r r� S s � r� )r )Nrd )$r� r ra �loggingr r_ r �collections.abcr Z salt.loaderr/ Zsalt.utils.filesZsalt.utils.pathZsalt.utils.urlZsalt.utils.versionsZsalt.utils.argsr r� Zsalt.utils.decoratorsr � getLoggerr� r r r r# r: r> r? rM rQ r^ rC rj rk r� r r r r �<module> sB .C '
| ver. 1.4 |
Github
|
.
| PHP 8.2.30 | Generation time: 0 |
proxy
|
phpinfo
|
Settings