File manager - Edit - /opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/index/__pycache__/collector.cpython-310.pyc
Back
o ;j9? � @ sN d Z ddlmZ ddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddlmZmZmZ ddlmZ ddlmZ ddlmZ ddlmZmZmZ dd l m Z dd l m Z dd l m!Z!m"Z" dd l#m$Z$ dd l%m&Z& ddl'm(Z( ddl)m*Z* ddl+m,Z, ddl-m.Z. ddl/m0Z0 ddl1m2Z2 ddl3m4Z4m5Z5m6Z6 e �7e8�Z9ee:e:f Z;dSdd�Z<G d d � d e=�Z>dTd!d"�Z?G d#d$� d$e=�Z@dUd'd(�ZAdVd)d*�ZBdWd-d.�ZCG d/d0� d0�ZDG d1d2� d2e�ZEdXd4d5�ZFeFdYd9d:��ZGed;d<�G d=d7� d7��ZHG d>d?� d?e�ZI dZd[dFdG�ZJ ;d\d]dJdK�ZKd^dMdN�ZLG dOdP� dPe�ZMG dQdR� dR�ZNdS )_zO The main purpose of this module is to expose LinkCollector.collect_sources(). � )� annotationsN)�Iterable�MutableMapping�Sequence)� dataclass)� HTMLParser)�Values)�Callable� NamedTuple�Protocol)�requests)�Response)� RetryError�SSLError)�NetworkConnectionError)�Link)� SearchScope)� PipSession)�raise_for_status)�is_archive_file��redact_auth_from_url)�vcs� )�CandidatesFromPage� LinkSource� build_source�url�str�return� str | Nonec C s6 t jD ]}| �� �|�r| t|� dv r| S qdS )zgLook for VCS schemes in the URL. Returns the matched VCS scheme, or None if there's no match. z+:N)r Zschemes�lower� startswith�len)r �scheme� r% �Q/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/index/collector.py�_match_vcs_scheme. s �r' c s e Zd Zd� fdd� Z� ZS ) �_NotAPIContent� content_typer � request_descr �Nonec s t � �||� || _|| _d S �N)�super�__init__r) r* )�selfr) r* �� __class__r% r& r. : s z_NotAPIContent.__init__)r) r r* r r r+ )�__name__� __module__� __qualname__r. � __classcell__r% r% r0 r& r( 9 s r( �responser r+ c C s2 | j �dd�}|�� }|�d�rdS t|| jj��)z� Check the Content-Type header to ensure the response contains a Simple API Response. Raises `_NotAPIContent` if the content type is not a valid content-type. � Content-Type�Unknown)z text/htmlz#application/vnd.pypi.simple.v1+html�#application/vnd.pypi.simple.v1+jsonN)�headers�getr! r" r( �request�method)r6 r) �content_type_lr% r% r& �_ensure_api_header@ s �r? c @ s e Zd ZdS )�_NotHTTPN)r2 r3 r4 r% r% r% r& r@ V s r@ �sessionr c C sF t j�| �\}}}}}|dvrt� �|j| dd�}t|� t|� dS )z� Send a HEAD request to the URL, and ensure the response contains a simple API Response. Raises `_NotHTTP` if the URL is not available for a HEAD request, or `_NotAPIContent` if the content type is not a valid content type. > �http�httpsT)Zallow_redirectsN)�urllib�parse�urlsplitr@ �headr r? )r rA r$ �netloc�path�query�fragment�respr% r% r& �_ensure_api_responseZ s rM c C sx t t| �j�r t| |d� t�dt| �� |j| d�g d��dd�d�}t |� t |� t�dt| �|j �d d �� |S ) aY Access an Simple API response with GET, and return the response. This consists of three parts: 1. If the URL looks suspiciously like an archive, send a HEAD first to check the Content-Type is HTML or Simple API, to avoid downloading a large file. Raise `_NotHTTP` if the content type cannot be determined, or `_NotAPIContent` if it is not HTML or a Simple API. 2. Actually perform the request. Raise HTTP exceptions on network failures. 3. Check the Content-Type header to make sure we got a Simple API response, and raise `_NotAPIContent` otherwise. �rA zGetting page %sz, )r9 z*application/vnd.pypi.simple.v1+html; q=0.1ztext/html; q=0.01z max-age=0)ZAcceptz Cache-Control)r: zFetched page %s as %sr7 r8 ) r r �filenamerM �logger�debugr r; �joinr r? r: )r rA rL r% r% r& �_get_simple_responsel s& ��� �rS r: �ResponseHeadersc C s< | r d| v r t j�� }| d |d< |�d�}|r t|�S dS )z=Determine if we have any encoding information in our headers.r7 z content-type�charsetN)�email�message�Message� get_paramr )r: �mrU r% r% r& �_get_encoding_from_headers� s r[ c @ s* e Zd Zddd�Zdd d �Zdd d�ZdS )�CacheablePageContent�page� IndexContentr r+ c C s |j sJ �|| _d S r, )�cache_link_parsingr] �r/ r] r% r% r& r. � s z CacheablePageContent.__init__�other�object�boolc C s t |t| ��o| jj|jjkS r, )� isinstance�typer] r )r/ ra r% r% r& �__eq__� s zCacheablePageContent.__eq__�intc C s t | jj�S r, )�hashr] r �r/ r% r% r& �__hash__� s z CacheablePageContent.__hash__N)r] r^ r r+ )ra rb r rc )r rg )r2 r3 r4 r. rf rj r% r% r% r& r\ � s r\ c @ s e Zd Zddd�ZdS ) � ParseLinksr] r^ r �Iterable[Link]c C s d S r, r% r` r% r% r& �__call__� s zParseLinks.__call__N�r] r^ r rl )r2 r3 r4 rm r% r% r% r& rk � s rk �fnc s2 t jd � fdd� ��t �� �d � �fd d � �}|S )z� Given a function that parses an Iterable[Link] from an IndexContent, cache the function's result (keyed by CacheablePageContent), unless the IndexContent `page` has `page.cache_link_parsing == False`. �cacheable_pager\ r � list[Link]c s t � | j��S r, )�listr] )rp )ro r% r& �wrapper� s z*with_cached_index_content.<locals>.wrapperr] r^ c s | j r �t| ��S t� | ��S r, )r_ r\ rr )r] �ro rs r% r& �wrapper_wrapper� s z2with_cached_index_content.<locals>.wrapper_wrapperN)rp r\ r rq )r] r^ r rq )� functools�cache�wraps)ro ru r% rt r& �with_cached_index_content� s ry r] r^ rl c c s� � | j �� }|�d�r+t�| j�}|�dg �D ]}t�|| j �}|du r%q|V qdS t | j �}| j p4d}|� | j� |�� | j }|jpE|}|jD ]} tj| ||d�}|du rXqI|V qIdS )z\ Parse a Simple API's Index Content, and yield its anchor elements as Link objects. r9 �filesNzutf-8)Zpage_url�base_url)r) r! r" �json�loads�contentr; r Z from_jsonr �HTMLLinkParser�encoding�feed�decoder{ �anchorsZ from_element) r] r> �data�file�link�parserr� r r{ �anchorr% r% r& � parse_links� s* � �r� T)�frozenc @ sH e Zd ZU dZded< ded< ded< ded< d Zd ed < dd d�ZdS )r^ a� Represents one response (or page), along with its URL. :param encoding: the encoding to decode the given content. :param url: the URL from which the HTML was downloaded. :param cache_link_parsing: whether links parsed from this page's url should be cached. PyPI index urls should have this set to False, for example. �bytesr~ r r) r r� r Trc r_ r c C s t | j�S r, )r r ri r% r% r& �__str__ s zIndexContent.__str__N)r r )r2 r3 r4 �__doc__�__annotations__r_ r� r% r% r% r& r^ � s c s6 e Zd ZdZd� fdd� Zdd d �Zddd�Z� ZS )r zf HTMLParser that keeps the first base HREF and a list of all anchor elements' attributes. r r r r+ c s$ t � jdd� || _d | _g | _d S )NT)Zconvert_charrefs)r- r. r r{ r� )r/ r r0 r% r& r. s zHTMLLinkParser.__init__�tag�attrs� list[tuple[str, str | None]]c C sR |dkr| j d u r| �|�}|d ur|| _ d S d S |dkr'| j�t|�� d S d S )N�base�a)r{ �get_hrefr� �append�dict)r/ r� r� �hrefr% r% r& �handle_starttag s ��z HTMLLinkParser.handle_starttagr c C s" |D ] \}}|dkr| S qd S )Nr� r% )r/ r� �name�valuer% r% r& r� ! s �zHTMLLinkParser.get_href)r r r r+ )r� r r� r� r r+ )r� r� r r )r2 r3 r4 r� r. r� r� r5 r% r% r0 r& r s r r� r �reason�str | Exception�meth�Callable[..., None] | Nonec C s |d u rt j}|d| |� d S )Nz%Could not fetch URL %s: %s - skipping)rP rQ )r� r� r� r% r% r& �_handle_get_simple_fail( s r� r_ rc c C s& t | j�}t| j| jd || j|d�S )Nr7 )r� r r_ )r[ r: r^ r~ r )r6 r_ r� r% r% r& �_make_index_content2 s �r� �IndexContent | Nonec C s� | j �dd�d }t|�}|rt�d|| � d S tj�|�\}}}}}}|dkrHtj � tj � |��rH|� d�s;|d7 }tj�|d�}t�d|� zt||d �}W n� ty` t�d | � Y d S ty| } zt�d | |j|j� W Y d }~d S d }~w ty� } z t| |� W Y d }~d S d }~w ty� } z t| |� W Y d }~d S d }~w ty� } zd } | t|�7 } t| | tjd � W Y d }~d S d }~w tj y� } zt| d|� �� W Y d }~d S d }~w tj y� t| d� Y d S w t || j d�S )N�#r r zICannot look at %s URL %s because it does not support lookup as web pages.r� �/z index.htmlz# file: URL is directory, getting %srN z`Skipping page %s because it looks like an archive, and cannot be checked by a HTTP HEAD request.z�Skipping page %s because the %s request got Content-Type: %s. The only supported Content-Types are application/vnd.pypi.simple.v1+json, application/vnd.pypi.simple.v1+html, and text/htmlz4There was a problem confirming the ssl certificate: )r� zconnection error: z timed out)r_ ) r �splitr' rP �warningrD rE �urlparse�osrI �isdirr< Z url2pathname�endswith�urljoinrQ rS r@ r( r* r) r r� r r r �infor �ConnectionErrorZTimeoutr� r_ ) r� rA r Z vcs_schemer$ �_rI rL �excr� r% r% r& �_get_index_content? sn � � ���� �� �� ���� �r� c @ s e Zd ZU ded< ded< dS )�CollectedSourceszSequence[LinkSource | None]� find_links� index_urlsN)r2 r3 r4 r� r% r% r% r& r� | s r� c @ sN e Zd ZdZd!dd �Ze d"d#dd��Zed$dd��Zd%dd�Z d&d d �Z d S )'� LinkCollectorz� Responsible for collecting Link objects from all configured locations, making network requests as needed. The class's main method is its collect_sources() method. rA r � search_scoper r r+ c C s || _ || _d S r, )r� rA )r/ rA r� r% r% r& r. � s zLinkCollector.__init__F�optionsr �suppress_no_indexrc c C sd |j g|j }|jr |s t�dd�dd� |D ��� g }|jp g }tj|||jd�}t ||d�}|S )z� :param session: The Session to use to make requests. :param suppress_no_index: Whether to ignore the --no-index option when constructing the SearchScope object. zIgnoring indexes: %s�,c s s � | ]}t |�V qd S r, r )�.0r r% r% r& � <genexpr>� s � z'LinkCollector.create.<locals>.<genexpr>)r� r� �no_index)rA r� ) Z index_urlZextra_index_urlsr� rP rQ rR r� r �creater� )�clsrA r� r� r� r� r� Zlink_collectorr% r% r& r� � s$ � ��zLinkCollector.create� list[str]c C s | j jS r, )r� r� ri r% r% r& r� � s zLinkCollector.find_links�locationr r� c C s t || jd�S )z> Fetch an HTML page containing package links. rN )r� rA )r/ r� r% r% r& �fetch_response� s z LinkCollector.fetch_response� project_namer �candidates_from_pager r� c s� t �� ��fdd��j���D ���� }t �� ��fdd��jD ���� }t�tj �rKdd� t � ||�D �}t |�� d�� d�g| }t� d�|�� tt|�t|�d �S ) Nc 3 �( � | ]}t |� �jjd d �d�V qdS )F�r� Zpage_validatorZ expand_dirr_ r� N�r rA Zis_secure_origin�r� �loc�r� r� r/ r% r& r� � � � �� �z0LinkCollector.collect_sources.<locals>.<genexpr>c 3 r� )Tr� Nr� r� r� r% r& r� � r� c S s* g | ]}|d ur|j d urd|j � ��qS )Nz* )r� )r� �sr% r% r& � <listcomp>� s �z1LinkCollector.collect_sources.<locals>.<listcomp>z' location(s) to search for versions of �:� )r� r� )� collections� OrderedDictr� Zget_index_urls_locations�valuesr� rP � isEnabledFor�logging�DEBUG� itertools�chainr# rQ rR r� rr )r/ r� r� Zindex_url_sourcesZfind_links_sources�linesr% r� r& �collect_sources� s2 � � � � � ����z LinkCollector.collect_sourcesN)rA r r� r r r+ )F)rA r r� r r� rc r r� )r r� )r� r r r� )r� r r� r r r� ) r2 r3 r4 r� r. � classmethodr� �propertyr� r� r� r% r% r% r& r� � s �! r� )r r r r )r6 r r r+ )r r rA r r r+ )r r rA r r r )r: rT r r )ro rk r rk rn r, )r� r r� r� r� r� r r+ )T)r6 r r_ rc r r^ )r� r rA r r r� )Or� � __future__r r� � email.messagerV rv r� r| r� r� � urllib.parserD �urllib.request�collections.abcr r r Z dataclassesr Z html.parserr �optparser �typingr r r Z pip._vendorr Zpip._vendor.requestsr Z pip._vendor.requests.exceptionsr r Zpip._internal.exceptionsr Zpip._internal.models.linkr Z!pip._internal.models.search_scoper Z pip._internal.network.sessionr Zpip._internal.network.utilsr Z pip._internal.utils.filetypesr Zpip._internal.utils.miscr Zpip._internal.vcsr Zsourcesr r r � getLoggerr2 rP r rT r' � Exceptionr( r? r@ rM rS r[ r\ rk ry r� r^ r r� r� r� r� r� r% r% r% r& �<module> sh ? � � =
| ver. 1.4 |
Github
|
.
| PHP 8.2.30 | Generation time: 0 |
proxy
|
phpinfo
|
Settings