File: //usr/lib/python3/dist-packages/uaclient/http/__pycache__/__init__.cpython-38.pyc
U
��Jhi; � @ sx d dl Zd dlZd dlZd dlZd dlZd dlZd dlZd dl Z d dl
m Z m Z m
Z
mZmZmZ d dlmZmZ d dlmZmZ d dlmZmZmZmZ dZ dZ dZ d Z d
Z e�!e�"e#��Z$ed d e%fd
e e&e&f fde'fg�Z(edd e%fd
e e&e&f fde&fde e&e f fde
e fg�Z)e&e*d�dd�Z+e&ee& e&ee& d�dd�Z,i a-ee& ee& dd�dd�Z.e e&e&f d�d d �Z/ej0j1e e&e&f d �d d �Z2d;ej3ee% e(d!�d"d#�Z4d$d%� Z5d&d'� Z6d<ej3ee% ee& e(d(�d)d*�Z7ee d�d+d,�Z8e&d-�d.d/�Z9d=e&ee% ee'e&f d0�d1d2�Z:di ddd3fe&ee' e e&e&f ee& ee% e*e)d4�d5d6�Z;d>e&e&e&e&e)d8�d9d:�Z<dS )?� N)�Any�Dict�List�
NamedTuple�Optional�Tuple)�error�request)� ParseResult�urlparse)�defaults�
exceptions�system�util)z169.254.169.254Zmetadataz[fd00:ec2::254]zhttp://archive.ubuntu.comzhttps://esm.ubuntu.comzhttp://api.snapcraft.iozhttps://api.snapcraft.io�UnparsedHTTPResponse�code�headers�body� HTTPResponse� json_dict� json_list)�url�returnc C sX z t | �}W n tk
r" Y dS X |jdkr2dS z
|j W n tk
rR Y dS X dS )NF)�https�httpT)r �
ValueError�schemeZport)r Z
parsed_url� r �8/usr/lib/python3/dist-packages/uaclient/http/__init__.py�is_service_url+ s
r )�protocol�proxy�test_urlr c
C sr |sd S t |�s tj|d��tj|dd�}| dkr�t|�jdkr�zt||d�}W n� tjk
rh � Y nn tj k
r~ � Y nX tj
k
r� � Y nB t k
r� } z$t �
d||t|�� tj|d��W 5 d }~X Y nX |jdkr�|S tj|d��t�| |i�}t�|�}z|�|� |W S tjt
jfk
�rl } z,t �
d||t|d t|��� tj|d��W 5 d }~X Y nX d S )
N)r! �HEAD)�methodr �� https_proxyz:Error trying to use "%s" as pycurl proxy to reach "%s": %s�� z:Error trying to use "%s" as urllib proxy to reach "%s": %s�reason)r r
ZProxyInvalidUrlr �Requestr r � _readurl_pycurl_https_in_https�PycurlRequiredError�ProxyAuthenticationFailed�PycurlCACertificatesError� Exception�LOGr �strZProxyNotWorkingErrorr � ProxyHandler� build_opener�open�socket�timeout�URLError�getattr)r r! r" �req�response�e�
proxy_handler�openerr r r �validate_proxy< sL �
�r= )�
http_proxyr& r c C s� i }| r| |d< |r ||d< d� tt��}dD ]6}tj�|�}|r.d� tt|�d���tt����}q.t �
d|� |tjd< |tjd< |r�t � |�}t �
|�}t �|� t j
dd |id
� |ad S ) aW
Globally configure pro-client to use http and https proxies.
- sets global proxy configuration for urllib
- sets the no_proxy environment variable for the current process
which gets inherited for all subprocesses
- sets module variable for use in https-in-https pycurl requests
this is retrieved later using get_configured_web_proxy
:param http_proxy: http proxy to be used by urllib. If None, it will
not be configured
:param https_proxy: https proxy to be used by urllib. If None, it will
not be configured
r r �,)�no_proxy�NO_PROXYzSetting no_proxy: %sr@ rA zSetting global proxy dict�extra)rB N)�join�sorted�UA_NO_PROXY_URLS�os�environ�get�set�split�unionr/ �debugr r1 r2 Zinstall_opener�_global_proxy_dict)r> r& Z
proxy_dictr@ Zenv_varZ proxy_valuer; r< r r r �configure_web_proxyq s. ��
rN )r c C s t S )N)rM r r r r �get_configured_web_proxy� s rO )r r c C s dd� | � � D �S )Nc S s i | ]\}}|� � |�qS r )�lower)�.0�k�vr r r �
<dictcomp>� s z$_headers_to_dict.<locals>.<dictcomp>)�items�r r r r �_headers_to_dict� s rW )r8 r5 r c
C s� zt j| |d�}W nj tjk
r: } z|}W 5 d }~X Y nD tjk
r| } z$t�t|j�� t j
|| j d��W 5 d }~X Y nX |� � }t
|jt|j�|d�S )N�r5 )�causer �r r r )r �urlopenr � HTTPErrorr6 r/ Z exceptionr0 r( r
ZConnectivityErrorZfull_url�readr r rW r )r8 r5 �respr: r r r r �_readurl_urllib� s "�r_ c C sJ t |�}t| �}|jdko8t�|j� o8|dk o8|jdk}t�d|� |S )a�
We only want to use pycurl if all of the following are true
- The target url scheme is https
- The target host is not in no_proxy
- An https_proxy is configured either via pro's config or via environment
- The https_proxy url scheme is https
urllib.request provides some helpful functions that we re-use here.
This function also returns the https_proxy to use, since it is calculated
here anyway.
r NzShould use pycurl: %r)r �_parse_https_proxyr r Z proxy_bypassZhostnamer/ rL )r& Z
target_urlZparsed_target_url�parsed_https_proxyZretr r r �should_use_pycurl� s
��� rb c C s| d }d }t | j�dkr | jd }t | j�dkr8| jd }||krV|rVd|krVt�� �n"||krltj|d��n tj| d��d S )Nr � Z407�r )r: )�len�argsr
r, r- Z PycurlError)r r �authentication_error_code�ca_certificates_error_coder �msgr r r �_handle_pycurl_error� s
rj )r8 r5 r& r c
s z dd l }W n tk
r( t�� �Y nX |�� }| �� �� }|dkrV|�|jd� nT|dkrn|�|j d� n<|dkr�|�|j
d� | j r�|�|j | j � nt
d�|���|�|j| �� � dd� | �� D �}t|�dkr�|�|j|� |�|jd� |�|jtj� |�r|�|j|� |�rXt|�}|�r6|�� nd }|�|j|� |�|j d � n
t � d
� t � � }|�|j!|� i � � fd d �} |�|j"| � z |�#� W n> |j$k
�r� }
z t%|
| �� |j&|j'd
� W 5 d }
~
X Y nX t(|�)|j*��} |�+� } |�,� t-| � | d�S )Nr �GETTr# �POSTz5HTTP method "{}" not supported in HTTPS-in-HTTPS modec S s g | ]\}}d � ||��qS )z{}: {}��format)rQ �name�valr r r �
<listcomp> s z2_readurl_pycurl_https_in_https.<locals>.<listcomp>� z1in pycurl request function without an https proxyc sF | � d�} d| krd S | �dd�\}}|�� �� }|�� }|� |< d S )Nz
iso-8859-1�:rc )�decoderJ �striprP )Z header_lineZname_rawZ value_rawro �valuerV r r � save_header( s
z3_readurl_pycurl_https_in_https.<locals>.save_header)r rg rh rZ ).�pycurl� ImportErrorr
r+ ZCurlZ
get_method�upperZsetoptZHTTPGETZNOBODYrl �dataZCOPYPOSTFIELDSr rn ZURLZ get_full_urlZ header_itemsre Z
HTTPHEADERZFOLLOWLOCATIONZCAINFOr ZSSL_CERTS_PATHZTIMEOUTr` ZgeturlZPROXYZ PROXYTYPEr/ �warning�io�BytesIOZ WRITEDATAZHEADERFUNCTIONZperformr rj Z E_RECV_ERRORZE_SSL_CACERT_BADFILE�intZgetinfoZ
RESPONSE_CODE�getvalue�closer )
r8 r5 r& rx �cr$ Zheader_str_listra Z body_outputrw r: r r r rV r r* � st ��� �
��r* c C s" | st �� �d�} | r t| �S d S )Nr )r Z
getproxiesrH r r% r r r r` J s r` rd c C sH | j �d�}i }|si }n"tj�|�s,i }nt�t�|��}|�|g �S )NZserviceclient_url_responses) ZfeaturesrH rF �path�exists�json�loadsr Z load_file)�cfgr Zresponse_overlay_pathZresponse_overlayr r r �_get_overlay_dataP s r� )r r5 r c C s� t | |�}|rJ|�d�}t�|d d ��}|�� dfW 5 Q R � S Q R X t|�s^tj|d��t� d�
|�� t � � d�}i }|r�||d< t
||�r�ttj||d �||d
�}|jd kr�tj|d��t�|j�|j� d �fS tj||d �} zZt�| ��F}t�|��0}|�� |j� d
�fW 5 Q R � W 5 Q R � W S Q R X W 5 Q R X W nV tjk
�r� }
z4|
jd k�rptj|d��|
jdk�r�t�� �n� W 5 d }
~
X Y nX d S )Nr r9 Z file_path� rd z
URL [GET]: {}r z
If-None-MatchrV �r5 r& i0 �etagZETagi� )r� �pop�lzmar3 r] r r
�
InvalidUrlr/ rL rn rO rH rb r* r r) r Z
ETagUnchanged�
decompressr r r[ r r\ ZVulnerabilityDataNotFound) r� r r5 r� Zoverlay_responser9 �fr& r r8 r: r r r �download_xz_file_from_url^ sH
�
�
�2
r� T)r r{ r r$ r5 �log_response_bodyr c
s� t | �stj| d��|r |s d}tj| |� |d�}d�� fdd�t� �D ��}t�d� |p\d| ||rn|�
d �nd �� t � � d
�}t
|| �r�t|||d ��n t||d ���jj
d d
d�} i }
g } d�j� dd�k�rtj| tjd�} t| t�r�| }
n"t| t��r| } nt�dt| �� d��fdd�t�j�D ��}d� |�p@d| |�}
|�rx�j}|
�r`|
}n
| �rj| }|
d� |�7 }
t�|
� t�j �j| |
| d�S )Nrd rl )r{ r r$ z, c s g | ]}d � |� | ��qS �z
'{}': '{}'rm �rQ rR rV r r rq � s zreadurl.<locals>.<listcomp>z'URL [{}]: {}, headers: {{{}}}, data: {}rk �utf-8r r� rX �ignore��errors�application/json� content-typer� ��cls� unexpected JSON response: %sc s g | ]}d � |� j| ��qS r� )rn r r� )r^ r r rq � s z&URL [{}] response: {}, headers: {{{}}}z
, data: {}�r r r r r ) r r
r� r r) rC rD r/ rL rn rt rO rH rb r* r_ r r r� r� r �DatetimeAwareJSONDecoder�
isinstance�dict�listr| r0 r r )r r{ r r$ r5 r� r8 Zsorted_header_strr& Z decoded_bodyr r � json_bodyZ debug_msgZ body_to_logr )r r^ r �readurl� sr ���
�
� �
�r� � localhost)� socket_path� http_method� http_path�
http_hostnamer c C s� t � t jt j�}|�| � tj�|�}||_z*|� ||� |�
� }|� � j ddd�}W 5 |�� |�� X i }g } d|j
�dd�kr�tj|tjd�}
t|
t�r�|
}n t|
t�r�|
} nt�dt|
�� t|jt|j
�||| d �S )
Nr� r� r� r� r� r� r� r� r� ) r4 ZAF_UNIXZ SOCK_STREAMZconnectr ZclientZHTTPConnection�sockr� r Z getresponser] rt r rH r� r� r r� r� r� r� r/ r| r0 r ZstatusrW ) r� r� r� r� r� Zconnr^ �outr r r� r r r �unix_socket_request� s4
�r� )N)NN)NN)r� )=Z
email.messageZemailZ http.clientr r} r� Zloggingr� rF r4 �typingr r r r r r Zurllibr r Z urllib.parser
r Zuaclientr r
r r rE Z PROXY_VALIDATION_APT_HTTP_URLZ PROXY_VALIDATION_APT_HTTPS_URLZ PROXY_VALIDATION_SNAP_HTTP_URLZ PROXY_VALIDATION_SNAP_HTTPS_URLZ getLoggerZ replace_top_level_logger_name�__name__r/ r r0 �bytesr r �boolr r= rM rN rO �messageZMessagerW r) r_ rb rj r* r` r� r� r� r� r r r r �<module> s� ��
�� � 2 � 1 �� �� _ �
� 6�
� M ��