File manager - Edit - /opt/saltstack/salt/lib/python3.10/site-packages/libcloud/compute/drivers/__pycache__/kubevirt.cpython-310.pyc
Back
o ;j ? � @ s d Z ddlZddlZddlZddlZddlZddlZddlmZm Z ddl m Z ddl m Z ddl mZmZmZmZmZmZmZmZ ddlmZmZ ddlmZmZm Z g d �Z d Z d Z h d �Z G d d� dee�Z!de"de"de"fdd�Z#dd� Z$ ddd�Z%dd� Z&dS )z. kubevirt driver with support for nodes (vms) � N)�Union�Optional)�datetime)� LibcloudError)�Node�NodeSize� NodeImage� NodeDriver� NodeLocation� StorageVolume�NodeAuthSSHKey�NodeAuthPassword)�Provider� NodeState)�VALID_RESPONSE_CODES�KubernetesDriverMixin� KubernetesBasicAuthConnection)�KubeVirtNodeDriver� DISK_TYPES�KubeVirtNodeSize�KubeVirtNodeImagez/api/v1/z/apis/kubevirt.io/v1alpha3/> ZhostDiskZ emptyDisk�cloudInitConfigDrive� containerDiskZ dataVolume�cloudInitNoCloudZ ephemeral�persistentVolumeClaimZ configMapc @ s� e Zd ZejZdZdZeZ dddgiZ e j e j e jd�ZdWdd �ZdXd d �Zd d � Zdd� Zdd� Zdd� ZdYdedefdd�ZedWdd��Zedd � �Ze dZd d ��Z ed d � �Z ed!d"� �Z ed#d$� �Z d%d&� Z ed'd(� �Z! ) d[d*d+�Z"dWd,d-�Z#d.d/� Z$dWd0d1�Z% 2 3 4 5 6 d\d7d8�Z& 3 4d]d9d:�Z'dYd;d<�Z(d=d>� Z)d^dAdB�Z*dCdD� Z+dYdEdF�Z,dGdH� Z-dIdJ� Z.dKdL� Z/d_dNdO�Z0d`dPdQ�Z1 R MdadSdT�Z2dUdV� Z3dS )br Zkubevirtzhttps://www.kubevirt.io� create_nodeZssh_key�password)�pending�running�stoppedNc C s g }|d ur!t |t�r|�|j� n t |t�r |�|� ntd��| �� D ]}|�|j� q%g }g }|D ]1}td | d }| j� |�}|j dkrJq4|j }|d D ]}|d d s_|�|� qQ|�|� qQq4g } |D ] } | �| j | dd �� qj|D ] } | �| j | d d �� qy| S ) Nz+location must be a NodeLocation or a string� namespaces/z/virtualmachines�� �items�specr T)� is_stoppedF) � isinstancer �append�name�str� ValueError�list_locations� KUBEVIRT_URL� connection�request�status�object�_to_node) �self�location� namespaces�nsZdormantZlive�req�result�itemZvms�vm� r9 �U/opt/saltstack/salt/lib/python3.10/site-packages/libcloud/compute/drivers/kubevirt.py� list_nodesV s8 �z KubeVirtNodeDriver.list_nodesc sv � s�st d��| �� }d}� rt� fdd�|�}�r$t�fdd�|�}|s*t d��zt|�W S ty: t d��w )z�get a vm by name or id. :param id: id of the vm :type id: ``str`` :param name: name of the vm :type name: ``str`` z,This method needs id or name to be specifiedNc � | j � kS �N��id��xr> r9 r: �<lambda>� � z-KubeVirtNodeDriver.get_node.<locals>.<lambda>c r< r= �r' r@ rD r9 r: rB � rC znode_gen is not definedzNode does not exist)r) r; �filter�next� StopIteration)r1 r? r' �nodesZnode_genr9 )r? r' r: �get_node~ s �zKubeVirtNodeDriver.get_nodec C s� |j tju rdS |j}|jd }td | d | }dddii}ddi}z| jj|d t� |�|d �}|j t v W S t y? � w ) z�Starting a VM. :param node: The node to be started. :type node: :class:`Node` :return: True if the start was successful, False otherwise. :rtype: ``bool`` T� namespacer �/virtualmachines/r# r � Content-Type� application/merge-patch+json�PATCH��method�data�headers) �stater �RUNNINGr' �extrar+ r, r- �json�dumpsr. r � Exception)r1 �noder' rJ r5 rQ rR r6 r9 r9 r: � start_node� s � �z KubeVirtNodeDriver.start_nodec C s� |j tju rdS |j}|jd }td | d | }ddi}ddd ii}z| jj|d t� |�|d �}|j t v W S t y? � w ) z�Stopping a VM. :param node: The node to be stopped. :type node: :class:`Node` :return: True if the stop was successful, False otherwise. :rtype: ``bool`` TrJ r rK rL rM r# r FrN rO ) rS r �STOPPEDr' rU r+ r, r- rV rW r. r rX )r1 rY r' rJ r5 rR rQ r6 r9 r9 r: � stop_node� s � �z KubeVirtNodeDriver.stop_nodec C sR |j d }|j}d}z| jjtd | d | |d�}|jtv W S ty( � w )z� Rebooting a node. :param node: The node to be rebooted. :type node: :class:`Node` :return: True if the reboot was successful, False otherwise. :rtype: ``bool`` rJ �DELETEr z/virtualmachineinstances/�rP )rU r' r, r- r+ r. r rX )r1 rY rJ r' rP r6 r9 r9 r: � reboot_node� s � �z KubeVirtNodeDriver.reboot_nodec C s� |j d }|j}| j||d�}|D ]}|d d }| j||d� q| �|� z| jjtd | d | dd �}|jt v W S t yD � w ) a Terminating a VMI and deleting the VM resource backing it. :param node: The node to be destroyed. :type node: :class:`Node` :return: True if the destruction was successful, False otherwise. :rtype: ``bool`` rJ �rJ � node_name�metadatar' �rJ � service_namer rK r] r^ ) rU r' �ex_list_services�ex_delete_servicer\ r, r- r+ r. r rX )r1 rY rJ r' �services�servicerd r6 r9 r9 r: � destroy_node� s � �z KubeVirtNodeDriver.destroy_node�defaultr' �templatec C s |� dd�dkr td��|� dd�dkrtd��||� di �� d �kr2td �||� di �� d ����|� d i �� d d �rGt�d� d |d d < |}d}t�|�}td | d }z | jj |||d� W n t yk � w | j |d�}|D ]} | j |kr�| � | � | S qttd��)a� Creating a VM defined by the template. The template must be a dictionary of kubernetes object that defines the KubeVirt VM. Following are the keys: - ``apiVersion``: ``str`` - ``kind``: ``str`` - ``metadata``: ``dict`` - ``spec``: ``dict`` - ``domain``: ``dict`` - ``volumes``: ``list`` - ... - ... See also: - https://kubernetes.io/docs/concepts/overview/working-with-objects/ - https://kubevirt.io/api-reference/ :param name: A name to give the VM. The VM will be identified by this name, and it must be the same as the name in the ``template["metadata"]["name"]``. Atm, it cannot be changed after it is set. :type name: ``str`` :param template: A dictionary of kubernetes object that defines the VM. :type template: ``dict`` with keys: ``apiVersion: str``, ``kind: str``, ``metadata: dict``, ``spec: dict`` etc. :param namespace: The namespace where the VM will live. (default is 'default') :return: � apiVersion� �kubevirt.io/v1alpha3z:The template must have an apiVersion: kubevirt.io/v1alpha3�kind�VirtualMachinez.The template must contain kind: VirtualMachinerb r' zeThe name of the VM must be the same as the name in the template. (name={}, template.metadata.name={})r# r FzoThe VM will be created in a stopped state, and then started. Ignoring the `spec.running: True` in the template.�POSTr rK �rP rQ )r2 zeThe node was not found after creation, create_node may have failed. Please check the kubernetes logs.)�getr) �format�warnings�warnrV rW r+ r, r- rX r; r' rZ ) r1 r' rk rJ r8 rP rQ r5 rH rY r9 r9 r: �_create_node_with_template sD &��� � ��z-KubeVirtNodeDriver._create_node_with_templatec C s\ | st �� } ddd| i| d�ddd| iig g dd�dd ii i d �d �g d g d �d�d�d�S )a� A skeleton VM template to be used for creating VMs. :param name: A name to give the VM. The VM will be identified by this name. If not provided, a random uuid4 will be used. The generated name can be gotten from the returned dict with the key ``["metadata"]["name"]``. :type name: ``str`` :return: dict: A skeleton VM template. rn rp �kubevirt.io/vm)�labelsr' Fry )�disks� interfacesZnetworkInterfaceMultiqueue�typerm )�requests�limits)�devices�machine� resourcesr )�domain�networks� terminationGracePeriodSeconds�volumes)rb r# )r rk �rl ro rb r# )�uuid�uuid4rD r9 r9 r: �_base_vm_templateb s* �� ����z$KubeVirtNodeDriver._base_vm_templatec s� t |t�s J d��|�d�|�d�|�d�|�d�|�d�|�d�|�d�d �� tt� fd d �� ��}|r=t�d �|�� t� |�}|�d �du rMi |d < |d �d�du r^| |d d< |S |d d | kryt�d�|d d | �� | |d d< |S )a A part of create_node that deals with the VM template. Returns the VM template with the name set. :param name: A name to give the VM. The VM will be identified by this name and atm it cannot be changed after it is set. See also the name parameter in create_node. :type name: ``str`` :param ex_template: A dictionary of kubernetes object that defines the KubeVirt VM. See also the ex_template parameter in create_node. :type ex_template: ``dict`` with keys: ``apiVersion: str``, ``kind: str``, ``metadata: dict`` and ``spec: dict`` :param other_args: Other parameters passed to the create_node method. This is used to warn the user about ignored parameters. See also the parameters of create_node. :type other_args: ``dict`` :return: dict: The VM template with the name set. z ex_template must be a dictionary�size�image�auth�ex_disks� ex_network�ex_termination_grace_period�ex_ports)r� r� r� r� r� r� r� c s � | d uS r= r9 r@ �Z other_paramsr9 r: rB � s zEKubeVirtNodeDriver._create_node_vm_from_ex_template.<locals>.<lambda>zGex_template is provided, ignoring the following non-None parameters: {}rb Nr' zgThe name in the ex_template ({}) will be ignored. The name provided in the arguments ({}) will be used.) r% �dictrs �listrE ru rv rt �copy�deepcopy)r' � ex_template� other_argsZignored_non_none_param_keysr8 r9 r� r: � _create_node_vm_from_ex_template� s@ � �� � �� z3KubeVirtNodeDriver._create_node_vm_from_ex_templatec C sV d } } }}|dur/t |t�sJ d��|jd }|j}|j�dd�p%|}|j�dd�p.|}|dur7|}|}dd� }|durS||�} | | d d d d d d d < |durk||�} | | d d d d d d d < |durs|}|}dd� } |dur�| |�} | | d d d d d d d< |dur�| |�} | | d d d d d d d< dS dS )a� A part of create_node that deals with the size of the VM. It will fill the vm with size information. :param size: The size of the VM in terms of CPU and memory. See also the size parameter in create_node. :type size: ``NodeSize`` with :param ex_cpu: The number of CPU cores to allocate to the VM. See also the ex_cpu parameter in create_node. :type ex_cpu: ``int`` or ``str`` :param ex_memory: The amount of memory to allocate to the VM in MiB. See also the ex_memory parameter in create_node. :type ex_memory: ``int`` :return: None Nzsize must be a NodeSize�cpu� cpu_request� ram_requestc S s t | t�s J d��t| �d S )Nz memory must be an int in MiB�Mi)r% �intr( )Z memory_valuer9 r9 r: �_format_memory� s z<KubeVirtNodeDriver._create_node_size.<locals>._format_memoryr# rk r� r� r~ �memoryr} c S s: t | t�r | �d�r | S zt| �W S ty td��w )N�mz0cpu must be a number or a string ending with 'm')r% r( �endswith�floatr) )Z cpu_valuer9 r9 r: � _format_cpu s �z9KubeVirtNodeDriver._create_node_size.<locals>._format_cpu)r% r rU �ramrs ) r8 r� �ex_cpu� ex_memoryZex_memory_limitZex_memory_requestZ ex_cpu_limitZex_cpu_requestr� r� r� r� r9 r9 r: �_create_node_size� s8 $�z$KubeVirtNodeDriver._create_node_sizec C s* t |t�s J d��|| d d d d< dS )a A part of create_node that deals with the termination grace period of the VM. It will fill the vm with the termination grace period information. :param vm: The VM template to be filled. :type vm: ``dict`` :param ex_termination_grace_period: The termination grace period of the VM in seconds. See also the ex_termination_grace_period parameter in create_node. :type ex_termination_grace_period: ``int`` :return: None z*ex_termination_grace_period must be an intr# rk r� N)r% r� )r8 r� r9 r9 r: �%_create_node_termination_grace_period s ����z8KubeVirtNodeDriver._create_node_termination_grace_periodc C sf |dur>z,t |t�r|d }|d }|d }nt |ttf�r+|d }|d }|d }ntd��W n ty= d }t|��w d }d }d }|i d|i}|i d|i}|pSi }|�d �rpg } |d D ] } | �| dd�� q_| || d< |�d�r�|| �dg �} |�d�D ] } | �| dd�� q�| || d< | d d d d �|� | d d d d d d �|� dS )a9 A part of create_node that deals with the network of the VM. It will fill the vm with network information. :param vm: The VM template to be filled. :type vm: ``dict`` :param ex_network: The network configuration of the VM. See also the ex_network parameter in create_node. :type ex_network: ``dict`` :param ex_ports: The ports to expose in the VM. See also the ex_ports parameter in create_node. :type ex_ports: ``dict`` :return: None N� interfacer' � network_typer � � z/ex_network must be a dictionary or a tuple/listzYex_network: You must provide a dictionary with keys: 'interface', 'name', 'network_type'.Z masqueradeZnetw1�podZ ports_tcp�TCP)�port�protocol�portsZ ports_udpZUDPr# rk r� r� r r{ )r% r� �tupler� �KeyErrorrs r& ) r8 r� r� r� Z network_namer� �msgZ network_dictZinterface_dict�ports_to_exposer� r9 r9 r: �_create_node_network0 sF � �� &z'KubeVirtNodeDriver._create_node_networkc C s | d d d d D ]}d|v sd|v rt d��q dtt�� � }dd i|d �}|d d id �}t|t�rD|j�� }t� |�}d� |�}nt|t �rY|j �� }t� |�}d� |�}nt d��||d d < | d d d d d d � |� | d d d d � |� dS )a� A part of create_node that deals with the authentication of the VM. It will fill the vm with a cloud-init volume that injects the authentication. :param vm: The VM template to be filled. :param auth: The authentication method for the VM. See also the auth parameter in create_node. :type auth: ``NodeAuthSSHKey`` or ``NodeAuthPassword`` :return: None r# rk r� r r zWSetting auth and cloudInit at the same time is not supported.Use deploy_node() instead.zauth-cloudinit-�bus�virtio��diskr' ZuserDatarm )r' r z*#cloud-config ssh_authorized_keys: - {} zJ#cloud-config password: {} chpasswd: {{ expire: False }} ssh_pwauth: True z/auth must be NodeAuthSSHKey or NodeAuthPasswordr� r rz N)r) r( r� r� r% r Zpubkey�striprV rW rt r r r& ) r8 r� �volumeZcloud_init_volume� disk_dictZ volume_dictZ public_keyZcloud_init_configr r9 r9 r: �_create_node_authq s6 ��� � � " z$KubeVirtNodeDriver._create_node_authc C s� |pg }t |�D ]�\}}|�d�}|�dd�} |�dd�|��} |�dd�} |tvr/tdt��|d krOz|d d }W n tyF td ��w d |i| d �} ny|dkr�d |vr[td��d|d vrgd} t| ��|d d }|| j|d�vr�d|d vs�d|d vr�d} t| ��|d d }|d d }|d �dd�}|d �dd�}| j||||||d� d|i| d �} nt� d �|�� ||�d i �d| i} | d| id| i}|d d d d d! d" � |� |d d d d# � | � q| � ||� d$S )%a� A part of create_node that deals with the disks (and volumes) of the VM. It will fill the vm with disk information. OS image is added to the disks as well. :param vm: The VM template to be filled. :type vm: ``dict`` :param ex_disks: The disk configuration of the VM. See also the ex_disks parameter in create_node. :type ex_disks: ``list`` :param image: The image to be used as the OS disk. See also the image parameter in create_node. :param namespace: The namespace where the VM will live. See also the location parameter in create_node. :type namespace: ``str`` :param location: The location where the VM will live. See also the location parameter in create_node. :type location: ``str`` :return: None � disk_typer� r� r' zdisk{}�devicer� z,The possible values for this parameter are: r Z volume_specr� z,A container disk needs a containerized image�r r' r z)You must provide a volume_spec dictionary� claim_namea4 You must provide either a claim_name of an existing claim or if you want one to be created you must additionally provide size and the storage_class_name of the cluster, which allows dynamic provisioning, so a Persistent Volume Claim can be created. In the latter case please provide the desired size as well.�rJ r� �storage_class_namezrdisk['volume_spec']['size'] and disk['volume_spec']['storage_class_name'] are both required to create a new claim.� volume_mode� Filesystem� access_mode� ReadWriteOnce)r� r' r2 �ex_storage_class_name�ex_volume_mode�ex_access_mode� claimName�r r' z5The disk type {} is not tested. Use at your own risk.r# rk r� r rz r� N) � enumeraters rt r r) r� � ex_list_persistent_volume_claims� create_volumeru rv r& �_create_node_image)r1 r8 r� r� rJ r2 �ir� r� r� Z disk_namer� � volumes_dictr� r� r� � storage_classr� r� r� r9 r9 r: �_create_node_disks� sh � � � � ��" z%KubeVirtNodeDriver._create_node_disksc C s� t |t�r|j}dtt�� � }d|i|d�}ddi|d�}| d d d d d d �d |� | d d d d �d |� dS )a� A part of create_node that deals with the image of the VM. It will fill the vm with the OS image as the first disk. :param vm: The VM template to be filled. :param image: The image to be used as the OS disk. See also the image parameter in create_node. :type image: ``NodeImage`` or ``str`` :return: None z boot-disk-r� r� r� r� r� r# rk r� r rz r r� N)r% r r' r( r� r� �insert)r8 r� Zboot_disk_namer� r� r9 r9 r: r� s $ z%KubeVirtNodeDriver._create_node_imager c C s� t |t�r|jtdd� | �� �vrtd��|j} nd} | dur0| j|| t� d�}| j||| d�S | j |d�}| j ||||d � | � |||| |� |durR| � ||� | � || | � | durc| �|| � | j||| d�S ) a�. Creating a VM with a containerDisk. @inherits: :class:`NodeDriver.create_node` The ``size`` parameter should be a ``NodeSize`` object that defines the CPU and memory limits of the VM. The ``NodeSize`` object must have the following attributes: - ram: int (in MiB) - extra["cpu"]: int (in cores) An example NodeSize for a VM with 1 CPU and 2GiB of memory (attributes except for ``ram`` and ``extra["cpu"]`` are not supported atm, but they are required by the ``NodeSize`` object):: >>> size = NodeSize( >>> id='small', # str: Size ID >>> name='small', # str: Size name. Not used atm >>> ram=2048, # int: Amount of memory (in MB) >>> disk=20, # int: Amount of disk storage (in GB). Not used atm >>> bandwidth=0, # int: Amount of bandwidth. Not used atm >>> price=0, # float: Price (in US dollars) of running this node for an hour. Not used atm >>> driver=KubeVirtNodeDriver # NodeDriver: Driver this size belongs to >>> extra={ >>> 'cpu': 1, # int: Number of CPUs provided by this size. >>> }, >>> ) The ``KubeVirtNodeSize`` wrapper can be used to create the ``NodeSize`` object more easily:: >>> size = KubeVirtNodeSize( >>> cpu=1, # int: Number of CPUs provided by this size. >>> ram=2048, # int: Amount of memory (in MB) >>> ) For legacy support, the ``ex_cpu`` and ``ex_memory`` parameters can be used instead of ``size``: - ``ex_cpu``: The number of CPU cores to allocate to the VM. ex_cpu must be a number (cores) or a string ending with 'm' (miliCPUs). - ``ex_memory``: The amount of memory to allocate to the VM in MiB. The ``image`` parameter can be either a ``NodeImage`` object with a ``name`` attribute that points to a containerDisk image, or a string representing the image URI. In both cases, it must point to a Docker image with an embedded disk. May be a URI like `kubevirt/cirros-registry-disk-demo`, kubevirt will automatically pull it from https://hub.docker.com/u/URI. For more info visit: https://kubevirt.io/user-guide/docs/latest/creating-virtual-machines/disks-and-volumes.html#containerdisk An example ``NodeImage``:: >>> image = NodeImage( >>> id="something_unique", >>> name="quay.io/containerdisks/ubuntu:22.04" >>> driver=KubeVirtNodeDriver, >>> ) The ``KubeVirtNodeImage`` wrapper can be used to create the ``NodeImage`` object more easily:: >>> image = KubeVirtNodeImage( >>> name="quay.io/containerdisks/ubuntu:22.04" >>> ) The ``location`` parameter is a NodeLocation object with the name being the kubernetes namespace where the VM will live. If not provided, the VM will be created in the default namespace. It can be created as the following:: >>> location = NodeLocation( >>> name='default', # str: namespace >>> ... # Other attributes are ignored >>> ) The ``auth`` parameter is the authentication method for the VM, either a ``NodeAuthSSHKey`` or a ``NodeAuthPassword``: - ``NodeAuthSSHKey(pubkey='pubkey data here')`` - ``NodeAuthPassword(password='mysecretpassword')`` If the ``auth`` parameter is provided, the VM will be created with a cloud-init volume that will inject the provided authentication into the VM. For details, see: https://kubevirt.io/user-guide/virtual_machines/startup_scripts/#injecting-ssh-keys-with-cloud-inits-cloud-config The ``ex_disks`` parameter is a list of disk dictionaries that define the disks of the VM. Each dictionary should have specific keys for disk configuration. The following are optional keys: - ``"bus"``: can be "virtio", "sata", or "scsi" - ``"device"``: can be "lun" or "disk" The following are required keys: - ``"disk_type"``: One of the supported ``DISK_TYPES``. Atm only "persistentVolumeClaim" and "containerDisk" are promised to work. Other types may work but are not tested. - ``"name"``: The name of the disk configuration - ``"voulme_spec"``: the dictionary that defines the volume of the disk. The content is depending on the disk_type: For `containerDisk` the dictionary should have a key `image` with the value being the image URI. For `persistentVolumeClaim` the dictionary should have a key `claim_name` with the value being the name of the claim. If you wish, a new Persistent Volume Claim can be created by providing the following: - required: - size: the desired size (implied in GB) - storage_class_name: the name of the storage class to # NOQA be used for the creation of the Persistent Volume Claim. Make sure it allows for dynamic provisioning. - optional: - access_mode: default is ReadWriteOnce - volume_mode: default is `Filesystem`, it can also be `Block` For other disk types, it will be translated to a KubeVirt volume object as is:: {disk_type: <volume_spec>, "name": disk_name} This dict will be translated to KubeVirt API object:: volumes: - name: <disk_name> <disk_type>: <voulme_spec> Please refer to the KubeVirt API documentation for volume specifications: https://kubevirt.io/user-guide/virtual_machines/disks_and_volumes An example ``ex_disks`` list with a hostDisk that will create a new img file in the host machine:: >>> ex_disks = [ >>> { >>> "bus": "virtio", >>> "device": "disk", >>> "disk_type": "hostDisk", >>> "name": "disk114514", >>> "volume_spec": { >>> "capacity": "10Gi", >>> "path": "/tmp/kubevirt/data/disk114514.img", >>> "type": "DiskOrCreate", >>> }, >>> }, >>> ] The ``ex_network`` parameter is a dictionary that defines the network of the VM. Only the ``pod`` type is supported, and in the configuration ``masquerade`` or ``bridge`` are the accepted values. The parameter must be a dict:: { "network_type": "pod", "interface": "masquerade | bridge", "name": "network_name" } For advanced configurations not covered by the other parameters, the ``ex_template`` parameter can be used to provide a dictionary of kubernetes object that defines the KubeVirt VM. Notice, If provided, ``ex_template`` will override all other parameters except for ``name`` and ``location``. The ``ex_template`` parameter is a dictionary of kubernetes object: - ``apiVersion``: ``str`` - ``kind``: ``str`` - ``metadata``: ``dict`` - ``spec``: ``dict`` - ``domain``: ``dict`` - ``volumes``: ``list`` - ... See also: - https://kubernetes.io/docs/concepts/overview/working-with-objects/ - https://kubevirt.io/api-reference/ :param name: A name to give the VM. The VM will be identified by this name and atm it cannot be changed after it is set. :type name: ``str`` :param size: The size of the VM in terms of CPU and memory. A NodeSize object with the attributes ``ram`` (int in MiB) and ``extra["cpu"]`` (int in cores). :type size: ``NodeSize`` with :param image: Either a libcloud NodeImage or a string. In both cases, it must a URL to the containerDisk image. :type image: ``str`` or ``NodeImage`` :param location: The namespace where the VM will live. (default is ``'default'``) :type location: `NodeLocation`` in which the name is the namespace :param auth: authentication to a node. :type auth: ``NodeAuthSSHKey`` or ``NodeAuthPassword``. :param ex_cpu: The number of CPU cores to allocate to the VM. (Legacy support, consider using ``size`` instead) :type ex_cpu: ``int`` or ``str`` :param ex_memory: The amount of memory to allocate to the VM in MiB. (Legacy support, consider using ``size`` instead) :type ex_memory: ``int`` :param ex_disks: A list containing disk dictionaries. Each dictionary should have the following keys: ``bus``: can be ``"virtio"``, ``"sata"``, or ``"scsi"``; ``device``: can be ``"lun"`` or ``"disk"``; ``disk_type``: One of the supported DISK_TYPES; ``name``: The name of the disk configuration; ``voulme_spec``: the dictionary that defines the volume of the disk. :type ex_disks: ``list`` of ``dict`` with keys: ``bus: str``, ``device: str``, ``disk_type: str``, ``name: str`` and ``voulme_spec: dict`` :param ex_network: a dictionary that defines the network of the VM. The following keys are required: ``network_type``: ``"pod"``; ``interface``: ``"masquerade"`` or ``"bridge"``; ``name``: ``"network_name"``. :type ex_network: ``dict`` with keys: ``network_type: str``, ``interface: str`` and ``name: str`` :param ex_termination_grace_period: The grace period in seconds before the VM is forcefully terminated. (default is 0) :type ex_termination_grace_period: `int` :param ex_ports: A dictionary with keys: ``"ports_tcp"`` and ``"ports_udp"`` ``"ports_tcp"`` value is a list of ints that indicate the ports to be exposed with TCP protocol, and ``"ports_udp"`` is a list of ints that indicate the ports to be exposed with UDP protocol. :type ex_ports: `dict` with keys ``ports_tcp``: ``list`` of ``int``; ``ports_udp``: ``list`` of ``int``. :param ex_template: A dictionary of kubernetes object that defines the KubeVirt VM. This is for advanced vm specifications that are not covered by the other parameters. If provided, it will override all other parameters except for `name` and `location`. :type ex_template: ``dict`` with keys: ``apiVersion: str``, ``kind: str``, ``metadata: dict`` and ``spec: dict`` c S s | j S r= rD r@ r9 r9 r: rB S s z0KubeVirtNodeDriver.create_node.<locals>.<lambda>z4The location must be one of the available namespacesrj N)r' r� r� )r' rk rJ rD )r8 r� r� r� )r% r r' �mapr* r) r� �localsrw r� r� r� r� r� r� )r1 r' r� r� r2 r� r� r� r� r� r� r� r� rJ r8 r9 r9 r: r / s* %� z KubeVirtNodeDriver.create_nodec sh | � � }|r|j� tt� fdd�|��}t� }g }|D ]}|jj|v r$q|�|jj� |�|j� q|S )z� If location (namespace) is provided only the images in that location will be provided. Otherwise all of them. c s | d d � kS )NrU rJ r9 r@ r� r9 r: rB � s z0KubeVirtNodeDriver.list_images.<locals>.<lambda>)r; r' r� rE �setr� �addr& )r1 r2 rH Zname_setZimagesrY r9 r� r: � list_images{ s z KubeVirtNodeDriver.list_imagesc C s^ t d }g }| j�|�j}|d D ]}|d d }|d d }|�t||d| jjd�� q|S )z; By locations here it is meant namespaces. r3 r" rb r' �uidrm )r? r' Zcountry�driver)�ROOT_URLr, r- r/ r& r r� )r1 r5 r3 r6 r7 r' �IDr9 r9 r: r* � s �z!KubeVirtNodeDriver.list_locationsc C sT d}|r|j }| �� }g }|D ]}|s|�|j� q||jd kr'|�|j� q|S )Nrm rJ )r' r; r& r� rU )r1 r2 rJ rH ZsizesrY r9 r9 r: � list_sizes� s �z KubeVirtNodeDriver.list_sizesrm r� r� T�Recyclec C s� |r|du r d} t | ��| j||||j||d�} | S | du s"| du r(d} t | ��ddd|idd t|�d id |d |gd |d|dg | | id�} td }d}t�| �}z | jj|||d� W n t yd � w | � � }|D ] }|j|krv| S qkdS )uv :param size: The size in Gigabytes :type size: `int` :param volume_type: This is the type of volume to be created that is dependent on the underlying cloud where Kubernetes is deployed. K8s is supporting the following types: -gcePersistentDisk -awsElasticBlockStore -azureFile -azureDisk -csi -fc (Fibre Channel) -flexVolume -flocker -nfs -iSCSI -rbd (Ceph Block Device) -cephFS -cinder (OpenStack block storage) -glusterfs -vsphereVolume -quobyte Volumes -hostPath (Single node testing only – local storage is not supported in any way and WILL NOT WORK in a multi-node cluster) # NOQA -portworx Volumes -scaleIO Volumes -storageOS This parameter is a dict in the form {type: {key1:value1, key2:value2,...}}, where type is one of the above and key1, key2... are type specific keys and their corresponding values. eg: {nsf: {server: "172.0.0.0", path: "/tmp"}} {awsElasticBlockStore: {fsType: 'ext4', volumeID: "1234"}} :type volume_type: `str` :param volume_params: A dict with the key:value that the volume_type needs. This parameter is a dict in the form {key1:value1, key2:value2,...}, where type is one of the above and key1, key2... are type specific keys and their corresponding values. eg: for nsf volume_type {server: "172.0.0.0", path: "/tmp"} for awsElasticBlockStore volume_type {fsType: 'ext4', volumeID: "1234"} Nz'Please provide a namespace for the PVC.)r� r' r� rJ r� r� z�An ex_volume_type must be provided from the list of supported clouds, as well as the ex_volume_params necessary for your volume type choice.�v1ZPersistentVolumer' �capacity�storage�Gi� volumeMode� accessModesZ persistentVolumeReclaimPolicy�storageClassNameZ mountOptionsr� �persistentvolumes/rq rr ) r) �_create_volume_dynamicr' r( r� rV rW r, r- rX � list_volumes)r1 r� r' r2 r� r� r� Z ex_dynamicZex_reclaim_policyZex_volume_typeZex_volume_paramsr� �vol�pvr5 rP rQ r� r� r9 r9 r: r� � sV ;� ��� � ��z KubeVirtNodeDriver.create_volumec C sZ ddd|ig |dddiid�d�}|d d � |� |d ur%||d d < ntd ��t|�d |d d d d< d}td | d } t�|�} z | jj| || d�} W n tyY � w | j d d dkr�t d�D ]-} td | d | } z | j�| �j } W n ty� � w | d d dkr� nt � d� qg| � � } | D ]}|jd d |kr�| S q�d S )ag Method to create a Persistent Volume Claim for storage, thus storage is required in the arguments. This method assumes dynamic provisioning of the Persistent Volume so the storage_class given should allow for it (by default it usually is), or already have unbounded Persistent Volumes created by an admin. :param name: The name of the pvc an arbitrary string of lower letters :type name: `str` :param size: An int of the amount of gigabytes desired :type size: `int` :param namespace: The namespace where the claim will live :type namespace: `str` :param storage_class_name: If you want the pvc to be bound to a particular class of PVs specified here. :type storage_class_name: `str` :param access_mode: The desired access mode, ie "ReadOnlyMany" :type access_mode: `str` :param matchLabels: A dictionary with the labels, ie: {'release': 'stable,} :type matchLabels: `dict` with keys `str` and values `str` r� ZPersistentVolumeClaimr' r} r� rm )r� r� r� r� r# r� Nr� z_The storage class name must be provided of astorage class which allows for dynamic provisioningr� r� rq r �/persistentvolumeclaimsrr r. �phase�Bound� �/persistentvolumeclaims/�pvc)r& r) r( r� rV rW r, r- rX r/ �range�time�sleepr� rU )r1 r� r' r� r� rJ r� r� rP r5 rQ r6 �_r� r� r9 r9 r: r� " sP & �� � � � ��z)KubeVirtNodeDriver._create_volume_dynamicc C sZ |j d rdS |j d }|j}|jd }|j d }|j d d }| j||||||d�}|S ) z� This method is for unbound volumes that were statically made. It will bind them to a pvc so they can be used by a kubernetes resource. �is_boundNr� z-pvcr� � access_modesr )r� r' r� r� rJ r� )rU r� r' r� ) r1 r� rJ r� r� r' r� r� r� r9 r9 r: � _bind_volumey s � z KubeVirtNodeDriver._bind_volumec C s� d}|j d r2|j d d }|j d d }td | d | }z | jj||d�}W n ty1 � w |j}td | }z | jj||d�}|jW S tyO � w ) Nr] r� r� r' rJ r r� r^ r� )rU r� r, r- rX r' r. )r1 r� rP r� rJ r5 r6 r� r9 r9 r: �destroy_volume� s$ � �z!KubeVirtNodeDriver.destroy_volumer� r� c C s� |j d s| �||j d �}|du rtd| d��|j d d }|du r&|}n|}|j d d }| �|� |j d |krAd} t| ��|j} td | d | } |d |id|i} d |i|d �} z | j�| �j }W n t yn � w |d d d d d d }|d d d d }|� | � |� | � ddi}ddd|dd|iid�iii}z&| jj| dt � |�|d�}d|j v r�|j d � |� n|g|j d< |tv W S t y� � w )z: params: bus, name , device (disk or lun) r� rJ NzPSelected Volume (PV) could not be bound (to a PVC), please select another volume)r� r� r' z0The PVC and the VM must be in the same namespacer rK r� r� r� r# rk r� r rz r� rL rM �r� r� rN rO �pvcs)rU r� r r\ r) r' r+ r, r- r/ rX r& rV rW r )r1 rY r� r� Zex_busZex_namer� r' rJ r� r8 r5 r� r� r6 rz r� rR rQ r9 r9 r: � attach_volume� sd � � ���� � �z KubeVirtNodeDriver.attach_volumec C sl | � |� |jd d }|j}|jd }td | d | }ddi}z | j�|�j}W n ty3 � w |d d d d d d } |d d d d } d} | D ]}d|v ri|d d |kri|d } | �|� nqP| srd} t | ��| D ]} | d | kr�| �| � nqtdd d| d d | iid�iii}z| jj|dt � |�|d�}|jd �|� |t v W S ty� � w )z� Detaches a volume from a node but the node must be given since a PVC can have more than one VMI's pointing to it r� r' rJ r rK rL rM r# rk r� r rz r� Nr r� z0The given volume is not attached to the given VMr� rN rO r� ) r\ rU r' r+ r, r- r/ rX �remover) rV rW r )r1 r� Zex_noder� r' rJ r5 rR r6 rz r� Z to_deleter� r� rQ r9 r9 r: � detach_volume� s^ � � � ���� � �z KubeVirtNodeDriver.detach_volumec C sJ t d | d }z | j�|�j}W n ty � w dd� |d D �}|S )Nr r� c S � g | ]}|d d �qS �rb r' r9 ��.0r7 r9 r9 r: � <listcomp>4 � zGKubeVirtNodeDriver.ex_list_persistent_volume_claims.<locals>.<listcomp>r" )r� r, r- r/ rX )r1 rJ Zpvc_reqr6 r� r9 r9 r: r� . s �z3KubeVirtNodeDriver.ex_list_persistent_volume_claimsc C s> d}z | j �|�j}W n ty � w dd� |d D �}|S )Nz&/apis/storage.k8s.io/v1/storageclassesc S r r r9 r r9 r9 r: r ? r z>KubeVirtNodeDriver.ex_list_storage_classes.<locals>.<listcomp>r" )r, r- r/ rX )r1 Zsc_reqr6 Zscsr9 r9 r: �ex_list_storage_classes8 s �z*KubeVirtNodeDriver.ex_list_storage_classesc C sT g }t d }z | j�|�j}W n ty � w |d D ]�}|d d dvr'q |d d }|d d d }t|�d ��}d i i}|d d |d< |d d dk|d< |d d |d< |d d |d< |d r�|d d d |d d< |d d d |d d< |d d d |d d< |d d }n|d d }t|||| jj|d�} |� | � q |S )z9 Location is a namespace of the cluster. z/persistentvolumes/r" r. r� > r� Z Availablerb r� r# r� r� r� r� r� r� r� r� r� r� r� r� ZclaimRefr' rJ )r? r' r� r� rU ) r� r, r- r/ rX r� �rstripr r� r& ) r1 r� Zpv_recr6 r7 r� r� rU r' r� r9 r9 r: r� C s: � � z KubeVirtNodeDriver.list_volumesc C s0 i }t | d�r | j|d< t | d�r| j|d< |S )N�key_file� cert_file)�hasattrr r )r1 �kwargsr9 r9 r: �_ex_connection_class_kwargsj s z.KubeVirtNodeDriver._ex_connection_class_kwargsFc % C s� |d d }|d d }| j j}d|d d i}g |d< d}d|d d d d d v rMd |d d d d d d v rL|d d d d d d d }n2|d d d d d �d d�r|d d d d d d �d d�r|d d d d d d d }t|�}|d d d d d �d i ��d d�}|r�t|�}n|}d} |d d d d d �dd�r�|d d d d d d �dd�r�|d d d d d d d } nV|d d d d d �d d��r |d d d d d d �dd��r |d d d d d d d } | } n |d d d d �dd��r+|d d d d d �dd�} t| t��s8t| �d��} |d d d d d �d i ��dd�} | du �rS| } | | ||d�} d�t| �t|��} t � | � d��� � } t | | |ddd|| d�}||d < | |d< d}|d d d d D ] }|�� D ]\}}t|�tu �r�d|v �r�|d }�q��q�t|||�}d|d d d v �r�|d d d d D ]}d|v �r�|d �|d d � �q�g }| j|d |d �}|D ]@}|d �d �}|d d D ]/}|�d �}|�d �}|�d!�}zt|� W n t�y |}Y nw |�||||d"�� �q��q�||d#< |�rDtj} d} d} t||| | | ||||d$� S td% |d d& } | j �| �j} d}!| d' D ] }"d|"v �rxd(|"d v �rx|"d d( d d |k�rx|"}!�q[|!du �s�d)|!d* v�r�tj} d} d} t||| | | ||||d$� S d|!d d i|d+< |!d* d) D ] }#|#d d,k�r�t||#d |�}d-|#d. v �r�tj ntj} �q�d} t � |d d/ d0�}$d1|!d* v �r�d2d3� |!d* d1 D �} ng } t||| | | |||||$d4� S )5a. converts a vm from the kubevirt API to a libcloud node :param vm: kubevirt vm object :type vm: dict :param is_stopped: if the vm is stopped, it will not have a pod :type is_stopped: bool :return: a libcloud node :rtype: :class:`Node` rb r� r' rJ r� r r~ r# rk r� r� r� r} Nr� r� Zcoresr� )r� r� r� r� �{} vCPUs, {}MB Ram�utf-8�r? r' r� r� Z bandwidthZpricer� rU Z undefinedr� r� r r� r` r| r� r� r� � targetPort)� local_port� public_portr� � service_type� port_forwards) r? r' rS � public_ips� private_ipsr� r� r� rU r z/podsr" ZownerReferencesZcontainerStatusesr. r� Zcomputer rS ZcreationTimestampz%Y-%m-%dT%H:%M:%SZZpodIPsc S s g | ]}|d �qS )�ipr9 )r r r9 r9 r: r ( s z/KubeVirtNodeDriver._to_node.<locals>.<listcomp>) r? r' rS r r r� r� r� rU � created_at) r, r� rs � _memory_in_MBr% r� r rt r( �hashlib�md5�encode� hexdigestr r" r| r� r r& re r) r r[ r r� r- r/ �PENDINGrT r �strptime)%r1 r8 r$ r� r' r� rU r� Z memory_reqr� Zcpu_req� extra_sizeZ size_name�size_idr� Z image_namer� �k�vr� r rg rh r Z port_pairr� r r rS r r r5 r6 r� �pdZ cont_statusr r9 r9 r: r0 u sP � � � $ �$������� � � �� � ����� �� ���zKubeVirtNodeDriver._to_nodec C sv d}|dur dd� |�i}td� |� }| jj||d�jd }|r9g }|D ]}||d �dd �v r6|�|� q%|S |S ) zo If node_name is given then the services returned will be those that concern the node. NZ fieldSelectorzmetadata.name={}z/namespaces/{}/services)�paramsr" rb r' rm )rt r� r, r- r/ rs r& ) r1 rJ ra rd r& r5 r6 �resrh r9 r9 r: re 9 s �z#KubeVirtNodeDriver.ex_list_services�NodePortc C s |j �dd�}d�|�� |j�}| j||d�} g } |s g }|D ]5} | �dd�s-| d | d< | �dd �s<d �| d �| d< | �| �d d �t| d �t| d �| d d �� q d} d} t| �dkr�|syd}| D ]}|d d }|ou| j ||d�}qe|S d}d| i}|s�| d d d }d|� | �i}t � d|i�} ddi} d�t ||�}nM|s�td�|���d}dd|dd id �d d |jig d �d �}| |d d< ||d d!< |dur�||d d"< |d#kr�|dur�||d d$< t � |�} d%�t |�}z | jj||| | d&�}W n t�y � w |jtv S )'a� Each node has a single service of one type on which the exposed ports are described. If a service exists then the port declared will be exposed alongside the existing ones, set override_existing_ports=True to delete existing exposed ports and expose just the ones in the port variable. :param node: the libcloud node for which the ports will be exposed :type node: libcloud `Node` class :param ports: a list of dictionaries with keys --> values: 'port' --> port to be exposed on the service; 'target_port' --> port on the pod/node, optional if empty then it gets the same value as 'port' value; 'protocol' ---> either 'UDP' or 'TCP', defaults to TCP; 'name' --> A name for the service; If ports is an empty `list` and a service exists of this type then the service will be deleted. :type ports: `list` of `dict` where each `dict` has keys --> values: 'port' --> `int`; 'target_port' --> `int`; 'protocol' --> `str`; 'name' --> `str`; :param service_type: Valid types are ClusterIP, NodePort, LoadBalancer :type service_type: `str` :param cluster_ip: This can be set with an IP string value if you want manually set the service's internal IP. If the value is not correct the method will fail, this value can't be updated. :type cluster_ip: `str` :param override_existing_ports: Set to True if you want to delete the existing ports exposed by the service and keep just the ones declared in the present ports argument. By default it is false and if the service already exists the ports will be added to the existing ones. :type override_existing_ports: `boolean` rJ rj z service-{}-{}rc Z target_portNr� r' rm zport-{}r� r� )r� r� r r' r Trb rN r� r# rL rM � {}/namespaces/{}/services/{}zHArgument ports is empty but there is no service of {} type to be deletedrq ZServicer� rh z kubevirt.io)r' ry rx )r| �selectorr� )ro rl rb r# r| Z clusterIPZ LoadBalancerZloadBalancerIPz{}/namespaces/{}/servicesrO )rU rs rt �lowerr' re r& r� �lenrf �extendrV rW r� r) r, r- rX r. r )r1 rY r� r Z cluster_ipZload_balancer_ipZoverride_existing_portsrJ rd Z service_listr� Z port_grouprR rQ r6 rh rP r# Zexisting_portsr5 r9 r9 r: �ex_create_serviceP s� 5 �� ����� � z$KubeVirtNodeDriver.ex_create_servicec C sH d� t||�}ddi}z | jj|d|d�}W n ty � w |jtv S )Nr) rL zapplication/yamlr] )rP rR )rt r� r, r- rX r. r )r1 rJ rd r5 rR r6 r9 r9 r: rf � s � z$KubeVirtNodeDriver.ex_delete_servicer= �NN)rj )NNN) NNNNNNNNr NN)Nrm r� r� Tr� NN)r� rj r� )r� r� N)F)rj NN)r( NNF)4�__name__� __module__� __qualname__r ZKUBEVIRTr| r' Zwebsiter Z connectionClsZfeaturesr r rT r[ ZNODE_STATE_MAPr; rI rZ r\ r_ ri r( r� rw � staticmethodr� r� r� r� r� r� r� r� r r� r* r� r� r� r� r� r� r r� r r� r r0 re r. rf r9 r9 r9 r: r I s� � ( W * @ �J @ 7l � N �u � W E = ' E � r �source� destination�returnc C sV | � � D ]$\}}t|t�r|�|i �}t||� q||vr"|||< q|r(|||< q|S )a Deep merge two dictionaries: source into destination. For conflicts, prefer source's non-zero values over destination's. (By non-zero, we mean that bool(value) is True.) Extended from https://stackoverflow.com/a/20666342, added zero value handling. Example:: >>> a = {"domain": {"devices": 0}, "volumes": [1, 2, 3], "network": {}} >>> b = {"domain": {"machine": "non-exist-in-a", "devices": 1024}, "volumes": [4, 5, 6]} >>> _deep_merge_dict(a, b) {'domain': {'machine': 'non-exist-in-a', 'devices': 1024}, 'volumes': [1, 2, 3], 'network': {}} In the above example: - network: exists in source (a) but not in destination (b): add source (a)'s - volumes: exists in both, both are non-zero: prefer source (a)'s - devices: exists in both: source (a) is zero, destination (b) is non-zero: keep destination (b)'s - machine: exists in destination (b) but not in source (a): reserve destination (b)'s :param source: RO: A dict to be merged into another. Do not use circular dict (e.g. d = {}; d['d'] = d) as source, otherwise a RecursionError will be raised. :param destination: RW: A dict to be merged into. (the value will be modified). :return: dict: Updated destination. )r" r% r� � setdefault�_deep_merge_dict)r4 r5 �key�valuerY r9 r9 r: r8 � s �r8 c C s� z t | �}|d d W S ty Y nw t| t�s td��| �d�r2t | �d��d d d d S | �d�rDt | �d��d d d S | �d�rTt | �d��d d S | �d�rbt | �d��d S | �d�rnt | �d��S | �d�r|t | �d��d S | �d �r�t | �d ��d d d d S | �d �r�t | �d ��d d d S | �d �r�t | �d ��d d S | �d �r�t | �d ��d S | �d�r�t | �d��S | �d�r�t | �d��d S td�| ���)an parse k8s memory resource units to MiB or MB (depending on input) Note: - 1 MiB = 1024 KiB = 1024 * 1024 B = 1048576 B - 1 MB = 1000 KB = 1000 * 1000 B = 1000000 B Reference: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory :param memory: Limits and requests for memory are measured in bytes. You can express memory as a plain integer or as a fixed-point integer using one of these suffixes: E, P, T, G, M, K, Ei, Pi, Ti, Gi, Mi, Ki For example, the following represent roughly the same value: 128974848, 129e6, 129M, 123Mi :type memory: ``str`` or ``int`` :return: memory in MiB (if input is `int` bytes or `str` with suffix `?i`) or in MB (if input is `str` with suffix `?`) :rtype: ``int`` i zmemory must be int or strZEiZPiZTir� r� ZKi�Ei� �P�T�G�M�Kz memory unit not supported {})r� r) r% r( r� r rt )r� Z mem_bytesr9 r9 r: r sB � r c C s` d| i}|p| |d< |p ||d< d� t| �t|��}t�|�d���� }t|||dddt|d�S )a2 Create a NodeSize object for KubeVirt driver. This function is just a shorthand for ``NodeSize(ram=ram, extra={"cpu": cpu})``. :param cpu: number of virtual CPUs (max limit) :type cpu: ``int`` :param ram: amount of RAM in MiB (max limit) :type ram: ``int`` :param cpu_request: number of virtual CPUs (min request) :type cpu_request: ``int`` :param ram_request: amount of RAM in MiB (min request) :type ram_request: ``int`` :return: a NodeSize object with ram and extra.cpu set :rtype: :class:`NodeSize` r� r� r� r r r r )rt r( r r r r r r )r� r� r� r� rU r' r" r9 r9 r: r L s �r c C s" t �| �d���� }t|| td�S )af Create a NodeImage object for KubeVirt driver. This function is just a shorthand for ``NodeImage(name=name)``. :param name: image source :type name: ``str`` :return: a NodeImage object with the name set to the source to a containerDisk image (e.g. ``"quay.io/containerdisks/ubuntu:22.04"``) :rtype: :class:`NodeImage` r )r? r' r� )r r r r r r )r' Zimage_idr9 r9 r: r w s r r/ )'�__doc__r� rV r� r� r ru �typingr r r Zlibcloud.common.typesr Zlibcloud.compute.baser r r r r r r r Zlibcloud.compute.typesr r Zlibcloud.common.kubernetesr r r �__all__r� r+ r r r� r8 r r r r9 r9 r9 r: �<module> sL ( +*> � +
| ver. 1.4 |
Github
|
.
| PHP 8.2.30 | Generation time: 0 |
proxy
|
phpinfo
|
Settings