☸️Kubernetes 的子部分

Prepare k8s Cluster

    There are many ways to build a kubernetes cluster.

    Install Kuberctl

    MIRROR="files.m.daocloud.io/"
    VERSION=$(curl -L -s https://${MIRROR}dl.k8s.io/release/stable.txt)
    [ $(uname -m) = x86_64 ] && curl -sSLo kubectl "https://${MIRROR}dl.k8s.io/release/${VERSION}/bin/linux/amd64/kubectl"
    [ $(uname -m) = aarch64 ] && curl -sSLo kubectl "https://${MIRROR}dl.k8s.io/release/${VERSION}/bin/linux/arm64/kubectl"
    chmod u+x kubectl
    mkdir -p ${HOME}/bin
    mv -f kubectl ${HOME}/bin

    Build Cluster

    MIRROR="files.m.daocloud.io/"
    VERSION=v0.20.0
    [ $(uname -m) = x86_64 ] && curl -sSLo kind "https://${MIRROR}github.com/kubernetes-sigs/kind/releases/download/${VERSION}/kind-linux-amd64"
    [ $(uname -m) = aarch64 ] && curl -sSLo kind "https://${MIRROR}github.com/kubernetes-sigs/kind/releases/download/${VERSION}/kind-linux-arm64"
    chmod u+x kind
    mkdir -p ${HOME}/bin
    mv -f kind ${HOME}/bin

    Creating a Kubernetes cluster is as simple as kind create cluster

    kind create cluster --name test

    and the you can visit https://kind.sigs.k8s.io/docs/user/quick-start/ for mode detail.

    MIRROR="files.m.daocloud.io/"
    [ $(uname -m) = x86_64 ] && curl -sSLo minikube "https://${MIRROR}storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64"
    [ $(uname -m) = aarch64 ] && curl -sSLo minikube "https://${MIRROR}storage.googleapis.com/minikube/releases/latest/minikube-linux-arm64"
    chmod u+x minikube
    mkdir -p ${HOME}/bin
    mv -f minikube ${HOME}/bin

    [Optional] disable aegis service and reboot system for aliyun

    sudo systemctl disable aegis && sudo reboot

    after you download binary, you can start your cluster

    minikube start --kubernetes-version=v1.27.10 --image-mirror-country=cn --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers --cpus=6 --memory=24g --disk-size=100g

    add alias for convinence

    alias kubectl="minikube kubectl --"

    and then you can visit https://minikube.sigs.k8s.io/docs/start/ for more detail.

    Prerequisites

    • Hardware Requirements:

      1. At least 2 GB of RAM per machine (minimum 1 GB)
      2. 2 CPUs on the master node
      3. Full network connectivity among all machines (public or private network)
    • Operating System:

      1. Ubuntu 20.04/18.04, CentOS 7/8, or any other supported Linux distribution.
    • Network Requirements:

      1. Unique hostname, MAC address, and product_uuid for each node.
      2. Certain ports need to be open (e.g., 6443, 2379-2380, 10250, 10251, 10252, 10255, etc.)
    • Disable Swap:

      sudo swapoff -a

    Steps to Setup Kubernetes Cluster

    1. Prepare Your Servers Update the Package Index and Install Necessary Packages On all your nodes (both master and worker):
    sudo apt-get update
    sudo apt-get install -y apt-transport-https ca-certificates curl

    Add the Kubernetes APT Repository

    curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
    cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
    deb http://apt.kubernetes.io/ kubernetes-xenial main
    EOF

    Install kubeadm, kubelet, and kubectl

    sudo apt-get update
    sudo apt-get install -y kubelet kubeadm kubectl
    sudo apt-mark hold kubelet kubeadm kubectl
    1. Initialize the Master Node On the master node, initialize the Kubernetes control plane:
    sudo kubeadm init --pod-network-cidr=192.168.0.0/16

    The –pod-network-cidr flag is used to set the Pod network range. You might need to adjust this based on your network provider

    Set up Local kubeconfig

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
    1. Install a Pod Network Add-on You can install a network add-on like Flannel, Calico, or Weave. For example, to install Calico:

    ```shell kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml ```

    ```shell kubectl apply -f https://docs.projectcalico.org/v3.14/manifests/calico.yaml ```

    1. Join Worker Nodes to the Cluster On each worker node, run the kubeadm join command provided at the end of the kubeadm init output on the master node. It will look something like this:
    sudo kubeadm join <master-ip>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>

    If you lost the join command, you can create a new token on the master node:

    sudo kubeadm token create --print-join-command
    1. Verify the Cluster Once all nodes have joined, you can verify the cluster status from the master node:
    kubectl get nodes

    This command should list all your nodes with the status “Ready”.

    2024年3月7日

    Command

      2024年3月7日

      Container 的子部分

      CheatShett

      type:
      1. remove specific image
      podman rmi <$image_id>
      1. remove all <none> images
      podman rmi `podamn images | grep  '<none>' | awk '{print $3}'`
      1. remove all stopped containers
      podman container prune
      1. remove all docker images not used
      podman image prune
      1. find ip address of a container
      podman inspect --format='{{.NetworkSettings.IPAddress}}' minio-server
      1. exec into container
      podman run -it <$container_id> /bin/bash
      1. run with environment
      podman run -d --replace 
          -p 18123:8123 -p 19000:9000 \
          --name clickhouse-server \
          -e ALLOW_EMPTY_PASSWORD=yes \
          --ulimit nofile=262144:262144 \
          quay.m.daocloud.io/kryptonite/clickhouse-docker-rootless:20.9.3.45 

      --ulimit nofile=262144:262144: 262144 is the maximum users process or for showing maximum user process limit for the logged-in user

      ulimit is admin access required Linux shell command which is used to see, set, or limit the resource usage of the current user. It is used to return the number of open file descriptors for each process. It is also used to set restrictions on the resources used by a process.

      1. login registry
      podman login --tls-verify=false --username=ascm-org-1710208820455 cr.registry.res.cloud.zhejianglab.com -p 'xxxx'
      1. tag image
      podman tag 76fdac66291c cr.registry.res.cloud.zhejianglab.com/ay-dev/datahub-s3-fits:1.0.0
      1. push image
      podman push cr.registry.res.cloud.zhejianglab.com/ay-dev/datahub-s3-fits:1.0.0
      1. remove specific image
      docker rmi <$image_id>
      1. remove all <none> images
      docker rmi `docker images | grep  '<none>' | awk '{print $3}'`
      1. remove all stopped containers
      docker container prune
      1. remove all docker images not used
      docker image prune
      1. find ip address of a container
      docker inspect --format='{{.NetworkSettings.IPAddress}}' minio-server
      1. exec into container
      docker exec -it <$container_id> /bin/bash
      1. run with environment
      docker run -d --replace -p 18123:8123 -p 19000:9000 --name clickhouse-server -e ALLOW_EMPTY_PASSWORD=yes --ulimit nofile=262144:262144 quay.m.daocloud.io/kryptonite/clickhouse-docker-rootless:20.9.3.45 

      --ulimit nofile=262144:262144: sssss

      1. copy file

        Copy a local file into container

        docker cp ./some_file CONTAINER:/work

        or copy files from container to local path

        docker cp CONTAINER:/var/logs/ /tmp/app_logs
      2. load a volume

      docker run --rm \
          --entrypoint bash \
          -v $PWD/data:/app:ro \
          -it docker.io/minio/mc:latest \
          -c "mc --insecure alias set minio https://oss-cn-hangzhou-zjy-d01-a.ops.cloud.zhejianglab.com/ g83B2sji1CbAfjQO 2h8NisFRELiwOn41iXc6sgufED1n1A \
              && mc --insecure ls minio/csst-prod/ \
              && mc --insecure mb --ignore-existing minio/csst-prod/crp-test \
              && mc --insecure cp /app/modify.pdf minio/csst-prod/crp-test/ \
              && mc --insecure ls --recursive minio/csst-prod/"
      2024年3月7日

      Template 的子部分

      DevContainer Template

        2024年3月7日

        DEV

          2024年3月7日

          Operator 的子部分

          KubeBuilder

            2024年3月7日

            Proxy 的子部分

            Daocloud

            1. install container tools

            systemctl stop firewalld && systemctl disable firewalld
            sudo dnf install -y podman
            podman run -d -P m.daocloud.io/docker.io/library/nginx
            2024年3月7日

            Serverless 的子部分

            Knative 的子部分

            Eventing 的子部分

            Broker

              2024年3月7日

              Plugin 的子部分

              Eventing Kafka Broker

                2024年3月7日

                Kserve 的子部分

                Serving 的子部分

                Inference

                  2024年3月7日

                  Generative

                    2024年3月7日

                    Canary Policy

                      2024年3月7日

                      Auto Scaling