Networking

DNS Setup

Overview

For any self-hosted service to be accessible via browser, you need:

A domain name (e.g. 72602.online)
DNS A records pointing each subdomain to your public IP
(Optional) cert-manager for automatic TLS certificates

72602 Cluster DNS

All subdomains use A records → 47.110.67.161 (ECS public IP):

72602.online                  A  47.110.67.161
argocd.72602.online           A  47.110.67.161
ops.docs.72602.online         A  47.110.67.161
token.72602.space             A  47.110.67.161
home.72602.online             A  47.110.67.161
n8n.72602.online              A  47.110.67.161
webhook.n8n.72602.online      A  47.110.67.161
ops.agent.72602.online        A  47.110.67.161
uptime.72602.space            A  47.110.67.161
clash.72602.online            A  47.110.67.161
api.minio.72602.online        A  47.110.67.161
console.minio.72602.online    A  47.110.67.161

Add these records in your DNS provider (Cloudflare / Aliyun DNS / etc.).

Traffic Flow

Browser → A record → 47.110.67.161:443
                         ↓ ECS sshd (reverse tunnel from minipc)
                       127.0.0.1:32443 (minipc ingress-nginx NodePort via SSH -R)
                         ↓ ingress-nginx NodePort
                      k3s ingress controller
                         ↓ Ingress rules
                      Service → Pod

TLS Certificates

With ingress-nginx + cert-manager, TLS is automatic:

Install cert-manager (see Installation/Networking/Cert_Manager.md)
Create a ClusterIssuer named lets-encrypt
Each Ingress gets annotation cert-manager.io/cluster-issuer: lets-encrypt
cert-manager automatically provisions TLS certificates via HTTP-01 challenge

⚠️ For clusters behind NAT/SSH reverse tunnel (like 72602), HTTP-01 validation requires the domain to resolve to the public IP and port 80/443 to reach the cluster. This works because ECS forwards 80/443 → minipc NodePort via SSH reverse tunnel (managed by autossh systemd services).

Prerequisites

Before deploying any app, ensure:

Domain registered and DNS manageable
k3s cluster running
ingress-nginx installed (NodePort mode)
cert-manager installed with lets-encrypt ClusterIssuer
ArgoCD installed
DNS A records added for each target subdomain

Cert Manager

Install Cert Manager

Installation

Install By

Preliminary

1. Kubernetes is installed; if not, check 🔗link

2. Helm binary is installed; if not, check 🔗link

1.get helm repo

helm repo add cert-manager-repo https://charts.jetstack.io
helm repo update

2.install chart

helm install cert-manager-repo/cert-manager --generate-name --version 1.20.2

Using Mirror

helm repo add ay-helm-mirror https://aaronyang0628.github.io/helm-chart-mirror/charts \
  && helm install ay-helm-mirror/cert-manager --generate-name --version 1.20.2

for more information, you can check 🔗https://aaronyang0628.github.io/helm-chart-mirror/

Preliminary

1. Kubernetes is installed; if not, check 🔗link

2. ArgoCD is installed; if not, check 🔗link

3. Helm binary is installed; if not, check 🔗link

1.prepare `cert-manager.yaml`

kubectl -n argocd apply -f https://raw.githubusercontent.com/AaronYang0628/docs/main/manifests/zjlab/cert-manager.yaml

kubectl -n argocd apply -f - << EOF
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: cert-manager
spec:
  syncPolicy:
    syncOptions:
    - CreateNamespace=true
  project: default
  source:
    repoURL: https://aaronyang0628.github.io/helm-chart-mirror/charts
    chart: cert-manager
    targetRevision: 1.20.2
    helm:
      releaseName: cert-manager
      values: |
        installCRDs: true
        image:
          repository: m.daocloud.io/quay.io/jetstack/cert-manager-controller
          tag: v1.20.2
        webhook:
          image:
            repository: m.daocloud.io/quay.io/jetstack/cert-manager-webhook
            tag: v1.20.2
        cainjector:
          image:
            repository: m.daocloud.io/quay.io/jetstack/cert-manager-cainjector
            tag: v1.20.2
        acmesolver:
          image:
            repository: m.daocloud.io/quay.io/jetstack/cert-manager-acmesolver
            tag: v1.20.2
        startupapicheck:
          image:
            repository: m.daocloud.io/quay.io/jetstack/cert-manager-startupapicheck
            tag: v1.20.2
  destination:
    server: https://kubernetes.default.svc
    namespace: basic-components
EOF

kubectl -n argocd apply -f - << EOF
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: cert-manager
spec:
  syncPolicy:
    syncOptions:
    - CreateNamespace=true
  project: default
  source:
    repoURL: https://charts.jetstack.io
    chart: cert-manager
    targetRevision: 1.20.2
    helm:
      releaseName: cert-manager
      values: |
        installCRDs: true
        image:
          repository: m.daocloud.io/quay.io/jetstack/cert-manager-controller
          tag: v1.20.2
        webhook:
          image:
            repository: m.daocloud.io/quay.io/jetstack/cert-manager-webhook
            tag: v1.20.2
        cainjector:
          image:
            repository: m.daocloud.io/quay.io/jetstack/cert-manager-cainjector
            tag: v1.20.2
        acmesolver:
          image:
            repository: m.daocloud.io/quay.io/jetstack/cert-manager-acmesolver
            tag: v1.20.2
        startupapicheck:
          image:
            repository: m.daocloud.io/quay.io/jetstack/cert-manager-startupapicheck
            tag: v1.20.2        
  destination:
    server: https://kubernetes.default.svc
    namespace: basic-components
EOF

2.sync by argocd

argocd app sync argocd/cert-manager

Preliminary

1. Docker|Podman|Buildah is installed; if not, check 🔗link

1.just run

docker run --name cert-manager -e ALLOW_EMPTY_PASSWORD=yes bitnami/cert-manager:latest

Using Proxy

you can run an addinational daocloud image to accelerate your pulling, check Daocloud Proxy

docker run --name cert-manager \
  -e ALLOW_EMPTY_PASSWORD=yes 
  m.daocloud.io/docker.io/bitnami/cert-manager:latest

Preliminary

1. Kubernetes is installed; if not, check 🔗link

1.just run

kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.20.2/cert-manager.yaml

Prepare Certificate Issuer

kubectl apply  -f - <<EOF
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  namespace: basic-components
  name: self-signed-issuer
spec:
  selfSigned: {}

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  namespace: basic-components
  name: my-self-signed-ca
spec:
  isCA: true
  commonName: my-self-signed-ca
  secretName: root-secret
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: self-signed-issuer
    kind: Issuer
    group: cert-manager.io

---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: self-signed-ca-issuer
spec:
  ca:
    secretName: root-secret
EOF

kubectl apply -f - << EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: lets-encrypt
spec:
  acme:
    email: byang628@zhejianglab.org
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-account-key
    solvers:
    - http01:
        ingress:
          class: nginx
EOF

kubectl -n kube-system apply -f - << EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: lets-encrypt
spec:
  acme:
    email: aaron19940628@gmail.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-account-key
    solvers:
    - http01:
        ingress:
          class: nginx
EOF

FAQ

Q1: cert-manager Pods are not ready

Symptom

cert-manager, cert-manager-webhook, or cert-manager-cainjector Pods keep restarting.

Check

kubectl -n basic-components get pods | grep cert-manager
kubectl -n basic-components describe pod -l app.kubernetes.io/name=cert-manager
kubectl -n basic-components logs deploy/cert-manager --tail=100

Fix

Ensure CRDs are installed (installCRDs: true in Helm values).
Re-sync app: argocd app sync argocd/cert-manager.
If image pull fails, switch to mirror image settings shown above.

Expected

All cert-manager Pods become Running and READY 1/1.

Q2: Certificate stays in Pending or Ingress has no TLS secret

Symptom

Ingress TLS secret is not created, or Certificate status does not become Ready.

Check

kubectl get clusterissuer
kubectl -n basic-components get certificate
kubectl -n basic-components describe certificate <certificate-name>
kubectl -n basic-components get secret root-secret

Fix

Confirm ClusterIssuer exists (self-signed-ca-issuer or lets-encrypt).
Ensure Ingress annotation matches issuer name: cert-manager.io/cluster-issuer.
Re-apply issuer manifest and then re-apply ingress.

Expected

Target certificate shows Ready=True and TLS secret exists.

FAQ

Q1: The browser doesn’t trust this self-signed certificate

Basically, you need to import the certificate into your browser.

kubectl -n basic-components get secret root-secret -o jsonpath='{.data.tls\.crt}' | base64 -d > cert-manager-self-signed-ca-secret.crt

And then import it into your browser.

Q2: Show me almost endless possibilities

You can add standard markdown syntax:

multiple paragraphs
bullet point lists
emphasized, bold and even bold emphasized text
links
etc.

...and even source code

the possibilities are endless (almost - including other shortcodes may or may not work)

Install HAProxy

🚀Installation

Environment

Install By

1.prepare ECS host and backups

ssh root@47.110.67.161 hostname
sudo ss -ltnp
sudo test -e /etc/haproxy || echo absent

Back up /etc/haproxy before installing. Keep the backup outside tracked source. The 72602 mail change also keeps the reverse-tunnel unit backup on the minipc.

2.install Ubuntu package

sudo apt-get install --no-install-recommends haproxy

Do not run upgrade, full-upgrade, or dist-upgrade. Ubuntu may start the package service during installation; stop it before replacing the default configuration and before the tunnel backends are ready.

3.configure TCP passthrough

/etc/haproxy/haproxy.cfg must use mode tcp, bind public IPv4 and IPv6 25, 465, 587, and 993, and send PROXY v2 to:

127.0.0.1:10225
127.0.0.1:10465
127.0.0.1:10587
127.0.0.1:10993

Do not terminate TLS or configure HTTP, relay, or authentication in HAProxy. Validate before starting it:

haproxy -c -f /etc/haproxy/haproxy.cfg

4.start after tunnel verification

# Confirm the four ECS loopback backends are sshd listeners first.
sudo ss -ltnp
sudo systemctl enable --now haproxy
sudo systemctl is-active haproxy
sudo systemctl is-enabled haproxy
sudo ss -ltnp

The public mail ports must belong to haproxy; only the four high ports on 127.0.0.1 may belong to the tunnel sshd.

5.rollback

sudo systemctl stop haproxy
# Restore the saved 10022 unit on 72602-minipc, daemon-reload, restart only 10022.

Do not restore an old HAProxy configuration or uninstall the package as part of this rollback. Preserve Mailu Secrets and PVCs.

🛎️FAQ

Mailu does not accept PROXY protocol

Check the transport source that reaches the Mailu front Pod. The current 72602 hostPort path is observed as 10.42.0.1, while the current Git values use realIpFrom=127.0.0.1/32. Dovecot logs Client not trusted when these do not match, and TLS/STARTTLS can close before negotiation. Correct the source or trust boundary in Git, wait for automatic ArgoCD reconciliation, and repeat the protocol and no-DATA relay checks. Do not manually patch or restart Mailu.

Ingress

Install Ingress

Installation

Install By

Preliminary

1. Kubernetes is installed; if not, check 🔗link

2. Helm is installed; if not, check 🔗link

1.get helm repo

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

2.install chart

helm install ingress-nginx/ingress-nginx --generate-name

Using Mirror

helm repo add ay-helm-mirror https://aaronyang0628.github.io/helm-chart-mirror/charts &&
  helm install ay-helm-mirror/ingress-nginx --generate-name --version 4.15.1

for more information, you can check 🔗https://aaronyang0628.github.io/helm-chart-mirror/

Preliminary

1. Kubernetes is installed; if not, check 🔗link

2. ArgoCD is installed; if not, check 🔗link

1.prepare `ingress-nginx.yaml`

```shell kubectl -n argocd apply -f https://raw.githubusercontent.com/AaronYa ```

kubectl -n argocd apply -f - <<EOF class=w>apiVersion: argoproj.io/v1alpha1 class=w>kind: Application class=w>metadata: name: ingress-nginx class=w>spec: syncPolicy: syncOptions: - CreateNamespace=true project: default source: repoURL: https://kubernetes.github.io/ingress-nginx chart: ingress-nginx targetRevision: 4.15.1 helm: releaseName: ingress-nginx values: | controller: image: registry: m.daocloud.io/registry.k8s.io service: enabled: true type: NodePort nodePorts: http: 32080 https: 32443 tcp: 8080: 32808 5324: 33224 #pg resources: requests: cpu: 100m memory: 128Mi admissionWebhooks: enabled: true patch: enabled: true image: registry: m.daocloud.io/registry.k8s.io metrics: enabled: false defaultBackend: enabled: false image: registry: m.daocloud.io/registry.k8s.io destination: server: https://kubernetes.default.svc namespace: basic-components class=w>EOF

[Optional] 2.apply to k8s

kubectl -n argocd apply -f ingress-nginx.yaml

3.sync by argocd

argocd app sync argocd/ingress-nginx

FAQ

Q1: Ingress created but cannot access the domain

Symptom

curl -k https://<your-domain> fails or times out.

Check

kubectl -n basic-components get svc
kubectl -n basic-components get pod -l app.kubernetes.io/component=controller
kubectl -n basic-components get ingress

Fix

Confirm your domain resolves to the node IP (or add /etc/hosts entry).
Ensure ingress-nginx Service exposes NodePort 32443 for HTTPS.
Re-sync app: argocd app sync argocd/ingress-nginx.

Expected

Ingress controller Pods are Running and HTTPS endpoint is reachable.

Q2: Using minikube and NodePort is unreachable

Symptom

Browser cannot open https://$(minikube ip):32443.

Check and fix

kubectl -n basic-components get svc
ssh -i ~/.minikube/machines/minikube/id_rsa docker@$(minikube ip) -L '*:30443:0.0.0.0:30443' -N -f
ssh -i ~/.minikube/machines/minikube/id_rsa docker@$(minikube ip) -L '*:32443:0.0.0.0:32443' -N -f
ssh -i ~/.minikube/machines/minikube/id_rsa docker@$(minikube ip) -L '*:32080:0.0.0.0:32080' -N -f

Expected

You can access the ingress endpoint through forwarded ports.

Istio

Install Istio

Installation

Install By

Preliminary

1. Kubernetes is installed; if not, check 🔗link

2. Helm is installed; if not, check 🔗link

1.get helm repo

helm repo add ay-helm-mirror https://aaronyang0628.github.io/helm-chart-mirror/charts
helm repo update

2.install chart

helm install ay-helm-mirror/kube-prometheus-stack --generate-name

Using Proxy

for more information, you can check 🔗https://artifacthub.io/packages/helm/prometheus-community/prometheus

Preliminary

1. Kubernetes is installed; if not, check 🔗link

2. Helm is installed; if not, check 🔗link

3. ArgoCD is installed; if not, check 🔗link

1.prepare `deploy-istio-base.yaml`

kubectl -n argocd apply -f - << EOF
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: istio-base
spec:
  syncPolicy:
    syncOptions:
    - CreateNamespace=true
  project: default
  source:
    repoURL: https://istio-release.storage.googleapis.com/charts
    chart: base
    targetRevision: 1.23.2
    helm:
      releaseName: istio-base
      values: |
        defaults:
          global:
            istioNamespace: istio-system
          base:
            enableCRDTemplates: false
            enableIstioConfigCRDs: true
          defaultRevision: "default"
  destination:
    server: https://kubernetes.default.svc
    namespace: istio-system
EOF

2.sync by argocd

argocd app sync argocd/istio-base

3.prepare `deploy-istiod.yaml`

kubectl -n argocd apply -f - << EOF
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: istiod
spec:
  syncPolicy:
    syncOptions:
    - CreateNamespace=true
  project: default
  source:
    repoURL: https://istio-release.storage.googleapis.com/charts
    chart: istiod
    targetRevision: 1.23.2
    helm:
      releaseName: istiod
      values: |
        defaults:
          global:
            istioNamespace: istio-system
            defaultResources:
              requests:
                cpu: 10m
                memory: 128Mi
              limits:
                cpu: 100m
                memory: 128Mi
            hub: m.daocloud.io/docker.io/istio
            proxy:
              autoInject: disabled
              resources:
                requests:
                  cpu: 100m
                  memory: 128Mi
                limits:
                  cpu: 2000m
                  memory: 1024Mi
          pilot:
            autoscaleEnabled: true
            resources:
              requests:
                cpu: 500m
                memory: 2048Mi
            cpu:
              targetAverageUtilization: 80
            podAnnotations:
              cluster-autoscaler.kubernetes.io/safe-to-evict: "true"
  destination:
    server: https://kubernetes.default.svc
    namespace: istio-system
EOF

4.sync by argocd

argocd app sync argocd/istiod

5.prepare `deploy-istio-ingressgateway.yaml`

kubectl -n argocd apply -f - << EOF
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: istio-ingressgateway
spec:
  syncPolicy:
    syncOptions:
    - CreateNamespace=true
  project: default
  source:
    repoURL: https://istio-release.storage.googleapis.com/charts
    chart: gateway
    targetRevision: 1.23.2
    helm:
      releaseName: istio-ingressgateway
      values: |
        defaults:
          replicaCount: 1
          podAnnotations:
            inject.istio.io/templates: "gateway"
            sidecar.istio.io/inject: "true"
            cluster-autoscaler.kubernetes.io/safe-to-evict: "true"
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 2000m
              memory: 1024Mi
          service:
            type: LoadBalancer
            ports:
            - name: status-port
              port: 15021
              protocol: TCP
              targetPort: 15021
            - name: http2
              port: 80
              protocol: TCP
              targetPort: 80
            - name: https
              port: 443
              protocol: TCP
              targetPort: 443
          autoscaling:
            enabled: true
            minReplicas: 1
            maxReplicas: 5
  destination:
    server: https://kubernetes.default.svc
    namespace: istio-system
EOF

6.sync by argocd

argocd app sync argocd/istio-ingressgateway

Preliminary

1. Kubernetes is installed; if not, check 🔗link

2. Helm is installed; if not, check 🔗link

3. ArgoCD is installed; if not, check 🔗link

4. Argo Workflow is installed; if not, check 🔗link

1.prepare `argocd-login-credentials`

kubectl get namespaces database > /dev/null 2>&1 || kubectl create namespace database

2.apply rolebinding to k8s

kubectl apply -f - <<EOF
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: application-administrator
rules:
  - apiGroups:
      - argoproj.io
    resources:
      - applications
    verbs:
      - '*'
  - apiGroups:
      - apps
    resources:
      - deployments
    verbs:
      - '*'

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: application-administration
  namespace: argocd
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: application-administrator
subjects:
  - kind: ServiceAccount
    name: argo-workflow
    namespace: business-workflows

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: application-administration
  namespace: application
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: application-administrator
subjects:
  - kind: ServiceAccount
    name: argo-workflow
    namespace: business-workflows
EOF

4.prepare `deploy-xxxx-flow.yaml`

6.submit to argo workflow client

argo -n business-workflows submit deploy-xxxx-flow.yaml

7.decode password

kubectl -n application get secret xxxx-credentials -o jsonpath='{.data.xxx-password}' | base64 -d

FAQ

Q1: Show me almost endless possibilities

You can add standard markdown syntax:

multiple paragraphs
bullet point lists
emphasized, bold and even bold emphasized text
links
etc.

...and even source code

the possibilities are endless (almost - including other shortcodes may or may not work)

Q2: Show me almost endless possibilities

You can add standard markdown syntax:

multiple paragraphs
bullet point lists
emphasized, bold and even bold emphasized text
links
etc.

...and even source code

the possibilities are endless (almost - including other shortcodes may or may not work)

Install Nginx

1. prepare `server.conf`

cat << EOF > default.conf
server {
  listen 80;
  location / {
      root   /usr/share/nginx/html;
      autoindex on;
  }
}
EOF

2. install

mkdir $(pwd)/data
podman run --rm -p 8080:80 \
    -v $(pwd)/data:/usr/share/nginx/html:ro \
    -v $(pwd)/default.conf:/etc/nginx/conf.d/default.conf:ro \
    -d docker.io/library/nginx:1.19.9-alpine
echo 'this is a test' > $(pwd)/data/some-data.txt

Tip

you can run an addinational daocloud image to accelerate your pulling, check Daocloud Proxy

visit http://localhost:8080

Networking

Subsections of Networking

DNS Setup

Overview

72602 Cluster DNS

Traffic Flow

TLS Certificates

Prerequisites

Install Cert Manager

Installation

Prepare Certificate Issuer

FAQ

FAQ

Install HAProxy

🚀Installation

🛎️FAQ

Install Ingress

Installation

FAQ

Install Istio

Installation

FAQ

Install Nginx

1. prepare server.conf

2. install

Install Traefik

1. prepare `server.conf`