Kubernetes 核心实战之三（精华篇 3/3）

6、Ingress ★

Ingress：Service 的统一网关入口，底层就是 nginx。（服务）

官网地址：https://kubernetes.github.io/ingress-nginx/ （都是从这里看的）

所有的请求都先通过 Ingress，由 Ingress 来 打理这些请求。类似微服务中的 网关。

6.1 安装 Ingress

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.47.0/deploy/static/provider/baremetal/deploy.yaml

## 这里我喜欢把depoly.yaml名字修改为ingress.yaml

# 修改镜像
vi ingress.yaml
# 将 image 的值改为如下值
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/ingress-nginx-controller:v0.46.0

# 安装资源
kubectl apply -f ingress.yaml

# 检查安装的结果
kubectl get pod,svc -n ingress-nginx

# 最后别忘记把 svc 暴露的端口 在安全组放行

ingress.yaml整个文件内容：

apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx

---
# Source: ingress-nginx/templates/controller-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx
  namespace: ingress-nginx
automountServiceAccountToken: true
---
# Source: ingress-nginx/templates/controller-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
data:
---
# Source: ingress-nginx/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
  name: ingress-nginx
rules:
  - apiGroups:
      - ''
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ''
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ''
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ''
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - extensions
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingressclasses
    verbs:
      - get
      - list
      - watch
---
# Source: ingress-nginx/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
  - kind: ServiceAccount
    name: ingress-nginx
    namespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ''
    resources:
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ''
    resources:
      - configmaps
      - pods
      - secrets
      - endpoints
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ''
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingressclasses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ''
    resources:
      - configmaps
    resourceNames:
      - ingress-controller-leader-nginx
    verbs:
      - get
      - update
  - apiGroups:
      - ''
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ''
    resources:
      - events
    verbs:
      - create
      - patch
---
# Source: ingress-nginx/templates/controller-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
  - kind: ServiceAccount
    name: ingress-nginx
    namespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-service-webhook.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  type: ClusterIP
  ports:
    - name: https-webhook
      port: 443
      targetPort: webhook
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/component: controller
  revisionHistoryLimit: 10
  minReadySeconds: 0
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/component: controller
    spec:
      dnsPolicy: ClusterFirst
      containers:
        - name: controller
          image: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/ingress-nginx-controller:v0.46.0
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          args:
            - /nginx-ingress-controller
            - --election-id=ingress-controller-leader
            - --ingress-class=nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            runAsUser: 101
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          livenessProbe:
            failureThreshold: 5
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
            - name: webhook
              containerPort: 8443
              protocol: TCP
          volumeMounts:
            - name: webhook-cert
              mountPath: /usr/local/certificates/
              readOnly: true
          resources:
            requests:
              cpu: 100m
              memory: 90Mi
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
        - name: webhook-cert
          secret:
            secretName: ingress-nginx-admission
---
# Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
# before changing this value, check the required kubernetes version
# https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  name: ingress-nginx-admission
webhooks:
  - name: validate.nginx.ingress.kubernetes.io
    matchPolicy: Equivalent
    rules:
      - apiGroups:
          - networking.k8s.io
        apiVersions:
          - v1beta1
        operations:
          - CREATE
          - UPDATE
        resources:
          - ingresses
    failurePolicy: Fail
    sideEffects: None
    admissionReviewVersions:
      - v1
      - v1beta1
    clientConfig:
      service:
        namespace: ingress-nginx
        name: ingress-nginx-controller-admission
        path: /networking/v1beta1/ingresses
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  namespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
rules:
  - apiGroups:
      - admissionregistration.k8s.io
    resources:
      - validatingwebhookconfigurations
    verbs:
      - get
      - update
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
  - kind: ServiceAccount
    name: ingress-nginx-admission
    namespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ''
    resources:
      - secrets
    verbs:
      - get
      - create
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
  - kind: ServiceAccount
    name: ingress-nginx-admission
    namespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: ingress-nginx-admission-create
  annotations:
    helm.sh/hook: pre-install,pre-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  namespace: ingress-nginx
spec:
  template:
    metadata:
      name: ingress-nginx-admission-create
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
    spec:
      containers:
        - name: create
          image: docker.io/jettech/kube-webhook-certgen:v1.5.1
          imagePullPolicy: IfNotPresent
          args:
            - create
            - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
            - --namespace=$(POD_NAMESPACE)
            - --secret-name=ingress-nginx-admission
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx-admission
      securityContext:
        runAsNonRoot: true
        runAsUser: 2000
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: ingress-nginx-admission-patch
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  namespace: ingress-nginx
spec:
  template:
    metadata:
      name: ingress-nginx-admission-patch
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
    spec:
      containers:
        - name: patch
          image: docker.io/jettech/kube-webhook-certgen:v1.5.1
          imagePullPolicy: IfNotPresent
          args:
            - patch
            - --webhook-name=ingress-nginx-admission
            - --namespace=$(POD_NAMESPACE)
            - --patch-mutating=false
            - --secret-name=ingress-nginx-admission
            - --patch-failure-policy=Fail
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx-admission
      securityContext:
        runAsNonRoot: true
        runAsUser: 2000

耐心等待pod创建完成，所有状态变成running。

开放青云服务器端口 30000-32767

查看 映射后的 IP

kubectl get pod,svc -n ingress-nginx

6.2 访问

每台服务器 都开放 映射后的 端口：
31737、32641

https://xxxxxxxx:32641
http://xxxxxxxxx:31737

6.3 安装不成功的bug解决

查看pod，一直在创建中。

使用describe，查看详细问题日志：

kubectl describe pod ingress-nginx-controller-65bf56f7fc-nwgff -n ingress-nginx

解决办法：



成功启动：

重新查看端口并访问

6.4 测试使用

官网地址：https://kubernetes.github.io/ingress-nginx/
ingress底层其实就是nginx。

6.4.1 搭建测试环境

创建两个 Service和deployment

vi test.yaml

# 复制下面

kubectl apply -f test.yaml

test.yaml整个文件内容：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-server
spec:
  replicas: 2
  selector:
    matchLabels:
      app: hello-server
  template:
    metadata:
      labels:
        app: hello-server
    spec:
      containers:
      - name: hello-server
        image: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/hello-server
        ports:
        - containerPort: 9000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx-demo
  name: nginx-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-demo
  template:
    metadata:
      labels:
        app: nginx-demo
    spec:
      containers:
      - image: nginx
        name: nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx-demo
  name: nginx-demo
spec:
  selector:
    app: nginx-demo
  ports:
  - port: 8000
    protocol: TCP
    targetPort: 80
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: hello-server
  name: hello-server
spec:
  selector:
    app: hello-server
  ports:
  - port: 8000
    protocol: TCP
    targetPort: 9000

6.4.2 配置 Ingress的规则

vi ingress-rule.yaml

# 复制下面配置

kubectl apply -f ingress-rule.yaml

# 查看 集群中的 Ingress
kubectl get ingress

ingress-rule.yaml完整内容：

apiVersion: networking.k8s.io/v1
kind: Ingress  
metadata:
  name: ingress-host-bar
spec:
  ingressClassName: nginx
  rules:
  - host: "hello.tigerhhzz.com"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: hello-server
            port:
              number: 8000 # hello-server （service） 的端口是 8000
  - host: "demo.tigerhhzz.com"
    http:
      paths:
      - pathType: Prefix
        path: "/"  # 把请求会转给下面的服务，下面的服务一定要能处理这个路径，不能处理就是404
        backend:
          service:
            name: nginx-demo  #java，比如使用路径重写，去掉前缀nginx
            port:
              number: 8000


apiVersion: networking.k8s.io/v1
kind: Ingress  
metadata:
  name: ingress-host-bar
spec:
  ingressClassName: nginx
  rules:
    - host: "hello.tigerhhzz.com"
      http:
        paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: hello-server
                port:
                  number: 8000 # hello-server 的端口是 8000
    - host: "demo.tigerhhzz.com"
      http:
        paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: nginx-demo
                port:
                  number: 8000

解决上面错误的方法：

kubectl get ValidatingWebhookConfiguration
# 把该死的 admission 删掉
kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission

6.4.3 测试I

在 自己电脑（不是虚拟机） hosts 中增加映射：
master的公网IP hello.tigerhhzz.com
master的公网IP demo.tigerhhzz.com

6.4.4 测试II

# kubectl get ingress
kubectl get ing

kubectl edit ing ingress的NAME -n

# 改变匹配的 path
  - host: "demo.tigerhhzz.com"
    http:
      paths:
      - pathType: Prefix
        path: "/nginx" # 匹配请求 /nginx 的，并且查找 nginx 文件.
        backend:
          service:
            name: nginx-demo
            port:
              number: 8000

随便写 /xxx 不匹配 nginx的，都返回 Ingress的 404的nginx

下面这个是 通过了 Ingress，Service 里的 Pod 没匹配到，才返回的 404（下面打印的 nginx 版本不一样的）

页面 进入 Pod 的那个nginx

cd /usr/share/nginx/html
ls
echo "hello tigerhhzz" > nginx

6.4.5 路径重写

修改ingress配置规则，下面这个这样子 就和 SpringCloud Gateway 网管转发一样的效果了。
ingress-rule.yaml的完整内容如下：

apiVersion: networking.k8s.io/v1
kind: Ingress  
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
  name: ingress-host-bar
spec:
  ingressClassName: nginx
  rules:
  - host: "hello.tigerhhzz.com"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: hello-server
            port:
              number: 8000
  - host: "demo.tigerhhzz.com"
    http:
      paths:
      - pathType: Prefix
        path: "/nginx(/|$)(.*)" 
        backend:
          service:
            name: nginx-demo  
            port:
              number: 8000

在可视化界面 创建

6.4.6 限流

官网文档：https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-limit-rate
  annotations:
    # 限流
    nginx.ingress.kubernetes.io/limit-rps: "1"
spec:
  ingressClassName: nginx
  rules:
  - host: "haha.tigerhhzz.com"
    http:
      paths:
      - pathType: Exact
        path: "/"
        backend:
          service:
            name: nginx-demo
            port:
              number: 8000

vim ingress-rule-2.yaml

# 复制上面配置

kubectl apply -f ingress-rule-2.yaml

kubect get ing

在可视化界面 创建ingress


在 自己电脑（不是虚拟机） hosts 中增加映射：
公网IP haha.tigerhhzz.com

访问测试：

http://haha.tigerhhzz.com:30808/

刷新过快 返回 503，官网文档也写了。


速率限制
这些注释定义了连接和传输速率的限制。这些可用于缓解DDoS 攻击

• nginx.ingress.kubernetes.io/limit-connections:允许来自单个IP 地址的并发连接数超过此限制时返回 503 错误
• nginx.ingress.kubernetes.io/limit-rps :每秒从给定IP 接受的请求数。突发限制设置为此限制乘以突发倍数，默认倍数为 5。当客户端超过此限制时，返回limit-reg-status-codedefault: 503
• nginx.ingress,kubernetes.io/limit-rpm:每分钟从给定IP 接受的请求数。突发限制设署为此限制乘以突发倍数，默认倍数为 5。当客户端超过此限制时，返回limit-reg-status-codedefault: 503
• nginx.ingress.kubernetes.io/limit-burst-multiplier:突发大小限制率的乘数。默认突发乘数为5，此注释覆盖默认乘数。当客户端超过此限制时，返回limit-req-status-codedefault: 503
• nginx.ingress.kubernetes.io/limit-rate-after :初始千字节数，之后对给定连接的响应的进一步传输将受到速率限制。此功能必须在启用代理缓冲的情况下使用
• nginx.ingress.kubernetes.io/limit-rate :每秒允许发送到给定连接的千字节数。零值禁用速率限制。此功能必须在启用代理缓冲的情况下使用。
• nginx.ingress.kubernetes.io/limit-whitelist :要从速率限制中排除的客户端IP源范围。该值是一个逗号分隔的 CIDR 列表

如果您在单个Ingress 规则中指定多个注释，则会按顺序应用限制
limit-connections，limit-rpm，limit-rps。

要为所有Ingress 规则全局配置设置，可以在NGINX ConfigMap limit-rate-after
中设置和值.Ingress注解中设置的值将覆盖全局设置。limit-rate

客户端IP 地址将根据PROXY 协议的使用或启用use-forwarded-headers X-Forwarded-For
时的标头值设置。

7. Kubernetes 存储抽象

类似于 Docker 中的 挂载。但要考虑 自愈、故障转移 时的情况

7.1 NFS 搭建

网络文件系统

1、所有节点
安装nfs-utils

# 所有机器执行
yum install -y nfs-utils

2、主节点

# 只在 mster 机器执行：nfs主节点，rw 读写
echo "/nfs/data/ *(insecure,rw,sync,no_root_squash)" > /etc/exports

mkdir -p /nfs/data
systemctl enable rpcbind --now
systemctl enable nfs-server --now

# 配置生效
exportfs -r

3、从节点

# 检查，下面的 IP 是master IP
showmount -e xxx.xxx.xxx.xxx

# 在 2 个从服务器 执行，执行以下命令挂载 nfs 服务器上的共享目录到本机路径 /root/nfsmount
mkdir -p /nfs/data

# 在 2 个从服务器执行，将远程 和本地的 文件夹 挂载
mount -t nfs 139.198.36.162:/nfs/data /nfs/data

# 在 master 服务器，写入一个测试文件
echo "hello nfs server" > /nfs/data/test.txt

# 在 2 个从服务器查看
cd /nfs/data
ls

# 在 从服务器 修改，然后去 其他 服务器 查看，也能 同步

7.2 原生方式 数据挂载

在 /nfs/data/nginx-pv 挂载，然后 修改， 里面 两个 Pod 也会 同步修改。
问题：删掉之后，文件还在，内容也在，是没法管理大小的。

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx-pv-demo
  name: nginx-pv-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-pv-demo
  template:
    metadata:
      labels:
        app: nginx-pv-demo
    spec:
      containers:
      - image: nginx
        name: nginx
        volumeMounts:
        - name: html
          mountPath: /usr/share/nginx/html # 挂载目录
      volumes:
        # 和 volumeMounts.name 一样
        - name: html
          nfs:
            # master IP
            server: 192.168.27.251  #master节点ip
            path: /nfs/data/nginx-pv # 要提前创建好文件夹，否则挂载失败

cd /nfs/data
mkdir -p nginx-pv
ls

vi deploy.yaml

# 复制上面配置

kubectl apply -f deploy.yaml

kubectl get pod -owide

cd /nfs/data/
ls
cd nginx-pv/
echo "cgxin" > index.html

# 进入 pod 里面查看

问题：占用空间，删掉之后，文件还在，内容也在，是没法管理大小的。

7.3 PV 和 PVC ★

PV：持久卷（Persistent Volume），将应用需要持久化的数据保存到指定位置
PVC：持久卷申明（Persistent Volume Claim），申明需要使用的持久卷规格

挂载目录。ConfigMap 挂载配置文件。
这里是 是 静态的， 就是自己创建好了 容量，然后 PVC 去挑。 还有 动态供应的，不用手动去创建 PV池子。

7.3.1 创建 PV 池

静态供应

# 在 nfs主节点（master服务器） 执行
mkdir -p /nfs/data/01
mkdir -p /nfs/data/02
mkdir -p /nfs/data/03

使用pv.yaml 创建 3个 PV

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv01-10m
spec:
  # 限制容量
  capacity:
    storage: 10M
  # 读写模式：可读可写
  accessModes:
    - ReadWriteMany
  storageClassName: nfs
  nfs:
    # 挂载 上面创建过的文件夹
    path: /nfs/data/01
    # nfs 主节点服务器的 IP
    server: 139.198.36.162
---
apiVersion: v1
kind: PersistentVolume
metadata:
  # 这个name 要小写，如 Gi 大写就不行
  name: pv02-1gi
spec:
  capacity:
    storage: 1gi
  accessModes:
    - ReadWriteMany
  storageClassName: nfs
  nfs:
    path: /nfs/data/02
    # nfs 主节点服务器的 IP
    server: 139.198.36.162
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv03-3gi
spec:
  capacity:
    storage: 3gi
  accessModes:
    - ReadWriteMany
  storageClassName: nfs
  nfs:
    path: /nfs/data/03
    # nfs 主节点服务器的 IP
    server: 139.198.36.162

vi pv.yaml

# 复制上面文件

kubectl apply -f pv.yaml

# 查看 pv， kubectl get pv
kubectl get persistentvolume

7.3.2 创建、绑定 PCV

相当于创建设用pv的申请书

创建pvc

pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nginx-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      # 需要 200M的 PV
      storage: 200Mi
  # 上面 PV 写的什么 这里就写什么    
  storageClassName: nfs

vi pvc.yaml

# 复制上面配置

kubectl get pv

kubectl apply -f pvc.yaml

kubectl get pv

kubectl get pvc

绑定了， 绑定了1G的，10M 不够，3G太大，就选择了 1G

7.3.3 创建 Pod 绑定 PVC

创建 Pod，绑定 PVC

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx-deploy-pvc
  name: nginx-deploy-pvc
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-deploy-pvc
  template:
    metadata:
      labels:
        app: nginx-deploy-pvc
    spec:
      containers:
      - image: nginx
        name: nginx
        volumeMounts:
        - name: html
          mountPath: /usr/share/nginx/html
      volumes:
        - name: html
          # 之前是 nfs，这里用 pvc
          persistentVolumeClaim:
            claimName: nginx-pvc

vi dep02.yaml

# 复制上面 yaml

kubectl apply -f dep02.yaml

kubectl get pod

kubectl get pv

kubectl get pvc

挂载后，测试


进入 Pod 内部查看 同步的文件

7.4 ConfigMap ★

ConfigMap：抽取应用配置，并且可以自动更新。挂载配置文件， PV 和 PVC 是挂载目录的。

7.4.1 redis示例

1. 创建 ConfigMap
   创建 / 删除 cm

vi redis.conf
# 写
appendonly yes

# 创建配置，redis保存到k8s的etcd；
kubectl create cm redis-conf --from-file=redis.conf

# 查看
kubectl get cm

rm -rf redis.conf

# 查看 ConfigMap 的 yaml 配置咋写的
kubectl get cm redis-conf -oyaml

apiVersion: v1
data:    # data是所有真正的数据，key：默认是文件名   value：配置文件的内容（appendonly yes 是随便写的）
  redis.conf: |
    appendonly yes
kind: ConfigMap
metadata:
  name: redis-conf
  namespace: default

2. 创建 Pod
   redis.yaml

apiVersion: v1
kind: Pod
metadata:
  name: redis
spec:
  containers:
  - name: redis
    image: redis
    command:
      # 启动命令
      - redis-server
      # 指的是redis容器内部的位置
      - "/redis-master/redis.conf"  
    ports:
    - containerPort: 6379
    volumeMounts:
    - mountPath: /data
      name: data
    - mountPath: /redis-master
      name: config
  volumes:
    - name: data
      emptyDir: {}
    - name: config
      configMap:
        name: redis-conf
        items:
        - key: redis.conf
          path: redis.conf

redis.conf 会放在 /redis-master 下

vi redis.yaml

# 复制上面配置

kubectl apply -f redis.yaml

kubectl get pod

页面中 进入刚才创建的 pod redis 内部

查看 redis.conf 配置文件 内容

kubectl get cm

# 修改配置 里 redis.conf 的内容
kubectl edit cm redis-conf

修改 redis-conf 的 redis.conf 内容

修改 redis-conf 的 redis.conf 内容

过了一会， 就同步了

3. 检查默认配置

kubectl exec -it redis -- redis-cli

127.0.0.1:6379> CONFIG GET appendonly
127.0.0.1:6379> CONFIG GET requirepass

和 命令行一样的

删除，重新创建 Pod，更新 配置文件的 配置值

查看 更新的 配置值

总结：
● 修改了 ConfigMap，Pod里面的配置文件会跟着同步。
● 但配置值 未更改，需要重新启动 Pod 才能从关联的ConfigMap 中获取 更新的值。 Pod 部署的中间件 自己本身没有热更新能力。

7.5 Secret

Secret ：是对象类型，用来保存敏感信息，例如密码、OAuth 令牌和 SSH 密钥。 将这些信息放在 secret 中比放在 Pod 的定义或者 容器镜像 中来说更加安全和灵活。

7.5.1 拉取失败

Docker hub 仓库中，自己的仓库设置成私有的。 然后去 下载私有的。下载不了（未登录）。

自己的仓库设置成私有的
查看 拉取命令

拒绝拉取

mypod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: private-cgxin-docker
spec:
  containers:
  - name: private-cgxin-docker
    image: cgxin/cgxin_docker:1.0

vi mypod.yaml

# 复制上面配置

kubectl apply -f mypod.yaml

kubectl get pod

提示 镜像拉取失败

可视化界面 查看错误描述：也是没有权限。

删除配置文件 创建的错误 Pod

7.5.2 创建 Secret

创建 Secret

kubectl create secret docker-registry cgxin-docker-secret \
--docker-username=leifengyang \
--docker-password=Lfy123456 \
--docker-email=534096094@qq.com

##命令格式
kubectl create secret docker-registry regcred \
  --docker-server=<你的镜像仓库服务器> \
  --docker-username=<你的用户名> \
  --docker-password=<你的密码> \
  --docker-email=<你的邮箱地址>

# 查看
kubectl get secret

kubectl get secret cgxin-docker-secret -oyaml

重新修改 配置文件，加入 Secret

apiVersion: v1
kind: Pod
metadata:
  name: private-cgxin-docker
spec:
  containers:
  - name: private-cgxin-docker
    image: cgxin/cgxin_docker:1.0
  # 加上 Secret  
  imagePullSecrets:
  - name: cgxin-docker-secret

vi mypod.yaml

# 复制上面配置

kubectl apply -f mypod.yaml

kubectl get pod

使用 Secret 后，可以成功 拉取下来了。

Docker Hub 镜像 复原成 public

总结：
可视化界面 操作很方便