linux/mac 本地环境(使用sshuttle)通过sshd访问k8s内网

linux/mac 本地环境(使用sshuttle)通过sshd访问k8s内网

参考：提高开发效率：打通 K8s 与本地之间的网络 - 陪她去流浪

sshuttle：搭建基于 SSH 的简易 VPN - 早起搬砖 morning.work

sshuttle · PyPI

分别生成公钥和私钥

K8S所在的服务器（推荐使用非root用户）和本地开发环境(linux/mac)执行

$ ssh-keygen

进入~/.ssh/目录查看

部署一个sshd服务，并暴露NodePort端口供本地客户端访问

本例中的sshd镜像在运行时会安装ptyhon，时间会比较长，为了省时间，

可以考虑先自己打个镜像，并推送到自己的仓库里，避免反复调试时等待python安装过程

vi dockerfile
FROM docker.io/panubo/sshd:latest

RUN apk add python2

# https://blog.csdn.net/qq_42533216/article/details/108225616
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories
RUN apk upgrade
RUN apk del curl
RUN apk add curl

# https://github.com/panubo/docker-sshd/blob/main/Dockerfile
ENTRYPOINT ["/entry.sh"]

CMD ["/usr/sbin/sshd", "-D", "-e", "-f", "/etc/ssh/sshd_config"]

自己打个镜像

docker build -t my-sshd:latest .

docker tag

docker push

准备部署sshd服务

有镜像后，将yaml文件中的

image: panubo/sshd修改成本地仓库的镜像路径

删除容器中安装python的代码apk add python2

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sshd
  labels:
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: sshd
      app.kubernetes.io/instance: sshd
  template:
    metadata:
      labels:
        app.kubernetes.io/name: sshd
        app.kubernetes.io/instance: sshd
    spec:
      hostname: pipeline
      containers:
        - name: sshd
          image: panubo/sshd
          imagePullPolicy: Always
          env:
          - name: SSH_ENABLE_ROOT
            value: "true"
          command:
          - /bin/sh
          - -c
          args:
          - |
            set -euo pipefail
            /entry.sh
            echo 'ssh-rsa xxx远程K8S所在机器的公钥xxx= yourname@yourmachine' > /etc/ssh/keys/ssh_host_ecdsa_key.pub
            echo '-----BEGIN OPENSSH PRIVATE KEY-----
            b4BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAaAAAABNlY2RzYS
            xxx远程K8S所在机器的私钥xxx
            sAAAAhANr5qpUS1qt0Thpli78qrLD61kUki9V2+ss3KlTPmsW/AAAAEXJvb3RAZTY5NzVl
            MjgyODE0AQIDBAUG
            -----END OPENSSH PRIVATE KEY-----
            ' > /etc/ssh/keys/ssh_host_ecdsa_key
            echo 'xxx本地开发环境机器的公钥xxx= yourname@yourmachine' >> /root/.ssh/authorized_keys
            sed -i 's/GatewayPorts no/GatewayPorts yes/' /etc/ssh/sshd_config
            sed -i 's/AllowTcpForwarding no/AllowTcpForwarding yes/' /etc/ssh/sshd_config
            apk add python2
            /usr/sbin/sshd -D -e -f /etc/ssh/sshd_config
---
apiVersion: v1
kind: Service
metadata:
  name: sshd
  labels:
spec:
  type: NodePort
  ports:
    - name: sshd
      nodePort: 32222
      port: 22
      targetPort: 22
      protocol: TCP
  selector:
    app.kubernetes.io/name: sshd
    app.kubernetes.io/instance: sshd

部署应用，推荐部署在sshd-system系统空间下，避免其它用户的骚操作

kubectl create ns sshd-system

kubectl apply -f sshd.yaml -n sshd-system

本地开发环境~/.ssh/config添加ssh快捷登录记录

Host sshd
    # 这是 K8s 所在机器某一节点
    HostName youripxxx
        # sshd 服务的 NodePort
    Port 32222
    User developer
        # 客户端机器的 的私钥
    IdentityFile ~/.ssh/sshd_id_ed25519

ssh 到 K8s 内部的 sshd 服务：

$ ssh sshd
Welcome to Alpine!

The Alpine Wiki contains a large amount of how-to guides and general
information about administrating Alpine systems.
See <http://wiki.alpinelinux.org/>.

You can setup the system with the command: setup-alpine

You may change this message by editing /etc/motd.

ssh:~# 

开发环境机器安装sshuttle

参考:sshuttle · PyPI

git clone https://github.com/sshuttle/sshuttle.git
cd sshuttle
 ./setup.py install

启动sshuttle服务

$ sshuttle -r sshd 0.0.0.0/0 --disable-ipv6

Connected to server.

换个窗口查看svc

$ kubectl get svc -A

找出各service所对应的ip，并将其配置到本地开发环境的hosts中

例如：

[root@master02 ~]# cat /etc/hosts

10.124.112.188 harbor-portal.harbor

刚部署的sshd服务如果与要访问service不在一个空间，则需要在域名后补齐空间（例如上例中的harbor就是一个空间名）

本地开发环境访问服务

$ curl harbor-portal.harbor

返回80端口对应的服务HTML代码

不要使用ping去访问这个service，根本就ping不通，sshd所在的pod与其它pod是一个网段可以ping通，但pod与service并不在一个网段

参考：Access Services Running on Clusters | Kubernetes

Access services, nodes, or pods using the Proxy Verb

Only works for HTTP/HTTPS.

Some clusters may allow you to ssh to a node in the cluster（本例是ssh到其中一个pod）. From there you may be able to access cluster services. This is a non-standard method, and will work on some clusters but not others. Browsers and other tools may or may not be installed. Cluster DNS may not work(本例中需要自己配置本地host)

将其它机器添加到白名单中

复制出白名单

kubectl cp -n sshd-system sshd-5fffbbc7dd-bhv9g:/root/.ssh/authorized_keys authorized_keys

vi authorized_keys

在白名单中加入新的公钥

再将白名单复制到容器中

kubectl cp -n sshd-system authorized_keys sshd-5fffbbc7dd-bhv9g:/root/.ssh/authorized_keys

本地执行

ssh sshd

Welcome to Alpine!

The Alpine Wiki contains a large amount of how-to guides and general
information about administrating Alpine systems.
See <http://wiki.alpinelinux.org/>.

You can setup the system with the command: setup-alpine

You may change this message by editing /etc/motd.

pipeline:~#

其它参考

完美解决 Could not find a version that satisfies the requirement 安装包名字 (from versions: )-CSDN博客

k8s部署openvpn打通k8s网络将Kubernetes集群网络暴露给本地开发网络 - Myki的博客

打通Kubernetes内网与局域网的N种方法 - 知乎