kubernetes存储类迁移-数据拷贝
2024-01-09 18:26:24
背景介绍
kubernetes集群最开始使用了nfs作为存储，随着后续使用过程中数据量逐渐增加，nfs存储性能逐步出现不足，现增加了基于csi的分布式块存储后，需要对原有基于nfs存储类下的pv迁移到新的存储类下。
测试环境
- k8s集群版本：1.25.12
- 节点数：4
- 旧存储类：nfs-storage
- 新存储类：rook-ceph-block
- 测试应用：mysql
- 需求：kubectl、具有pv、pvc、pod、sts、deploy控制权限
测试方法
- 基于旧存储类创建pvc和mysql应用
apiVersion: v1
kind: Service
metadata:
name: mysql
labels:
app: mysql
spec:
ports:
- port: 3306
selector:
app: mysql
clusterIP: None
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-pv-claim
labels:
app: mysql
spec:
storageClassName: nfs-storage
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
labels:
app: mysql
spec:
selector:
matchLabels:
app: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: mysql
spec:
containers:
- image: mysql:5.7
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: Passw0rd#1
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: mysql-pv-claim
- mysql建库表，插入数据
mysql> create database test;
Query OK, 1 row affected (0.00 sec)
mysql> use test;
Database changed
mysql> create table newtb (id int);
Query OK, 0 rows affected (0.02 sec)
mysql> insert into newtb (id) values (1);
Query OK, 1 row affected (0.00 sec)
mysql> insert into newtb (id) values (2);
Query OK, 1 row affected (0.00 sec)
mysql> insert into newtb (id) values (3);
Query OK, 1 row affected (0.00 sec)
mysql> select id from newtb;
+------+
| id |
+------+
| 1 |
| 2 |
| 3 |
+------+
3 rows in set (0.00 sec)
mysql>
- 关闭mysql服务
- 基于新存储类创建相同大小的pvc
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: new-mysql-pv-claim
labels:
app: mysql
spec:
storageClassName: rook-ceph-block
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
- 创建同时挂载新旧pvc的迁移任务pod
apiVersion: v1
kind: Pod
metadata:
name: datacp
spec:
containers:
- name: busybox
image: busybox
command:
- 'sleep'
args:
- infinity
volumeMounts:
- name: old-pvc
mountPath: /old
- name: new-pvc
mountPath: /new
volumes:
- name: old-pvc
persistentVolumeClaim:
claimName: mysql-pv-claim
- name: new-pvc
persistentVolumeClaim:
claimName: new-mysql-pv-claim
- 在任务pod中执行数据拷贝
[root@rocky8 examples]# kubectl exec -it datacp -- sh
/ #
/ # ls /old
auto.cnf client-cert.pem ib_logfile0 mysql private_key.pem server-key.pem
ca-key.pem client-key.pem ib_logfile1 mysql.sock public_key.pem sys
ca.pem ib_buffer_pool ibdata1 performance_schema server-cert.pem test
/ # ls /new/
lost+found
/ # rmdir /new/lost+found/
/ # (cd /old; tar -cf - .) | (cd /new; tar -xpf -)
/ # ls -l /old/
total 176196
-rw-r----- 1 999 999 56 Dec 27 07:26 auto.cnf
-rw------- 1 999 999 1680 Dec 27 07:26 ca-key.pem
-rw-r--r-- 1 999 999 1112 Dec 27 07:26 ca.pem
-rw-r--r-- 1 999 999 1112 Dec 27 07:26 client-cert.pem
-rw------- 1 999 999 1680 Dec 27 07:26 client-key.pem
-rw-r----- 1 999 999 679 Dec 27 07:33 ib_buffer_pool
-rw-r----- 1 999 999 50331648 Dec 27 07:33 ib_logfile0
-rw-r----- 1 999 999 50331648 Dec 27 07:26 ib_logfile1
-rw-r----- 1 999 999 79691776 Dec 27 07:33 ibdata1
drwxr-x--- 2 999 999 4096 Dec 27 07:26 mysql
lrwxrwxrwx 1 999 999 27 Dec 27 07:26 mysql.sock -> /var/run/mysqld/mysqld.sock
drwxr-x--- 2 999 999 8192 Dec 27 07:26 performance_schema
-rw------- 1 999 999 1676 Dec 27 07:26 private_key.pem
-rw-r--r-- 1 999 999 452 Dec 27 07:26 public_key.pem
-rw-r--r-- 1 999 999 1112 Dec 27 07:26 server-cert.pem
-rw------- 1 999 999 1676 Dec 27 07:26 server-key.pem
drwxr-x--- 2 999 999 8192 Dec 27 07:26 sys
drwxr-x--- 2 999 999 54 Dec 27 07:29 test
/ # ls -l /new/
total 176192
-rw-r----- 1 999 999 56 Dec 27 07:26 auto.cnf
-rw------- 1 999 999 1680 Dec 27 07:26 ca-key.pem
-rw-r--r-- 1 999 999 1112 Dec 27 07:26 ca.pem
-rw-r--r-- 1 999 999 1112 Dec 27 07:26 client-cert.pem
-rw------- 1 999 999 1680 Dec 27 07:26 client-key.pem
-rw-r----- 1 999 999 679 Dec 27 07:33 ib_buffer_pool
-rw-r----- 1 999 999 50331648 Dec 27 07:33 ib_logfile0
-rw-r----- 1 999 999 50331648 Dec 27 07:26 ib_logfile1
-rw-r----- 1 999 999 79691776 Dec 27 07:33 ibdata1
drwxr-x--- 2 999 999 4096 Dec 27 07:43 mysql
lrwxrwxrwx 1 root root 27 Dec 27 07:43 mysql.sock -> /var/run/mysqld/mysqld.sock
drwxr-x--- 2 999 999 4096 Dec 27 07:43 performance_schema
-rw------- 1 999 999 1676 Dec 27 07:26 private_key.pem
-rw-r--r-- 1 999 999 452 Dec 27 07:26 public_key.pem
-rw-r--r-- 1 999 999 1112 Dec 27 07:26 server-cert.pem
-rw------- 1 999 999 1676 Dec 27 07:26 server-key.pem
drwxr-x--- 2 999 999 12288 Dec 27 07:43 sys
drwxr-x--- 2 999 999 4096 Dec 27 07:43 test
/ # diff /old/ /new/
diff: /old/mysql.sock: No such file or directory
diff: /new/mysql.sock: No such file or directory
Common subdirectories: /old/mysql and /new/mysql
Common subdirectories: /old/performance_schema and /new/performance_schema
Common subdirectories: /old/sys and /new/sys
Common subdirectories: /old/test and /new/test
- 关闭任务pod
- 修改新旧pv的删除策略，保证为Retain
[root@rocky8 examples]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
mysql-pv-claim Bound pvc-05a52d71-4289-4ef3-aa8e-da13c6b9e835 1Gi RWO nfs-storage 20m
new-mysql-pv-claim Bound pvc-7bf42770-ca3d-433c-9b44-71dd5549563d 1Gi RWO rook-ceph-block 12m
root@rocky8 examples]# kubectl describe pv pvc-05a52d71-4289-4ef3-aa8e-da13c6b9e835 | grep Reclaim
Reclaim Policy: Delete
[root@rocky8 examples]# kubectl describe pv pvc-7bf42770-ca3d-433c-9b44-71dd5549563d | grep Reclaim
Reclaim Policy: Delete
[root@rocky8 examples]#
[root@rocky8 examples]# kubectl patch pv pvc-05a52d71-4289-4ef3-aa8e-da13c6b9e835 -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
persistentvolume/pvc-05a52d71-4289-4ef3-aa8e-da13c6b9e835 patched
[root@rocky8 examples]# kubectl patch pv pvc-7bf42770-ca3d-433c-9b44-71dd5549563d -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
persistentvolume/pvc-7bf42770-ca3d-433c-9b44-71dd5549563d patched
[root@rocky8 examples]# kubectl describe pv pvc-05a52d71-4289-4ef3-aa8e-da13c6b9e835 | grep Reclaim
Reclaim Policy: Retain
[root@rocky8 examples]# kubectl describe pv pvc-7bf42770-ca3d-433c-9b44-71dd5549563d | grep Reclaim
Reclaim Policy: Retain
[root@rocky8 examples]#
- 备份旧pvc定义，删除新旧pvc
[root@rocky8 examples]# kubectl get pvc mysql-pv-claim -o yaml >old-pvc.yaml
[root@rocky8 examples]# kubectl delete pvc mysql-pv-claim
persistentvolumeclaim "mysql-pv-claim" deleted
[root@rocky8 examples]# kubectl delete pvc new-mysql-pv-claim
persistentvolumeclaim "new-mysql-pv-claim" deleted
[root@rocky8 examples]#
- 解绑新pv和pvc的绑定关系
[root@rocky8 examples]# kubectl edit pv pvc-7bf42770-ca3d-433c-9b44-71dd5549563d
# 删除claimRef定义，保存退出
[root@rocky8 examples]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-05a52d71-4289-4ef3-aa8e-da13c6b9e835 1Gi RWO Retain Released default/mysql-pv-claim nfs-storage 29m
pvc-7bf42770-ca3d-433c-9b44-71dd5549563d 1Gi RWO Retain Available rook-ceph-block 21m
[root@rocky8 examples]#
- 创建和旧pvc相同的使用新存储类的新pvc
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-pv-claim
labels:
app: mysql
spec:
storageClassName: rook-ceph-block
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
# 查看新pvc和旧pv的绑定情况
[root@rocky8 examples]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
mysql-pv-claim Bound pvc-7bf42770-ca3d-433c-9b44-71dd5549563d 1Gi RWO rook-ceph-block 55s
[root@rocky8 examples]#
- 恢复mysql应用，检查数据
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
| test |
+--------------------+
5 rows in set (0.00 sec)
mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| newtb |
+----------------+
1 row in set (0.00 sec)
mysql> select id from newtb;
+------+
| id |
+------+
| 1 |
| 2 |
| 3 |
+------+
3 rows in set (0.00 sec)
mysql>
文章来源:https://blog.csdn.net/longtds/article/details/135485650
本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。 如若内容造成侵权/违法违规/事实不符，请联系我的编程经验分享网邮箱：veading@qq.com进行投诉反馈，一经查实，立即删除！
本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。 如若内容造成侵权/违法违规/事实不符，请联系我的编程经验分享网邮箱：veading@qq.com进行投诉反馈，一经查实，立即删除！