Kubernetes 集群備份工具 Velero 使用指南

作者：whale_life 2022-09-23 17:26:04
云計(jì)算
云原生 Velero 對(duì)存儲(chǔ)的支持較好，可以支持很多種存儲(chǔ)資源，比如 AWS S3、Azure Blob、Google Cloud Storage、Alibaba Cloud OSS、Swift、MinIO 等等。

??想了解更多關(guān)于開源的內(nèi)容，請(qǐng)?jiān)L問：??

??51CTO 開源基礎(chǔ)軟件社區(qū)??

??https://ost.XWCX.NET??

簡介

Velero 是一款可以安全的備份、恢復(fù)和遷移 Kubernetes 集群資源和持久卷等資源的備份恢復(fù)軟件。

Velero 實(shí)現(xiàn)的 kubernetes 資源備份能力，可以輕松實(shí)現(xiàn) Kubernetes 集群的數(shù)據(jù)備份和恢復(fù)、復(fù)制 kubernetes 集群資源到其他 kubernetes 集群或者快速復(fù)制生產(chǎn)環(huán)境到測試環(huán)境等功能，這種備份就類似于把資源的 yaml 文件進(jìn)行整體備份，從而保障資源的完整性。

Velero 對(duì)存儲(chǔ)的支持較好，可以支持很多種存儲(chǔ)資源，比如 AWS S3、Azure Blob、Google Cloud Storage、Alibaba Cloud OSS、Swift、MinIO 等等.

??Storage-support??。

流程圖

注意：本文檔采用 MinIO 作為后端存儲(chǔ)。

Docker 部署 MinIO

因?yàn)槲覀円ＷC存儲(chǔ)和集群隔離，這樣能保證在集群出問題的時(shí)候，不影響備份，所以就將 MinIO 部署在其他服務(wù)器上，與 Kubernetes 環(huán)境隔離。避免因意外情況宕機(jī)的時(shí)候影響更多。

mkdir -pv /data/{minio,config}
docker pull minio/minio
docker pull minio/mc:latest

docker run -d -p 9000:9000 -p 9090:9090 --name=minio --restart=always -e "MINIO_ROOT_USER=admin" -e "MINIO_ROOT_PASSWORD=admin123456" -v /data/minio:/data -v /data/config:/root/.minio  minio/minio server /data --console-address ":9090" --address ":9000"

# 對(duì)minio 授權(quán)
docker run -it --entrypoint=/bin/sh -v /data/config:/root/.minio minio/mc

mc config host add velero http://127.0.0.1:9000 admin admin123456 --api S3v4 
mc mb -p velero/velero

注意，mc 使用的是 api 端口，并不是 console 界面端口，所以 mc 在授權(quán)的時(shí)候需要使用 API 端口。

然后瀏覽器通過配置的賬號(hào)和 password 即可登錄：

安裝 velero 在 k8s master 上

下載 velero：

https://github.com/vmware-tanzu/velero/releases/download/v1.9.0/velero-v1.9.0-linux-amd64.tar.gz。

1、創(chuàng)建密鑰并安裝velero

cat > credentials-velero <[default]
aws_access_key_id = admin
aws_secret_access_key = admin123456
EOF
# 安裝velero，配置自動(dòng)補(bǔ)全
cp velero /usr/local/bin/
echo 'source <(velero completion bash)' >> /etc/profile && source /etc/profile

2、K8s集群安裝velero

# 需要提前下載鏡像 velero/velero-plugin-for-aws:v1.2.1 和velero/velero:v1.9.0
velero install \
    --provider aws \
    --plugins velero/velero-plugin-for-aws:v1.2.1 \
    --bucket velero \
    --secret-file ./credentials-velero \
    --use-volume-snapshots=false \
    --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://10.0.2.196:9000

安裝時(shí)產(chǎn)生的日志。

CustomResourceDefinition/backups.velero.io: attempting to create resource
CustomResourceDefinition/backups.velero.io: attempting to create resource client
CustomResourceDefinition/backups.velero.io: created
CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource
CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource client
CustomResourceDefinition/backupstoragelocations.velero.io: created
CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource
CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource client
CustomResourceDefinition/deletebackuprequests.velero.io: created
CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource
CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource client
CustomResourceDefinition/downloadrequests.velero.io: created
CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource
CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource client
CustomResourceDefinition/podvolumebackups.velero.io: created
CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource
CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource client
CustomResourceDefinition/podvolumerestores.velero.io: created
CustomResourceDefinition/resticrepositories.velero.io: attempting to create resource
CustomResourceDefinition/resticrepositories.velero.io: attempting to create resource client
CustomResourceDefinition/resticrepositories.velero.io: created
CustomResourceDefinition/restores.velero.io: attempting to create resource
CustomResourceDefinition/restores.velero.io: attempting to create resource client
CustomResourceDefinition/restores.velero.io: created
CustomResourceDefinition/schedules.velero.io: attempting to create resource
CustomResourceDefinition/schedules.velero.io: attempting to create resource client
CustomResourceDefinition/schedules.velero.io: created
CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource
CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource client
CustomResourceDefinition/serverstatusrequests.velero.io: created
CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource
CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource client
CustomResourceDefinition/volumesnapshotlocations.velero.io: created
Waiting for resources to be ready in cluster...
Namespace/velero: attempting to create resource
Namespace/velero: attempting to create resource client
Namespace/velero: created
ClusterRoleBinding/velero: attempting to create resource
ClusterRoleBinding/velero: attempting to create resource client
ClusterRoleBinding/velero: created
ServiceAccount/velero: attempting to create resource
ServiceAccount/velero: attempting to create resource client
ServiceAccount/velero: created
Secret/cloud-credentials: attempting to create resource
Secret/cloud-credentials: attempting to create resource client
Secret/cloud-credentials: created
BackupStorageLocation/default: attempting to create resource
BackupStorageLocation/default: attempting to create resource client
BackupStorageLocation/default: created
Deployment/velero: attempting to create resource
Deployment/velero: attempting to create resource client
Deployment/velero: created
Velero is installed!  Use 'kubectl logs deployment/velero -n velero' to view the status.

驗(yàn)證是否安裝完畢。

[root@master ~]# kubectl get pod -n velero 
NAME                     READY   STATUS    RESTARTS   AGE
velero-cf79784ff-zdfp9   1/1     Running   0          47s
[root@master ~]# kubectl  api-versions  | grep velero
velero.io/v1

3、velero 命令

velero create backup NAME [flags]
# 剔除 namespace
--exclude-namespaces stringArray                  namespaces to exclude from the backup
# 剔除資源類型
--exclude-resources stringArray                   resources to exclude from the backup, formatted as resource.group, such as storageclasses.storage.k8s.io
# 包含集群資源類型 
--include-cluster-resources optionalBool[=true]   include cluster-scoped resources in the backup
# 包含 namespace
--include-namespaces stringArray                  namespaces to include in the backup (use '*' for all namespaces) (default *)
# 包含 namespace 資源類型
--include-resources stringArray                   resources to include in the backup, formatted as resource.group, such as storageclasses.storage.k8s.io (use '*' for all resources)
# 給這個(gè)備份加上標(biāo)簽
--labels mapStringString                          labels to apply to the backup
-o, --output string                               Output display format. For create commands, display the object but do not send it to the server. Valid formats are 'table', 'json', and 'yaml'. 'table' is not valid for the install command.
# 對(duì)指定標(biāo)簽的資源進(jìn)行備份
-l, --selector labelSelector                      only back up resources matching this label selector (default )
# 對(duì) PV 創(chuàng)建快照
--snapshot-volumes optionalBool[=true]            take snapshots of PersistentVolumes as part of the backup
# 指定備份的位置
--storage-location string                         location in which to store the backup
# 備份數(shù)據(jù)多久刪掉
--ttl duration                                    how long before the backup can be garbage collected (default 720h0m0s)
# 指定快照的位置，也就是哪一個(gè)公有云驅(qū)動(dòng)
--volume-snapshot-locations strings               list of locations (at most one per provider) where volume snapshots should be stored

4、測試

velero 在安裝包中有一個(gè)測試demo，我們可以利用測試demo來進(jìn)行測試驗(yàn)證。

[root@master ~]# cd velero-v1.9.0-linux-amd64/examples/nginx-app/
[root@master nginx-app]# ls
base.yaml  README.md  with-pv.yaml
# 注意修改 with-pv 中的 storageclass
storageClassName: longhorn
# 修改 svc 類型為 ClusterIP
type: ClusterIP
[root@master nginx-app]# kubectl apply -f with-pv.yaml 
namespace/nginx-example created
persistentvolumeclaim/nginx-logs created
deployment.apps/nginx-deployment created
service/my-nginx created
[root@master nginx-app]# kubectl get svc
NAME       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
my-nginx   ClusterIP   10.108.189.178           80/TCP    79s
[root@master nginx-app]# kubectl get all -n nginx-example 
NAME                                    READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-6779884c68-4nv2s   2/2     Running   0          2m16s

NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
service/my-nginx   ClusterIP   10.108.189.178           80/TCP    2m16s

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   1/1     1            1           2m16s

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-deployment-6779884c68   1         1         1       2m16s

5、備份

[root@master nginx-app]# velero backup create nginx-backup --include-namespaces nginx-example
Backup request "nginx-backup" submitted successfully.
Run `velero backup describe nginx-backup` or `velero backup logs nginx-backup` for more details.

[root@master nginx-app]# velero backup describe nginx-backup
Name:         nginx-backup
Namespace:    velero
Labels:       velero.io/storage-location=default
Annotations:  velero.io/source-cluster-k8s-gitversion=v1.23.5
              velero.io/source-cluster-k8s-major-version=1
              velero.io/source-cluster-k8s-minor-version=23
Phase:  Failed (run `velero backup logs nginx-backup` for more information)
Errors:    0
Warnings:  0
Namespaces:
  Included:  nginx-example
  Excluded:  
Resources:
  Included:        *
  Excluded:        
  Cluster-scoped:  auto
Label selector:  
Storage Location:  default
Velero-Native Snapshot PVs:  auto
TTL:  720h0m0s
Hooks:  
Backup Format Version:  1.1.0
Started:    2022-06-30 17:44:49 +0800 CST
Completed:  2022-06-30 17:44:49 +0800 CST
Expiration:  2022-07-30 17:44:49 +0800 CST
Velero-Native Snapshots: 
[root@master nginx-app]# velero backup logs nginx-backup
An error occurred: gzip: invalid header

登錄 minio 查看，備份記錄。

6、恢復(fù)測試

刪除 nginx-app：

[root@master velero-v1.9.0-linux-amd64]# kubectl delete -f examples/nginx-app/base.yaml

恢復(fù) nginx：

[root@master velero-v1.9.0-linux-amd64]# velero restore create --from-backup nginx-backup --wait
Restore request "nginx-backup-20220630192521" submitted successfully.
Waiting for restore to complete. You may safely press ctrl-c to stop waiting - your restore will continue in the background.

Restore completed with status: Completed. You may check for more information using the commands `velero restore describe nginx-backup-20220630192521` and `velero restore logs nginx-backup-20220630192521`.

[root@master velero-v1.9.0-linux-amd64]# kubectl  get pods -n nginx-example
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-6779884c68-4nv2s   2/2     Running   0          27s

恢復(fù)會(huì)新建一個(gè) restores。

目錄注意：??velero restore??? 恢復(fù)不會(huì)覆蓋??已有的資源???，只恢復(fù)當(dāng)前集群中??不存在的資源??。已有的資源不會(huì)回滾到之前的版本，如需要回滾，需在restore之前提前刪除現(xiàn)有的資源。

備份所有命名空間：

velero backup logs k8s-all

7、定時(shí)備份

比較推薦做定時(shí)全量備份，這個(gè)能保證我們是一直全量最新備份的狀態(tài)。

# 每天12點(diǎn)備份一次
velero create schedule k8s-all --schedule="0 0 * * *"

查看備份：

velero  get  backup   #備份查看
velero  get  schedule #查看定時(shí)備份
velero  get  restore  #查看已有的恢復(fù)
velero  get  plugins  #查看插件

本文標(biāo)題：Kubernetes集群備份工具velero使用指南
分享路徑：http://fisionsoft.com.cn/article/cdispps.html