Kubernetes Federation
Kubernetes聯邦集群是由https://github.com/kubernetes-sigs主導開發，目前已經進入Kubefed v2的0.2.0-alpha.1版本。通過OpenShift內置的Kubefed Operator，我們可以在OpenShift之上運行聯邦集群。在Kubefed中通過Controller Manager維護聯邦集群中每個Kubernetes集群成員的資源狀態。
Kubenetes Federation可以多個Kubernetes集群之間實現以下高可用功能：
- 簡化管理多個集群的Kubernetes 組件(如Deployment, Service 等)。
- 在多個集群之間分散工作負載(容器)，以提升應用(服務)的可靠性。
- 跨集群的資源編排，依據編排策略在多個集群進行應用(服務)部署。
- 在不同集群中，能更快速更容易地遷移應用(服務)。
- 跨集群的服務發現，服務可以提供給當地存取，以降低延遲。
- 實踐多雲(Multi-cloud)或混合雲(Hybird Cloud)的部署。
準備環境
OpenShift集群環境
我們需要2個基於4.x的OpenShift集群環境：cluster1和cluster2，其中在cluster1部署KubeFed（即Federation Manager）。
配置客戶端環境
我們用Linux作為客戶端運行環境，需要在客戶端上配置2個集群的context，並且安裝kubefedctl客戶端。
1. 在客戶端登錄OpenShift集群cluster1和cluster2。
<code>$ oc login/<code>
$ oc config rename-context $(oc config current-context) cluster1
$ oc login
$ oc config rename-context $(oc config current-context) cluster2
2. 先查看將客戶端的Config Context，然後把客戶端登錄的兩個Context Name分別設為cluster1和cluster1。（可參照《 》）。
3. 設置完後先將缺省Config設置到cluster1，然後再查看Config Context為以下cluster1和cluster2。
<code>$ oc config use-context cluster1
$ oc config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAM ESPACE
* cluster1 api-cluster-beijing-c3c2-beijing-c3c2-example-opentlc-com:6443 opentlc-mgr/api-cluster-beijing-c3c2-beijing-c3c2-example-opentlc-com:6443 def ault
cluster2 api-cluster-beijing-3beb-beijing-3beb-sandbox1785-opentlc-com:6443 opentlc-mgr/api-cluster-beijing-3beb-beijing-3beb-sandbox1785-opentlc-com:6443 def ault/<code>
4. 安裝kubefedctl客戶端
<code>$ curl -LO https://github.com/kubernetes-sigs/kubefed/releases/download/v0.1.0-rc6/kubefedctl-0.1.0-rc6-linux-amd64.tgz
$ tar -xvf kubefedctl-0.1.0-rc6-linux-amd64.tgz
$ sudo mv kubefedctl /usr/local/bin//<code>
安裝配置KubeFed Operator
1. 在cluster1上創建運行KubeFed Operator的Namespace。**說明**：KubeFed缺省使用名為“kube-federation-system”的項目作為其運行環境。因此在缺省的時候，本文以下部分中的“--kubefed-namespace kube-federation-system”其實都可以省掉。如果KubeFed運行在其它項目名中，則以下相關命令都需要提供“--kubefed-namespace”參數。
<code>$ oc --context=cluster1 create ns kube-federation-system/<code>
2. 用集群管理員進入OpenShift Console的Administrator視圖中的OperatorHub（確認此時是在kube-federation-system項目中）。然後找到“Kubefed Operator”點擊進入。
3. 點擊Install，然後選擇部署在kube-federation-system項目中，最後點擊Subscribe。
4. 在Installed Operators中等到狀態變為“Up to date”後進入該Kubefed Operator。
5. 查看當前運行的Pod。
<code>$ oc --context=cluster1 -n kube-federation-system get pods
NAME READY STATUS RESTARTS AGE
kubefed-operator-76b674b9fd-dsjkl 1/1 Running 0 88s/<code>
6. 在Kubefed Operator的Overview頁面中找到KubeFedWebHook，點擊Create Instance，然後接受缺省配置創建KubeFedWebHook。最後查看Pod的狀態應該如下。
<code>$ oc --context=cluster1 -n kube-federation-system get pods
NAME \t\tREADY STATUS RESTARTS AGE
kubefed-operator-76b674b9fd-dsjkl \t\t1/1 Running 0 88s
kubefed-admission-webhook-89fbcbb74-dvdq6 1/1 Running 1 10h/<code>
7. 在Kubefed Operator的Overview頁面中找到KubeFed，點擊Create Instance，然後接受缺省配置創建KubeFed。最後查看Pod的狀態應該如下。
<code>$ oc --context=cluster1 -n kube-federation-system get pods
NAME READY STATUS RESTARTS AGE
kubefed-admission-webhook-89fbcbb74-dvdq6 1/1 Running 1 10h
kubefed-controller-manager-6d5f46d745-nd8fw 1/1 Running 1 10h
kubefed-controller-manager-6d5f46d745-p6f6b 1/1 Running 1 10h
kubefed-operator-76b674b9fd-dsjkl \t\t 1/1 Running 0 88s/<code>
8. 執行命令（也可在下圖中的Kubefed Opeator中的All Instance中查看），查看聯邦集群中的對象實例。
<code>$ oc --context=cluster1 -n kube-federation-system get kubefedconfig
NAME AGE
kubefed 31h
$ oc --context=cluster1 -n kube-federation-system get kubefedwebhook
NAME AGE
kubefedwebhook-resource 31h
$ oc --context=cluster1 -n kube-federation-system get kubefed
NAME AGE
kubefed-resource 31h/<code>
將OpenShift集群加入到聯邦集群
1. 執行以下命令，將cluster1和cluster2加入到聯邦集群(用cluster1作為主管)中。
<code>$ kubefedctl join cluster1 --cluster-context cluster1 --host-cluster-context cluster1 --kubefed-namespace=kube-federation-system --v=2
I0313 02:09:12.434998 1289 join.go:159] Args and flags: name cluster1, host: cluster1, host-system-namespace: kube-federation-system, kubeconfig: , cluster-context: cluster1, secret-name: , dry-run: false
I0313 02:09:12.535522 1289 join.go:240] Performing preflight checks.
I0313 02:09:12.539877 1289 join.go:246] Creating kube-federation-system namespace in joining cluster
I0313 02:09:12.544239 1289 join.go:382] Already existing kube-federation-system namespace
I0313 02:09:12.544257 1289 join.go:254] Created kube-federation-system namespace in joining cluster
I0313 02:09:12.544267 1289 join.go:403] Creating service account in joining cluster: cluster1
I0313 02:09:12.553436 1289 join.go:413] Created service account: cluster1-cluster1 in joining cluster: cluster1
I0313 02:09:12.553451 1289 join.go:441] Creating cluster role and binding for service account: cluster1-cluster1 in joining cluster: cluster1
I0313 02:09:12.577425 1289 join.go:450] Created cluster role and binding for service account: cluster1-cluster1 in joining cluster: cluster1
I0313 02:09:12.577455 1289 join.go:809] Creating cluster credentials secret in host cluster
I0313 02:09:13.596025 1289 join.go:835] Using secret named: cluster1-cluster1-token-8dxrm
I0313 02:09:13.608147 1289 join.go:878] Created secret in host cluster named: cluster1-9d4j7
I0313 02:09:13.638964 1289 join.go:282] Created federated cluster resource
$ kubefedctl join cluster2 --cluster-context cluster2 --host-cluster-context cluster1 --kubefed-namespace=kube-federation-system --v=2
I0315 08:54:52.816368 799 join.go:159] Args and flags: name cluster2, host: cluster1, host-system-namespace: kube-federation-system, kubeconfig: , cluster-context: cluster2, secret-name: , dry-run: false
I0315 08:54:53.178787 799 join.go:238] Performing preflight checks.
I0315 08:54:53.411417 799 join.go:244] Creating kube-federation-system namespace in joining cluster
I0315 08:54:53.665019 799 join.go:252] Created kube-federation-system namespace in joining cluster
I0315 08:54:53.665060 799 join.go:398] Creating service account in joining cluster: cluster2
I0315 08:54:53.854303 799 join.go:408] Created service account: cluster2-cluster1 in joining cluster: cluster2
I0315 08:54:53.854334 799 join.go:436] Creating cluster role and binding for service account: cluster2-cluster1 in joining cluster: cluster2
I0315 08:54:54.413824 799 join.go:445] Created cluster role and binding for service account: cluster2-cluster1 in joining cluster: cluster2
I0315 08:54:54.413852 799 join.go:804] Creating cluster credentials secret in host cluster
I0315 08:54:54.743515 799 join.go:830] Using secret named: cluster2-cluster1-token-jhprj
I0315 08:54:54.754980 799 join.go:873] Created secret in host cluster named: cluster2-jpsq9
I0315 08:54:54.782983 799 join.go:280] Created federated cluster resource/<code>
2. (可選步驟)-執行命令可將cluster2從聯邦集群中去掉。
<code>$ kubefedctl unjoin cluster2 --cluster-context cluster2 --host-cluster-context cluster1 --kubefed-namespace=kube-federation-system --v=2/<code>
3. 查看聯邦集群中的OpenShift集群都是READY=True狀態。
<code>$ oc --context=cluster1 -n kube-federation-system get kubefedclusters
NAME READY AGE
cluster1 True 1m
cluster2 True 1m/<code>
4. 執行命令設置允許運行聯邦對象類型。任何Kubernetes API支持的類型都可以成為對應的聯邦類型（包括CRD類型），例如以下的FederatedDeployment、FederatedService。**注意**：這裡只是**允許部署**這些聯邦對象類型，而不是在聯邦集群中對這些聯邦對象**執行復制操作**。
<code>for type in namespaces secrets serviceaccounts services configmaps deployments.apps scc
do
kubefedctl enable $type --kubefed-namespace kube-federation-system
done/<code>
執行後會有以下輸出，說明這些類型都已經註冊到聯邦集群的FederatedTypeConfig中了。
<code>customresourcedefinition.apiextensions.k8s.io/federatednamespaces.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/namespaces updated in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federatedsecrets.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/secrets updated in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federatedserviceaccounts.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/serviceaccounts created in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federatedservices.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/services created in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federatedconfigmaps.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/configmaps created in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federateddeployments.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/deployments.apps created in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federatedsecuritycontextconstraints.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/securitycontextconstraints.security.openshift.io created in namespace kube-federation-system/<code>
5. 查看聯邦集群中FederatedTypeConfigAPI包括的API類型，結果對應上一步設置的聯邦集群允許的API資源對象類型。
<code>$ oc --context=cluster1 -n kube-federation-system get FederatedTypeConfig
NAME AGE
configmaps 3m47s
deployments.apps 3m46s
namespaces 6m28s
secrets 5m56s
securitycontextconstraints.security.openshift.io 4m3s
serviceaccounts 3m48s
services 3m48s/<code>
6. （可選）：可以用以下命令diable一種Kubernetes API的類型。例如下面命令會從FederatedTypeConfig中將Deployment類型刪除（並同時刪除構成其聯邦部署能力的相關CRD對象），使其不再具有在聯邦集群中的傳播複製能力。
<code>$ kubefedctl disable deployments.apps --delete-crd/<code>
7. （可選）*：以下命令可以查看集群支持的所有Kubernetes API類型。
<code>$ oc api-resources/<code>
向聯邦集群部署應用資源
概念說明
kubefedctl的federate命令是用來根據Kubernetes API Tpye對象生成對應的Federated化對象。其中Federated化對象中template部分就是對應一般的Kubernetes API Tpye對象。
<code>kubefedctl federate <target> <target> [flags]/<target>/<target>/<code>
以下每個“kubefedctl federate”命令都將一個特定的Kubernetes API類型Federated化，即生成對應的Federated\\<type>對象。/<type>
<code>kubefedctl federate namespace test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate secrets test-secret -n test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate serviceaccounts test-serviceaccount -n test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate services test-service -n test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate configmaps test-configmap -n test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate deployments.apps test-deployment -n test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate federatedsecuritycontextconstraints test-anyuid -n test-namespace --kubefed-namespace kube-federation-system/<code>
可以用“kubefedctl federate”命令的“--output string”參數將Federated化對象轉為YAML格式輸出。
操作
參照以下操作指導可實現在集群聯邦中部署應用。
1. 先確認cluster1和cluster2上都沒有test-namespace項目。
<code>$ oc --context=cluster1 get project test-namespace
Error from server (NotFound): namespaces "test-namespace" not found
$ oc --context=cluster2 get project test-namespace
Error from server (NotFound): namespaces "test-namespace" not found/<code>
2. 在cluster1上創建test-namespace項目，然後確認cluster2還沒有test-namespace項目。這是由於test-namespace項目還未加入到KubeFed中。
<code>$ oc --context=cluster1 get project test-namespace
NAME DISPLAY NAME STATUS
test-namespace Active
$ oc --context=cluster2 get project test-namespace
Error from server (NotFound): namespaces "test-namespace" not found/<code>
3. 執行命令，把名為“test-namespace”的namespace對象Federated化，既生成了一個名為“test-namespace”的federatednamespace對象。然後查看這個對象。
**說明**：只有Federated化資源對象（例如下面的test-namespace項目）才能被Kubefed的Controller Manager複製到聯邦集群的Member Cluster上。
<code>$ kubefedctl federate namespace test-namespace --kubefed-namespace kube-federation-system
W0315 13:00:01.270320 1080 federate.go:410] Annotations defined for Namespace "test-namespace" will not appear in the template of the federated resource: map[openshift.io/sa.scc.mcs:s0:c23,c7 openshift.io/sa.scc.supplemental-groups:1000520000/10000 openshift.io/sa.scc.uid-range:1000520000/10000]
I0315 13:00:01.270486 1080 federate.go:474] Resource to federate is a namespace. Given namespace will itself be the container for the federated namespace
I0315 13:00:01.394914 1080 federate.go:503] Successfully created FederatedNamespace "test-namespace/test-namespace" from Namespace\t/<code>
**說明**：如果一個namespace中已經有資源了，可以使用“--contents”參數將項目中的已有的資源類型自動Federated化。
<code>kubefedctl federate namespace my-namespace --contents --kubefed-namespace kube-federation-system/<code>
4. 分別執行以下命令，查看上一步從名為test-namespace的namespace生成的同名聯邦對象federatednamespace實例和器詳細信息。可以從詳細信息中看到federatednamespace的傳播範圍其是cluster1和cluster。
<code>$ oc --context=cluster1 -n test-namespace get federatednamespace
NAME AGE
test-namespace 3m
$ oc --context=cluster1 -n test-namespace get federatednamespace test-namespace -o yaml
apiVersion: types.kubefed.io/v1beta1
kind: FederatedNamespace
metadata:
creationTimestamp: "2020-03-20T16:43:32Z"
finalizers:
- kubefed.io/sync-controller
generation: 1
name: test-namespace
namespace: test-namespace
resourceVersion: "1126829"
selfLink: /apis/types.kubefed.io/v1beta1/namespaces/test-namespace/federatednamespaces/test-namespace
uid: ef74af24-6ac9-11ea-92df-1222d546a5b5
spec:
placement:
clusterSelector:
matchLabels: {}
template:
spec: {}
status:
clusters:
- name: cluster2
- name: cluster1
conditions:
- lastTransitionTime: "2020-03-21T09:15:03Z"
lastUpdateTime: "2020-03-21T09:15:03Z"
status: "True"
type: Propagation/<code>
5. 確認此時cluster2上也有了test-namespace項目。
<code>$ oc --context=cluster2 get project test-namespace
NAME DISPLAY NAME STATUS
test-namespace Active/<code>
6. 將項目名改為test-namespace1，然後重複以上（1）-（5）步驟，確認test-namespace1項目可以自動從cluster1複製到cluster2。
7. 下載測試應用項目。注意：以下地址的上游項目中sample-app目錄裡所有文件內容的apiversion命名空間已經不適合OpenShift 4。
<code>$ git clone https://github.com/liuxiaoyu-git/federation-dev
$ cd federation-dev/archive/<code>
8. 由於我們後面部署的測試應用容器需要更高運行權限，因此需要執行以下命令為test-namespace項目提權。
<code>$ oc --context=cluster1 -n test-namespace adm policy add-scc-to-user anyuid -z default
securitycontextconstraints.security.openshift.io/anyuid added to: ["system:serviceaccount:test-namespace:default"]
$ oc --context=cluster2 -n test-namespace adm policy add-scc-to-user anyuid -z default
securitycontextconstraints.security.openshift.io/anyuid added to: ["system:serviceaccount:test-namespace:default"]/<code>
9. 確認“federation-dev/archive/sample-app”中的所有文件內容中的“namespace”設置的是我們本次需要同步的項目“test-namespace”。
10. 部署聯邦應用資源，包括federatedconfigmap、federateddeployment、federatedsecret、federatedservice、federatedserviceaccount。然後依次查看這些聯邦對象。
<code>$ oc --context=cluster1 -n test-namespace apply -R -f sample-app/
federatedconfigmap.types.kubefed.k8s.io/test-configmap created
federateddeployment.types.kubefed.k8s.io/test-deployment created
federatedsecret.types.kubefed.k8s.io/test-secret created
federatedservice.types.kubefed.k8s.io/test-service created
federatedserviceaccount.types.kubefed.k8s.io/test-serviceaccount created
$ oc --context=cluster1 -n test-namespace get federatedconfigmap
NAME AGE
test-configmap 3m
$ oc --context=cluster1 -n test-namespace get federateddeployment
NAME AGE
test-deployment 3m
$ oc --context=cluster1 -n test-namespace get federatedsecret
NAME AGE
test-secret 3m
$ o --context=cluster1 -n test-namespacec get federatedserviceaccount
NAME AGE
test-serviceaccount 3m
$ oc --context=cluster1 -n test-namespace get federatedservice
NAME AGE
test-service 3m/<code>
11. 查看以上每個YAML文件，以federateddeployment.yaml文件為例，確認對象類型是“FederatedDeployment”；部署的“placement”包括cluster1和cluster2；cluster2的配置通過“overrides將運行Pod副本“/spec/replicas”修改為“5”份。
<code>$ more sample-app/federateddeployment.yaml
apiVersion: types.kubefed.io/v1beta1
kind: FederatedDeployment
metadata:
name: test-deployment
namespace: test-namespace
spec:
template:
metadata:
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
placement:
clusters:
- name: cluster1
- name: cluster2
overrides:
12. clusterName: cluster2
clusterOverrides:
- path: "/spec/replicas"
value: 5/<code>
13. 查看cluster1和cluster2上的應用運行Pod副本數量。
<code>$ oc --context=cluster1 -n test-namespace get deployment test-deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
test-deployment 3 3 3 3 11m
$ oc --context=cluster2 -n test-namespace get deployment test-deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
test-deployment 5 5 5 5 11m/<code>
14. 由於OpenShift的Route是和每個集群的域名相關，因此我們先採用命令的方式為cluster1和cluster2的Service生成各自的Route。
<code>$ oc --context=cluster1 -n test-namespace expose svc test-service
$ oc --context=cluster2 -n test-namespace expose svc test-service/<code>
15. 重複以上（8）-（13）步，將sample-app應用部署到在（7）創建test-namespace1項目中（需要將以上相關步驟中的所有命令和部署YAML文件中的“test-namespace”替換為“test-namespace1”）。
操控聯邦集群上的應用部署
控制應用在聯邦集群中的部署範圍
1. 執行命令，將cluster2從聯邦集群中去掉。
<code>$ oc --context=cluster1 -n test-namespace patch federateddeployment test-deployment --type=merge -p '{"spec":{"placement":{"clusters": [{"name":"cluster1"}]}}}'
$ oc get kubefedclusters -n kube-federation-system --context=cluster1
NAME READY AGE
cluster1 True 29m/<code>2. 執行命令，將cluster2加入到聯邦集群中。
<code>$ oc --context=cluster1 -n test-namespace patch federateddeployment test-deployment --type=merge -p '{"spec":{"placement":{"clusters": [{"name":"cluster1"}, {"name":"cluster2"}]}}}'
$ oc get kubefedclusters -n kube-federation-system --context=cluster1
NAME READY AGE
cluster1 True 20m
cluster2 True 1m/<code>控制聯邦集群中部署的Pod數量
場景1-聯邦部署優先級高於集群部署
1. 手動直接將cluster2上應用的Deployment運行的Pod部分改為3，然後馬上查看Deployment的Pod變化情況。確認“DESIRED”的Pod會先變為3個，然後又會變為5，最後“AVAILABLE”也會變為5。這說明是在聯邦集群中即便可以通過手動修改特性OpenShift集群中的應用配置，但是聯邦集群機制還會根據聯邦集群上的配置將其修正過來，即FederatedDeployment的優先級高於Deployment。
<code>$ oc --context=cluster2 -n test-namespace patch deployment test-deployment --type=merge -p '{"spec":{"replicas":"3"}}'
$ oc get deployment test-deployment --context=cluster2 -n test-namespace -w
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
test-deployment 3 3 3 3 27m
test-deployment 5 3 3 3 27m
test-deployment 5 5 5 3 27m
test-deployment 5 5 5 5 28m/<code>場景2-ReplicaSchedulingPreference優先級高於聯邦部署
在聯邦集群中，我們可以用ReplicaSchedulingPreference對象來制定在不同集群上分配部署副本數量的策略。當使用了ReplicaSchedulingPreference，系統將忽略FederatedDeloyment對象中在spec.replicas部分指定的部署副本數量。
1. 創建包含以下內容的rsp.yaml文件。它定義了一個ReplicaSchedulingPreference，讓應用運行的副本為6個，且cluster1和cluster2上運行的Pod數量比例是1:2。
<code>apiVersion: scheduling.kubefed.io/v1alpha1
kind: ReplicaSchedulingPreference
metadata:
name: test-deployment
namespace: test-namespace
spec:
targetKind: FederatedDeployment
totalReplicas: 6
clusters:
cluster1:
weight: 1
cluster2:
weight: 2/<code>
2. 執行命令，在test-namespace項目中創建ReplicaSchedulingPreference對象。
<code>$ oc --context=cluster apply -f rsp.yaml/<code>
3. 稍後執行命令，查看cluster1和cluster2的Deployment副本數量。可以確認聯邦集群中部署這個應用的Pod數量已經成為6個，且在cluster1和cluster2上運行的Pod數量比例是1:2。
<code>$ oc --context=cluster1 -n test-namespace get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
test-deployment 2/2 2 2 13h
$ oc --context=cluster2 -n test-namespace get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
test-deployment 4/4 4 4 13h/<code>
4. 查看名為test-deployment的federateddeployment。可以看到FederatedDeployment的缺省部署數量（即每個集群的replicas為3個）被Override了（即cluster1的replicas為2、cluster2的replicas為4）。另外從最下面的Events可以看出KubeFed的federateddeployment-controller先後更新了cluster1和cluster2中的Deployment。
<code>$ oc --context=cluster1 -n test-namespace describe federateddeployment test-deployment
Name: test-deployment
Namespace: test-namespace
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"types.kubefed.io/v1beta1","kind":"FederatedDeployment","metadata":{"annotations":{},"name":"test-deployment","namespace":"t..."
API Version: types.kubefed.io/v1beta1
Kind: FederatedDeployment
Metadata:
Creation Timestamp: 2020-03-21T02:28:30Z
Finalizers:
kubefed.io/sync-controller
Generation: 3
Resource Version: 1285158
Self Link: /apis/types.kubefed.io/v1beta1/namespaces/test-namespace/federateddeployments/test-deployment
UID: a7a800d7-6b1b-11ea-abac-1222d546a5b5
Spec:
Overrides:
Cluster Name: cluster1
Cluster Overrides:
Path: /spec/replicas
Value: 2
Cluster Name: cluster2
Cluster Overrides:
Path: /spec/replicas
Value: 4
Path: /spec/template/spec/containers/0/image
Value: nginx:1.17.0-alpine
Op: add
Path: /metadata/annotations
Value:
Foo: bar
Op: remove
Path: /metadata/annotations/foo
Placement:
Clusters:
Name: cluster2
Name: cluster1
Template:
Metadata:
Labels:
App: nginx
Spec:
Replicas: 3
Selector:
Match Labels:
App: nginx
Template:
Metadata:
Labels:
App: nginx
Spec:
Containers:
Image: nginx
Name: nginx
Status:
Clusters:
Name: cluster2
Name: cluster1
Conditions:
Last Transition Time: 2020-03-21T15:38:30Z
Last Update Time: 2020-03-21T15:38:30Z
Status: True
Type: Propagation
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal UpdateInCluster 32m (x2 over 46m) federateddeployment-controller Updating Deployment "test-namespace/test-deployment" in cluster "cluster1"
Normal UpdateInCluster 7m35s (x172 over 3h47m) federateddeployment-controller Updating Deployment "test-namespace/test-deployment" in cluster "cluster2"/<none>/<code>
場景3-關閉聯邦集群中的一個集群
在完成以上場景2後，如果關閉聯邦集群中的一個集群（例如cluster2），KubeFed會將6個Pod全部運行在cluster1上。
kubefedctl 命令
kubefedctl enable和kubefedctl disable子命令
這兩個命令可以分別打開和關閉在聯邦集群中傳播Kubernetes API類型。
kubefedctl federate子命令
Kubernetes API Type轉成Federated Type
我們可以用“kubefedctl federate”命令將一個一般的Kubernetes API Type轉成Federated Type，例如將configmap變成federatedconfigmap。在實現過程中有以下兩種方法：
1. 根據YAML定義文件
1. 查看一個定義ConfigMap對象的YAML文件configmap.yaml
<code>apiVersion: v1
kind: ConfigMap
metadata:
name: web-file
data:
index.html: "Hello from Kubernetes Cluster Federation!"/<code>
2. 用“kubefedctl federate”命令轉換成FederatedConfigMap，輸出便是FederatedConfigMap的YAML。可以根據輸出結果創建FederatedConfigMap。
<code>$ kubefedctl federate -f configmap.yaml
---
apiVersion: types.kubefed.io/v1beta1
kind: FederatedConfigMap
metadata:
name: web-file
spec:
placement:
clusterSelector:
matchLabels: {}
template:
data:
index.html: Hello from Kubernetes Cluster Federation!/<code>
2. 根據已有對象，生成Federated類型的對象
1. 執行命令，根據當前已有名為web-file的ConfigMap對象，生成對應的同名FederatedConfigMap對象。
<code>$ kubefedctl federate cm web-file
W0322 01:57:10.549787 3583 federate.go:410] Annotations defined for ConfigMap "test-namespace/web-file" will not appear in the template of the federated resource: map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","data":{"index.html":"Hello from Kubernetes Cluster Federation!"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"web-file","namespace":"test-namespace"}}
]
I0322 01:57:10.560774 3583 federate.go:503] Successfully created FederatedConfigMap "test-namespace/web-file" from ConfigMap/<code>
2. 確認已經生成FederatedConfigMap對象。
<code>$ oc get fcm web-file
NAME AGE
web-file 2m32s/<code>
刪除Federated類型的對象
如果刪除Federated類型的對象，那麼會在聯邦集群範圍內的所有相關集群中刪除其對應的Kubernetes API對象。例如當執行命令刪除了fsvc（即FederatedService），OpenShift會刪除cluster1和cluster2中和FederatedService對應的Service對象。
<code>$ oc delete fsvc --all
federatedservice.types.kubefed.io "test-service" deleted/<code>
參考文檔
4. https://github.com/kubernetes-sigs/kubefed/blob/master/docs/userguide.md
5. https://github.com/openshift/federation-dev/tree/master/archive
6. https://github.com/openshift/federation-dev/blob/master/archive/docs/ocp4-namespace-scoped.md
7. https://github.com/openshift/federation-dev/blob/master/archive/docs/ocp4-cluster-scoped.md
8. https://github.com/redhat-gpte-devopsautomation/rhte2019_multi_cloud
9. https://www.linkedin.com/pulse/openshift-4x-foundations-fundamentals-kubefed-dewan-i-ahmed
10. https://juejin.im/post/5dafe985f265da5b560e0d2d
11. https://juejin.im/post/5db6a43be51d452a01253454
12. https://www.kubernetes.org.cn/5702.html
閱讀更多 OpenShift之家 的文章
