Kubernetesは、すべての操作とコンポーネント間の通信、および外部ユーザーコマンドは REST API で通信していますが
kubectlコマンドを使うと、そのことを意識せずにKubernetesを操作できます。

具体的にどんなAPIで通信しているのか気になったので、culでapiの呼び出しを試してみました。

API Overview
- https://kubernetes.io/docs/reference/using-api/

説明

すること
- 認証はkubectl proxyを使用
- 基本的な操作のみ
しないこと
- コマンドインストール等は説明しません
- kubectlで操作できるKubernetes環境の作成(検証ではminikubeを使います)

culでアクセスする際のクラスタと資格情報

Kubernetes clusterとAPIで通信するには、クラスタと資格情報の設定が必要ですが、
kubectl proxy コマンドを使えば、kubectlの情報を使い簡単にCurlで通信できる
ようなので、今回はこれを使用します。

Directly accessing the REST API
- https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/#directly-accessing-the-rest-api

プロキシについて詳しい説明はここを参照

command proxy
- https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#proxy
Kubernetesのプロキシー
- https://kubernetes.io/ja/docs/concepts/cluster-administration/proxies/

APIのURI、メソッドについて

HTTPメソッドのPOST、PUT、PATCH、DELETE、GETで取得、作成、更新、削除等の操作ができるみたい

Kubernetes API Concepts
- https://kubernetes.io/docs/reference/using-api/api-concepts/

リクエストするURIのルールは/apis/GROUP/VERSION/namespaces/NAMESPACE/みたい

Resource URIs
- https://kubernetes.io/docs/reference/using-api/api-concepts/#resource-uris

具体的なHTTP Requestは、以下サイトで調べて実行する感じですね

API OVERVIEW
- https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/

呼び出すリソースにより、バージョンが違ったりしますがkubectl api-versionsで確認できます
※kubectl の実行結果を元に検証するので、別に確認しなくてもよい

api-versions
- https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#api-versions

crulで操作してみる

以下でプロキシを有効にします。
例だとアクセス先が”127.0.0.1:8001”になります。

kubectl proxy

例

$ kubectl proxy
Starting to serve on 127.0.0.1:8001

別タブを開いて、curl でAPIを確認してみる。
呼び出しの結果はこんな感じになりますね。

curl http://127.0.0.1:8001/api

例

$ curl http://127.0.0.1:8001/api
{
  "kind": "APIVersions",
  "versions": [
    "v1"
  ],
  "serverAddressByClientCIDRs": [
    {
      "clientCIDR": "0.0.0.0/0",
      "serverAddress": "192.168.58.2:8443"
    }
  ]
}$

#サーバのIPアドレスと一致していることが分かる
$ kubectl get nodes -o wide
NAME   STATUS   ROLES           AGE    VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION                      CONTAINER-RUNTIME
test   Ready    control-plane   6d4h   v1.27.4   192.168.58.2   <none>        Ubuntu 22.04.2 LTS   5.15.90.1-microsoft-standard-WSL2   docker://24.0.4
$

crulでいろんな操作をしてみる

kubectlでしていることcurlで再現してみる
kubectl optionsで実行結果の詳細を出力して、それをcurlで再現してみます。

詳細を出すオプションは「-v, --v Level」ですね。
※Level＝数字でこれを増やすとより詳細な情報が表示されます

kubectl options
- https://kubernetes.io/docs/reference/kubectl/kubectl/
- https://kubernetes.io/docs/concepts/cluster-administration/system-logs/#log-verbosity-level

get

namespaces の default Podをkubectlで確認すると以下で
実行結果から参考にする箇所はGET https://127.0.0.1:32779/api/v1/namespaces/default/podsですね

#通常
$ kubectl get po
No resources found in default namespace.
$

#ログレベル6
$ kubectl get po --v 6
I1104 07:04:31.991710 2947784 loader.go:395] Config loaded from file:  /home/tech-0222/.kube/config
I1104 07:04:31.992394 2947784 cert_rotation.go:137] Starting client certificate rotation controller
I1104 07:04:31.998063 2947784 round_trippers.go:553] GET https://127.0.0.1:32779/api?timeout=32s 200 OK in 5 milliseconds
I1104 07:04:32.000597 2947784 round_trippers.go:553] GET https://127.0.0.1:32779/apis?timeout=32s 200 OK in 1 milliseconds
I1104 07:04:32.007583 2947784 round_trippers.go:553] GET https://127.0.0.1:32779/api/v1/namespaces/default/pods?limit=500 200 OK in 1 milliseconds
No resources found in default namespace.
$

https://127.0.0.1:32779をkubectl proxyのhttp://127.0.0.1:8001 に置き換え実行します

$ curl http://127.0.0.1:8001/api/v1/namespaces/default/pods
{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": {
    "resourceVersion": "231632"
  },
  "items": []
}$

create

kubectlでdeploymentを作成して実行結果を確認しても
POST で何しているのかイマイチ分からない・・

bodyの値はこれを参考にするみたい
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#deployment-v1-apps

kubectl create deployment test --image=nginx --v 6
$ kubectl create deployment test --image=nginx --v 6
I1104 07:07:15.771745 2949470 loader.go:395] Config loaded from file:  /home/tech-0222/.kube/config
I1104 07:07:15.772452 2949470 cert_rotation.go:137] Starting client certificate rotation controller
I1104 07:07:15.796065 2949470 round_trippers.go:553] POST https://127.0.0.1:32779/apis/apps/v1/namespaces/default/deployments?fieldManager=kubectl-create&fieldValidation=Strict 201 Created in 22 milliseconds
deployment.apps/test created
$

ログレベルを上げると結構情報が出てくる

$ kubectl create deployment test --image=nginx --v 9
I1104 07:11:09.193858 2952517 loader.go:395] Config loaded from file:  /home/tech-0222/.kube/config
I1104 07:11:09.194497 2952517 cert_rotation.go:137] Starting client certificate rotation controller
I1104 07:11:09.196339 2952517 request.go:1212] Request Body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"test","creationTimestamp":null,"labels":{"app":"test"}},"spec":{"replicas":1,"selector":{"matchLabels":{"app":"test"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"app":"test"}},"spec":{"containers":[{"name":"nginx","image":"nginx","resources":{}}]}},"strategy":{}},"status":{}}
I1104 07:11:09.196408 2952517 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubectl/v1.28.1 (linux/amd64) kubernetes/8dc49c4" 'https://127.0.0.1:32779/apis/apps/v1/namespaces/default/deployments?fieldManager=kubectl-create&fieldValidation=Strict'
I1104 07:11:09.196636 2952517 round_trippers.go:510] HTTP Trace: Dial to tcp:127.0.0.1:32779 succeed
I1104 07:11:09.205189 2952517 round_trippers.go:553] POST https://127.0.0.1:32779/apis/apps/v1/namespaces/default/deployments?fieldManager=kubectl-create&fieldValidation=Strict 201 Created in 8 milliseconds
I1104 07:11:09.205223 2952517 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 4 ms ServerProcessing 4 ms Duration 8 ms
I1104 07:11:09.205243 2952517 round_trippers.go:577] Response Headers:
I1104 07:11:09.205263 2952517 round_trippers.go:580]     Audit-Id: 5f69b6ad-ebe8-4994-891b-6772ca04b7dc
I1104 07:11:09.205283 2952517 round_trippers.go:580]     Cache-Control: no-cache, private
I1104 07:11:09.205302 2952517 round_trippers.go:580]     Content-Type: application/json
I1104 07:11:09.205321 2952517 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 1755fe49-0f63-4c1a-907a-9b0a8f579e96
I1104 07:11:09.205344 2952517 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: dac6b1ea-44da-4c4b-add9-1923a92f74a3
I1104 07:11:09.205354 2952517 round_trippers.go:580]     Content-Length: 1641
I1104 07:11:09.205376 2952517 round_trippers.go:580]     Date: Fri, 03 Nov 2023 22:11:09 GMT
I1104 07:11:09.205429 2952517 request.go:1212] Response Body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"test","namespace":"default","uid":"99e04c85-0cf9-473e-ae25-227fc2584dba","resourceVersion":"234374","generation":1,"creationTimestamp":"2023-11-03T22:11:09Z","labels":{"app":"test"},"managedFields":[{"manager":"kubectl-create","operation":"Update","apiVersion":"apps/v1","time":"2023-11-03T22:11:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:progressDeadlineSeconds":{},"f:replicas":{},"f:revisionHistoryLimit":{},"f:selector":{},"f:strategy":{"f:rollingUpdate":{".":{},"f:maxSurge":{},"f:maxUnavailable":{}},"f:type":{}},"f:template":{"f:metadata":{"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"nginx\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}}}}]},"spec":{"replicas":1,"selector":{"matchLabels":{"app":"test"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"app":"test"}},"spec":{"containers":[{"name":"nginx","image":"nginx","resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}}
deployment.apps/test created
$

結果の"Request Body:"があるのでここらへんが関係してそう

Request Body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"test","creationTimestamp":null,"labels":{"app":"test"}},"spec":{"replicas":1,"selector":{"matchLabels":{"app":"test"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"app":"test"}},"spec":{"containers":[{"name":"nginx","image":"nginx","resources":{}}]}},"strategy":{}},"status":{}}
curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubectl/v1.28.1 (linux/amd64) kubernetes/8dc49c4" 'https://127.0.0.1:32779/apis/apps/v1/namespaces/default/deployments?fieldManager=kubectl-create&fieldValidation=Strict'

curlのPOSTでdeploymentを作成をしてみる

参考
- JSON形式なので「-H "Content-Type: application/json"」を設定
- POSTでデータを遅れるようcurlのオプションの「 -d 」でそれ以降にjsonでリソースのデータを指定

＃deployments リソースがないことを確認
$ kubectl get deployments.apps
No resources found in default namespace.
$

＃curlで成
$ curl -X POST -H "Content-Type: application/json" -d '{"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"test","creationTimestamp":null,"labels":{"app":"test"}},"spec":{"replicas":1,"selector":{"matchLabels":{"app":"test"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"app":"test"}},"spec":{"containers":[{"name":"nginx","image":"nginx","resources":{}}]}},"strategy":{}},"status":{}}' http://127.0.0.1:8001/apis/apps/v1/namespaces/default/deployments
{
  "kind": "Deployment",
  "apiVersion": "apps/v1",
  "metadata": {
    "name": "test",
    "namespace": "default",
    "uid": "510c0c80-745d-4192-b61f-95883c2cccab",
    "resourceVersion": "248029",
    "generation": 1,
    "creationTimestamp": "2023-11-04T02:52:10Z",
    "labels": {
      "app": "test"
    },
    "managedFields": [
      {
        "manager": "curl",
        "operation": "Update",
        "apiVersion": "apps/v1",
        "time": "2023-11-04T02:52:10Z",
        "fieldsType": "FieldsV1",
        "fieldsV1": {
          "f:metadata": {
            "f:labels": {
              ".": {},
              "f:app": {}
            }
          },
          "f:spec": {
            "f:progressDeadlineSeconds": {},
            "f:replicas": {},
            "f:revisionHistoryLimit": {},
            "f:selector": {},
            "f:strategy": {
              "f:rollingUpdate": {
                ".": {},
                "f:maxSurge": {},
                "f:maxUnavailable": {}
              },
              "f:type": {}
            },
            "f:template": {
              "f:metadata": {
                "f:labels": {
                  ".": {},
                  "f:app": {}
                }
              },
              "f:spec": {
                "f:containers": {
                  "k:{\"name\":\"nginx\"}": {
                    ".": {},
                    "f:image": {},
                    "f:imagePullPolicy": {},
                    "f:name": {},
                    "f:resources": {},
                    "f:terminationMessagePath": {},
                    "f:terminationMessagePolicy": {}
                  }
                },
                "f:dnsPolicy": {},
                "f:restartPolicy": {},
                "f:schedulerName": {},
                "f:securityContext": {},
                "f:terminationGracePeriodSeconds": {}
              }
            }
          }
        }
      }
    ]
  },
  "spec": {
    "replicas": 1,
    "selector": {
      "matchLabels": {
        "app": "test"
      }
    },
    "template": {
      "metadata": {
        "creationTimestamp": null,
        "labels": {
          "app": "test"
        }
      },
      "spec": {
        "containers": [
          {
            "name": "nginx",
            "image": "nginx",
            "resources": {},
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "Always"
          }
        ],
        "restartPolicy": "Always",
        "terminationGracePeriodSeconds": 30,
        "dnsPolicy": "ClusterFirst",
        "securityContext": {},
        "schedulerName": "default-scheduler"
      }
    },
    "strategy": {
      "type": "RollingUpdate",
      "rollingUpdate": {
        "maxUnavailable": "25%",
        "maxSurge": "25%"
      }
    },
    "revisionHistoryLimit": 10,
    "progressDeadlineSeconds": 600
  },
  "status": {}
}$

＃作成されたことを確認
$ kubectl get deployments.apps
NAME   READY   UP-TO-DATE   AVAILABLE   AGE
test   1/1     1            1           4s
$

delete

kubectlで作成したリソースを削除するには"kubectl delete"で以下結果となり
DELETE https://127.0.0.1:32779/apis/apps/v1/namespaces/default/deployments/test が関係ありそう

$ kubectl delete deployments.apps test --v 6

I1104 07:08:17.833647 2950433 loader.go:395] Config loaded from file:  /home/tech-0222/.kube/config
I1104 07:08:17.834363 2950433 cert_rotation.go:137] Starting client certificate rotation controller
I1104 07:08:17.857512 2950433 round_trippers.go:553] DELETE https://127.0.0.1:32779/apis/apps/v1/namespaces/default/deployments/test 200 OK in 19 milliseconds
deployment.apps "test" deleted
I1104 07:08:17.859948 2950433 round_trippers.go:553] GET https://127.0.0.1:32779/apis/apps/v1/namespaces/default/deployments/test 404 Not Found in 1 milliseconds
$

curlでの実行は以下のようになります。

＃deploymentsがある
$ kubectl get deployments.apps
NAME   READY   UP-TO-DATE   AVAILABLE   AGE
test   1/1     1            1           36s
$

＃削除
  $ curl -X DELETE http://127.0.0.1:8001/apis/apps/v1/namespaces/default/deployments/test
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Success",
  "details": {
    "name": "test",
    "group": "apps",
    "kind": "deployments",
    "uid": "e94883d0-8050-4e27-9c8c-a4edb0b05683"
  }
}$

＃deploymentsがなくなる
$ kubectl get deployments.apps
No resources found in default namespace.
$

＃curlのGETで確認すると以下
$ curl http://127.0.0.1:8001/apis/apps/v1/namespaces/default/deployments/test
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "deployments.apps \"test\" not found",
  "reason": "NotFound",
  "details": {
    "name": "test",
    "group": "apps",
    "kind": "deployments"
  },
  "code": 404
}$

Kubernetes APIをkubectlではなくcrulで呼び出してみる

説明

culでアクセスする際のクラスタと資格情報

APIのURI、メソッドについて

crulで操作してみる

crulでいろんな操作をしてみる

get

create

delete

Kubernetes kubectl rollout コマンドでdeploymentをロールバックしてみる

tfenvでterraformインストールしてバージョンを切替える

terraformのインストール方法について

Kubernetes kubectl version コマンドでバージョンを調べてみる