Software Engineer

Building a k8s CRD Go (golang) Client

· by jsnby · Read in about 5 min · (859 Words)
kubernetes golang

UPDATE: I’ve redone this work with kubebuilder and documented that here. That process is much smoother.

This document is mainly intended for myself to reference in the future the next time I have to build a CRD. The code can be found in the jasonhancock/examplecrd repo on Github.

I was recently building something and decided that I was going to store some data as a Custom Resource inside the k8s cluster. It’s 2023 and it seems like this should be an easy task. My application that will be creating the custom resources is written in Go (golang), so I’ll need to be able to generate a Go client for my custom resource type. This turned out to be harder than it really should be. There were some other articles covering things, but they haven’t aged well:

The tooling also leaves a lot to be desired like not supporting Go modules/having to have your code inside the $GOPATH is just one example.

Let’s start by working through the CustomResourceDefinition. First, I needed to pick some unique name for the API group. This should be unique but informative. I’m going to select Now we should decide on a Kind. I’m going to call it ExampleResource, which translates into a name of You can see those in the CRD below.

We then need to decide how to scope our resources…should they be namespaced, or cluster-wide? I’m going to go with namespaced.

Now we decide what data we want on the resource. Keeping this example simple and generic, I’m going to say each resource has a color and a size, both of which are strings and both of which are required. Our CRD looks like this:

apiVersion: ""
kind: "CustomResourceDefinition"
  name: ""
  group: ""
  scope: "Namespaced"
    plural: "exampleresources"
    singular: "exampleresource"
    kind: "ExampleResource"
  - name: v1
    served: true
    storage: true
        type: object
        - spec
            type: object
            - color
            - size
                type: string
                type: string

I’m going to set up a workspace to contain this work:

cd ~/development
mkdir examplecrd
cd examplecrd
go mod init

I’m going to write the CRD from above into a file crd.yaml, then apply it with kubectl:

kubectl apply -f crd.yaml

Let’s manually create an ExampleResource with kubectl. example.yaml:

apiVersion: ""
kind: "ExampleResource"
  name: "testing"
  color: red
  size: large
kubectl apply -f example.yaml

And we can list our resource:

$ kubectl get exampleresources
testing   39s
$ kubectl get exampleresource testing -o yaml
kind: ExampleResource
  annotations: |
  creationTimestamp: "2023-07-26T22:11:39Z"
  generation: 1
  name: testing
  namespace: default
  resourceVersion: "6950"
  uid: be066c5d-7cd2-4a2e-916e-3b6f17f37879
  color: red
  size: large

So that’s great. But now I want to do the same thing from some Go code, so let’s get started generating our client. Let’s create a types.go file in the path from the root of our repo of pkg/apis/

package v1

import metav1 ""

//go:generate controller-gen object paths=$GOFILE

// ExampleResourceSpec is the spec of the ExampleResource object.
// +kubebuilder:object:generate=true
type ExampleResourceSpec struct {
	Color string `json:"color"`
	Size  string `json:"size"`

// +genclient

// ExampleResource is our example resource.
type ExampleResource struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec ExampleResourceSpec `json:"spec"`

// ExampleResourceList is a list of ExampleResource objects.
type ExampleResourceList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`

	Items []ExampleResource `json:"items"`

Create a doc.go in that same directory:

// +k8s:deepcopy-gen=package
// +k8s:defaulter-gen=TypeMeta

package v1

And a register.go:

package v1

import (
	metav1 ""

// SchemeGroupVersion defines the group info of the API.
var SchemeGroupVersion = schema.GroupVersion{
	Group:   "",
	Version: "v1",

var (
	schemeBuilder      runtime.SchemeBuilder
	localSchemeBuilder = &schemeBuilder

	// AddToScheme adds to the scheme.
	AddToScheme = localSchemeBuilder.AddToScheme

func init() {
	// We only register manually written functions here. The registration of the
	// generated functions takes place in the generated files. The separation
	// makes the code compile even when the generated files are missing.

// Resource takes an unqualified resource and returns a Group qualified GroupResource
func Resource(resource string) schema.GroupResource {
	return SchemeGroupVersion.WithResource(resource).GroupResource()

// Adds the list of known types to the given scheme.
func addKnownTypes(scheme *runtime.Scheme) error {



	return nil

Let’s update go.mod by running a tidy:

go mod tidy

Let’s set up some tooling….

cd /tmp
git clone

Configure our shell:

export CODEGEN_PKG=/tmp/code-generator

Fake the tooling to think we’re in a $GOPATH:

mkdir /tmp/go/src/
cd /tmp/go/src/
ln -s ~/development/examplecrd
cd /tmp/go/src/
export GOPATH=/tmp/go

The hack/ can be found in the examplecrd repo. I had some trouble getting the binaries from the code-generator to build.

Generate the deepcopy stuff:

cd ~/development/examplecrd/pkg/apis/
go generate
unset GOPATH
go mod tidy

You should then be able to build/run the main.go:

export KUBECONFIG=~/.kube/config
cd ~/development/examplecrd
go run main.go
testing red large

This entire process seems like it’s harder than it needs to be. It would be great if the k8s maintainers would at least drop the $GOPATH requirement and work with Go modules as that would likely make things a lot simpler/user friendly.