Helm from Java, Part 5

I’ve amended my prior class diagram to reflect the fact that ChartVersion “is a” Metadata:

Metadata is a generated structure—part of the Tiller API—that represents an actual Helm Chart.yaml file.

Why is this important?  Because now you can see the concepts: an IndexFile is basically a collection of all the Chart.yamls conceptually held (usually in gzip archives) by a ChartRepository.  And a ChartRepository is really not much more than an IndexFile.  This whole structure can be greatly simplified.

The way I see it, the fact that this notional collection of Chart.yamls is a file is incidental, and hence IndexFile is a very poor name indeed.  Structurally, all it is is a collection of things.  We can also tell that because of the Get() function signature, an item in that collection is uniquely identified (or at least should be!) by a name and a version.  This means that this notional collection of Chart.yamls is a set (and not, say, a list, or some sort of other collection type that permits duplicates).

Next, we can tell from the Add() function that really what you’re doing with this function is creating a new ChartVersion structure and adding it.  That furthers our suspicion that really what we’re dealing with here is some sort of collection type of ChartVersions, and nothing else.

If represented in Java, it really doesn’t need to be any more complicated than a Map.  In this case, it is a Map of ChartVersions (and hence Metadata instances) indexed by a key consisting of name and version.  So a ChartRepository too is really not much more than this Map located somewhere else, together with the information needed to talk to it.

Next, ChartVersion is also a poor name.  Something like ChartDescriptor is probably better: the fact that a Version attribute happens to be one of the things it logically contains doesn’t—it seems to me—elevate it to primacy here.  What this thing is is really a full description of the state of a Helm chart (the Metadata aspect of its nature) and some information about where you can find the real thing (the URLs attribute of its ChartVersion nature).

So instead of an IndexFile having many ChartVersions that it calls its Entries (?!), in Java I propose a Map (probably a SortedMap) of ChartDescriptors indexed by ChartDescriptorKey instances.  Obviously you can create one of these from an index.yaml file from a chart repository—but just as importantly you could create it from something else.

Helm from Java, Part 4

I’ve created a UML class diagram to help me decipher the main structures in the Go code of Helm around chart repositories and their related concepts:

For the most part I stayed true to UML notation.  Similarly-colored boxes are from the same source files:

• pkg/repo/repo.go: a source file dealing with RepoFile structures.
• pkg/repo/chartrepo.go: a source file dealing with ChartRepository and Entry structures.
• pkg/repo/index.go: a source file dealing with ChartVersion and IndexFile structures.

A few interesting things stand out here.

First, from a conceptual standpoint, a ChartRepository is basically a glorified wrapper around an IndexFile, which is, in turn, a glorified wrapper around a set of ChartVersions.  A ChartVersion, in turn, is also a Metadata, which is a gRPC/Protocol Buffers object you’ve seen before (I didn’t indicate that relationship here on this diagram, but probably should have).  ChartVersions are stored (well, should be stored) within their containing IndexFile sorted by their version.  Go maybe doesn’t have the same concept as Java’s sorted collections and Comparators, so there’s some additional sorting logic that really you don’t need in the IndexFile concept.

ChartRepository instances are notionally “stored” in a RepoFile, but really what the RepoFile stores is a “pointer” to a ChartRepository—a primary key, of sorts—called, confusingly, an Entry.  More confusingly, a ChartRepository refers to its identifying Entry as its Config!  But if you think of an Entry as a primary key of a ChartRepository you should do OK.  A RepoFile is basically a pile of Entry instances, so, again, notionally, a collection of ChartRepository pointers.  (There is some question about why this data structure permits duplicate Entry instances (i.e. with the same Name but different contents); see Helm issue #2606 for details.)

I’m in the process of translating this to Java, and I think the resulting object model is going to look a lot cleaner, while delivering the same functionality, and permitting alternate implementations of chart storage and discovery.  Stay tuned.

Helm from Java, Part 3

If you use microbean-helm, you are exposed to the gRPC-colored view of the Tiller server.  (If you don’t know what I’m talking about, see part 1 and part 2.)

That view of things has a lot of gRPC cruft in it that for the most part you won’t be concerned with.  To help show you what the conceptual structure of the Tiller object model really is, I put together a UML class view of the main object model exposed by Tiller, eliminating the gRPC methods you’re not likely to use:

So fundamentally in the Helm/Tiller ecosystem, you’re working with charts, that, when installed, result in releases.  In the gRPC-generated object model, a Chart has a Metadata, several files, many values, and many Templates.

Each of these objects represents one of the items in the Helm chart file structure.  But, obviously, because with microbean-helm these are Java objects you can create them from any source you like.

Of note here is the use of Any for non-template files (like NOTES.txt).  While there are apparently lots of different ways to use this general purpose class, the Helm ecosystem appears to encode a chart file’s simple relative filename as the return value of its getTypeUrl() method, and its textual content as the return value of its getData() method. It’s not entirely clear whether Helm and Tiller got this right, but that is currently the behavior, so there you go.

It strikes me that, fuzzily, there are interesting opportunities here that involve microbean-helm, fabric8’s DefaultKubernetesClient, fabric8’s DockerClient, and so on to create Tiller-compatible charts but using plain Java.

Helm from Java, part 2

In the previous post, I outlined how microbean-helm produces the Java bindings for Helm, the Kubernetes package manager.  Specifically, it creates and packages up the gRPC Java code that describes the API surfaced by the Tiller server-side component of Helm (the thing that does the heavy lifting in the Helm ecosystem).

In the latest revision of the project, I’ve done some road grading to make it easier to work with the (cumbersome) gRPC API.  Here is how you connect to Tiller from Java and ask it for a particular release.  That is, we’re basically doing the Java equivalent of helm history someRelease:

	import java.util.List;
	import hapi.release.ReleaseOuterClass.Release;
	import hapi.services.tiller.ReleaseServiceGrpc;
	import hapi.services.tiller.ReleaseServiceGrpc.ReleaseServiceBlockingStub;
	import hapi.services.tiller.Tiller.GetHistoryRequest;
	import hapi.services.tiller.Tiller.GetHistoryResponse;
	import io.fabric8.kubernetes.client.DefaultKubernetesClient;
	import org.microbean.helm.Tiller;
	// Use a DefaultKubernetesClient to set up a port forwarding situation
	// with a Tiller pod running in your cluster, and make sure both are
	// in a try-with-resources block so they're closed properly:
	try (final DefaultKubernetesClient client = new DefaultKubernetesClient();
	final Tiller tiller = new Tiller(client)) {
	// Let's do the equivalent of helm history someRelease.
	final GetHistoryRequest.Builder builder = GetHistoryRequest.newBuilder();
	assertNotNull(builder);
	builder.setMax(1);
	builder.setName(releaseName);
	final GetHistoryRequest request = builder.build();
	final ReleaseServiceBlockingStub stub = tiller.getReleaseServiceBlockingStub();
	final GetHistoryResponse response = stub.getHistory(request);
	final List<? extends Release> releasesList = response.getReleasesList();
	final Release release = releasesList.get(0);
	}

view raw

ConnectToTiller.java

hosted with ❤ by GitHub

Under the covers, this sets up a port forwarding connection to the first Pod in the cluster that is running a healthy instance of Tiller (hopefully there is exactly one) and establishes all the gRPC plumbing for you (which is not very straightforward).  As long as you ensure that (in this example) both the DefaultKubernetesClient and the Tiller object are closed, as is done in this example, you won’t leak resources.

Happy Helming from Java!

Helm from Java!

If you work with Kubernetes, you have surely encountered Helm, the more-official-than-the-alternatives package manager for the Kubernetes platform.

Helm is a command-line tool, written in Go, that interacts with a Kubernetes cluster to make managing all the various Kubernetes resources a little easier and less of a frantic exercise in watching things break and restart automatically.  It is indispensable.  You can read more about it at its Github repository.

Helm’s machinery consists of two parts: the command line tool, helm, and the server-side componentry that stays mostly hidden behind the scenes (Tiller).  (The nautical imagery gets very old very fast but there’s no escaping it.)

When you install helm, the first thing you do (typically) is to run helm init.  This sets up some housekeeping directories and such locally, and also, very conveniently, programmatically constructs Kubernetes Deployment, Service and (optionally) Secret resources that together cause Tiller to be deployed into Kubernetes so that the helm command line tool can talk to it.  It’s a very simple idempotent bootstrap operation and is key to helm‘s simplicity.

Once you have Tiller running in your Kubernetes cluster, the helm command line tool talks to it to do the heavy lifting.  Tiller, in other words, is the real workhorse, and helm is a glorified curl for it.  (Representing it this way is a great disservice to the Helm team, of course, and I’m not actually serious, but it should help you to put the pieces together mentally.)

Now suppose you wanted for whatever crazy reasons to do the following from a Java library:

• Install Tiller if it isn’t already there
• Talk to it using a Java API

If you use my (early days!) microbean-helm project, you can now do this—no command line tooling required.

The Tiller API itself is defined using protocol buffers, which means you can generate it using gRPC.  That is handled by microbean-helm’s pom.xml file, which arranges for the protocol buffers files to be checked out of the official Helm Github repository and compiled appropriately. (Then, because they’re now part of a regular old Maven Java project, you can make Javadocs out of them too.)

So the generated Tiller API lets you talk to Tiller, which is the real heart of the whole Helm system.  With that, you can write any number of helm-like tools in Java.

Of course, you need a Tiller server to talk to.  The installation part is something that I hand-tooled by following the logic in the Helm installer code (invoked indirectly via helm init) and making it idiomatic for a Java library.  Just as with helm init, you can install Tiller if it isn’t there, or upgrade it if it is.

The net effect is that armed with this Java library you can now install or upgrade Tiller, install Helm charts, and otherwise work with Helm artifacts without having to drop down to the command line. Happy Helming—from Java!