Configuration and Dependency Acquisition

I’ve written some prior posts on the strange state of affairs that got those of us in Java-land to MicroProfile Config and the many ways in which I find it bizarre, unpredictable, clumsy and deficient. In this post I’d like to talk about what configuration systems actually depend on so we can locate them in architectural space before proceeding further.

I’ve indirectly made the case that a Java configuration system is fundamentally in the business of loading objects and handing them to you. That’s it. You ask for an object; it is delivered unto you. It is not fundamentally about conversion, “properties” and all the usual stuff people tend to bring up when they talk about Java configuration because they are subconsciously thinking about System properties and environment variables, both of which are different things and are the way they are due to requirements that have nothing to do with Java configuration per se. (But I digress.)

Next little factoid: Most Java applications these days either knowingly use a dependency injection framework (Spring, CDI) or unknowingly invent a bad version of one. Dependency injection frameworks load and make objects and hand them to you. Hmm. That sounds familiar.

Suppose now you want to use a configuration system to configure, among possibly other things, the dependency injection framework you’re using. If configuration is all about loading objects and handing them to you, and, arguably, dependency injection is at least partially about the same thing, then how do the dependencies between these two architectural nuggets work?

Cutting to the chase, the DI framework has to depend on the configuration framework, or the configuration framework couldn’t configure it before it started. So no dependency injection for you while you’re in the configuration system. That much is obvious, and yet there are still configuration systems of all sorts of stripes that get this wrong.

What may be less obvious at first glance is how to leave object production as much as possible in the hands of the dependency injection framework, and configuration as much as possible in the hands of the configuration framework. In other words, in CDI I can produce a String, and in a configuration system I can look up a String. How are these operations different? Which should I use at which points?

Or to put it another way, for a well-designed configuration system to do its job, it does in fact need to be a very limited version of a dependency acquisition (if not injection) framework itself. Fortunately, the barrier to entry is much lower for configuration systems than full-blown dependency injection systems. For example, configuration systems only really have to worry about getting The Things That Make Configured Objects Of Particular Kinds, not any object in the world. This cuts down dramatically on scope and lifecycle requirements and proxying. Also, since configuration systems are bootstrappy by their very nature, they don’t really need injection or circular reference handling or any of the other things that justifiably make dependency injection systems inherently complicated.

One great absolutely minimal example of this sort of thing that is deliberately extremely limited is java.util.ServiceLoader. This “configuration framework” instantiates an assignable class given a superinterface (or superclass) that often you have defined, and hands it to you. That’s it. That’s still a dependency acquisition framework: wherever you are, whatever architectural layer you inhabit, you ask the ServiceLoader for a provider object, and then manipulate that provider object to get you your dependency. One of the many things that’s nice about this is you can supply the types involved, so the configuration system stays out of this business. The configuration system also tells you exactly how it’s going to make a match between what you asked for and what’s available. None of this involves type conversion, scalar munging, namespaces, weak typing or other frippery. Said frippery may of course reside “lower down” the architectural stack, but that’s not the business of the ServiceLoader nor should it be.

Suppose ServiceLoader were enhanced a little bit (let’s pretend) to be able to sort through multiple service providers to hand you one that was the most suitable for you in some fashion (right now, ServiceLoader just reports all the providers it found). That would be a great example of a helpful configuration system: you ask it for a Car, and it sorts through all the various Car providers to find the one that is suited to where your application happens to be located in configuration space and picks that one. Then it tells the selected provider to get you your Car. Is it a perfect match? Maybe, but maybe not. But it is a suitable match. Will the provider supply the same Car every time it’s asked for one? Maybe, but maybe not. If the application wants a stable Car, then it caches the Car. If it does not, then it asks the provider for a Car again. This is not the business of the enhanced hypothetical ServiceLoader either.

(As an aside: MicroProfile Config dimly glimpses that this is the way to go (a Config finds a suitable ConfigSource, but falls down by then forcing all such service providers to be in the business of producing strings that other service providers (Converters) convert into what you need. Now because you are producing things of a type the application developer didn’t hand you, you have to worry about names, weak typing and namespaces when that’s not anything at all that the application developer wants to work with, and that’s how you get such a strange API.)

Anyway, armed with such a thing, then you can cause those configured objects to surface in your dependency injection framework however you like. If, for example, you’re in the middle of writing a CDI portable extension (so something executing before the dependency injection system has completed initializing), you could use the configuration system to acquire objects you need to figure out how your extension should behave, and then if they’re relevant elsewhere in the application, you could write producer methods for them, or the provider that makes them, and now the rest of your ordinary CDI code could just use them as any other bean. If you carry the dependency injection mindset to its logical conclusion, an application really shouldn’t, itself, need to consult some special system for configuration data. It will just use its normal objects, some of which will have been configured and produced by DI system internals.

If you think of things this way, then you can start to see what shape a configuration system in Java should take. More (obviously) on this topic to come.

Some of the Things I Don’t Like About MicroProfile Config

Here are some things I don’t like about the MicroProfile Config APIs (and really also most APIs like them) and the ways in which they are defined (or not defined) to work. I’m writing them down in no particular order so I don’t forget. This is obviously related to my prior post.

If I’m handed an opaque application that uses MicroProfile Config, I don’t know the names or types of the configuration properties it will seek, if any. So I don’t know what configuration to author.

If your opaque library happens to seek “fred” (and it’s opaque so I can’t know this up front; see above; but let’s say I use strings or something and get a good guess) and my library also seeks “fred“, and yours expects a certain String format for “fred” (so that it can convert it to some specialized object), and mine expects a different String format for “fred“, and I combine the two libraries into an application, and there are Converters from you and Converters from me and various ConfigSources involved, there’s no way that I can see to reconcile this other than trial and error at runtime.

If I put a ConfigSource on the classpath with ordinal Integer.MAX_VALUE that returns a random String for every value sought, or do something well-intentioned but accidental like this, pretty much everything will fail at runtime and it’s difficult to understand why. (In general, if you have to resort to ordinals or priorities in anything, you have already lost; what you’re actually after is a reduction algorithm.)

A ConfigSource can report whatever it wants from getOrdinal, so even if you set a config_ordinal property in its backing data somehow, you don’t know if your setting will be used.

A ConfigSource can return whatever it likes as its name, and since this is used as a tiebreaker in sorting, sorting is indeterminate. So which ConfigSource “answers the phone” is also indeterminate from run to run.

We insist that developers model business objects because developers know the names and types of the data they use (Person has a getAgeInYears() method that returns a positive int, not a method like getProperty(String name, Class<?> type)). But MicroProfile Config forces configuration (where the application also knows the names and types of the configuration data it consumes) to be consumed via dynamically-typed, Map-like probe requests (Config.getValue("personAge", int.class)). That’s very odd.

A Config can return an empty set for the configuration property names it claims to support because the set of such available names may change from moment to moment, or may not be able to be determined for any reason. So an application developer doesn’t know what she can ask for via the probing APIs. (Can she ask for “fred“? If she asks for “fred” twice and gets two null responses, does that mean that “fred” is never supported? Or maybe just that coincidentally “fred” couldn’t be retrieved during that particular time window? She doesn’t know, and can’t know.)

A ConfigSource can do whatever it wants, as can a Converter, as can ConfigSourceProviders, and they can all come from the classpath, whether user-, application-, application-server- or library-supplied, or all four, and a Config is essentially just a shell around these ConfigSources and Converters and ConfigSourceProviders, so not only do you not know what you can ask for, you don’t know whether it will be delivered to you for any given call you make, or, if it can, whether it can be delivered to you with the type you want, or, if you make it this far, whether such typed data can be delivered to you again with the same type if you make the same call, or if you restart the application if the same things you observed last time will hold true this time.

Since a Converter can do whatever it wants and can be user- or system-supplied, then even with the Config::getConverter method you don’t know what kind of suitable conversion is available at any given point for any given property.

Given a configuration property named “fred“, there’s no way to know what type(s) its value may successfully be converted to (other than String).

Configuration, in other words, is by spec unpredictable in name, type, availability, idempotency and determinism. That’s bizarre.

While the APIs and specified behavior for the Config-locating-and-assembly subsystem (ConfigProviderResolver) to conceivably work with any ConfigSource and Converter and Config implementations, in practice doing things like mixing and matching one vendor’s ConfigProviderResolver with another’s ConfigProvider doesn’t work.

Applications are associated with ClassLoader instances in some vague way. They could be associated with Object instead (provided the Object-in-question’s equals and hashCode methods are stable) and everything would still work. This tells you that ClassLoader association is unnecessary.

When a classloader is unloaded, various internal resources inside any MicroProfile Config implementation must be released, but there’s no real specification about how this should be done. In practice, to do this properly

There’s no specified way to configure MicroProfile Config with MicroProfile Config. There are occasional places where the need for this is visible and an attempt has been made to address it (config_ordinal, ConfigSourceProvider), but with no real systemic approach.

Notions of how to close and otherwise release resources including Converters inside any MicroProfile Config implementation require absurdly complicated WeakReference machinations in any specification-compliant implementation for no really good reason (see above). Otherwise you leak ClassLoaders.

Since you don’t know from call to call whether “fred” is available, let alone what type it can be converted to (since its format could change or conversion could be random), then injection of its value via CDI should really be defined in terms of only Provider injection. Alas. So an Integer-typed, @ConfigProperty-annotated CDI injection point that was “validated early” can fail later, or not, at runtime, which is the very sort of thing CDI is designed to prevent.

Overall, the APIs conflate the problems of object acquisition (“get me a ham sandwich”) with those of object production (“make a ham sandwich out of its parts and hand it over”), binding (“to make a ham sandwich, find bread, ham, cheese, mayo, mustard and lettuce”) and sourcing (“you find the bread in the breadbox unless it’s already out and then it’s on the counter, the cheese is in the cheese drawer, and the mayo is usually on the top shelf of the fridge, unless you have to open a new bottle….”). Then the results are served up with unpredictable scalar probing methods (unless I’m using CDI injection, I have to ask for the bread, then the condiments, then the lettuce and put them together, assuming I actually got results for these). This is all very strange. I just want a ham sandwich. The person making the ham sandwich probably cares a great deal about all this stuff, but I wouldn’t be asking them for a ham sandwich if I wanted to make it myself! (Also I’m never going to convert bread to mustard.)

Finally, an application developer also basically just wants a ham sandwich. She wants to receive a HerApplicationConfigThatSheDesigned object in some ceremony-free manner that’s appropriate for the environment her application is in, and call typed methods on it at various points during the lifespan of her application to get the information she needs, handling errors in the way that she sees fit. You can see examples of this in the wild. Consider this, and then re-read what’s above and sit with it.

There are so, so, so many ways this whole area could be better. I hope to say more soon.