Configuration Service¶

The configuration service (and corresponding local components) allow for remote administration of a service or client. This remote administration is essential for enterprise deployments. The template main programs that etch creates for you have configuration information (service and listener uris) built into the code. If you go too far down this path then your programs will be hard to deploy as they will need to be recompiled to change settings. There is a natural evolutionary path often taken:

• Embedded code
• Command line (or plugin properties)
• Local configuration file
• Remote configuration file or database

At each level above we are trading off convenience for manageability. The last level is the most manageable for the operators, while the first level is the most convenient for the programmers.

The best practical systems often use a combination of the above techniques to achieve the goals of the project.

Another direction taken on configuration is whether the configuration information is static or dynamic. Static information is fixed once the program starts, dynamic configuration can change while the program is running. Consider a running service listener with some configured information for the listener uri and also the names, passwords, and other information for the users. Suppose it is statically configured. Some user information needs to be updated. The service must be stopped, the configuration updated, and the service started again. During that time the service is unavailable, and sometimes work in progress when the service went down is lost. Dynamic configuration would have allowed that service to remain available while the information was being updated.

Requirements¶

One of the goals of a configuration service should be to not make things so inconvenient for the programmers that the technique is abandoned. There are several ways that configuration interferes with programmers:

• The act of adding a new configuration item is burdensome.
• Development is hindered by requirement of using an external service.
• Different interfaces for local vs. remote configuration.
• Different interfaces for various languages.
• Parsing and checking configuration values.

From these we can come up with some requirements:

• Same api for local and remote configuration.
• Same api for various languages.
• Minimal information needed to get started during development.
• Self-contained / standalone development experience.
• Api delivers required data type.

Some other requirements come from extant ideas for configuration. XML brought us structured documents and this is perfect for configuration information as well. XML makes a lousy configuration medium because it is syntactically complex. Yaml has the advantage of being much simpler and just as powerful. Property files and command lines are initially easy but have no structure and run out of gas fast. Databases are good for structured data but are very complex to setup and use. File systems are nice and have a structured model but are perceived as heavyweight and difficult to manage. Let's go with yaml for a bit and see what we can use. Here's an example yaml file:

listenerUri: tcp://realty.net:4001
users:
    mary: { birthday: 1959-10-01, zip: 94001, pw: zowie1, active: true, interest: [ home, condo, apartment ] }
    jake: { birthday: 1967-01-19, zip: 78759, pw: flak33, active: false, interest: [ commercial ] }

Ok, we can see (or guess) the following:

• Structured organization (scalars, maps, lists)
• Easy syntax
• Simple scalar data types (boolean, integer, real, strings, dates)
• Easily supported in a variety of languages.

(I'm taking a subset of the possible yaml capabilities, ignoring references and complex language dependent data types.)

(Note that, while I'm talking about yaml, that I'm using it as a programmer friendly model.)

The data types supported by yaml match etch capabilities as well. So, let's call these a requirement.

• Data types supported include boolean, integers, reals, strings, dates, maps, sets, and lists.

In the example above, we see that users is a map with two elements, mary and jake. Mary (and jake) is a map with 5 elements. We can think of maps (and lists and scalars) as nodes in a directed acyclic graph, with maps and lists containing more nodes within and scalars only having values. If I have my finger on the node users, I can reference mary's birthday by using the path "mary/birthday". I could also globally reference jake's password as the path "/users/jake/pw". The second of mary's interests could be referenced as the path "interest/1" with value condo. This naming scheme looks like a file system or network db, and so it is.

• Configuration data is organized into a directed, acyclic graph of nodes. Each node has a unique id and is either a map, a list, or a scalar value. Each node has a single parent. Each node has a unique name relative to its parent, and a unique path of names relative to the root node.

This allows us to treat subtrees of the configuration data generically, such as the subtrees "users/mary" and "users/jake" which are each "a user". By this abstraction we can write code which maps configuration data onto objects.

Some generic operations are suggested for nodes:

• getParent, getName, getIndex, getPath, isRoot, isList, isMap, size

These operations work for all nodes. Given a node id, they return the appropriate property.

For nodes which are maps or lists, we need to be able to list the children:

• listConfigIds

Given a node id, lists the children of this node. A node id list is returned.

For nodes which are maps or lists, we need to be able to get a specific named (or indexed) child:

• getConfigPath, getConfigIndex

Given a node id and a path or index, gets the node id of the specified child node.

For nodes which are scalars, we need to be able to get the value:

• hasValue, getBoolean, getInteger, getDouble, getString, getDate, getMap, getList, getSet

The operation hasValue tests to see if the value is present, the get operations return the value as the requested data type if it is or can be converted to that type.

For convenience, the value getting operations include three extra methods: getMap, getList, and getSet. These take structural nodes and return them as best effort mappings into local equivalents of the raw underlying data. You lose the data conversion capabilities of the scalar access methods (getBoolean, etc.).

Example¶

Here is a code snippet to access a mary's birthday:

java.util.Date bd = service.getDate( service.getConfigPath( service.getRoot(), "users/mary/birthday" ) ) );

That's a bit of typing, eh? Generally you'd cache the value of root in a variable, and let's add a convenient operation to get a value given a node and a relative path without having to handle the node id ourselves:

Object root = service.getRoot();
...
java.util.Date bd = service.getDatePath( root, "users/mary/birthday" );

Complete Configuration IDL¶

Now, this interface I've defined, it includes only operations to access the configuration values, not change them. I've done it this way on purpose, so that we can quickly get started with a variety of sources and ignore the actual implementation. This interface could, in fact, be implemented in a variety of ways, including file system, network or relational db, property file, yaml file, xml file, command line, or environment variables. Where a particular service supports modification, it would perhaps require a specialized api for that. So, you'd define that api, mixin this basic access api, and that's your service. Clients (of the service) that only need to access the data would use just this basic access api.

I've not discussed operations relating to noticing updates (however accomplished). I'll catch up to that later. But they are in the access api. Here is a complete etch service idl.