Dan Newcome on technology

I'm bringing cyber back

Archive for November 2022

The quest for modularity

leave a comment »

I was reading some of Avery Pennarun’s blog posts on system architecture and design. There was a set of bullet points that stood out to me that I will copy below:

The top-line goals of module systems are always the same:

  1. Isolate each bit of code from the other bits.
  2. Re-connect those bits only where explicitly intended (through a well-defined interface).
  3. Guarantee that bits you change will still be compatible with the right other bits.
  4. Upgrade, downgrade, and scale some bits without having to upgrade all the other bits simultaneously.

This article has some interesting heuristics as well:

State – If you don’t have state, just pure logic, you probably have a library, not a service. If you have state, but it doesn’t change with user requests what you have is configuration, either global or environment-specific. If changes to configuration are not time-sensitive, it can be a parameter to the deployment, which gets triggered when it changes. If they are time-sensitive you might need a configuration service that serves them continuously, rarely a bespoke one. If you do have global state that can change with, and across requests, you may have a service.

I have thought about this quite a bit over the years. My own heuristic revolves around units of deployment vs units of reuse. It’s kind of the same logic around static vs dynamic dependencies in C++. We make tradeoffs that might make sense initially (size vs complexity) but change as constraints change (more RAM/Disk, lower cost).

In the past I have favored library-driven development. I’ve come to learn that many times people are creating monolithic services (even if they are “micro”) that are really business logic libraries. When I architected the UberNote service tier, I started with a monolithic service, but the logic was all in a .NET assembly that was nicely organized into namespaces that would loosely be called “domain driven” today I think. If we needed to split things into multiple services, we could just use that library anywhere we needed to and the service would only call what it needed, or we could have split it into more than one dll to decouple the “deployment”.

I think that the most important part of a system design is the logical decomposition initially, and secondarily the topology (servers and pods, networks, etc). If the aspects of the logical design (domain?) are correct we should be able to arrange things in different topologies as we grow.

Some of the other things that commonly become issues are configuration, provisioning/defining environments, tenancy and maybe some others. Some aspects of the “12 factor app” are orthogonal here. I’m not talking about logging, etc.

Written by newcome

November 28, 2022 at 10:31 am

Posted in Uncategorized

Middleware everywhere

leave a comment »

I’m a huge fan of composition via middleware. Some people call them plugins, as we did back when I was working on Ubernote. We had plugins for all sorts of things. The only thing you need is a stable API and a default way of invoking things.

This turns out to be important later when considering how to do things in the Node/Express ecosystem. Express doubled down on plugins and I think it worked out pretty well. React has something now called Hooks which I think of in a similar way. It’s a way of plugging behavior into the page render flow without relying on fixed page lifecycle methods. Page lifecycles are a pain. It’s one of my worst memories of ASP.NET (viewstate, etc) and initially React had a similar pain with things like componentWillRender.

My current role involves a large Ruby codebase. I’m not as familiar with that ecosystem yet and I was reading some code that used an HTTP library called Faraday. Recently a co-worker was walking me through some code that used this library and I commented on having seen it and noted that it must be like requests or similar.

Well, it is, but it has some pretty neat features like… plugins! So we can do things like OAuth token refreshes in the client middleware. Pretty cool. Middleware everywhere.

Written by newcome

November 14, 2022 at 4:38 pm

Posted in Uncategorized