BOSS Great Wall.jpg
Figure 1. In software, we have no view of the large scale structure

Engineering the large scale structure of code

John R Spray

Last update: 2020-02-27

I would like to acknowledge the help of Roopak Sinha at AUT (Auckland University of Technology) in the writing of the paper for ECSA 2018 including his academic perspective.

Organisation of this site:

  • Chapter One, "What problem does ALA solve" explains the current major problems in software engineering that ALA solves.

  • Chapter Two, "What does the structure look like?" gives several different perspectives on what the ALA structure looks like.

  • Chapter Three, "Why the structure works" explains why this structure solves the problems outlined in chapter one.

  • Chapter Four, "Execution models", explains the different ways to make the CPU execute the right code at the right time when the organisation of the code that emerges from ALA is completely independent of the execution flow.

  • Chapter Five, "Methodology", describes ALA in terms of where it fits into Agile software development.

  • Chapter Six, "The philosophy behind ALA", gives the theory of why ALA works from the perspective of complex systems and how our brains work.

  • Chapter Seven, "ALA compared with…​", compares ALA with existing architectural styles and patterns, both similarities and differences. Choose the one you are most familiar with to see how ALA uses it, changes it slightly, and adds to it.

Each chapter ends with an example project. These projects ground the architecture in real code. Unlike most pedagogical sized examples, these examples are progressively non-trivial. Yet because of ALA’s power, they remain small and easily readable.

1. Chapter one - What problem does it solve?

1.1. The Big Ball of Mud

ALA is an in-the-large strategy to structure code. It tells you how to organise your code so it doesn’t degenerate, little by little, into sphagetti code, or what Brian Foote and Joseph Yoder describe as a "big ball of mud" during its life cycle.

Existing architectural patterns, styles, or principles such as Layers, Decomposition, DSLs, Components, Models, Event-Driven, MVC, Inversion of Control, Functional Programming, Object Oriented Design) are insufficient by themselves. What is needed is a big organisational strategy. That strategy makes it clear where, how and why to use all these other styles.

1.2. An optimal reference architecture

ALA is a reference architecture. It is independent of any specific domain, so it is a general reference architecture. The reference architecture is 'optimal' for certain non-functional requirements. By optimal, I mean that it makes these qualities as good as they can be.

  • Readability

  • Complexity

  • Maintainability

  • Testability

If other non-functional requirements are also important, ALA provides a good starting point. Even if the ALA structure must be compromised for other qualities, it is still better to start with these quality attributes optimised and deviate from them as necessary. As it happens, the maintainability resulting from ALA frequently makes other quality attributes easy to achieve as well. For example, in an ALA application it is often easy to make local performance optimizations in the interfaces that don’t affect the application code. Or, you can port an application without changing the application code.

1.3. Readability

close up code.jpg
Figure 2. Code quickly becomes a big ball of mud

ALA code is readable, not because of style, convention, comments or documentation, but because any one piece of code appears to you as a separate little program.

1.4. Complexity

There is a meme in the software industry that says that the complexity of software must be some function of its size. This need not be so. With proper use of abstraction it is possible to have complexity that is constant regardless of program size. ALA makes use of this.

complexity curve.png

This is a qualitative graph comparing the complexity of an ALA application with that of a big ball of mud and an average loosely coupled application. This is further explained later here.

1.5. Maintainability

The maintainability effort over time should qualitatively follow the green curve in the graph below because as software artefacts are written, their reuse should reduce the effort required for other user stories. Product owners seem to have an innate sense that we manage to organise our code such that this happens. That is why they get so frustrated when things seem to take longer and longer over time, and they often ask us "haven’t we done this before". In practice, too often we follow the red curve. Maintenance eventually gets so difficult that we want to throw it away and start again. We reason we can do better. My experience is that we don’t do better when we rewrite. We just create another mess. It is just a psychological bias on the part of the developer caused by a combination of a) the Dunning Kruger effect and b) the fact that it is easier to read our own recently written code than someone else’s.

If we apply all the well known styles and principles, the best we seem to be able to manage is the orange curve, which still has maintenance effort continuously increasing with an exponential factor.

However, whenever we have done an experimental re-write using ALA, it comes out spectacularly better.

effort curve.png

ALA is based on the theoretical architectural constraints needed to follow the green curve.

1.6. Domain oriented

As has been found useful in other methodologies such as Domain Specific Languages, Domain Driven Design, Model Driven Software Development and Language Oriented Programming, ALA provides a way to be 'domain oriented'.

But unlike most of the other domain oriented methodologies, ALA provides a way to be domain oriented with ordinary code, and with the same development envorinment. It is just a way to organise ordinary code to be domain oriented.

1.7. The software engineer’s trap

Typical bright young engineers come out of university knowing C++ or Java (or other C*, low-level, imperative, language that mimics the silicon), and are confident that, because the language is Turing-complete, if they string together enough statements, they can accomplish anything. At first they can. Agile methods only require them to deliver an increment of functionality. There hardly seems a need for a software architect to be involved. And besides, we are told that any design can emerge through incremental refactoring.

Figure 3. Code can quickly get complex

As the program gets larger, things are getting a little more complicated, but the young developer’s brain is still up to the task, not realizing he has already surpassed anyone else’s ability to read the code. He is still able to get more and more features working. One day the code suddenly 'transitions'. It transitions from the complicated quadrant into the complex quadrant. And now it is trapped there. It is too complex for the in-the-large refactoring that would be required to make it transition back. This pattern happens over and over again in almost all software.

The incremental effort to maintain starts to eat away and eventually exceed the incremental increase in value. This now negative return causes the codebase itself to eventually lose value, until it is no longer an asset to the business.

When a new bright young engineer who knows C* arrives, he looks at the legacy codebase and is convinced that he can do better. And the cycle repeats. This is the CRAP cycle (Create, Repair, Abandon, rePlace). ALA is the only method I know that can prevent the CRAP cycle.

1.8. A short history of ALA

From early on in my career, I experienced the CRAP cycle many times. Each time I wanted to find a way to not fall into it. I would research and use all the architectural styles and principles I could find. I would come across things like 'loose coupling', and I remember asking myself, yes but how does one accomplish that?, and still fail.

I started searching for a pre-worked, generally applicable, 'template architecture' that would tell me what the organisation of the code should look like for any program. I searched for such a thing many times and never found one. Some would say that this is because the highest level structure depends on project specific requirements.

Forty years worth of mistakes later, I finally have that template meta structure that all programs should have. The turning point was when I noticed two (accidental) successes in parts of two projects. These successes were only noticed years later, 15 years in one case and 5 years in the other. They had each undergone considerable maintenance during that time. But their simplicity had never degraded and their maintenance had always been straightforward. It was like being at a rubbish dump and noticing two pieces of metal that had never rusted. "That’s weird", you think to yourself. "What is going on here?"

One of them had the same functionality as another piece of software that I had written years earlier. That software was the worst I had ever written. It was truly a big ball of mud, and maintenance had become completely impossible, causing the whole product to be abandoned. So it wasn’t what the software did that made the difference between good and bad. It was how it was done.

Analysing the common properties of those two code bases, gave clues that eventually resulted in a theoretical understanding of how to deal with complex systems. This meta-structure is what I now call Abstraction Layered Architecture.

Subsequently, I ran some experiments to see if the maintainability and non-complexity could be predictably reproduced. These experiments, which have worked spectacularly well so far, are discussed as a project at the end of every chapter.

1.9. Simplify the overwhelming software architecture styles, patterns & principles

Currently the problem of structuring software code to meet quality attributes involves mastering an overwhelming number of software engineering topics:

  • Complexity, Understandability, Readability, Maintainability, Modifiability, Testability, Extensibility, Dependability, Performance, Availability, Scalability, Portability, Security, usability, Fault-tolerance

  • Views, Styles, Patterns, Tactics, Models, UML, ADL’s, ADD, SAAM, ATAM, 4+1, Decomposition

  • CBD/CBSE, C&C, Pipes & Filters, n-tier, Client/Server, Plug-in, Microservices, Monolithic, Contracts, Message Bus

  • Modules, Components, Layers, Classes, Objects, Abstraction, Granularity

  • Information hiding, Separation of Concerns, Loose Coupling & High Cohesion

  • Semantic coupling, Syntax coupling, Temporal coupling, existence coupling, Dependencies, Interactions, Collaboration

  • Interfaces, Polymorphism, Encapsulation

  • Execution models, Event-Driven, Multithreaded, Mainloop, Data-driven, Concurrency, Reactor pattern, Race condition, Deadlock, Priority Inversion, Reactive

  • Principles: SRP, OCP, LSP, ISP, DIP; MVC, MVP, etc

  • Design Patterns: Layers, Whole-Part, Observer, Strategy, Factory method, Wrapper, Composite, Decorator, Dependency Injection, Callbacks, Chain of Responsibility, etc

  • Expressiveness, Fluency, DDD, Coding guidelines, Comments, Documentation

  • Programming Paradigms, Imperative, Declarative, OO, Activity-flow, Work-flow, Data-flow, Function blocks, Synchronous, State machine, GUI layout, Navigation-flow, Data Schema, Functional, Immutable objects, FRP, RX, Monads, AOP, Polyglot-Programming Paradigms

  • Messaging: Push, Pull, Synchronous, Asynchronous, Shared memory, Signals & Slots

  • Memory management, Heap, Persistence, Databases, ORMs

  • Up-front design, Agile, Use cases, User stories, TDD, BDD, MDSD

Mastering all these topics takes time. Even if you can, juggling them all and being able to use the right ones at the right time is extremely taxing on any developer. Add to that the mastering of technologies and tools, keeping to agile sprints deadlines, and commitment to your team and management, it is an almost impossible task. 'Working code' tends to be what the team is judged on, especially by project managers or product owners who have no direct interest in architecture or even the Definition of Done. They don’t want to know about the rather negative sounding term, "technical debt".

ALA works by pre-solving most of these software engineering topics into a single 'meta-style' This meta-style provides a simple set of architectural constraints.

Being a pre-worked recipe of the aforementioned list of styles and patterns, ALA contains no truly novel ideas. Some ingredients are accentuated in importance more than you might expect (such as abstraction). Some are relatively neutral. Some are putposefully left out. The biggest surprise for me during the conception process of ALA was that some well-established software engineering memes seemed to be in conflict. Eventually I concluded that they were in-fact plain wrong. We will discuss these in detail one at a time in subsequent chapters. But to wet your appetite here is one meme that ALA throws out: the UML class diagram. Read on to find out why.

Like any good recipe, the ingredients work together to form a whole that is greater than the sum of parts. The resulting code quality is significantly ahead of what the individual memes do. It continues to surprise me just how effective it is.

1.10. The first few strokes

As a software engineer contemplating a new project, I have often asked myself "Where do I start?" This also happens with legacy code, when contemplating the direction that refactorings should take. "If this software were structured optimally well, what would it look like?"

Christopher Alexander, the creator of the idea of design patterns in building architecture, said, "As any designer will tell you, it is the first steps in a design process which count for the most. The first few strokes which create the form, carry within them the destiny of the rest". This has been my experience too.

In Agile, where architecture is meant to emerge, this wisdom has been lost. ALA restores that wisdom to software development, and gives the software architect the exact process to follow for that little piece of up-front design. No more than one sprint is required to do this architectural work, regardless of the size of the project.

Furthermore, once this architectural work is done, the Agile process works significantly better thereafter. Furthermore, my experience over several projects so far is that the initial architecture does not need to change as the development proceeds.

1.11. Example project - Functional composition

In this example, we use 'functional composition' because it is a programming paradigm we all already know. However, keep in mind that simple functional composition (without monads) is not a suitable programming paradigm for the most part of a typical programs. It suits when a problem requires dedicated CPU to process a job as fast as it can in computer time, the sequence is known ahead of time (proactive not reactive), nothing else needs doing while this is happening, and it doesn’t have to wait for anything while it is being done. Nevertheless, this can be a solution for small programs.

Applying ALA to functional composition means three things:

  • Every function is an abstraction.

For our purpose here, an abstraction means that our brain can easily learn (by reading the function name or a comment) and retain what a function essentially does. It means that when other programmers are reading your code where a function is called, they don’t have to 'follow the indirection' - they can stay with the code unit they are in and read it like any other line of code. It means a single responsibility. It means it knows nothing about the content of any other abstractions. It means reuseable, and it means stable. The name of the function should not be generic ProcessData, or CalculateResult. It should not be the name of the event that caused it to be executed like PulseComplete.

  • Functions go in a small number of discrete abstraction levels.

This implies that function call depth is at most three (not counting library functions at a 4th level).

The first level function contains all knowledge about the application requirements. No implementation here, just describe the requirements in terms of other functions.

The second level is functions that contain knowledge about the domain. It has all the abstractions needed to make it possible for the first level to describe the requirements. No function at this level knows anything about the specific application. An example would be calculate mortgage repayments, or filter data.

The third level functions are at an even greater level of abstraction, things that would be potentially reusable in many domains. It should have the abstraction level of the types of programming problems being solved. Examples might be communications, persistence, logging. None of these functions can have any knowledge of the specific application, nor the domain. So the persistence functions are not persistence of specific domain objects. With configuration, they would know how to persist anything.

A function that doesn’t clearly belong at one of these abstraction levels should be split in two. Specific application knowledge generally becomes configuration parameters in the higher layer of a more abstract function in the lower layer.

For completeness, a 4th level would be your programming language library. No where in these levels is the underlying hardware, nor data. Later we will see where they go, but for now forget all preconceived notions of layers such as UI, business logic and Database. In ALA, these are incorrect.

  • The first layer just describes the requirements.

The top layer describes requirements and that’s all it does (like a DSL). It composes functions from the lower layers, and configures them for a specific purpose according to the requirements.

Let’s look at some bad code that breaks each of these constraints and then the corresponding code that fixes them.

1.11.1. Bad code

 void main()
    while (1)
        GetTemperatures(temperatures); // gets 100 temperatues (1)(2)
        temperatures = ProcessTemperatures(temperatures); (1)(2)
 // do everything needed to process an adc reading
 float ProcessTemperatures(temperatures)  (4)(5)
    for (i = 0; i<100; i++) {  (3)
        temperature = (temperatures + 4) * 8.3;  (3)
        rv = Smooth(temperature);  (6)(7)
    return rv;
 // smooth the reading before displaying
 float SmoothTemperature(temperature) (4)
    static filtered = 0;
    filtered = filtered*9/10 + temperature/10; (3)
    return filtered;
1 function name is specific to this application, destroying it as a potential abstraction
2 functions are collaborating to implement the 100 samples at a time requirement
3 details from requirements appearing inside functions (all the constants), destroying potential abstractions
4 function name doesn’t describe an abstraction
5 function has three responsibilities, process 100 samples at a time, convert to Celsius, and Filtering
6 function composition in wrong level (only the application knows this needs doing
7 function composition too deep (function composition should be shallow)
8 Temporal problems - if adc readings take 1 ms, main loop time is 100 ms

1.11.2. Better code

 void main() (1)
    while (1)
        adc = GetAdcReading(port=21);  (2)
        temperatureInCelcius = OffsetAndScale(adc, offset=4, slope=8.3); (5)
        smoothedTemperature = Filter(temperatureInCelcius, 10); (6)
        if (SampleEvery(100)) (4)
offsetandscale.c - (domain abstraction)
 // offset and scale a value
 void OffsetAndScale(data, offset, scale) (3)
    return (data + offset) * scale;
filter.c - (domain abstraction)
 // IIR 1st order filter, higher filterstrength is lower cutoff frequency
 float Filter(int input, int filterStrength)  (3)
    static float filtered = 0.0; (7)
    filtered = (filtered * (filterStrength-1) + input) / filterStrength
    return filtered;
sample.c - (domain abstraction)
 // Returns true every n times it is called
 bool SampleEvery(int n)  (3)
    static counter = 0; (7)
    if (counter>=n)
       counter = 0;
       rv = true;
       rv =  false;
    return rv;

The code is now arranged clearly into two abstraction layers, the application layer and the domain abstractions layer. Other domain abstractions used by the application are not shown: the ADC, and the Display.

1 The application is readable in isolation (once you know what the abstraction it is composed of do) without having to go and read code inside any of the abstractions.
2 The application describes the requirements, all of the requirements, and does nothing else. It delegates all the actual work to domain abstractions. Only the application knows it is a thermometer. The application knows nothing of how the abstractions work, only what they do.
3 None of the abstractions know anything about each other or anything about the application. They don’t know they are being used to make a thermometer. They are readable in isolation.
4 The application knows the thermometer requirement detail of how many ADC readings are needed for each temperature display update.
5 Application knows the thermometer specific conversion factors from ADC to Celsius but not how to do offsetting or scaling.
6 Application knows the configuration of the filtering it needs to be a smooth thermometer but not how to do filtering.
7 The emphasis is on 'abstraction' not on 'zero side effects'. Filter and SampleEvery are good abstractions despite having a side effect.
8 Temporal problems are mitigated somewhat - the main loop now goes much faster. (But the main loop period still includes the ADC conversion time and Display output time. To fix that we will need to compose with monads or objects generally.)

These are more properties of the abstraction layered version:

  • The application can easily be rewired to do things like the following examples:

    • swap the order of processing of the scaling and the filtering

    • insert a new data processing operation between say the scaling and the filter

    • add a logging output destination at a higher data rate

    • switch to a different type of ADC or display

    • add adapters or wrappers for using 3rd party components

  • If the requirements of the thermometer change, no domain abstractions would change - because, as abstractions, they don’t know anything specific about thermometers.

  • In this 'functional composition', at run-time, data comes up into the application code layer and back down into the domain abstractions layer at each step. That’s why the application has some local variables to identify the data at various points during the processing. In most other programming paradigms we will use, the data will not come up to the application layer at run-time. Instead, it will go directly between the instances of the domain abstractions. The application will then only wire together instances.

1.11.3. Composing with monads

Remember that while we have composed with functions in this example, it is not generally a useful programming paradigm or execution model. We have composed with imperative functions here because they are a familiar programming paradigm that we all learnt immediately we started programming. But we won’t get far doing composition with functions. The problem is that functional composition forces the execution flow to follow exactly the composition flow. This only suits a narrow range of problems. Usually we will need to use composition in other programming paradigms (execution models) that separate execution flow from composition flow.

For example, the most common programming paradigm we will use is Data-flow. When we compose domain abstractions together, we mean that at run-time data will pass between adjacently wired instances. A common Data-flow paradigm is monads. We wont learn further about monads, except to say that this is what the Thermometer example might look like using them.

 void main()
    program = new ADC()
    .OffsetAndScale(adc, offset=4, slope=8.3)


Notice how the while loop we had before is gone. The code now just describes the flow of data. How it actually executes has be abstracted away and is handled separately.

1.11.4. Composing with plain old objects

Here is how you may write the program using plain objects. The wireIn method allows us to compose with objects:

 void main()
    program = new ADC()
       .wireIn(new OffsetAndScale(adc, offset=4, slope=8.3))
       .wireIn(new SampleEvery(100))
       .wireIn(new Display());

Monads are a way of composing data-flows. It’s generally one flow of data in one direction. But in ALA, when we compose, each connection may carry multiple flows, or in both directions. Or it may represent something else entirely. So while composing with functions, or composing with monads can be compliant with ALA, we want the power of composition in a much more general way.

To illustrate this lets put some UI on our thermometer:

Figure 4. Thermometer application complete with UI

We have added a little bit of UI to show that we are composing with more than just data-flow. We are specifying the UI objects as well. For UI objects, the lines don’t mean data-flow, they mean 'display inside'. So now different lines have different meanings. Here is how that diagram is represented as text.

 void main()
    program = new ADC()
       .wireIn(new OffsetAndScale(adc, offset=4, slope=8.3))
       .wireIn(new SampleEvery(100))
       .wireIn(temperature = new FloatField());

    mainwindow = new Window()
       .wireTo(new Label("Temperture:"))


There are two programming paradigms here - the meaning of the wiring is data-flow in some parts, and UI layout in other parts. This is all done in the one cohesive piece of code that represents the thermometer application.

The examples to follow will use a range of different programming paradigms and consequently 'composition' will mean different things. Sometimes we will use custom programming paradigms - whatever allows us to describe those requirements in the best way.

2. Chapter two - What does the structure look like?

In this section we just describe the anatomy of the ALA structure without trying to explain too much about why it looks that way. That will be covered in Chapter three.

We describe it in several different ways because we all have different experience or different prior knowledge on which we build comparative ideas. So we each need a different perspectives. This chapter has about 10 different perspectives, some concrete, some more abstract, some metaphores, and finally some code. Use the one that best explains the insight for you.

2.1. Four important SW engineering ideas brought together

ALA is the bringing togther of four important software engineering ideas. All are absolutely essential:

  • Abstraction

  • Composition

  • Direct expression of requirements

  • Polyglot Programming Paradigms (execution models)

Surprisingly, none of these four are particularly main-stream (relative to other memes in SW engineering). And I have never seen all four used together anywhere else, so that is what makes ALA unique. Using all four together is incredibly powerful.

(Polyglot = 'uses multiple different')

2.2. Code organisation into folders

We start with a practical viewpoint of ALA - how it organises code into folders.

If you see an ALA application, you will find three folders called:

  • Application

  • DomainAbstractions

  • ProgrammingParadigms

There should also be a readme file that points to this website (or equivalent documentation). In ALA, we are explicit about what knowledge is needed before a given piece of code can be understood (knowledge dependencies). To understand an ALA structured application, you need a basic understanding of ALA (from this chapter). So that’s why there should be a readme file.

Continuing with the idea of knowledge dependencies, the class in the Application folder will have knowledge dependencies on the classes in the DomainAbstractions folder. In other words, you need to know what the classes in the DomainAbstractions folder do in order to read the application code. Similarly the classes in the DomainAbstractons folder have knowledge dependencies on the interfaces in the ProgrammingParadigms folder. There are no dependencies between classes within a folder.

In the Application folder, you will usually find a diagram. This diagram describes the requirements. The diagram is 'complete' in that it describes all details of the requirements - it is not just an overview. The diagram is itself 'executable'. ALA is just a way of writing requirements that are executable.

It should be quite easy to read the diagram as it only describes the requirements and does not involve itself with implementation. The boxes are instances of the DomainAbstractions (objects). The lines make a specific composition of instances.

There should be a code file that exactly represents the diagram. It is generated from the diagram. So the diagram is the source. However, looking at this code file may clarify how the diagram is represented in code.

Every box in the diagram is an instance of one of the classes in the DomainAbstractions folder. These classes are called abstractions rather than modules or components because they have zero knowledge of each other and zero knowledge about the specific application. Their abstraction level is more general than the application, and so they are reusable within a domain. For now a domain can just mean your company.

The lines in the diagram represent connections using one of the interfaces from the ProgrammingParadigms folder. There is usually more than one interface, but no more than a few. Each represents a 'programming paradigm' such as event flow, data flow, a UI composition, or a schema relationship. The abstraction level of the ProgrammingParadigms folder is more general again than the DomainAbstractions - each paradigm should be useful for a type of computing problem in many different domains. This is the 'abstract interactions' pattern.

This small set of interfaces allows instances of domain abstractions to be wired together in an infinite variety of ways - the property of composability.

2.3. How classes are used

this is another practical viewpoint, this time on how classes are used in ALA programs.

In ALA, a class’s public interface (it’s public methods and properties) are only used to instantiate and configure the class. It is not used for anything the class actually does. The public interface is 'owned' by the class so is specific to the configuration of that class. The public interface is only used from a class in a layer above. Only that layer knows what should be instantiated, how it should be configured, and how the instantiated objects are composed together to make a system.

All other operations are done through interfaces. Class don’t 'own' these interfaces - they are not specific to any one class. They are not about what any one class does, or needs. They are more general so that typically many different classes will implement/accept them. Objects of different classes can then be connected together using these more general interfaces in a variety of ways. The implication is that classes do not have association relationships. The lines that you would normally see dominating most UML class diagrams are completely absent if you drew a class diagram of an ALA application.

ALA doesn’t need or use inheritance either. So the only relationship between classes is composition. If you drew a class diagram in ALA, you wouldn’t draw lines for composition. This is because you are composing abstractions. You wouldn’t draw a line to a square-root function every time you used it. It’s the same thing when using any abstraction. So it turns out that if you did try to draw a class diagram in ALA, it would have no lines at all. So there’s no point.

Any given class will typically implement/accept more than one of the generic interfaces. For the data-flow interfaces at least, think of them as I/O ports. This is the interface segregation principle, except that we do not refer to the other objects as clients. Only the class in the layer above (that uses the public interface) has the status of a client. The objects to which an object is wired are peers.

2.4. Abstraction Layers

In contrast to the previous two sections that talk about the use of folders and classes, this section gives the most abstract perspective we will use. I introduce it now because it is the one that gives ALA its name.

This figure shows the abstraction layers:

Layers diagram
Figure 5. The four ALA layers

The first problem in understanding abstraction layers is understanding what abstraction means. Unfortunately the software industry has misused the word to the point where we get things upside down. This comes about because it sees hardware or alternatively the database at the bottom, and since hardware and databases are 'concrete', we argue are the least abstract. And so we build things on top of those that supposedly get more abstract. Whatever is at the very top, we argue, being the farthest away from the concrete silicon, must be the most abstract.

This thinking is completely wrong. We will look in depth at what 'abstract' means in a later section, but for now, just suspend everything you think you know about abstraction. In ALA we will say that 'more abstract' means 'more ubiquitous', 'more reusable' and 'more stable'. The application, at the top, is the least abstract. Also suspend everything you think you know about layers. In ALA, the hardware is never at the bottom. And neither is the database. Your programming language is.

Because this perspective probably doesn’t really connect with anything you already do, we will just list three key takeaway points from this section. These will become clearer later. In ALA:

  1. The only dependencies you are allowed are on abstractions (shown as green arrows on the figure) and referred to as 'knowledge dependencies' or 'design-time dependencies' (as opposed to run-time dependencies).

  2. The first three abstraction layers are Application, Domain Abstractions, and Programming Paradigms.

  3. The layers get more abstract as you go down, and therefore more ubiquitous, more reusable, and more stable.

2.5. Executable Description of Requirements

If I had just two minutes to explain what ALA is, this is the perspective I would use:

This perspective puts the focus on your input information - the requirements. ALA is a methodology that finds a way to directly describe requirements. It describes all the details in the requirements. Instead of having two artefacts, one for requirements capture and one for software source, ALA combines them as a single document and a single source of truth. BDD (Behavioural Driven Design) does something similar, but only achieves it for requirements and their tests. ALA goes one step further and makes the expressed requirements also the solution.

The description of the requirements itself has no implementation details. It just describes all details of requirements. The amount of code that describes requirements is typically about 2% of the entire application. When requirements change, you only need to change this 2%.

The description of the requirements is executable. Even though the description has no implementation details, it still executes directly. The expressed requirements is also the application.

No Separate Architecture

The executable description of requirements in the top layer is also the architecture or the design. (I do not make a distinction between architecture and design.) There is no separate artefact or documentation of the architecture, no model, no "high level" design. The same artefact that describes the requirements and is executable is also the application’s architecture. One source of truth for everything.

2.6. Create and Compose

If I had ten minutes to explain what ALA is, this is the perspective I would use.

A common cliché for tackling complexity is "divide and conquer". Now here is a surprise. In ALA we do not divide and conquer. Instead we use a different cliché, "Create and Compose"

Here are a few examples of composing:

  • When we write code in a general purpose programming language, we are composing with statements. Statements are low level (fine grained) elements and only support a single programming paradigm, which we could describe as 'imperative', but by composing instances of them in different ways we can create something. The structure is linear or a tree.

  • In functional programming, we are composing with functions, so the elements are higher level things that you create. But the programming paradigm is still imperative. The structure is linear or a tree.

  • When programming with monads, we are composing with amplified data types. These are usually low-level elements. But the programming paradigm has changed from imperative to data-flow. The structure is usually linear. (You don’t need to understand or use Monads to use ALA. however, See my method to understand Monads in Chapter Six

  • When programming using the UML class diagram, we are composing high-level classes. The programming paradigm is associations. The syntax is graphical. The structure is a network.

  • When programming with XAML, we are composing with fundamental UI elements. The programming paradigm is UI layout.

Let’s list the different properties present in these composition methods:

  • low-level/high-level - A fixed set of fundamental elements versus elements that you can create.

  • Programming paradigm: The meaning of a composition relationship is fixed in each case. It can be Imperative, Data-flow, UI layout etc.

  • Linear/Tree/Network: The structure built by the composition relationships can be linear, a tree structure or a general network.

  • Syntax: The syntax for the composition relationship can be using spaces, dots or boxes and lines and we can use various types of bracketing or indenting for the text form of tree structures.

In ALA, we are setting up the top layer so we can do composition that

  • Composes high-level elements that you create.

  • Allows use of many programming paradigms, and allows new ones that you can create.

  • Uses the same syntax for all composition relationships.

  • Allows linear, tree or network structures.

ALA can therefore be described as 'generalised create and compose'.

Generally, compositions are 'instances of abstractions' 'connected' together in a specific way. This can be thought of as a graph. A graph is most easily imagined as a box and line drawing. In the common examples of composition that we mentioned above, sequential execution flow, monads, UI layout etc, the composition using text readily supports graphs that are linear or small tree structures. Arbitrary graph structures can usually be done by adding connections in a special way - by naming some of the nodes and then connecting by their identifier. However this method is somewhat inconvenient and unreadable in text form. ALA therefore can use diagrams to allow compositions to be arbitrary graphs. We are going to need that in any non-trivial application.

To support generalized composition, ALA dedicates the top layer to the composition itself, a layer below it for the abstractions from which instances can be composed, and a layer below that for the different types of composition paradigms. The middle layer is usually plain old classes and the bottom layer is usually plain old interfaces, although there are many other ways to do ALA.

2.7. Layers instead of decomposition

In the previous section, we discussed how ALA uses 'Create and Compose' rather than 'Divide and Conquer'.

In this section, let’s have a look at the other side of that coin and explore what is wrong with decomposition.

Consider this phrase, often found near definitions of software architecture.

"decomposition of a system into elements and their relations".

Notice the word 'their', which I have italicised to emphasis that the relations are inferred to be between the said elements. It implies that the elements know something about each other. It implies they collaborate. This is a really bad meme. ALA is the antithesis of this meme.

Here is how to reword the meme for ALA:

"abstractions and composition of their instances".

Strictly speaking the wording of the decomposition meme does not preclude this meaning, but it is at best misleading. This seemingly subtle shift causes a huge change in the structure, as described in the two contrasting diagrams below:

2.7.1. ALA structure is not this

An architecture based on decomposition into elements and their relations looks something like this:

Decomposition into elements and their relations
Figure 6. Decomposition into elements and their relations

The figure shows five modules (or components) and their relations (as interactions). Study almost any piece of software, and this is what you will find (even if it adheres to the so-called layering pattern).

The structure generally can be viewed as 'clumping'. Like galaxies, certain areas have higher cohesion, and so go inside boxes. Other areas are more loosely coupled, and so are represented by lines between the boxes. The difference between high cohesion and loose coupling is only quantitative.

Software health in this type of architecture is effectively management of the resulting coupling between the cohesive clumps. Allocate code to boxes in such a way as to minimize coupling. This coupling management has two conflicting forces. One is the need to have interactions to make the modules work as a system. The other is to minimize the interactions to keep the modules as loosely coupled as possible. As maintenance proceeds, the number of interactions inevitably increases, and the interfaces get fatter. The clumping is gradually eroded. Any so-called encapsulations become more or less transparent.

Various architectural styles are aimed at managing this conflict. Most notably:

  • layering pattern

  • MVC pattern

  • Dependency rules

    1. Avoid circular dependencies.

    2. Avoid high fan-in and high fan-out on a single module.

    3. Avoid dependencies on unstable interfaces.

Note that none of this 'dependency management' actually avoids circular coupling. To some extent there will always be 'implicit coupling' in both directions between modules of a decomposed system. This is because the modules are the opposite of abstractions - specific 'parts' designed to interact and therefore collaborate. For example, a function of a decomposed system will tend to be written to do what its caller requires even if there is no explicit compile-time dependency on its caller. So circular coupling may be avoided at compile-time, but will still be present at design-time. That is why in the diagram above, couplings are drawn from the insides of each of the modules in both directions. This indicates that the code inside has some inherent design-time collaborative coupling. To the compiler or a dependency graphing tool, the lines may appear to be in one direction, and therefore 'layered', but it is not telling you the whole story of the coupling.

2.7.2. ALA structure looks like this

When you use abstractions instead of modules, the qualitative difference is that there are no interactions, no collaboration, no coupling between your abstractions at all:

Abstractions do not interact
Figure 7. Abstraction do not interact

The word 'modules' has been changed to the word 'abstractions'. All the dependencies are gone. And with them all their problems, and all their management. The implicit coupling that we talked about earlier is also gone. It no longer has a 'clumping' structure. Loose coupling is replaced with zero coupling.

The obvious question now is how can the system work? Where do all the interactions between elements that we had before go? The answer is they become normal code, completely contained inside one additional abstraction:

Abstractions and composition of their instances
Figure 8. Abstractions and composition of their instances

Interactions or collaboration should never be implemented in your abstractions. That just destroys them as abstractions. They are implemented inside another new abstraction at a different, more specific, abstraction level. Being contained inside that new abstraction the interactions are not coupling. They are just a composition of instances. They are cohesively together in one place where they belong because they are the specific information about the specific application. That small amount of code has all the knowledge about the specific application. The abstractions no longer know about the specific application.

ALA overturns the conventional meme about decomposition into elements and their relations. It is unnecessary to write software that way. The only relationship that remains is the 'use of an abstraction'. This is, of course, a dependency but it is a good dependency. We will discuss from the point of view of good and bad dependencies in a later section. For now, dependencies are good if we want more of them. The more of them the better. For example if you have a library function or class, say squareroot, the more it is used the better, because the more useful the library function must have been. This type of dependency, the 'use of an abstraction', is the only one you need to build a system.

Software engineering should not be about managing coupling.

It should be about inventing abstractions.

2.8. DSLs

ALA’s succinct expression of requirements in the top layer sounds similar to the way requirements might be represented in a DSL (Domain Specific Language). Under the broader definition of a DSL, ALA’s domain abstractions layer is a DSL. But ALA is also different from a DSL. ALA, as its name suggests, is fundamentally about layering of abstractions. It layers them in a small number of layers, according to their abstraction level. When you do this, the top two layers emerge as the specific application and the domain. Therefore ALA happens to converge on the same solution as DSLs for these top two layers.

In coming to this same solution from a different direction it has a different emphasis than a DSL has. It does not pursue the idea of an external DSL (new syntax), nor even the syntactic elegance of DSLs. It doesn’t move application development away from the developer as DSLs are often designed to do. You don’t get a different language such as XAML that a UI specialist designer can learn. These things may still be desirable qualities and ALA does not preclude them, it is just not what ALA is about. ALA says that just getting the abstraction layering right is enough to deal with complexity and maintainability.

As a DSL, in ALA you usually just wire together plain old objects, or functions in a way that is confined by a grammar. The classes (the domain abstractions) and the 3rd layer interfaces collectively form the DSL. The grammar is defined by which classes use which interfaces. This sets the rules for composition.

By the way, ALA also emerges other already discovered architectural styles such as CBE (Component Based Engineering), and composability. These are discussed later.

2.9. SMITA (Structure Missing in the Action)

The problem in most large code bases is that the system structure, the in-the-large structure, is not explicit. It is distributed inside the modules themselves. If there is any collaboration between modules, it is implicitly hidden inside them. Finding this structure, even for a single user story can be time consuming. I have often spent a whole day doing that, doing countless all-files searches, just to change one line of code. Many developers I have spoken to can identify with this experience.

It can get a lot worse as the system gets larger. In a bizarre twist, the more loosely coupled you make the elements, the harder it gets to trace a user story through them (because of the indirections). Some people conclude that loose coupling and being able to trace through a user-story are naturally in conflict.

I call this situation SMITA (Structure Missing in the Action). This hidden structure is sometimes partially brought out as a model, a sort of high-level documentation of the internal structure. But such models are a secondary source of truth.

ALA completely eliminates this problem and this conflict. The structure is explicitly coded in one place, without any indirections. Yet the abstractions are zero-coupled.

2.10. Diagrams vs text

In ALA we will often use a diagram instead of text for the source code in the application (top) layer.

Text is effective only when the relationships between instances of abstractions (words in the text) is a linear sequence or a relatively shallow tree (represented by indenting). If the relationships are an arbitrary graph or a deep tree, diagrams are far more effective. Becasue of this, part of what ALA is about is easily supporting programming with diagrams (sometimes called models, but I will avoid this ambiguous term). ALA diagrams show everything in an applciation, UI, event flow, dataflows, state machines, etc.

If a diagram is used, it is the 'source'. A code form of the diagram is generated from it for execution.

We will delve into greater detail on why our brains work better with diagrams, and graphing tools to support diagrams in chapter three.

2.11. Real world metaphors

2.11.1. Atoms and molecules

Here are two atom abstractions: Oxygen atom Hydrogen atom

Instances can be composed to make a molecule: Water molecule

If water was implemented in the same way we typically write software, there would be no water molecule per se; the oxygen atom would be modified to instantiate hydrogen atoms and interact with them. Even if dependency injection is used to avoid the instantiating, it is still unlikely that a water abstraction would be invented to do that, and there would still be the problem of the oxygen module being modified to interact with hydrogen’s specific interface. Either way, the oxygen module ends up with some implicit knowledge of hydrogen. And hydrogen probably ends up with some implicit knowledge of oxygen in providing what it needs.

This implicit knowledge is represented by the following diagram. The relationship is shown coming from the inner parts of the modules to represent implicit knowledge of each other.

diagram o h.png

While oxygen and hydrogen are modules, they are not abstractions because oxygen is implicitly tied to hydrogen and vice-versa. They can’t be used as building blocks for any other molecules.

To keep oxygen as abstract as it is in the real world, an interface must be conceived that is even more abstract than oxygen or hydrogen. In the molecule world this is called a polar bond.

The corresponding software would look like this:


The water molecule has a "uses instances of" relationship with the two atoms, and the atoms have a "uses instance of" relationship with the even more abstract polar bond. Polar bond is an example of what we call an 'abstract interaction'.

2.11.2. Lego

The second real world metaphor is Lego. Shown in the image below is the same three layers we had above for molecules, atoms and bonds.


The domain abstractions are the various lego pieces, instances of which can be assembled together to make things. Lego pieces themselves have instances of an abstract interface, which is the stud and tube. There is a second abstract interface, the axle and hole. We also call the abstract interface the 'execution model' and here with the lego metaphor we start to see why it can be thought of in this way - when the model runs, stud and tube interface executes the holding of the model together and the axle and hole interface executes turning.

2.11.3. Electronic schematic

The third real world metaphor comes from electronics. The abstractions are electronic parts, instances of which can be composed as a schematic diagram:


In this domain, the abstract interfaces (execution models) are both digital signals and analog voltage levels.

2.11.4. A clock

The forth and final real world metaphor is a clock. In this diagram, we show the process of composition of abstractions to make a new abstraction. The process is a circle because instances of the new abstraction can themselves be used to make still more specific abstractions. Each time around the circle adds one layer to the abstraction layering.


Lets go round the circle once. We start with abstract parts such as cog wheels and hands. Instances of these have abstract interfaces that allow them to interact at run-time, such as spinning on axles and meshing teeth. The next step is to instantiate some of these abstractions and configure them. For example, configure the size and number of teeth of the cog wheels. Next comes the composition step, where they are assembled. Finally we have a new abstraction, the clock. Instances of them can be used to compose other things such as a scheduling things during your day, but that is a whole different abstraction.

There are many other instances of this pattern in the real world, and in nature. In fact almost everything is composed in this way.

2.12. Example project - Ten-pin bowling

The ten-pin bowling problem is a common coding kata. Usually the problem presented is just to return the total score, but in this example we will tackle the more complicated problem of keeping the score required for a real scorecard, which means we need to keep all the individual frame ball scores. We can afford to do this even for a pedagogical sized example because ALA can provide a simple enough solution.

bowling scorecard2.png

The ALA method starts by "describing the requirements in terms of abstractions that you invent". When we start describing the requirements of ten-pin bowling, we immediately find that "a game consists of multiple frames", and a "frame consists of multiple balls". Let’s invent an abstraction to express that. Let’s call it a "Frame". Instances of Frame can be wired together by a "ConsistsOf" relationship. So let’s invent an abstract interface to represent that, and call it 'IConsistsOf'.

Here is the diagram of what we have so far.

diagram bowling 1.png

This is the first time we are using a diagram for an ALA application, so lets go through the conventions used.

The name in the top of the boxes is the abstraction name. The name just beneath that is the name of an instance of the abstraction. For the bowling application above, we are using two instances of the Frame abstraction, one called "game" and one called "frame". Below the abstraction name and instance name go any configuration information of the instance.

The Frame abstraction is configured with a lambda function to tell it when it is finished. The Frame abstraction works like this - when its last child is complete it will create a new one. It will stop doing that when the lambda expression is true. It will tell its parent it is complete when both the lambda expression is true and its last child Frame is complete.

The end of the chain is terminated with a leaf abstraction that also implements the 'IConsistsof' interface called 'SinglePlay'. It represents the most indivisible play of a game, which in bowling is one throw. Its job is to record the number of pins downed.

The concept in the Frame abstraction is that at run-time it will form a composite pattern. As each down-stream child frame completes, a Frame will copy it to start a new one. This will form a tree structure. The "game" instance will end up with 10 "frames", and each frame instance will end up with 1, 2 or 3 SinglePlays.

Note, in reference to the ALA layers, this diagram sits entirely in the top layer, the Application layer. The boxes are instances of abstractions that come from the second layer, the Domain Abstractions layer. The arrows are instances of the programming paradigm, 'InConsistsOf', which comes from the third layer, the ProgrammingParadigms layer.

This diagram will score 10 frames of ten-pin bowling but does not yet handle strikes and spares. So let’s do some 'maintenance' of our application. Because the application so far consists of simple abstractions, which are inherently stable, maintenance should be possible without changing these abstractions.

The way a ten-pin bowling scorecard works, bonuses are scored in a different way for the first 9 frames than for the last frame. In the first nine frames, the bonus ball scores come from following frames, and just appear added to the frame’s total. They do no appear as explicit throws. In the last frame, they are shown as explicit throws on the scorecard. That is why there are up to 3 throws in that last frame.

To handle the different last frame, we just need to modify the completion lambda expression to this.

frameNum<9 && (balls==2 || pins==10) // completion condition for frames 1..9
|| (balls==2 && pins<10 || balls==3) // completion condition for frame 10

To handle bonuses for the first 9 frames, we introduce a new abstraction. Let’s call it Bonuses. Although we are inventing it first for the game of ten-pin bowling, it is important to think of it as a general purpose, potentially reusable abstraction.

What the Bonus abstraction does is, after its child frame completes, it continues adding plays to the score until its own lambda function returns true.

The completed ten-pin bowling scorer is this:

diagram bowling 2.png

Note that the "game" instance (the left box of the diagram) implements IConsistsOf. This is where the outside world interfaces to this scoring engine. During a game, the number of pins knocked down by each throw is sent to this IConsistsOf interface. To get the score out, we would call a GetScore method in this interface. The hard architectural work is done. We have invented abstractions to make it easy to express requirements. We have a diagram that describes the requirements. And the diagram is executable. All we have to do is put some implementation code inside those abstractions and the application will actually execute.

First lets turn the diagram into equivalent code. At the moment, there are no automated tools for converting such diagrams to code. But it is a simple matter to do it manually. We get the code below:

private IConsistsOf game = new Frame("game")
    .setIsFrameCompleteLambda((gameNumber, frames, score) => frames==10)
    .WireTo(new Bonus("bonus")
        .setIsBonusesCompleteLambda((plays, score) => score<10 || plays==3)
        .WireTo(new Frame("frame")
            .setIsFrameCompleteLambda((frameNumber, balls, pins) => frameNumber<9 && (balls==2 || pins[0]==10) || (balls==2 && pins[0]<10 || balls == 3))
            .WireTo(new SinglePlay("SinglePlay")

All we have done is use the 'new' keyword for every box in the diagram. We have made the constructor take the instance name as a string. (This name is not used except to identify instances during debugging.) We use a method called "WireTo" for every line in the diagram. More on that in a minute. And we pass any optional configuration into the instances using setter methods. The WireTo method and the configuration setter methods all return the 'this' pointer, which allows us to write this code in fluent style. If you are not familiar with fluent style it is just making methods return the this reference, or another object, so that you can chain together method calls using dot operators.

Not all ALA applications will be put together using the method in the previous paragraph, but I have found it a fairly good way to do it for most of them, so we will see this same method used for other example projects to come.

So far, this has been a fairly top-down, waterfall-like approach. We have something that describes all the details of the requirements, but we haven’t considered implementation at all. Past experience tells us this may lead us into dangerous territory. Will the devil be in the details? Will the design have to change once we start implementing the abstractions? The first few times I did this, I was unsure. I was not even sure it could actually be made to work. The reason it does work is because of the way we have handled details. Firstly all details from requirements are in the diagram. The diagram is not an overview of the structure. It is the actual application. All other details, implementation details, are inside abstractions, where they are hidden even at design-time. Being inside abstractions isolates them from affecting anything else. So, it should now be a simple matter of writing classes for those three abstractions and the whole thing will come to life. Implementing the three abstractions turns out to be straightforward.

First, design some methods for the IConsistOf interface that we think we will need to make the execution model work:

    public interface IConsistsOf
        void Ball(int score);
        bool IsComplete();
        int GetScore();
        int GetnPlays();
        IConsistsOf GetCopy(int frameNumber);
        List<IConsistsOf> GetSubFrames();

The first four methods are fairly obvious. The Ball method receives the score on a play. The Complete, GetScore and GetnPlays methods return the state of the sub-part of the game. The GetCopy method asks the object to return a copy of itself (prototype pattern). When a child frame completes, we will call this to get another one. The GetSubFrames method is there to allow getting the scores from all the individual parts of the game as required.

The SinglePlay and Bonus abstractions are very straightforward.

So let’s code the Frame abstraction. Firstly, Frame both implements and accepts IConsistsOf. A field is needed to accept an IConsistsOf. The WireTo method will set this field:

// Frame.cs
private IConsistsOf downstream;

Frame has one 'state' variable which is the list of subframes. This is the composite pattern we referred to earlier, and what ends up forming the tree.

// Frame.cs

private List<IConsistsOf> subFrames;
private readonly Func<int, int, int, bool> isFrameComplete;
private readonly int frameNumber = 0;

The second variable is the lambda expression that is a configuration passed to us by the application. It would be readonly (immutable) except that I wanted to use a setter method to pass it in, not the constructor, to indicate it is optional.

The third variable is the frameNumber, also immutable. It allows frame objects to know which child they are to their parent - e.g. 1st frame, 2nd frame etc. This value is passed to the lambda expression in case it wants to use it. For example, the lambda expression for a bowling frame needs to know if it is the last frame.

The methods of the IConsistsOf interface are now straightforward to write. Lets go over a few of them to get the idea. Here is the most complicated of them, the Ball method:

public void Ball(int player, int score)
    // 1. Check if our frame is complete, and do nothing
    // 2. See if our last subframe is complete, if so, start a new subframe
    // 3. Pass the ball score to all subframes

    if (IsComplete()) return;

    if (subFrames.Count==0 || subFrames.Last().IsComplete())

    foreach (IConsistsOf s in subFrames)
        s.Ball(player, score);

It looks to see if the last child frame has completed, and if so starts a new child frame. Then it just passes on the ball score to all the child objects. Any that have completed will ignore it.

The IsComplete method checks two things: 1) that the last child object is complete and 2) that the lambda expression says we are complete:

private bool IsComplete()
    if (subFrames.Count == 0) return false; // no plays yet
    return (subFrames.Last().IsComplete()) &&
        (isLambdaComplete == null ||
         isLambdaComplete(frameNumber, GetnPlays(), GetScore()));

GetScore simply gets the sum of the scores of all the child objects:

private int GetScore()
    return subFrames.Select(sf => sf.GetScore()).Sum();

The GetCopy method must make a copy of ourself. This is where the prototype pattern is used. This involves making a copy of our child as well. We will be given a new frameNumber by our parent.

IConsistsOf GetCopy(int frameNumber)
    var gf = new Frame(frameNumber);
    gf.objectName = this.objectName;
    gf.subFrames = new List<IConsistsOf>();
    gf.downstream = downstream.GetCopy(0);
    gf.isLambdaComplete = this.isLambdaComplete;
    return gf as IConsistsOf;

The few remaining methods of the IConsistOf interface are trivial. The implementation of IConsistsOf for the other two abstractions, SinglePlay and Bonuses, is similarly straightforward. Note that whereas Frame uses the composite pattern, Bonuses uses the decorator pattern. It implements and requires the IConsistsOf interface. The SinglePlay abstraction, being a leaf abstraction, only implements the IConsistsOf interface.

One method we haven’t discussed is the wireTo method that we used extensively in the application code to wire together instances of our domain abstractions. The wireTo method for Frame is shown below:

public Frame WireTo(IConsistsOf c)
    downstream = c;
    return this;

This method does not need to be implemented in every domain abstraction. I use an extension method for WireTo. The WireTo extension method uses reflection to find the local variable to assign to.

The WireTo method will turn out to be useful in many ALA designs. Remember in ALA we "express requirements by composing instances of abstractions". If the 'instances' of 'abstractions' are implemented as 'objects' of 'classes', then we will use the wireTo method. If the 'instances' of 'abstractions' are 'invocations' of 'functions', as we did in the example project in Chapter One, we wont use WireTo obviously. In the coffeemaker example to come, 'instances' of 'abstractions' are 'references' to 'modules' because a given application would only have one of each abstraction.

The wireTo method returns 'this', which is what allows the fluent coding style used in the application code. The configuration setter methods also return the this reference so that they too can be used in the fluent style.

Here is the full code for the Frame abstraction (with comments removed as we just explained everything above):

// Frame.c
using System;
using System.Collections.Generic;
using System.Linq;
using GameScoring.ProgrammingParadigms;
using System.Text;

namespace GameScoring.DomainAbstractions

    public class Frame : IConsistsOf
        private Func<int, int, int[], bool> isLambdaComplete;
        private readonly int frameNumber = 0;
        private IConsistsOf downstream;
        private string objectName;
        private List<IConsistsOf> subFrames = new List<IConsistsOf>();

        public Frame(string name)
            objectName = name;

        public Frame(int frameNumber)
            this.frameNumber = frameNumber;

        // Configuration setters follow.

        public Frame setIsFrameCompleteLambda(Func<int, int, int[], bool> lambda)
            isLambdaComplete = lambda;
            return this;

        // Methods to implement the IConsistsOf interface follow

        public void Ball(int player, int score)
            if (IsComplete()) return;

            if (subFrames.Count==0 || subFrames.Last().IsComplete())

            foreach (IConsistsOf s in subFrames)
                s.Ball(player, score);

        public bool IsComplete()
            if (subFrames.Count == 0) return false;
            return (subFrames.Last().IsComplete()) &&
                (isLambdaComplete == null ||
                 isLambdaComplete(frameNumber, GetnPlays(), GetScore()));

        public int GetnPlays()
            return subFrames.Count();

        public int[] GetScore()
            return subFrames.Select(sf => sf.GetScore()).Sum();

        List<IConsistsOf> IConsistsOf.GetSubFrames()
            return subFrames;

        IConsistsOf IConsistsOf.GetCopy(int frameNumber)
            var gf = new Frame(frameNumber);
            gf.objectName = this.objectName;
            gf.subFrames = new List<IConsistsOf>();
            gf.downstream = downstream.GetCopy(0);
            gf.isLambdaComplete = this.isLambdaComplete;
            return gf as IConsistsOf;


The full source code for the bowling application can be viewed or downloaded from here: GameScoring code

2.13. Example project - Tennis

Now let’s modify the bowling application to score tennis. If the bowling game hadn’t been implemented using ALA, you probably wouldn’t contemplate doing this. But ALA excels for maintainability, and I want to show that off by changing Bowling to Tennis. The Frame and IConsistsOf abstractions look like they could be pretty handy for Tennis. A match consists of sets, which consists of games, which consists of SinglePlays.

We will need to make a small generalization to the Frame abstraction first. This will allow it to keep score for two players. We just change the type of the score from int to int[]. The Ball method will be generalized to take a player parameter to indicate which player won a play. A generalization of an abstraction to make it more reusable is a common operation in ALA.

The only other thing we will need to do is invent a new abstraction to convert a score such as 6,4 into a score like 1,0, because, for example, the winner of a game takes one point into the set score. This new abstraction is called WinnerTakesPoint (WTP in the diagram).

Here is the tennis scoring game:


The diagram expresses all the details of the requirements of tennis except the tiebreak.

Here is the diagram’s corresponding code:

private IConsistsOf match = new Frame()
    .setIsFrameCompleteLambda((matchNumber, nSets, score) => score.Max()==3)
    .WireTo(new WinnerTakesPoint()
        .WireTo(new Frame()
            .setIsFrameCompleteLambda((setNumber, nGames, score) => score.Max()>=6 && Math.Abs(score[0]-score[1])>=2)
            .WireTo(new WinnerTakesPoint()
                .WireTo(new Frame()
                    .setIsFrameCompleteLambda((gameNumber, nBalls, score) => score.Max()>=4 && Math.Abs(score[0]-score[1])>=2)
                    .WireTo(new SinglePlay()))))));

The new WinnerTakesPoint abstraction is easy to write. It is a decorator that implements and requires the IConsistsOf interface. Most methods pass through except the GetScore, which returns 0,0 until the down-stream object completes, then it returns either 1,0 or 0,1 depending on which player has the higher score.

And just like that, the tennis application will now execute. The frame abstraction we invented for bowling is already done.

2.13.1. Add tiebreak

Now let’s switch our attention back to another example of maintenance. Let’s add the tiebreak feature. Another instance of Frame will score the tiebreak quite nicely. However we will need an abstraction that can switch us from playing the set to the tie break. Let’s call it Switch, and give it a lambda function to configure it with when to switch from one subframe tree to another. Switch simply returns the sum of scores of its two subtrees. Here then is the full description of the rules of tennis:


And here is the code version of that diagram. This application passes an exhaustive set of tests for the scoring of tennis.

private IConsistsOf match = new Frame("match")
    .setIsFrameCompleteLambda((matchNumber, nSets, score) => score.Max()==3)
    .WireTo(new WinnerTakesPoint("winnerOfSet")
        .WireTo(new Switch("switch")
            .setSwitchLambda((setNumber, nGames, score) => (setNumber<4 && score[0]==6 && score[1]==6))
            .WireTo(new Frame("set")
                .setIsFrameCompleteLambda((setNumber, nGames, score) => score.Max()>=6 && Math.Abs(score[0]-score[1])>=2)
                .WireTo(new WinnerTakesPoint("winnerOfGame")
                    .WireTo(new Frame("game")
                        .setIsFrameCompleteLambda((gameNumber, nBalls, score) => score.Max()>=4 && Math.Abs(score[0]-score[1])>=2)
                        .WireTo(new SinglePlay("singlePlayGame"))
            .WireTo(new WinnerTakesPoint("winnerOfTieBreak")
                .WireTo(new Frame("tiebreak")
                    .setIsFrameCompleteLambda((setNumber, nBalls, score) => score.Max()==7)
                    .WireTo(new SinglePlay("singlePlayTiebreak"))

And just like that we have a full featured executable tennis scoring engine.

2.13.2. Final notes

Notice that I have added string names to the instances of Frame and other objects. This is not required to make the program function, but generally is a good habit to get into in ALA. It is because in ALA we typically use multiple instances of abstractions in different parts of the program. The names give us a way of identifying the different instances during any debugging. Using them I can Console.Writeline debugging information along with the object’s name.

Around 8 lines of code express the rules of ten-pin bowling and around 15 lines of code express the rules of tennis. That sounds about right for the inherent complexity of the two games. The two rule descriptions actually execute and pass a large battery of tests.

The domain abstractions are zero-coupled with one another, and are each straightforward to write by just implementing the methods of the IConsistOf interface according to what the abstraction does. The abstractions are simple and stable. So no part of the program is more complex than its own local part.

The domain abstractions are reusable in the domain of game scoring. And, my experience was that as the details inside the abstractions were implemented, the application design didn’t have to change. Here is a link to the code on Github: GameScoring code

Why two example applications? The reason for doing two applications in this example is two-fold.

  1. To show the decreasing maintenance effort. The Tennis game was done easily because it reused domain building blocks we had already created for bowling.

  2. To emphasis where all the details of the requirements end up. The only difference between the bowling and tennis applications is the two diagrams, which are translated into two code files: bowling.cs and tennis.cs of 8 lines and 15 lines respectively. These two files completely express the detailed requirements of their respective games. No other source files have any knowledge of these specific games. Furthermore, Bowling.cs and Tennis.cs do not do anything other than express requirements. All implementation to actually make it execute is hidden in domain abstractions and programming paradigm abstractions.

3. Chapter three - Why the structure works

In the previous chapter we described what the structure, the anatomy, of ALA looks like as if we were dissecting a dead body. We see where things are but we don’t yet understand why they are there. In this chapter we explain why that structure works. Why does this way of organising code result in software that meets those non-functional requirements we listed in Chapter one?

3.1. Good versus bad dependencies

We can distinguish two types of dependencies. One is run-time dependencies. These are dependencies in the code that are there because one module will need another module to be present at run-time for the system to work. The other is design-time dependencies. These are knowledge you must have to even understand a given piece of code. I will often refer to this type as a "knowledge dependency" or "use of an abstraction". It is also sometimes called "semantic coupling".

Run-time dependencies are bad.

Design-time dependencies are good.

A simple example of a run-time dependency is a module that calculates the average rainfall then calls a display module to display the result. The Display module needs to be present at run-time. But to understand the code that calculates the average rainfall requires no knowledge about displays, nor even where the result will be sent. The dependency is only there to make the system work at run-time.

A simple example of a design-time dependency is some code that calculates the standard deviation. To understand the code needs knowledge of squareroot. This is a design-time, or knowledge dependency. Any standard deviation code should use and have a dependency on squareroot.

We find both types of dependencies in conventional code. A typical program is chock full of the run-time dependencies. But whether a knowledge dependency or a run-time dependency, they all just look like a function call or a 'new' keyword. We generally don’t distinguish between them. In fact we are not normally taught to tell the difference. They are all just called dependencies. We lump them together when we talk about dependency management, loose coupling, layering, fan-in & fan-out, or circular dependencies. Dependency graphing tools just show them both.

These two different types of dependencies are not just good and bad. They are really good and really bad. So it’s well worth the effort to learn how to tell the difference. What’s more it’s entirely possible to build a system using only the good dependencies.

A knowledge dependency is good because it makes use of an abstraction (which by implication will be more abstract than itself). The more knowledge dependencies you have, the more you are using those reusable abstract building blocks. The more the better. If something is used a lot, it follows that it is useful, and a good abstraction. And if something is a good abstraction, it is stable and it is easy to learn what it is.

Run-time dependencies are bad because they completely destroy abstractions. They are bad because they cause explicit and implicit coupling. And they are bad because they obscure the structure.

In ALA we eliminate all run-time dependencies.

Consider the diagram below:

dependency diagram.png

There are four run-time dependencies.

Now consider this diagram.

dependency diagram 1.png

There are five knowledge dependencies (the top layer uses five abstractions in the second layer), but no run-time dependencies (because the connections between the instances are completely inside another abstraction).

The letters used in the top layer represent instances. (In UML they would be underlined.) You never draw arrows for knowledge dependencies - only ever refer to the abstraction by its name. (Just as you would never draw an arrow to a box representing the squareroot function - you would just use SQRT by its name.)

In common programming languages, the run-time dependencies in the first diagram and the knowledge dependencies in the second diagram are both syntactically written in the same form, either new A() or just a function call, A(). The only difference is in where those function calls or new keywords are. This simple change makes a huge difference in the quality of the code.

3.1.1. Comparison of good versus bad dependencies.

Table 1. Comparison of two approaches
Run-time dependencies version Knowledge dependencies version

There is no diagram of the arrangement between A, B, C, D, E, or if there is, it is likely a high level overview, lacking in detail, and a second source of truth that gets out of date.

There is a diagram. It is the one source of truth. It includes all details about the specific application.

Knowledge about the specific application is spread through all modules.

Knowledge about the specific application is only in one place. The abstractions no nothing of each other or the specific application.

The class or function names A, B, D and E will relate to what they do (which is fine). For example, they may be the specific hardware chips used in the case of drivers. The calling module must know these names, creating a fixed arrangement between the modules. The modules are loosely coupled.

No peer abstractions refer to these names. There is no fixed arrangement between abstractions. Abstractions are zero coupled. The code that knows that a particular hardware chip is used in this application is where it belongs, in the application code.

Since there is a fixed arrangement, responsibilities can be blurred. For example, it may be unclear whether to add something to B or C. Or C can make assumptions about details in B, causing collaborative coupling.

With no relations between abstractions, responsibilities are clear. Something to be added clearly belongs in one or other of the abstractions, or in a new abstraction. C cannot make any assumptions about some details of B. It cannot have collaborative coupling with B

Although there is no explicit dependency from, for example, B to C, the fixed arrangement is likely, over time, to make B implicitly collaborate with C (do what C requires), resulting in implicit coupling.

No implicit coupling can develop over time because there is no relationship between them. B cannot collaborate with C (do what C specifically requires), or have implicit coupling with C.

The arrangement between A, B, C, D and E is not obvious in the code. It is buried inside of B, C and D.

The arrangement between instances of A, B, C, D and E is explicitly coded in one place.

Only A and E can potentially be abstractions.

All of A, B, C, D and E are abstractions.

Arbitrarily, only the two ends of the data flow chain can be reused independently .

All of A, B, C, D and E are independently reusable.

Difficult to insert another module between, say, B and C.

Easy to insert a new instance of some operator between B and C, etc.

If the observer pattern is used (in the mistaken belief that it reduces the coupling), it only mirrors the same problems. For example B would now have a dependency on C when it registers. But because it adds indirection, the observer pattern makes the program even harder to understand.

If the observer pattern is used (as the means to implement the wiring between the instances), the receivers do not do the registering, the application does (not strictly the observer pattern). The abstractions themselves don’t get more difficult to understand because, being abstractions, they only have knowledge as far as their interfaces anyway. The application does not get harder to understand either. The arrangement of the instances is still explicitly and in one place.

If dependency injection is used with automatic wiring, the arrangement is still somewhat fixed, but is now even more obscure. All classes can still be collaborating with one another. A smell that this is happening is that over time the interfaces, IA, IB, ID and IE change as the requirements of the system change.

If dependency injection is used, the application does the wiring explicitly. It is the only place that should know who will talk to whom at run-time for this specific application. There are no specific interfaces between pairs of modules to change over time, because they all just use a stable abstract interface.

Each module has its own interface. But they are all doing essentially the same thing, getting data.

Uses a single more abstract interface called IDataflow.

The arrangement between the modules cannot easily be changed, both because the wiring code is buried inside the modules and because the interfaces are essentially specific to pairs of modules.

The composition can very easily be changed. Instances of abstractions can be re-wired in any combination.

During code creation, run-time dependencies are easily introduced, and never seem too terrible at the time. But when they accumulate to hundreds or even thousands of them, as they do in most typical applications, that’s when the system, as described on the left side of the table, just appears as a monolith.

3.1.2. Notes

The application level module either moves the data between the instances of A, B, C, D, E itself, or wires them together using the even more abstract interfaces, such as the one shown called IDataflow. These abstract interfaces are not specific to any of A, B, C, D or E. This is the abstract interactions pattern. The interface design is such that there could potentially be many abstractions that implement it or accept it, or both.

If dependency injection is used, I prefer not to use XML for the explicit wiring. XML is not very readable, and it only handles tree structures. If you must use text, use normal code. But there are situations where a diagram is the only readable way to go. I will go into these in a later section.

When you are comparing the left and right sides of the table above, you may be wondering, where did the free lunch come from? Where did the runtime dependencies go? Is this some kind of magic? How can the program work without them? Or haven’t I just moved them somewhere else? No there are no tricks. The answer is that we have been taught to do programming in a very bad way. The knowledge that A will talk to B, B to C etc is there, but it is now in ordinary code, not as dependencies between anything. They are no longer dependencies because that code is fully contained in one place, inside a single new abstraction. Doing this makes a huge difference to any code. If you haven’t yet got your head around this, keep reading because we will present the same insight in other ways.

The only dependencies we have used on the right side of the table are knowledge dependencies:

  1. The application should and must 'know' at design-time what abstractions it needs to compose to make a specific application.

  2. The domain abstractions should and must know what kind of abstract interfaces to use for its inputs and outputs.

3.1.3. No loose coupling

Since our conventional programs are typically full of coupling of all sorts, this constraint on the architecture will obviously change how we write programs significantly. But surprisingly, things quickly get easier, not harder with these constraints, a lot easier.

When we say no loose coupling, it means there is zero coupling. Zero coupling between the details contained inside any two abstractions. Abstractions are therefore free floating little independent programs. To understand any part of the code involves understanding only that part of the code.

To understand any part of the code should involve understanding only that part of the code..

3.1.4. Knowledge dependency layers

The one type of dependency allowed is when you use an abstraction.

The code inside an abstraction in a higher layer makes use of an abstraction from a lower layer.

We call it a knowledge dependency because to understand the code in the higher layer, you must know about the abstraction. You don’t have to know about the details inside the abstraction, you just need to know about the abstraction. This is the way the world works and the way our brains have evolved to make sense of it. And it’s the way we need to structure our programs.

When we write our programs using only knowledge dependencies, all the knowledge needed to understand a piece of code is explicit. It is right there in any function calls or new keywords. There is no knowledge needed from anywhere else, because there is no implicit coupling.

In ALA, knowledge dependencies form the layers. There are no run-time dependencies present, so that is why the ALA layers are significantly different from the layers you would normally find in a program trying to use the layering pattern.

The bottom layer is your general purpose programming language. You must know its abstractions such as if-else statements before you can understand any layer above. You can generally learn this once for a whole career.

You also need to know the next layer, which is at the abstraction level of programming paradigms. Examples are data-flow, state machines, database schemas, and UI trees. You would generally learn these as needed for different programming problems. A given domain will typically make use of several of them.

You also need to know the next layer, the domain layer where you have useful building blocks for solving problems in a specific domain. You would learn these when you start a new job.

Finally we come to the top layer. Its abstraction level is a single application. The abstraction is what the user sees - a tool that does a job by meeting a set of requirements. The abstraction level of the top layer is the details of the requirements.

To get the insight of ALA, you need to throw away any previous conceptions of layering you may have had as these will contain run-time dependencies. Think of run-time dependencies as just wiring in the top layer, tipped on its side.

3.1.5. Stability of dependencies.

Because all dependencies used in ALA are just 'uses of abstractions', dependencies are always toward the more stable. Even if the implenetation details inside an abstraction change, the abstraction itself stays stable, because an abstraction is just an idea. ALA therefore naturally conforms with the Stable Dependencies Principle (depend in the direction of stability) and the Stable Abstractions Principle (Entities should be as abstract as they are stable).

3.1.6. Dependency fan-in and fan-out

One of the guidelines sometimes used for dependencies is that a class that has high fan-in should not have high fan-out. The argument goes that a class with high fan-in should have high stability but one with high fan-out would have low stability (presumably because dependencies are thought to be things that cause changes to propagate). Knowledge dependencies, becasue they are on abstractions do not have this property. An abstraction is something that insulates its dependants from its internal details. In ALA, it is perfectly fine, in fact really really good if a class in the middle layer can indeed have both high fan-in and high fan-out. It simply means that it is both useful to its users in higher layers, and making use of even more abstract things in lower layers.

If you think about your programming language as the bottom layer (on which everything depends), every reusable class you write has both high fan-in and high fan-out. This meme that not having high fan-in and high fan-out for the same class does not apply to knowledge dependencies. And if you apply it to run-time dependencies, what the meme should say is zero fan-in and zero fan-out.

3.1.7. Circular dependencies

Of course in ALA, with only knowledge dependencies present in the system, and the abstraction layering being formed from them, you obviously cannot have circular knowledge dependencies. Nor would that even make sense. (Well actually it can make sense when we use knowledge recursion, in the same way that a mathematician might use recursion to define something. We will visit that in the last chapter.)

Since there are no run-time dependencies, the issue of circular dependencies with them does not arise at all.

But let’s just take look at the wiring that we create inside the application. (This is the wiring up of instances of abstractions to make a composition.) Can this wiring be circular? Yes it can, with the proviso that the execution model handles the execution of it in the way you intend. The execution model is a completely different story and is covered in the next section. In principle it is absolutely fine to have circular wiring. The electronics guys could not do without it - they call it feedback. And programs need it too. So why use a programming system that makes it awkward by constantly having to breaking the circle somewhere so there is no circle at at compile-time, but allowing the circle at run-time? ALA simply eliminates all that non-sense.

That concludes our discussion on why the ALA structure works from the point of view of good and bad dependencies.

3.2. Expression of requirements

We have previously discussed this aspect of ALA in terms of structure. It is the top layer. And we have used this aspect as the starting point in the method to develop the example projects. But why does the succinct description of requirements in that top layer work?

In conventional software development, we typically break a user story (or feature or functional requirement) up into different implementation responsibilities. For example, layers like GUI, business logic and database, or a pattern such as MVC (Model, View, Controller). But a user story or feature actually starts out as cohesive knowledge n the requirements. And its not a huge amount of cohesive knowledge, so it doesn’t need breaking up. Cohesive knowledge, knowledge that is by its nature highly coupled within itself should be kept together. All we need to do to keep it together is find a way to describe it so that it is executable. Don’t try to do any implementation, just get it described in a concise and complete form. If you can do that, the chances are you will be able to find a way to make it execute.

In ALA we want to find a way to express the user story with about the same level of expressiveness as when the user story was explained in English by the product owner. The language he used would have contained domain specific terms to enable him to explain it concisely. The same thing ought to be possible in the code. Anything that does not come directly from the requirements and starts to look like implementation detail is separated out. It comes out into abstractions. These abstractions typically contain knowledge of how user stories in general are implemented - how things can be displayed, how things can be saved, how data can be processed.

It turns out that abstractions that know how to implement useful things for expressing user stories are not only reusable for different user stories, but can be reusable for other applications. In other words, they are domain level abstractions. A typical user story might be composed of several of them, some to implement the user story’s UI, some to implement the user story’s business, and some to implement the user story’s saving of data. A user story instantiates the abstractions, configures them with the specific knowledge from the requirement, and then wires them together.

Most maintenance is probably changing, adding or fixing user stories or features. When those features are described entirely in one place instead of distributed through a lot of modules, you have a direct understanding of how the user story is represented by code, and therefore of how to change it or fix it.

Of course application code makes heavy use, in fact is entirely composed of, instances of domain abstractions. When fixing a bug, it quickly becomes clear if the application code itself doesn’t represent the requirements as intended, or one of the abstractions is not doing its job properly. Again the maintenance is easy.

3.3. Abstractions are design-time encapsulations

The maintainability quality attribute is often thought of in terms of ripple effects of change. I don’t think that is quite the right way to look at it. I have often had to make changes across a number of modules in poorly written code. The changes themselves just don’t take that long. The problem I see is the time you have to spend understanding enough of the system to know where to make those changes and to be confident they wont break anything. Even if the change is one line of code (which it often is), you may have had to understand a lot of code to figure that out. You have to understand all the code that is potentially coupled to that one line of code, which is essentially the complexity.

Unlike modules, abstractions encapsulate complexity at design-time. They give boundaries to how far you have to read code to understand code.

3.3.1. Abstractions and Instances

All software architectures should contain two concepts for its elements equivalent to abstractions and instances.

Abstractions are design-time elements. Instances are run-time elements. Object oriented programming has the two concepts in classes and objects. But many discussions on software architecture seem to combine them into one term, such as modules, components or layers. They may implicitly contain the separate concepts, as components may, but not having them explicit will inevitably lead to confusion.

The problem is their different dependencies. Dependencies between Instances are run-time dependencies. Dependencies between Abstractions are knowledge dependencies. If we don’t have separate terms for design-time and run-time elements, we will tend to implement run-time dependencies in the design-time elements, destroying them as abstractions.

Nearly all common layering schemes have this problem. A common example of the problem is associations between classes. The most important idea that OOP brought us, the idea of different design-time elements and run-time elements, has been ruined by associations. They encourage you to implement run-time dependencies between classes, an anti-pattern in ALA.

What we should be doing is representing the knowledge of run-time dependencies between instances inside another abstraction. By using the terms Abstraction and Instance, ALA honours the separation of run-time elements and design-time elements, and reminds us not to implement runtime dependences between abstractions. at design-time they are not dependencies at all.

3.4. Composition versus decomposition

Here we revisit the important idea introduced in section 2.6 to do with the pitfalls of thinking in terms of hierarchical decomposition.

In decomposition methods, we are taught to decomposes the system into smaller elements or components with relations between them. Then decompose those into still smaller ones. The process continues until the pieces are simple enough to understand and implement. Each decomposition is completely contained inside its parent component, so it forms a fractal or hierarchical structure.

Decomposition of the system into elements and their interactions.

The decomposition approach is often the de facto or informal method used by developers because it is encouraged by many architecture styles and patterns, for example components or MVC. It is the method used in ADD (Attribute Driven Design). Indeed some definitions of software architecture sound like this meme:

  • From Wikipedia quoting from Clements, Paul; Felix Bachmann; Len Bass; David Garlan; James Ivers; Reed Little; Paulo Merson; Robert Nord; Judith Stafford (2010:

    "Each structure comprises software elements, relations among them, and properties of both elements and relations."

    "Architecture is the fundamental organization of a system embodied in its components, their relationships to each other, and to the environment, and the principles guiding its design and evolution. [IEEE 1471]

    "Architecture also focuses on how the elements and components within a system interact with one another."
  • From an article on coupling by Martin Fowler

    "You can break a program into modules, but these modules will need to communicate in some way—otherwise, you’d just have multiple programs."
  • Loose coupling and high cohesion

Is loose coupling the best we can do? We are told that modules or components must collaborate in some way. It seems reasonable and even self-evident. So why is it completely wrong? It’s becasue we are thinking in terms of decomposition. There is another way - composition.

To be fair, some of the examples above are vague enough to be interpreted in either way. But all are misleading in that they are suggestive of the idea of decomposition.

To fix the problem, we should re-word the meme:

Expression of the requirements by composition of abstractions.

All four big words are changed and some are exact opposites. Indeed, the architecture that comes out of this method is "inside out" when compared to the decomposition method.

Let’s contrast two pseudo-structures: one that results from the decomposition approach and one that results from the composition approach.

3.4.1. Decomposition of the system into elements and their interactions

This diagram shows a decomposition structure. The outer box is the system. It shows decomposition into four elements, and then those in turn are decomposed into four elements each.

Decomposition structure
Figure 9. Decomposition Structure

The outer elements correctly only refer to the outer interface of the components - their package or namespace interface, facade, or aggregate root - however you want to think of it. Encapsulation is used at every level of the structure to hide implementation details.

The elements are labelled with numbers to emphasise that they are not good abstractions. Of course, in practice these elements have a name.

The next diagram shows the same structure but with parts relevant to a user story marked in red. This is the "their interactions" part of the "The decomposition of your system into elements and their interactions".

Decomposition structure
Figure 10. Tracing a User story

The diagram shows both decomposition relationships (boxes inside boxes) and interaction relationships (lines).

3.4.2. Expression of the requirements by composition of abstractions

This diagram shows a composition structure.

Decomposition structure
Figure 11. Composition Structure

Only 'composition' relationships are present. We have shown some of them as lines even though you wouldn’t normally draw them. For example, the one from c to C. In practice we wouldn’t normally draw a diagram like this at all - the abstractions would be just referred to by name. But here we are trying to make a combined diagram of the meta-architecture and the specific architecture. The meta-architecture is the three layers, and the knowledge dependencies that go from the higher layers to the lower layers. The specific architecture consists of the diagrams inside the user stories in the top layer, the specific composition of instances.

Note that although we use lines in the diagrams in the top layer, those lines do not represent dependencies.

3.4.3. Comparison of the two approaches

Table 2. Comparison of Decomposition vs Composition approaches
Decomposition Composition

The maintenance cost (effort per user story or effort per change) increases over time. This is because complexity is increasing. Changes will tend to have ripple effects, but that isn’t the biggest problem. Even if a change ends up being in one place, reasoning about the system to determine where that change should be can require reasoning across the system.

The maintenance cost reduces as the system grows. This is because as the domain abstractions mature, the user stories become less and less work to do - they simply compose, configure and wire together instances of existing domain abstractions.

Decomposition structure

Decomposition structure

Hierarchical (fractal) structure

Layered structure

Elements become less abstract as you zoom in. They are specific parts of specific parts.

Parts become more abstract as you go down the layers.

Hides details through encapsulation, which works at compile-time.

Hides details through abstraction, which works at design-time.

Inner parts are increasingly private. They are encapsulated in increasingly smaller scopes. These private parts still need to be known about at design-time to understand the system (unless they happen to also be good abstractions).

Lower layers are increasing public. Only the abstractions themselves are needed to understand the system.

Dependencies go in the direction from the outermost element to the innermost. This is the direction of less abstract and therefore less stable.

Dependencies go down the layers. This is the direction of more abstract, and therefore more stable.

Dependencies also exist between parts at the same hierarchical level

There are no dependencies between abstractions at the same layer.

Encourages the same element to be used for both abstraction and instance - often called a module or component.

Clearly has two distinct types of elements - abstractions and instances.

Elements are loosely coupled.

Abstractions are zero coupled.

Discourages reuse. 16 elements all different from each other.

Encourages reuse. Only 5 abstractions. 16 instances of those five abstractions.

SMITA - Structure missing in the action. If you are interested in a particular user story, you will typically have to trace it through multiple elements, multiple interfaces, and their interactions across the structure. An example of this is shown by the diagram with the red lines.

Eliminates this problem. The structure is explicit and in one place.

Coupling increases during maintenance. This is because details are not hidden inside abstractions, only encapsulations. Any of them can be needed at any time by an outer part of the structure. So as maintenance proceeds, more of them will need to be brought into the interfaces, increasing the coupling as time goes on.

Coupling remains at zero during maintenenace. Abstractions represent ideas, and ideas are relatively stable even during maintenance. All the dependencies are relatively unaffected. An operation called generalizing an abstraction is sometimes done. This increases the versatility, reuse and ubiquity of abstractions over time.

Complexity increases as the system gets larger.

The complexity stays constant as the system gets larger. Each abstraction is its own stand-alone program. If we choose an ideal granularity of say 200 lines of code, the complexity in any one part of the program is that of 200 lines of code.

3.4.4. Transforming a decomposition structure into a composition structure

  • The structure turns inside out. Abstractions are found in the inner-most encapsulations. These are brought out to be made public, reusable, ubiquitous and stable at the domain abstractions layer.

  • The parts of the inner encapsulations that are specific to the application are factored out to become configuration information in the application layer, which it uses when instatiating abstractions.

  • Dependencies that existed between encapsulated elements for run-time communications are eliminated. They become simple wiring up of instances inside the application.

3.4.5. Smells of decomposition

  • Hierarchical diagrams

The tell-tale sign that this is happening is when we draw hierarchical diagrams. Boxes contained inside boxes. Even if we don’t draw them that way, the 'containment' or encapsulation is still implied. This is what package and component diagrams do. ALA has no use for package diagrams in the logical view. (However, they are still relevant in other views. There are several good reasons to have separately deployable binary code units such as exes or dlls.)

  • The dependency graph has many levels

If you have avoided circular dependencies, your application can be viewed as a (compile-time) dependency graph. Because it has run-time dependencies, it will have many 'levels'. These are not the hierarchical encapsulation levels, but just the strings of run-time dependencies within each level. In a composition system, the dependency graph will have a low number of layers.

  • Encapsulation without abstraction

Encapsulating details without an abstraction causes module or component boundaries to look relatively transparent at design-time. Their interfaces will tend to be specific to pairs of modules, and will tend to get increasingly wide as the software life cycle proceeds.

  • Modules have responsibility for who they communicate with

Either the sender knows who to send messages to, or, if using publish/subscribe, the receiver knows who to receive messages from. Understanding the system requires reading inside the parts to get the interconnection knowledge.

  • Compile-time indirection

If you find yourself doing many 'all files' searches to trace the flow of data or execution, this is a decomposition smell. The connections between the decomposed elements are mostly in the form of direct function calls or new keywords, and the name of another module. You have to find all these symbolic connections to trace through the system. In a composed structure, these connections are just adjacent elements in the text, or lines on a diagram. In both cases they are annonymous.

  • Run-time indirection

To avoid circular dependencies, many of the Compile-time indirections would have been changed to run-time indirections. This is often done using observer pattern of automatic dependency injection.

There is a meme that says something to the effect that such indirection is a two edged sword. On one hand it reduces coupling but on the it makes the structure even harder to see than it was when you has 'all files' searches. You may have to resot to a run-time debugger to see where the bugger goes next. At first this seems reasonable. It seems that you must always have this compromise between explicit structure and loose coupling. However it is just a result of decomposition., and unnecessary.

In ALA, there is no conflict between indirection and an explicit structure.

In a composition structure, at the top layer, all the structure is explicit in the form of the wiring. This is where all the design-time knowledge about the interactions between instances belongs, and where you can trace messages through the system at design-time with neither 'all files' searches, nor a debugger. When a message is processed by an instance of an abstraction, you know what that abstraction is supposed to do. You can tell if an issue is in the application or if an abstraction is not doing what is expected of it.

When you drop down inside an abstraction, you are now in a different program, bordered by its inputs and outputs. You don’t need to know where the execution flow goes outside its I/O ports to understand how it works because an abstraction has no knowledge of anything outside. If the abstraction calculates the squareroot and doesn’t do it correctly, you only need to debug to its interfaces.

3.5. Diagrams vs text

Why do we often use a diagram instead of text in the application (top) layer of an ALA application?

Diagrams and text are sometimes thought of as equivalent - and it’s a matter of personal preference which you use. I do not agree with this. From the point of view of how our brain’s work best, they are different, and each is powerful at its own job.

Consider an electronics engineer who uses a schematic diagram. Ask him to design a circuit using text and he will think you a simpleton. Electronics naturally has a network structure that is best viewed and reasoned about as a diagram. If you turn a diagram into a textual list of nodes and connections, the brain can no longer work with it directly. It is constantly interrupted to search for symbolic references when it should be free to just reason about the design.

Most software naturally has an arbitrary network structure. Think about whenever you are working with legacy code - how often to you need to do "all files searches" or "find all references". Try designing or reasoning about a state machine without a diagram.

Text can readily be used to compose elements in a linear chain or sequence. It is excellent for telling stories. White space is the normal connector between the elements. Sometimes periods or other symbols are used instead. Text can also handle shallow tree structures, simply by using indenting. Compilers may use brackets, usually () or {}. Interestingly, the brackets work for the compiler, but not for the brain. The brain doesn’t see them, it just sees the indenting. So I personally don’t agree that Python’s significant indenting is a mistake as many do.

When the tree gets deep, the indenting is too deep for our brains to follow. So text is suitable for linear structures and shallow trees. Structured programming and XAML are examples of tree structured code represented successfully in text.

Text becomes troublesome when there are arbitrary connections across the structure forming a mesh like network. It must be done with matching labels or identifiers. Most imperative programs are actually not a tree structure because of the variables. They must be done with labels. Local variables in a small scope are not too much of a problem. It only requires an editor that highlights all of them. For large scopes we end up spending too much time finding and trying to remember the connections, resorting to many all-files searches. It is a cumbersome way to try to reason about what is usually a simple structure when viewed as a diagram.

(When we talk about labels, we are talking about labels that are used for connecting two or more points. These labels are not abstractions. References to the names of abstractions are absolutely fine, and we don't draw lines for them even if we are using a diagram. We just use a box with the abstraction name inside it.)

When we need to compose instances of abstractions in an arbitrary network structure, our brains work much better using a diagram. The brain can readily see and follw the lines between the instances of the abstractions. Unlike with text labels, the lines are anonymous, as they should be. Lines don’t need encapsulation. To understand all uses of a variable in text, we need an encapsulation scope. To understand all places connected by a line, the brain just sees all the lines instead. Generally lines connect only two points or ports, but sometimes may connect three or four. More than that, and it starts to smell as if a new abstraction may be waiting to be discovered. The spacial positioning of elements is also something the brain readily remembers. So, diagrams can qualitatively do things that text simply cannot.

ALA does not require a diagram per se. It only requires abstraction layering, and it’s quite possible for user story to just consist of a linear sequence of operators or operations. For example, a sequence of movements by a robot or a "Pipes and Filters" sequence of operations on data. However, ALA is polyglot with respect to programming paradigms (computational models). This means that a user story will gnerally combine multiple programming paradigms: UI, event-flows, data-flows, state machines, data schemas, etc. These aspects of an application tend to be naturally interrelated (inherent in the requirements), which is what causes the resulting relationships among its instances of abstractions to be network like in nature. Diagrams, then, embrace the bringing together of all these different interrelationships of a user story in one place and view.

3.5.1. Diagramming tools

The ALA design process (which is describing your requirements and inventing the needed abstractions as you go) is an intense diagram generating activity, especially the first time in a new domain. It requires all your focus. I have found that hand drawing the diagram on paper is not good. The diagram quickly gets into a messy state which requires redrawing, and that interrupts your flow. I have found that a diagramming tool that constantly needs you to control the layout, such as Visio, is also not good.

So until there is a better tool, I have been using Xmind because as a mind-mapping tool, it is designed to not get in your way as you are creating. It lays itself out as a tree structure, and then allows cross connections on the tree to be added using a just key short-cut at the source and a mouse click at the destination node. It has its limitations, however I use some simple conventions to get around its these, such as using < and > to represent input or output ports.

Furthermore, the tree structure allows easy translation of the diagram into indented fluent code that represents the wiring. Any cross connections have to become identifiers that name the abstraction instances.

See the end of this chapter for an example project using Xmind.

Thoughts on the essentials of a diagramming tool.

It would have the low driving overhead of a mind mapping tool. As with a mind-mapping tool, you control the logical layout, and the tool does the actual spacial positioning. It would primarily use keypresses, but allow mouse clicks where it makes sense, for example, to specify the destination of a 'cross connection'. The tool would route the cross conenction for you.

A tree topology can be done with simple key presses. The tree would capture the primary relationships between instances, on their main ports.

You can make mutiple trees for different user stories that are disconnected logically, but for the purpose of automatic layout, are connected to the main tree (just an invisible line).

Abstractions are defined in a separate panel as stand-alone box with ports. Once a new abstraction is  defined, it can be instantiated in the diagram by its abstraction name with auto completion. Boxes represent these instances of abstarctions with the ports still lablled around their boundary.

The tool would allow subtrees to extend from a port in the direction it points, which could be up, down, left or right.

The tool's purpose is to aid creativity in the ALA process of representing a user story, inventing new abstraction as you go. Of course the tool would also automatically generate the corresponding code.

In my experience, a low overhead drawing tool is essential during the iteration zero design phase and during subsequent maintenance.

3.6. Composability and Compositionality

We have referred to the property "composability" a few times. By composability, we refer to the ability to create an infinite variety of applications by combining instances of a finite number of domain abstractions in different arrangements.

This is a very important property in ALA. Composability uses the Principle of Compositionality which states: In mathematics, semantics, and philosophy of language, the principle of compositionality is the principle that the meaning of a complex expression is determined by the meanings of its constituent expressions and the rules used to combine them.

Jules Hedges says of this property "I claim that compositionality is extremely delicate, and that it is so powerful that it is worth going to extreme lengths to achieve it."

The consequence of compositionality for software is that once a reader knows a finite number of abstractions, together with their rules of composition (grammar), they are able to understand a potentially infinite number of compositions, on first reading.

In software engineering, it is described by a pattern called "Abstract Interactions" or "Configurable Modularity" by Raoul de Campo and Nate Edwards - the ability to reuse independent components by changing their interconnections but not their internals. It is said that this characterises all successful reuse systems, and indeed all systems which can be described as "engineered".

ALA has these properties by using domain abstractions and programming paradigm interfaces.

As mentioned earlier, there are other software systems that have composability, usually using the data-flow paradigm, such RX (Reactive Extensions), or more generally monads. Most composability systems are restricted to a single paradigm. For ALA to have the correct level of expressiveness of all requirements, when inventing and composing domain abstractions, a variety of 'connection paradigms' are needed. Some examples of these are discussed in the next chapter on execution models.

We can make an analogy with Lego bricks. Some Lego parts have the familiar little stud and tube connectors. Some will support axles and holes connections, either tight or loose. These different ways of connecting Lego parts are analogous to different programming paradigms and different ways for parts of the model to 'execute' at run-time.

If the domain were for building model toys (the Lego domain), the non-ALA method would start with the imagined toy and decompose it into parts specific to that one toy. The solution would be brittle and hard to change and no other toys would be possible without the same huge effort all over again. The ALA method is to invent a finite set of building blocks and the mechanisms by which they connect. Then the initial toy can be easily changed, and other toys are possible with little effort.

3.7. Example project - Some real dependency graphs

Our example project for this chapter is a real legacy application (that was maintained for approximately 10 years) that we decided to re-write using ALA. Normally, for reasons I won’t go into here, I would never re-write an application. Maintenance had become difficult with this legacy code, and we wanted to run a research experiment to see if a rewrite using ALA could be successful. It would also give us a good basis for comparative metrics of the two code bases.

The original application has around 30 KLOC. Rather than look at any of the details of the application itself, we present here dependency graphs generated by Ndepend for the old legacy application and new ALA application.

3.7.1. Legacy application dependency graphs

One of the core tenets of ALA (as discussed in Section 3.2) is "Composition using layers" instead of "Decomposition using encapsulation". Unfortunately Ndepend is designed with the assumption that the application should be built using the latter approach. It likes to present a decomposition structure, starting with assemblies (packages) at the outermost level, then namespaces, and then classes. I’m not sure why it considers namespaces a viable encapsulation mechanism because they don’t provide encapsulation. Anyway, here is the namespace dependency graph for the main assembly of the legacy version of the application, as it comes out of ndepend.

Figure 12. Legacy application - namespaces

This graph is quite large, so if you like you can right click on it, and open it in a new tab in your browser. The red arrows are dependencies in both directions.

Each box represents a namespace. The thickness of the arrows is proportional to the number fo dependencies. The size of the boxes is proportional to the number of lines of code in the namespace.

If we drill down into the largest namespace, UIForms, we see the class relationships between classes inside that namespace:

classes in uiforms namespace.png
Figure 13. Legacy application - classes in uiforms namespace

Here you can see that ndepend is trying to make out the layers. The layers are vertical columns, going from left to right. I have left them vertical even through ALA abstraction layers are usually drawn horizontal because they come out more readable on the page. Again there are many dependencies in both directions drawn in red.

Here are the classes inside the DataStructure namespace:

classes in datastructure namespace.png
Figure 14. Legacy application - classes in datastructure namespace

Again, Ndepend is trying to make out the layers from left to right.

There is one class called Device which actually looks like it might be a good abstraction.

As mentioned, namespaces provide no useful decomposition structure. They do not make abstractions in themselves, nor do they implement a facade pattern or an aggregate root type of pattern with even logical encapsulation. Any classes inside each namespace can have unconstrained relationships with any classes in any other namespace.

So Ndepend is giving us a false picture here, because it is omitting all dependencies that go in or out of the namespaces. To really get an idea of what the big ball of mud looks like, I configured Ndepend to use a query that gives me all the classes in all the namespaces. Here finally is what this application truly looks like:

classes in all namespaces.png
Figure 15. Legacy application - all classes in all namespaces

This graph is very large. Right click on it, and open it in a new tab in your browser, so you can zoom in to see the dependencies in the background. It is truly frightening. Ndepend had no chance to find the dependency layers. There may be vaque onion type layers going outwards from the middle. It makes readily visible why continued maintenance on this application is so difficult. You have to read a lot of code to find even a tiny part of this hidden structure.

The developer who maintains the application tells me this is a fair projection of the complexity that he has to deal with.

To be fair, some of the dependencies in this diagram are 'good' dependencies (as described in Section 3.1 on good and bad dependencies). For example, the box near south-east called ScpProtocolManager has a lot of dependencies coming into it, which means it is possibly used a lot and therefore is a potential good abstraction. Ndepend does not know about the concept of good and bad dependencies, but if it did I would have it just display the bad ones.

3.7.2. New ALA application dependency graphs

Here is the equivalent Ndepend generated class dependency graph for the new ALA version of the application.

classes in all namespaces.png
Figure 16. New ALA application - classes in all namespaces

Ndepend has tried to find the three ALA layers which are vertical and go from left to right. Only the Application sits in the top layer. The DomainAbstractions layer contains the next two columns of classes and a few from the next column. And the ProgrammingParadigms layer contains the rest on the right. Actually there were a couple of bad dependencies present when this graph was generated which have since been fixed. (There should be no dependency between Panel and OptionBox, nor between Wizard and WizardItem.) With these removed, the graph would form into the three abstraction layers.

The newly rewritten application is a work in progress at this point. However, as features are added, this is all the dependencies you will ever see. The Application already uses most of the domain abstractions we will ever need, and the domain abstractions already use the programming paradigm interfaces they need. There are a few DomainAbstractions to be added, but this is essentially what the class dependency graph will look like.

This graph has the classes from all namespaces. But just for interest, here is ndpend’s namespace dependency graph.

Figure 17. New ALA application - namespaces

Remember in ALA, we do not use decomposition, so namespaces do not represent decomposition of the system. They represent layers. You can clearly see the three layers. The wiring namespace also goes in the programmingparadigms layer.

Lets drill inside the domain abstraction namespace to see the interdependencies within that layer. We expect to see no dependencies:

classes in domainabstractions namespace.png
Figure 18. New ALA application - classes in DomainAbstractions namespace

Ok here we see the two previously mentioned bad dependencies, and two other dependencies. They are on delegates or enums in the same source file, and so don’t count as bad dependencies.

And finally, let’s drill into the ProgrammingParadigms namespace

classes in programmingparadigms namespace.png
Figure 19. New ALA application - Classes in Programming Paradigms namespace

Again we see a few dependencies on delegates in the same source file which are ok. There is a couple of connector classes that depend on interfaces in this same layer. I consider them part of the interface from the programming paradigm point of view. They are in the same source file as a cohesive unit.

As of this writing, the new ALA version of the application is still a research project, but so far everything has gone smoothly with two weeks spent doing the description of the requirements as a diagram, and three months so far spent writing the domain abstractions. So far there are no issues getting it to actually execute. It is expected that we will actually commercialize the project soon and replace the old application.

3.7.3. The application’s diagram

As we said in this chapter, diagrams can be an important aspect of ALA when the user story naturally contains a network of relationships among its instances of abstractions. In this application this is the case. There are UI relationships between elements of the UI. There are data-flow relationships between UI elements, data processing elements, and data sources. There are event-flows from UI to wizards and between wizards and the SaveFileBrowser. and there are minor data-flows such as a the filepath from the file browser to the csvFileReaderWriter.

Here is a sample section from the application diagram that shows all the relationships that implement the user story:

Xmind used to design an application
Figure 20. Xmind being used to design an application

This diagram was drawn using Xmind. It shows a single user story. There is a UI with a menu item or a tool bar to start the user story. It then displays a browse dialogue to specify the location of the file. When the filepath has been selected, it gets data off a device on a COM port, using a protocol, and writes it to a CSV file. The data is also routed to be shown on a grid on the UI.

The user story diagram makes use of four different programming paradigms (which become four different interface types). Firstly there is the UI structure consisting of the window with its menubar, grid etc arranged inside it. Secondly, there is an event connection for when the menu is clicked which opens the browse dialog. Thirdly a dataflow connection carries the output of the browse dialog, a string containing the selected filepath, to the CSVFileReaderWriter. Another dataflow connection carries characters between the COM port and the SCPProtocol and another carries SCPcommands from the SessionDataSCP. The forth programming paradigm is a table data flow that carries dynamic columns and rows of data from the SessionDataSCP object to the grid object in the UI and to the CSVFileReaderWriter.

Having drawn the diagram to represent the user story, we need to make the diagram execute. When we started this particular project we had no tool for automatically generating the code from the diagram, but during the project, one of the interns wrote a tool to do this. It parsed the Json output from Xmind and generated C# wiring code equivalent to what we will show below.

However, at first we were hand generating code, and it is instructive to know what this hand generated code looks like, just so we know how the diagram actually executes.

When we were hand generating the code, it was important that the code was readable from the point of view of seeing how it corresponds exactly with the diagram. (It wasn’t important that the code was readable from the point of view of seeing how the user story works - that was the job of the diagram.) We had various conventions to support the one to one matching of diagram and code. One of these conventions was to indent the code to exactly mirror the tree structures in the diagram. Another was that whenever a new instance of an abstraction instantiated, all its ports would be wired immediately, and they would be wired in the order they were declared in the abstraction. This implies a depth first wiring strategy, analogous to walking the diagram tree depth first. Any ports with cross connections (the red lines in the diagram) would also be wired to their destinations at the time the abstraction were instantiated. If the destination instance did not already exist it would be pre-instantiated.

Using these conventions, it is a simple matter to hand generate the code below from the diagram.

using System;
using System.Windows.Media;
using DomainAbstractions;
using Wiring;

namespace Application
    class Application
        private MainWindow mainWindow = new MainWindow("App Name") { Icon = "XYXCompanyIcon"};

        public static void Main()
            new Application().Initialize().mainWindow.Run();

        private Application Initialize()
            return this;

        private Application()
            var getInfoWizard = new Wizard("Get information off device") { SecondTitle = "What information do you want to get off the device?" };
            Grid DataGrid;
            var sessionDataSCP = new SessionDataSCP();
            var csvFileReaderWriter = new CSVFileReaderWriter();

            // UI
                .WireTo(new Vertical()
                    .WireTo(new Menubar()
                        // XR3000
                        .WireTo(new Menu("File")
                            .WireTo(new MenuItem("Get information off device") { Icon = "GetDeviceIcon.png", ToolTip = "Get session data or LifeData or favourites from the device\nto save to a file or send to the cloud" }
                            .WireTo(new MenuItem("Put information onto device") { Icon = "PutDeviceIcon.png" })
                            .WireTo(new MenuItem("Exit") { Icon = "ExitIcon.png" })
                        .WireTo(new Menu("Tools"))
                        .WireTo(new Menu("Help"))
                    .WireTo(new Toolbar()
                        // XR3000
                        .WireTo(new Tool("GetDeviceIcon.png") { ToolTip = "Get information off device" }
                        .WireTo(new Tool("PutDeviceIcon.png") { ToolTip = "Put information onto device" })
                        .WireTo(new Tool("DeleteDeviceIcon.png") { ToolTip = "Delete information off device" })
                    .WireTo(new Horizontal()
                        .WireTo(new Grid() { InstanceName = "Sessions" })
                        .WireTo((DataGrid = new Grid() { InstanceName = "DataGrid" })
                    .WireTo(new Statusbar()
                        .WireTo(new Text() { Color = Brushes.Green }
                            .WireFrom(new LiteralString("Connected to device"))

                .WireTo(new WizardItem("Get selected session files") { Icon = "IconSession.png", Checked = true }
                    .WireTo(new Wizard("Select destination") { SecondTitle = "What do you want to do with the session files?", ShowBackButton = true }
                        .WireTo(new WizardItem("Save selected sessions as files on the PC") { Icon = "SessionDocumentIcon.png", Checked = true }
                            .WireTo(new SaveFileBrowser("Select location to save data") { Icon = "SaveIcon.png", InitialPath = "%ProgramData%\XYZCompany"}
                        .WireTo(new WizardItem("Send records to NAIT") { Icon = "NAIT.png" })
                        .WireTo(new WizardItem("Send sessions to NLIS") { Icon = "NLIS.png" })
                .WireTo(new WizardItem("Get Lifedata"));

            var comPorts =
                new ComPortAdapter()
                    .WireTo(new SCPProtocol()
                        .WireTo(new SessionDataSCP()



We used a 'diagram first' rule to keep the diagram and code in sync. Change the diagram first, then change the wiring code.

As of this writing, a graphical IDE is being developed for these types of ALA applications.

4. Chapter four - Execution models

4.1. Execution models (Programming paradigms)

ALA requires the use of multiple execution models, and models other than the native CPU’s hardware sequential instruction execution flow (called imperative).

The imperative paradigm of the hardware is the default paradigm in most general purpose high level languages. Some programmers are so used to thinking in terms of sequential execution of statements that it can be very difficult to think in terms of anything else. You keep wanting to know what the equivalent sequentially executed model is in order to understand what is going on instead of letting go and just thinking in terms of the different programming paradigm. Certainly it is nice to know what is going on under the covers from a performance or resourcing point of view. But from a logical point of view, it is better to let go and start expressing requirements directly in a range of different paradigms.

State machine execution model
Figure 21. State machine execution model

The canonical example is a state machine. At first it can be difficult to write a program as a state machine, even if a state machine is a more suitable execution model for expressing the requirements. It takes some getting used to.

It is an essential part of ALA to not only be able to use alternative execution models, but to use multiple execution models in the same application, and in the same user story. This is referred to as polyglot programming paradigms. For most user stories, the most common paradigm needed is not imperative but a combination of data-flow, UI layout, data schema and activity.

Within these basic paradigms, there are lots of variations on how they can actually be made to execute.

The following sections are a selection of common programming paradigms with a short description of each. This list will give you an idea of how varied they can be, and how powerful using them in combination would be. It is not an exhaustive list. In ALA you can invent your own programming paradigms when they better express the requirements, just as we did with the 'IConsistsOf' paradigm that we used in the project example for game scoring at the end of chapter two.

4.2. Imperative

The default provided by your programming language. It reflects the underlying machine. Connected elements are executed consecutively and synchronously as fast as the CPU can do them. The connected elements are language statements, especially function or method calls. Functions and methods are executed 'synchronously', which means that execution is always passed with the messages. The receiver of a message gets both the message and the CPU resource to process it. The sender waits, frozen, until the receiver finishes using the CPU resource and returns it. This paradigm is only suitable when you know ahead of time the order that things will happen, and those things should happen as fast possible with no temporal concerns whatsoever. It is the paradigm to use to execute short running algorithms.

4.3. Event driven

'Event driven' is an overloaded term in software engineering because it generally means both 'an asynchronous message' and 'decoupled'. Let’s clarify the decoupled part briefly because that part doesn’t involve the execution model.

4.3.1. coupled/decoupled

One perspective of 'Event Driven' is simply 'breaking out' of the imperative 'synchronous/coupled' paradigm. In Imperative programming, function calls are both synchronous (because the caller waits for it to return before continuing its own execution) and coupled (because the function caller refers directly to a function in another module by its name).

'Event driven' usually means both 'asynchronous' and 'decoupled'. It is asynchronous because the sender of the message does not wait for a response - it continues with its own execution immediately. It is decoupled because the sender does not name the recipient of the message. Instead the receiver usually names the event it is interested in. (Note that while this is considered 'decoupled' in Event Driven programming, it is not considered decoupled in ALA because the coupling is simply reversed - the receiver of the message names the sender (or the event name). In ALA neither coupling is allowed within a layer.)

These two notions, synchronous/asynchronous and coupled/decoupled are actually independent of each other. All four combinations are possible, and sensible. In fact all four combinations can be used in ALA. Although the term 'Event Driven' usually refers to the combination asynchronous/decoupled, in C#, events are synchronous/decoupled, being a synchronous implementation of the observer pattern - the sender waits until all receivers of the event have completed reacting to the event.

The coupling/decoupling aspect has nothing to do with execution models. We have already established ALA’s rules on coupling in chapter three (you must be fully decoupled in both directions within a layer).

So now let’s focus on the Event Driven programming paradigm from the perspective of being asynchronous.

4.3.2. synchronous/asynchronous

When something sends out an event or message there are two ways a receiver can can get the message from a timing point of view.

A real world example of an asynchronous event is

  • Synchronous - the receiver processes the message immediately when the message is sent. The receiver must be waiting, doing nothing, ready to process the message. The time of processing is therefore said to be synchronised with the sender.

  • Asynchronous - The receiver processes the message in its own time some time later. The receiver may be busy doing other things in the meantime. This implies the message must be stored somewhere until the receiver is ready. The time of processing is not synchronised with the sending of the event.

So synchronous is made of an event plus the receiver in a do-nothing-wait state. asynchronous is made up an event, a storage of the event, and a receiver taking the event when it is ready.

request/response pattern

A common pattern is an orchestration of two messages, a request and a response. Usually both messages are processed synchronously or both messages are processes asynchronously.

When both messages are processed synchronously it creates a very common pattern which we know of as a function call or method call. This pattern is efficient when running on a single thread because it is directly supported in silicon by the subroutine call instruction. This instruction passes the event and the execution CPU resource to the receiver at the same time, and passes them both back for the response at the same time. It is actually a very nice invention. However, because the function or method is so common in programming languages, and is so efficient, the synchronous request/response pattern appears to be fundamental to most developers and tends to be over used for messaging when asynchronous methods would be better.

real world messaging

In the real world we don’t normally think about synchronous. Events or messages happen. The sender sends it out. The receivers react to the events when they get around to it. It’s inherently asynchronous.

If there is to be a response to the message, the sender doesn’t wait around doing nothing while it waits for that response. Instead it reacts to the response when it arrives. If the response is not that important, like a request for a quote, the sender simply never notices that there was no response. If it is important to get the response, like a payment of an invoice, the sender will generally timeout if there is no response and send a new asynchronous message.

There can be synchronous messages in the real world too, such as when having a conversation.

When there is to be a response, the sender can be doing other things. We don’t have to think in terms of waiting at all. Instead we simply think of the sender reacting to the response event at it’s convenience. This is the asynchronous execution model.

Asynchronous events or messages are the fundamental form. An asynchronous message can be processes synchronously, but not the other way around.

Event driven as a paradigm

Having understood teh difference between synchronous and asynchronous, we have seen that events and message are terms we use for either synchronous or asynchronous.

The term 'Event driven' carries a higher level meaning that is an execution paradigm.

We will call the two alternatives Event Driven and Orchestration.

The orchestration paradigm will be covered in the Activity programming paradigm in the next section. For now it means that in our application, at design-time, we want to prescribe what will happen in what order.

The event driven paradigm is the opposite. In the application at design-time. We don’t know what will happen next. We will just wait until an event happens, and then react to it, usually changing some state.

The Orchestration paradigm generally uses synchronous messaging - because everything is orchestrated everyone waits for everyone else to complete before they go in their turn. The Event driven paradigm can use either synchronous or asynchronous messaging. If using synchronous, everything will complete reacting to one event before a second event gets any reaction.

Why use synchronous?

So if asynchronous is the more general and more flexible execution model, why use synchronous at all? As mentioned synchronous is efficient on a single thread, and is supported by the very common function or method statement of our common programming languages. There are other reasons to use synchronous.

  • Synchronous allows orchestration to be coded more conveniently. After a function call, the next thing to happen is coded in the next line of code, which is very nice. If using asynchronous messages for orchestration logic, the code for what happens next ends up in a different place (where the response event arrives) (although languages that have async/await can do code orchestration using asynchronous messaging just as nicely).

  • Single threaded synchronous messaging avoids certain potential problems because it is inherently deterministic - it orders the execution of everything that reacts to events whereas asynchronous leaves the ordering of execution to be determined separately, which will generally appear less deterministic.

Why use asynchronous?
  • Asynchronous avoid temporal coupling between sender and receiver.

  • Asynchronous allows separate explicit scheduling of execution for performance issues.

There are two reaons to use

The thread can’t do anything else until it gets the response. This causes performance issues for long running routines. The connonical example is the UI that freezes while the single thread application executes a long synchronous algorithm. Synchronous generally involves temporal coupling.

Synchronous can work across threads, processors or networks, but becomes even more problematic in temporal coupling. A synchronous call may block execution of the sender for an arbitrary length of time. The more general asynchronous form becomes preferred.

ALA can use both asynchronous and synchronous. It does not have rules for when to use one or the other. The rules remain more or less the same as in non-ALA applications. However, ALA is all about abstractions that are completely uncoupled - they know absolutely nothing about each other. It is therefore desirable that abstractions that generate events and abstractions that listen to events don’t have to be coupled by having to know whether they will both use synchronous function calls or both use some asynchronous event mechanism. The only way around this problem is for any interfaces that may need to be either to be asynchronous. That’s because an asynchronous interface can be used synchronously.

Abstractions that receive events can implement the asynchronous interface synchronously if they choose. If callbacks are used, this means that they will call the callback function in the interface synchronously. If Tasks or Promises are used, it means they will return a Task or Promise already in the complete state.

Abstractions that send events can use the asynchronous interface synchronously if they choose. They simply don’t do anything else until the response comes. If callbacks are used, it needn’t care if the callback is called back synchronously. If Tasks or Promises are used, it needn’t care if the Task or Promise it gets back after it sends the event is already in the complete state.

In this way senders and receivers do not need to be coupled with respect to synchronous/asynchronous. If two abstractions that don’t know each other have their instances wired together, the sender of the event determines whether it will be asynchronous or synchronous - do nothing while it waits for the response or not.

Unfortunately, if you make an interface asynchronous in order for it to handle either asynchronous or synchronous, both ends must be written in the coding style of asynchronous. If the sender wants to use synchronous, the coding style becomes awkward unless using async/await. This is especially true when there is a known sequence of activities to be done that is naturally expressed as sequential function calls. The other problem is that if you are in the happy position of having async/await available, the async functions can start spreading to everywhere.

On the receiver side, when it wants to implement an asynchronous interface in a synchronous manner, it can’t simplly return the vale - it must call the callback or set the result in the future object first.

All this only affects code that is written inside domain abstractions. It doesn’t affect things so much at the application layer. This is because in the application layer we have lifted ourself into the realm of composition of instances of abstractions. We have abstracted away the execution model in the interfaces in the Programming Paradigms layer. If you are wanting to do something sequential (but not Imperative) in the application layer module, you do it using the 'Activity' execution model which we will describe in a later section of this chapter.

4.3.3. Preemptive/non-premptive

Before leaving the 'Event Driven' execution model, we just need to clarify two variants of asynchronous - preemptive or non-preemptive. An Event or Message can be sent and end up being executed on another thread or the same thread.

In ALA, when using asynchronous, it must be non-pre-emptive by default. This is to prevent thread safety causing coupling between abstractions. Preemtive asynchronous (using multiple threads) would only be used when it is the only way to solve the temporal constraints of the problem, which would be understood to be compromising the decoupling between abstractions. This is the same criteria you should use in any type of programming style.

4.3.4. Examples of Event Driven

There are many examples of usage of event driven such as in Node.js, the reactor pattern, IEC 61499 function blocks and there is usually an Asynchronous Event Framework behind the scenes of state machines.

4.4. Activity-flow

The name Activity-flow comes from the UML activity diagram. Activities that are wired together execute in order. One starts when the previous one finishes. The activity itself may take a long time to complete (without holding the CPU). Activity flows can split, run in parallel or pseudo-parallel and recombine.

There are languages or libraries that support the Activity paradigm in text form so that the code looks like the Imperative paradigm but is actually more of the Activity paradigm. These are mechanisms such as async/await, yield, or coroutines such as Protothreads implemented using Duff’s device in C.

4.4.1. Work-flow

Persisted Activity-flow. This includes long running activities within a business process such as an insurance claim.

4.5. Data-flow

A data-flow model is a model in which wired instances in the program (or connected boxes on a diagram) are a path of data without being a path of execution-flow. The execution flow is like in another dimension relative to the data flow - it may go all over the place.

A stream of data flows between the connected components. Each component processes data at its inputs and sends it out of its outputs.

Each input and output can be operated in either push or pull mode. Usually the system prescribes all pull (LINQ), all push (RX), all inputs pull and outputs push (active objects with queues) or all outputs pull and inputs push (active connectors). In ALA we can use a mix of these different mechanism when we define the programming paradigm interfaces.

The network can be circular provided some kind of execution semantic finishes the underlying CPU execution at some point (see synchronous programming below).

The data-flow paradigm raises the question of type compatibility and type safety. Ideally the types used by the components are either parameterised and specified by the application at each connection or determined through type inference.

4.5.1. IDataFlow<T>

I frequently use data-flow execution models.

Here is one variation which works well:


This variation has these properties:

  • On a diagram, the line (wire) represents a variable that holds the value.

  • Fan-out - one output can connect to multiple inputs. All inputs read the same output variable.

  • Fan-in - multiple outputs cannot connect to one input.

  • Each output is implemented by a single memory variable whose scope is effectively all the places connected by the line (wire).

  • Receivers can get an event when the value changes

  • Receivers can read and re-read their inputs at any time.

  • Operator don’t need to have an output variable, they can pass the get through and recalculate every time instead.

Here is the version I use most often.


It has a number of useful properties.

4.5.2. ITable

This interface moves a whole table of data at once. The table has rows and columns. The columns are determined at runtime by the source.

TBD implemantation examples

4.5.3. IIterator

This data-flow interface allows moving a finite number of data values at once. It does so without having to save all the values anywhere in the stream, so has an efficient execution model that moves one data value at a time through the whole network.

This is the ALA equivalent of both IEnumerator and IObserver as used by monads. ALA uses the WireTo extension method that it already has to do the Bind operation. So the IIterator interface is wired in the same consistent way as all the other paradigm interfaces. There is no need for IEnumerable and IObservable type interfaces to support Also unlike monads, multiple arbirary interfaces can be wired between two objects with a single wiring operation.

IIterator has two variants that handle push and pull execution models. Either the A object can push data to the B object, or the A object can pull data from the B object.

TBD implementation examles

4.5.4. Glitches

All systems can have glitches when data flows are pushed in a diamond pattern. The diamond pattern occurs when an output is wired to two or more places, and then the outputs of those places eventually come back together. If they never come together, even both seen by a human, then we generally don’t care what order everything is executed in. But when they come together, the first input that arrives with new data will cause processing, and use old data on the other inputs. This unplanned combination of potentially inconsistent data processed together is a glitch. It even happens in electronic circuits.

The following composition of data-flow operators is meant to calculate (X+1)*(X+2)

diagram 25.png

When X changes, there can be a glitch, a short period of time, in which the output is (Cnew+1)*(Cold+2).

In imperative programming, this problem is up to the developer to manage. He will usually arrange the order of execution and arrange for a single function or method to be called at the place where the data-paths come back together. As he does this, he is introducing a lot of non-obvious coupling indisde the modules of the system, which is one of the big problems with imperative programming.

When we have composability, we don’t know inside the abstractions how data will propagate outside, and how it will arrive at its inputs. We want to execute whenever any of our inputs change, because as far as we know it may be the only change that might happen. So we really want the execution model to take care of eliminating glitches automatically for us.

This is a work in progress for the IDataFlow execution model described above. In the meantime, as a work-around I take care of it at the application level using a pattern. When I know data-flows will re-merge in a potentially inconsistent manner, I wire in an instance of an abstraction called 'Order' between the output and all its destination inputs. This instance of order is configured to explicitly control the order that the output date stream events are executed in. Then I will use a second abstraction called 'EventBlock' at the end of all data paths except one, the one that executes last.

diagram 26.png

By default multiple IDataFlows wired to a single output are executed in the order that they are wired anyway. On the diagram, they are drawn top to bottom in that order. This improves the determinism but is a little too implicit for my liking, so that is why I use the order abstraction.

4.6. Live-data-flow

As used in the coffee-maker example earlier, this paradigm simulates electronic circuits instead of using the concept of discrete messages. Semantically the inputs have the values of the outputs they are wired to at all times. This type of flow is readily implemented with shared memory variables.

FRP (Functional Reactive Programming) also is effectively a live data-flow execution model.

4.7. Synchronous data-flow

The use of the word synchronous here is different from its use in the discussion of synchronous/asynchronous events above. Here it means a master system clock clocks the data around the system on regular ticks. At each tick, every instance latches its own inputs and then processes them and places the results on their outputs. Data progresses through one operator per tick, so takes more time to get through the system from inputs to outputs. The result is a more deterministic and mathematically analysable system.

The execution timing and the timing of outputs occurs at a predictable tick time, albeit on a slower time scale than an asynchronous system. All timings are lifted into the normal design space.

Glitches that could occur in an asynchronous system (discussed earlier) are eliminated at the level of single clock ticks. A fast glitch could not occur. A glitch would occur when different data paths had different lengths, and would last for at least one tick duration. Controlling glitches is therefore lifted into the normal design space.

4.8. State machines (transition-flow)


4.9. UI layout


4.10. UI navigation flow


4.11. Data schema


4.12. Example Project - Coffee machine

Robert Martin posed an interesting pedagogical sized embedded system problem about a coffee maker in his book “Agile Software Development: Principles, Patterns and Practices”. The original chapter is called “Heuristics and Coffee”.

In the original chapter, the worked solution to this problem uses decomposition into three modules that collaborate or interact with one another. The ALA solution follows the opposite philosophy. It has three abstractions (which correspond with the three modules), but they do not collaborate or interact with one another. Being abstractions, they don’t know anything about each other. As domain abstractions, they also know nothing about the coffee machine. The coffee machine is then constructed (as another abstraction in the top layer) that makes use of the three domain abstractions.

This example uses different execution models from the 'consistsof' and 'dataflow' ones that we used in previous examples. Here we will use some extremely simple, yet quite interesting electronic-signal-like execution models that have a simple main-loop polling type implementation, just as Robert Martin’s original solution also had.

Reading an ALA application requires first knowing the pre-requisite knowledge you need from lower layer abstractions. So before presenting the application, lets first familiarise ourselves with the abstractions we need from the domain layer, and the Programming Paradigms layer.

4.12.1. Domain abstractions layer

Here are the three domain abstractions:

CoffeeMaker Domain Abstractions
Figure 22. Coffee maker domain abstractions

Take a moment to look at these three abstractions:

 — The UI has a lamp you can control, and a push button which outputs an event (should have been two separate abstractions).

 — There is a WarmerPlate. It tells you whether or not a container is on the warmer plate, and whether or not it is empty. It controls its own heater.

 — There is a Boiler. It can be turned on or off. It will tell you when it is empty of water. And you can stop water flow instantly with a steam release valve. It will turn its own heater off if it runs out of water, or the valve is opened.

That’s all there is to know about the three domain abstractions.

4.12.2. Programming Paradigms layer

We have three programming paradigms

 — live-data-flow (works like an electronic circuit)

 — events

 — simple state machine

The API for the Programming Paradigms layer is described in the key on the right of the diagram below. It gives you all the knowledge from this layer to be able to read the diagram. So, for example, a solid line is a data-flow; the rounded box is state with the states enumerated inside it.

The details of how to turn the diagram into code is explained in a project document, also provided in the Programming Paradigms layer.

4.12.3. Application layer

Now that we have understood the knowledge dependencies in all lower layers, we can read the diagram that resides in the top layer, the application layer:

CoffeeMaker Dataflow diagram
Figure 23. Coffee maker solution

The diagram to the left is the application itself. Instances of the three domain abstractions, UI, Boiler and Warmer plate are shown as boxes.

Follow me now as we go through the user stories by looking at the lines on the diagram:

  • When the UI push button is pressed, we set the state to Brewing, provided the Boiler has water and the pot is on the Warmerplate.

  • When the state is brewing, it turns on the boiler, and coffee making starts.

  • If someone takes the pot off, the valve is opened to momentarily release pressure from the boiling water, which stops the water flow.

  • When the boiler becomes empty, the state is set to Brewed. When the state is Brewed, the light in the UI is turned on.

  • When the coffee pot is replaced empty, the state goes back to the idle state where we began.

That’s all there is to reading this application. The code for the coffee machine can be read and understood in about one minute. Compare that with reading other solutions to the coffee machine problem.

Note that the paragraph above is pretty much a restatement of the requirements in English. It could have been the requirements. The amount of information in the English form (or the diagram form) is about the same, thus the Domain Abstractions gave us the correct level of expressiveness. Further confirmation of this is if the level of expressiveness allows us to modify it.

For example, say a requirement was added that a coin device was to enable the machine to be used. The coin device is an abstraction that provides an output when a coin is given, and has a reset input. Looking at the diagram, and being able to reason about its operation so easily, you can see that the coin device’s output would intercept the Pushbutton using another instance of an AND gate. And to reset the coin device, you could use the boiler empty output event.

4.12.4. Execution

To make it actually execute, we apply the manual procedure documented in “Execution models.doc”. This document is in the Programming Paradigms layer. It will generate these 6 lines of code:

if (userInterface.Button && warmerPlate.PotOnPlate && !boiler.Empty) { state = Brewing; } userInterface.Button = false;
boiler.OpenSteamReleaseValve = !warmerPlate.PotOnPlate;
boiler.On = state==Brewing;
if (boiler.Empty && !prevBoilerEmpty) { state = Brewed; } prevBoilerEmpty = boiler.Empty;
if (warmerPlate.PotEmpty && !prevPotEmpty) { state = Idle; } prevPotEmpty = warmerPlate.PotEmpty;
userInterface.LightOn = state==Brewed;

There is a one-to-one correspondence between the lines in the diagram and the lines in the code.

As you can see, the execution model is a simple one. The 6 lines of code are continually executed in a loop. This execution model is effective and appropriate for this small application.

The 6 lines of code can be built into a complete program shown below:

 #ifndef _COFFEE_MAKER_H_
 #define _COFFEE_MAKER_H_
 // Coffee Maker domain abstraction
 #include "CoffeeMakerAPI.h"  // original hardware abstraction supplied by hardware engineers
 // Knowledge dependencies :
 // "PolledDataFlowProgrammingParadigm.doc" -- explains how to hand compile a data flow diagram of this type to C code
 // Following are 3 Domain abstractions that the application has knowledge dependencies on

 #include "UserInterface.h"
 #include "Boiler.h"
 #include "WarmerPlate.h"

 class CoffeeMaker
    enum {Idle, Brewing, Brewed} state;
    Boiler boiler;
    UserInterface userInterface;
    WarmerPlate warmerPlate;
    bool prevBoilerEmpty, prevPotEmpty;
    void _Poll();
        : state(Idle), prevBoilerEmpty(boiler.Empty), prevPotEmpty(warmerPlate.PotEmpty)
    void Poll();
 #endif //_COFFEE_MAKER_H_
 // CoffeeMaker.c
 // This is not source code, it is code hand compiled from the CoffeeMaker application diagram
 #include "CoffeeMaker.h"

 void CoffeeMaker::_Poll() (1)
    if (userInterface.Button && warmerPlate.PotOnPlate && !boiler.Empty) { state = Brewing; } userInterface.Button = false;
    boiler.OpenSteamReleaseValve = !warmerPlate.PotOnPlate;
    boiler.On = state==Brewing;
    if (boiler.Empty && !prevBoilerEmpty) { state = Brewed; } prevBoilerEmpty = boiler.Empty;
    if (warmerPlate.PotEmpty && !prevPotEmpty) { state = Idle; } prevPotEmpty = warmerPlate.PotEmpty;
    userInterface.LightOn = state==Brewed;

 void CoffeeMaker::Poll()
    // get inputs processed
    // run application
    // get outputs processed
1 The 6 lines of code appear in the "CoffeeMaker::_Poll()" function.

If you are using a diagram as we are in this solution, you always change the diagram first when the requirements change. It provides the expressiveness needed to see the application’s requirements represented in a clear, concise and coherent way. There the logic can be ‘reasoned’ with. It is not documentation, it is the source code representation of the requirements, and executable, both important aspects of ALA.

The next step is to implement the three abstractions. These are straightforward using the same execution model as was used for the application, so are not shown here.

The resulting application passes all of Martin’s original acceptance tests plus a number of additional tests of behaviour gleaned from his original text.

5. Chapter five - Methodology

5.1. Agility

Apart from an iteration zero, ALA is inherently optimally agile. By agile, we mean it’s easy to change the functional requirements. ALA achieves this in its highest level separation of concerns. This primary separation is code that just describes requirements from code that just does implementation. The implementation code never has knowledge of any specific requirement, so it generally doesn’t change when requirements change. Only the code that describes requirements needs to change, and that code is optimally minimal over requirements, and so is optimally agile over changes to requirements.

5.1.1. Iteration zero

When a new project begins, the only new information we have is the requirements. Any design decisions that don’t depend on the requirement could already have been made beforehand. It is those decisions that form the ALA reference architecture. Therefore,, when we get the requirements, that is our immediate and total focus. We may not know all of them, but we will only need a sample to build a picture of the architecture.

Looking at the available new information as a whole first instead of taking it a bit at a time during the project’s sprints will make a huge difference to the eventual architecture quality.

The process in the first iteration takes requirements one by one, and represents them, in all their detail. Domain abstractions will be invented as you go, and they will have parameters or properties that will handle those details from requirements.

For the first green field application, you spend a maximum of one sprint. After that you do need to find out if your design works. So you may not get through all the known requirements. That does not matter.

To know whether knowledge from your design goes in the application layer of the domain abstractions layers, you consider what the scope of that knowledge is. Is the knowledge specific to this one requirement in the one application, or is it potentially reusable in the same or other applications? A softkey label is clearly specific. The concept of softkeys is clearly in the domain.

The output of the first sprint does not implement any of the invented abstractions, but it does include all details of the requirements that are looked at. In so doing, you design the first approximation of a DSL. The DSL may be refined later as more requirements are looked at.

Each abstraction will eventually be implemented as a class, but initially we just record the names of the abstractions, and a short explanation that provides the insight into what this abstraction does.

By the end of the first sprint the requirements will have become easier and easier to represent, as the set of abstractions will have taken shape. Sometimes you will generalize an abstraction further to enable it to be useful in for more things.

By keeping moving through the requirements at a much faster pace than in normal development (say one feature per hour instead of one per week), we can keep representing them in a coherent way, revising abstraction ideas we have already invented. Ideally, we will end up with a set of domain abstractions that can be wired together in different ways to represent a ‘domain space’ of requirements. That domain space will grow slightly as time goes on and it accommodates a growing family of products, but we don’t want it to grow beyond that. We don’t want to invent ways of implementing things we will never do. If you leave the invention of the set of abstractions to be done gradually during the longer time scale of the project as a whole, you will lose the opportunity to have a coherent set that will compose with each other in an infinite variety of ways.

The output of sprint zero is usually a diagram showing the wiring of instances of abstractions, together with annotated configuration information for those instances.

It doesn’t matter if some of the requirements are wrong. Chances are they are still useful for scoping out the domain space. What we are actually producing in this phase are the necessary abstractions for the Domain layer. If the requirements change later, it will be trivially simple to change them as only the Application layer wiring should change.

Once this process has started to become easy, which should happen within the first sprint, the burning question in our minds will become “Will all this actually work?” We have to trust that there will be a way to write those abstractions to make the whole system work.

5.1.2. 2nd Sprint

In the second sprint we start converging on normal Agile. You pick one feature to implement first. Agile would say it should be the most essential feature to the MVP (minimum viable product), but this can be tempered by the need to choose one that requires a fewer number of abstractions to be implemented. Next, you design the interfaces that will allow those abstractions to plug together according to the wiring you already have from the first sprint. What messages will these paradigm interfaces need to pass at runtime between the unknown clients to make them work?

It may take several sprints to produce the first working feature, depending on the number of abstractions it uses.

At first this sounds as if it might be just the waterfall method reincarnated. Do an overall design, document it or model it, and then write lots of code before everything suddenly starts working. But the design we created in iteration zero is very different from what a normal waterfall would produce, and is resilient to the sorts of problems waterfall creates. Instead of a ‘high level’ design of how the implementation will work which is lacking in detail, the design is a representation of requirements, in full detail. The design is not a model. It is executable.

There is one more important thing that the design phase in Iteration Zero does. While it deliberately doesn’t address any implementation, it does turn the remainder of the implementation into abstractions, and those abstractions are zero coupled. To convert from executable to actually executing, it only remains to implement these now completely separate parts. You can give these abstractions to any developer to write. Together the developers will also easily be able to figure out the paradigm interface methods needed to make them work, and the execution models to take care of the execution flow through them with adequate performance.

Often when a project is split into two phases, the first phase turns out to be waste. The devil is in the details so to speak. This happens because the implementation details in phase two are coupled back to and affect the design in phase one. As learnings take place during implementation, the design must change. In ALA the output from phase one is primarily abstractions, which are inherently stable and therefore hide details that can’t affect the overall design. If the abstractions are good, phase two will typically have little effect on the work done in phase one.

Once the first feature is working, several abstractions will have been implemented. The second feature will take less time because some of the abstractions are already done. In ALA velocity increases as time goes on and keeps increasing until new features only involve instantiating, configuring and wiring domain abstractions in new ways. This velocity acceleration is the complete opposite of what happens in monolithic code.

5.1.3. Later sprints

Imagine going into a sprint planning meeting with a Product Owner, a small team of developers, and a mature ALA domain that already has all the common domain abstractions done. As the Product Owner explains the requirements, one of the team members writes them down directly as they would be represented in terms of the domain abstractions. Another team member watches and remembers any lost details without slowing the product owner down. A third member implements the acceptance tests in similar fashion, and a fourth provides him with test data. It would be nice to have a tool that compiles the diagram into the equivalent wiring code. With such a tool, the team could have it executing by the end of the meeting. At the end of the planning meeting the development team say to the product owner "Is this what you had in mind?". The team can get immediate feedback from the Product Owner that the requirements have been interpreted correctly.

Of course, the planning meeting itself would only produce 'normal' functionality. Usually it is up to the development team, not the Product Owner, to uncover all the abnormal scenarios that can happen, and that is usually where most of the work in a software system goes. Having said that, in a mature domain, the validation of data already has decorator abstractions ready to go.

effort curve.png

The graph shows the effort per user story against months into a green-field project. The left axis is arbitrary - the shape of the curves is what is important. For a big ball of mud, experience tells us that the effort increases dramatically and can asymptote at around 2 years as our brains can no longer handle the complexity, and the project must be abandoned.

The COCOMO model, which is an average of industry experience, has a power relationship with program size, with an exponent of around 1.05 to 1.2. I have used the mid point, 1.1, for this graph. The model appears to imply that getting lower than 1.0 is a barrier, but there is no reason to believe this is the case. Reuse can make the power become less than 1. The range of 1.05 and 1.2 probably results from some reuse mitigating some ever increasing complexity.

ALA takes advantage of the fact that zero-coupled abstractions can keep complexity relatively constant and drastically increase reuse. A spectacular fall in effort per user story is thus possible.

5.2. Folder structure

|  `--application.cpp
|  `--application.cpp
|  |--abstraction1.cpp
|  |--abstraction1.h
|  |--abstraction2.cpp
|  `--abstraction2.h

This is a suggested folder structure for ALA. Because ALA does not use decomposition, you don’t end up with components that are contained by the applications, so there are no subfolders under the application. Instead, you end up with Domain Abstractions outside the application, so they go in their own folder in a flat structure.

Similarly, the Programming Paradigms code is not contained by an application, or even by the domain, so would not be contained by the domain’s projects folder.

5.3. Convention over configuration

When the application create an instance of an abstraction, most of the configuration of that abstraction should have defaults. In ALA, setters allow optional configuration, reducing the amount of information that would otherwise be required in the application to fully configure each abstraction. Any configuration that we wish to enforce goes into the constructor.

There is a counter argument that says that all configuration should be explicit so that nothing can be forgotten. ALA prefers optional configuration because we want the application to just express the requirements. Also optional configuration allows abstractions to default to their simplest form, making them easier to learn.

5.4. Knowledge prerequisites.

When other programmer are doing maintenance on your code, you should make sure they have the knowledge they need. They will need knowledge of ALA. They will need to know about the programming paradigms used. They will need to know about the domain abstractions, and the insight of what each one does. And then they should know that the application diagram is the source code. It is up to you that every develop that follows will know all this.

5.4.1. Intellisence

After they have modified the diagram, the maintaining developers will need to manually modify the corresponding code. Here they will see instances of abstractions being used all over the place, either 'new' keywords or function calls. If we have done our job with knowledge prerequisites, they will have been introduced to these abstractions. However, it doesn’t hurt to have brief reminders of what they are pop up when the mouse is hovered over them. So put in triple slash comments (or equivalent) describe the abstraction succinctly, with the intention of it being a reminder to someone who has already met the abstraction. Put a full explanation in the remarks and examples sections.

The class name after a new keyword is actually the constructor name, so you must duplicate the summary section there. Often in ALA, the class name is not referred to at all in the application.

5.5. Two roles

ALA requires two roles. Both can be done by the same person, but always he should be wearing only one hat at a time. There is the role of the architect, and the role of the developer.

The role of the architect is harder than that of the developer, that’s why we have the role. Expect it to be hard. Perhaps, surprisingly, the architect’s main job is to focus on the requirements, and the developers main job is to implement the abstractions (which know nothing of the requirements). In describing the requirements, the architect invents domain abstractions.

The architect also has a role in helping the developer to design the interface’s methods. In other words, how at runtime the system will be made to work.

This aspect of ALA can also be difficult at times. I have sometimes got stuck for a day or so trying to figure out how the interfaces should work, while still keeping them more abstract than the domain abstractions. The ALA constraints are that these interfaces should work between any two domain abstractions for which it may be meaningful if they are composed together in an application. However, the problem has always been solvable, and once solved, it always seems to have a certain elegance, as if you have created a myriad of possibilities at once. Implementation of the interfaces by the relevant domain abstractions becomes easy. Development then proceeds surprisingly quickly.

5.6. Example project - a real device

Unlike our previous example projects, this project is a real device and had previously been implemented without any knowledge of ALA. So this example serves to make comparisons between ALA and conventional software development. The original software was around 200 KLOC and took 3 people 4 years to write.

The actual device is used by farmers all over the world. It can weigh livestock and keeps a database about them for use in the field. It connects to many other devices and has a large number of features:

XR5000 image
Figure 24. Livestock weighing indicator

The architecture in the original software, was somewhat typically organised into modules and patterns by its developers. Also somewhat typically, it had ended up with a high cost of modifiability - a big ball of mud. After the first release, the first incremental requirement was a 'Treatments' feature, which involved several new tables, new columns in existing tables, new data pages, new settings pages and some new business logic. This feature took a further 3 months to complete (actually 6 calendar months), which seemed out of proportion for the size of the feature. Somehow the Product Owner and managers seemed to have a sort of intuition that if similar things had been done before, such as menus or database tables, those things were already done, and the only new work was in the specific details of the new requirements. Those requirements could be communicated in a relatively short time, say of the order of one hour or one day if you include follow up discussions of abnormal scenarios. So 6 months did not go down well. ALA, of course, works in exactly this intuitive way that managers hope for. All the things already done are sitting there in the domain abstractions, waiting to be reused, configured and composed into new requirements.

5.6.1. Iteration zero

During the development, there had a been a high number of changes required to the UI. It occurred to me at the time that the underlying elements of the UI were not changing. It was mainly the details of layout and navigation around the device’s many pages that were changing. The same could be said about the data and business logic. Only details were changing.

I took to representing the new designs using box and line drawings representing both the UI layouts and the navigation flows. I realized these diagrams were potentially executable, and wondered how far I could go representing the data and business logic in the same way. I decided to try to represent all of the functionality of the indicator in just one diagram.

It took two weeks to complete the diagram. I used Xmind because it laid itself out. I found that any drawing package that needed you to stop and do housekeeping such as rearranging the layout got in the way so much that you would lose your flow. Xmind allowed me to just enter in the nodes and it would automatically wire them in as either peers or chains and lay them out. The one disadvantage was that Xmind only does trees, so any cross tree relations had to be done manually, but this was also very quick in Xmind once you were used to it. I just let the cross wiring form arcs across parts of the tree.

Progress was extremely rapid once you had the abstractions and paradigms you needed. And many of them were obvious: softkeys, pages, grids, menus, actions, navigate-action, tables. etc. The programming paradigms would pop into play as needed. After the obvious UI-layout and navigation-flow ones came data-flow and data-flow of table types, events, and schema. The user of this device could set up custom fields, so the schema itself partially came from another table. At times I would get stuck not knowing how to proceed. The longest of these blocks was half a day. But every time the required abstractions or programming paradigms would firm up, and in the end anything seemed possible.

The diagram itself took shape on the right hand side of the Xmind tree. On the left side I had the invented domain abstractions and paradigm interfaces, with notes to explain them. The right side was mostly just a set of relatively independent features, but there was the odd coupling between them such as UI-navigation lines that were also present in the requirements.

The diagram contained around 2000 nodes (instances of the abstractions), which is about 1% of the size of the total original code. There were about 50 abstractions, and several paradigm interfaces.

Part of the diagram is shown below (laid out more nicely in Visio)

Application diagram for the All Animals View feature
Figure 25. Application diagram for the All Animals View feature

As I did the diagram, I deliberately left out anything to do with the aforementioned Treatments feature, so that I could see how easy it might have been to implement once the domain abstractions for the rest of the requirements had matured. So after the diagram was completed, I added the Treatments feature. This involved adding tables, columns to existing tables, a settings screen, a data screen, and some behaviours. No further abstractions needed to be invented. The incremental time for the diagram additions was of the order of one hour. Obviously testing would be needed on top of that, and the 'Table' abstraction would need additional work so it could migrate itself, a function it had not needed up until this point. Although somewhat theoretical, the evidence was that we could get at least an order of magnitude improvement in incremental maintenance effort.

At first the diagram seemed too good to be true. It had an elegance all of its own. It apparently captured all of the requirements, without any implementation at all, and yet seemed potentially executable. And if it worked, application modifications of all the kinds we had been doing were going to be almost trivial.

The burning question on my mind was, is it simply a matter now of writing a class for each of these abstractions and the whole job is done?

5.6.2. Translating the diagram to code

We hired a C++ student and proceeded with a 3-month experiment to answer this question.

It was a simple matter to translate the diagram into C++ code that instantiated the abstractions (classes), wired them together using dependency injection setters, configured the instances using some more setters, and used the fluent interface pattern to make all this straightforward and elegant. Part of the code for the diagram sample above is shown below to give you a feel for what it looked like.

	->wiredTo((new Softkeys())
		->wiredTo((new Softkey())
			->wiredTo(new Navigate(m_animalHistoryScreen))
		->wiredTo((skeyOptions = new Softkey())
			->wiredTo(new Menu()
				->wiredTo(new Navigate("Session...", m_sessionSummaryScreen))
				->wiredTo(new Navigate("Settings...", m_settingScreen1))
	->wiredTo((searchField = new TextDisplayField())
		->setField(VIDField = new Field(COLUMN_VID))
	->wiredTo(new Grid()
		->wiredTo(columnOrder = new ColumnOrder())
		->setRowMenus((new EventHandler())
			->wiredTo(new Menu()
				->wiredTo(new Navigate("View information for this animal", m_animalSummaryScreen))
				->wiredTo((new Action("Delete Record", AnimalsTable::DeleteRow))->wiredTo(AnimalsTable))

5.6.3. Writing the classes

We knew we wouldn’t have time to write all 50 classes, so we chose to implement the single feature shown below as a screen shot.

All Animals screen shot
Figure 26. All Animals view in the weighing indicator

The student’s job was to write 12 abstractions out of the 50. These 12 were the ones used by that feature. The initial brief was to make the new code work alongside the old code (as would be needed for an incremental legacy rewrite), but the old code was consuming too much time to integrate with, so this part was abandoned.

The learning curve for the student was done as daily code inspections, explaining to him where it violated the ALA constraints, and asking him to rework that code for the next day. It was his job to invent the methods he needed in the paradigm interfaces to make the system work, but at the same time keep them abstract by not writing methods or protocols for particular class pairs to communicate. It took about one month for him to fully 'get' ALA and no longer need the inspections.

The student completed the 12 classes and got the feature working in the device. The feature included showing data from one database table in a grid, sorting, searching, softkeys, and a menu.

Interestingly, as the student completed certain abstractions that allowed parts of other features to be done, he would quickly go and write the wiring code and have the other features working as well. For example, after the softkeys, actions, navigate, and page abstractions were done, he went through and added all the softkey navigations in the entire product as this only took minutes to do.

We wanted more funding to retain the student until we had enough to do the treatments feature, and indeed all 50 abstractions with the hope of making this implementation the production code and improving our ongoing maintenance effort. But that was not to be, despite the promising result.

We have about a quarter of a data point. Some of the abstractions done were among the most difficult, for example the Table abstraction, which had to use SQL and a real database to actually work. So it is not unreasonable to use extrapolation to estimate that the total time to do all 50 abstractions would be about one person-year. That compares with the original 12 person-years.

It seems that classes that are abstractions are faster to write. This seems intuitive because you don’t have any coupling to worry about. More importantly, the two phase design-then-code methodology of ALA allows the developer not to have to deal with large scale structure at the same time as writing code. This frees the developer to go ahead and write the code for the local problem.

I believe it is beneficial for each developer to be trained to be both an architect and a developer, but just don’t ask them to do both at the same time.

This practical result combined with the theory outlined earlier in this article suggests there ought to be a large improvement in incremental maintenance effort over a big-ball-of-mud architecture.

6. Chapter six - The philosophy behind ALA

6.1. The human brain

In this perspective of ALA, we look at the problem of complexity in software in the context of how the human brain works.

Software design involves our intelligence or brain power. Understandability, readability, complexity are all things very closely related to the brain. Yet in the field of software engineering we pay little attention to how the brain understands our complicated world in order to understand how we should do our software.

Our brains do it primarily through one mechanism which we have come to call 'abstraction'. We learn abstractions from the commonality of multiple examples, and we then use abstractions without those examples cluttering up the common notion that was learned.

Paintings from the Chauvet cave.jpg

Our ancestors could use a word like 'bring your spear' and it had a simple meaning to them only because all the detail and knowledge that went into building a spear was replaced with the abstraction 'spear'. Without the abstraction, the sentence would have had to be more like "bring the object that we made by joining the object we made by applying blows to the hard material we found at the place…​, with the long material we cut in the place with the tall…​, by tying it with the long grass material using the gooey stuff we found at the…​". Even this sentence was only made possible by other abstractions: joining, material, blows, hard, long (twice), cut, tall, tying, gooey, and found. If we expanded all of them until we were only using a few basic concepts like 'object' and 'place', we would have a sentence so long that we could never communicate at all. That’s what abstractions do, and how our brains make use of them. The word spear, in turn can be used to create a new abstraction, a hunting plan, while all of that other detail remains hidden from that new context.

The problem with software engineering is we are not making use of this way that the brain works. Simply put, we are not creating good abstractions. This lets the complexity inside one 'module' spill out into other modules. Abstractions, not modules, are the only mechanism that allows us to hide details at design-time.


As software engineers we do learn and use many abstractions. For example if we want to protect simultaneous access to the resource, our brain should conjure up 'mutex'.

If the brain already 'knows about' an abstraction, the abstraction is like any other single line of code, such as a mutex. We can make use of a mutex without having to deal with the details and complexities of how it works. We don’t have to think about the fact that we may have to wait for the resource. Nor that another thread may start running if we have to wait. Nor that if a higher priority thread preempts us while we have the resource, we may have it for a long time. Nor that if a still higher priority thread needs it during that long time, we will be given temporary priority to finish our use of it. We can just simply use the abstraction for protecting a resource.

Abstractions like mutex, regex, SQL are already invented by the 'culture' of software engineering, much like memes in the real world have been passed down to us. Where we fall down is when we get into a particular domain where the abstractions have not yet been invented, and we need to invent them. It is not easy to invent new abstractions, but invent them we must, at a rate far higher than is normal for cultural evolution.

Good domain abstractions, introduced and learned by new developers in the domain, then appear to them as normal program elements - things they can use with extraordinary convenience like any other line of code.

6.2. Abstraction

Of the overwhelming list of engineering topics that we listed in Chapter One, this topic that is the most fundamental to ALA, and the one most needed for explaining it. It’s also probably the vaguest and most misunderstood topic in software engineering, so we will spend some time understanding it.

Abstraction will be the king. The short reason why we start with abstraction is that our quality attributes, complexity and understandability are very much to do with how our brains work, and for 100,000 years at least, our brains have worked with abstractions to understand our world. Abstractions are the only mechanism our brains use for dealing with otherwise complex things.

As in a chess game, winning is only about protecting the king. But this Abstraction king is benevolent. If he is destroyed, you do not lose the game immediately. It will take time, but you will lose.

There are other contenders to be king in the engineering topics list. For example, it is said that the best thing about TDD is not the testing but the emergence of better abstractions. TDD is like a lord that serves the king. It usually serves the king, causing you to make better abstractions. But sometimes it just serves its own purpose and makes the abstraction worse. It just produces code that works where it passes and no more.

Another contender is microservices. It is popular because it improves your abstractions by making them harder to destroy with cross coupling. But it too is just a lord. Because it provides physical boundaries that in normal software would be crossed, it serves the king. But by serving the abstraction king directly we can have logical boundaries, and all their benefits, even in 'monolithic' code.

Another contender to be king is 'no side effects' used by the functional mathematical purity guys. There are those who talk as if disobeying this king is absolute treason. But again, this lord is only effective because he usually serves the abstraction king. But, again, there are times when he doesn’t, and 'no side effects' is not enough to make a good abstraction.

ALA always follows the one true king.

6.2.1. Classes, Modules, functions and encapsulation.

Classes, Modules, functions and encapsulation are artefacts of the language and do their thing at compile-time. They are not necessarily abstractions. They have been around for about 60 years, not enough time for our brains to see them in the same way as the compiler does. Although abstractions are implemented using these artefacts, ALA needs them to also be abstractions. In ALA "abstraction" is the term we use for the artefacts of our design instead of classes, modules, functions, or components, all of which are extremely fragile as abstractions.

6.2.2. Wikipedia on abstraction

"Thinking in abstractions is considered by anthropologists, archaeologists, and sociologists to be one of the key traits in modern human behaviour, which is believed to have developed between 50 000 and 100 000 years ago. Its development is likely to have been closely connected with the development of human language, which (whether spoken or written) appears to both involve and facilitate abstract thinking."

In the real world, new abstractions come along infrequently, and are conceived of by few. People quickly begin using them to understand new insights or compose new things. They become so natural to us that we forget that they are abstractions. In no other field do we need to create them as fast as in software engineering. It is the most important skill a developer needs to have.

6.2.3. Defining abstraction

The term abstraction is arguably one of software engineering’s vaguest and most overloaded terms. For the purpose of this article we will need a definition. I find the easiest way to define it is to provide a set of 'statements about', 'qualities of', and 'what it is nots':

  • What our brains evolved to use to understand things

  • Etymology: 'to draw out commonality'

  • The concept or notion represented by the commonality of many instances

  • Has inherent stability - as stable as the concept itself

  • Increases with ubiquity and reuse

  • Decreases as you get closer to your specific application

  • The only mechanism that separates and hides knowledge at design-time

6.2.4. The three stages of creativity

Creativity with abstractions
Figure 27. The creativity cycle

A good abstraction separates the knowledge of different worlds. A clock is a good abstraction. On one side is the world of cog wheels. On the other side is someone trying to be on time in their busy daily schedule. Neither knows anything about the details of the other. SQL is another good abstraction. On one side is the world of fast indexing algorithms. On the other is finding all the orders for a particular customer. Let us consider a domain abstraction - the calculation of loan repayments. On one side is the world of mathematics with the derivation and implementation of a formula. On the other, the code is about a person wanting to know if they can afford to buy a house. If your abstractions don’t separate knowledge of different worlds like this, then you are probably just factoring common code. Find the abstraction in that common code. Make it hide something complicated that’s really easy to use and really useful, like a clock.

The creativity cycle starts with abstractions, such as cogs and hands, instantiates them, configures them for a particular use, then composes them into a new abstraction. In ALA we usually go around the creativity cycle three times, creating three layers on top of our base programming language.

6.2.5. An illustration of abstraction at work

Imagine you are reading the following function, xyz123, at design-time and trying to understand it:

real xyz123(real)
    b = fubar(a)
real fubar(real)
    // complicated code

You don’t know what fubar is (fubar stands for messed up beyond all recognition), so you follow the indirection, an inconvenience at the least because you are really just interested in xyz123. You begin reading the code at fubar. It only has about 20 lines but it is complicated. A comment mentions that it uses a CORDIC algorithm and gives a reference. But before following that indirection as well, you note that fubar has the following properties:

  • a module

  • has a simple interface

  • encapsulated

  • uses nothing but what is in the interface

  • no side effects

  • hides information

  • is loosely coupled

  • separates two concerns

  • is small

  • follows the coding guidelines

  • has comments

It could even have a good name describing what it returns in terms of its input, something like Returns X such that X is …​., but that still wouldn’t solve the issue.

Why, if it has all these good software properties, is it destroying our ability to understand function xyz123? The missing ingredient is of course abstraction, the thing the brain needs for meaning. So let’s go ahead and add the abstraction property. To do that, change fubar to just 4 letters (it doesn’t even need to be a good name): SQRT. While we are at it, let’s add the abstraction property to xyz123 as well, by naming it StandardDeviation.

real StandardDeviation(real)
    b = SQRT(a)
real SQRT(real)
    // complicated code

Suddenly understandability in the first function is unlocked. The complicated code inside SQRT no longer matters. It is completely isolated by the abstraction. If your brain already knows the SQRT abstraction (I had to choose one that I knew you already knew), there is no need to follow the indirection. The reader of the code continues reading with the next line of code after the SQRT invocation as if it is just like any other line of code in their language. That’s what abstraction, at least as used in ALA, is.

All those other properties made no difference while the abstraction property was missing. But any one of them could destroy the abstraction. Our current programming languages provide nothing to help us with abstraction. At least for the meantime, it remains the one cerebral activity software engineers must do for themselves.

6.2.6. Abstractions need ports

In traditional programs, inputs (or at least incoming function calls) are typically part of the module or class’s interface but outputs (or at least outgoing function calls) are typically just buried in the code.

This is fine if calling functions or methods in a lower abstraction layer. However, it is absolutely not fine if calling functions or methods of a peer in the same abstraction layer.

In ALA all inputs and outputs to or from peers in the same layer must be 'ports'. There should be one port for each peer that can be wired. This is the Interface Segregation Principle. A port is a logical wirable connection point. A port either implements or accepts an interface. Outgoing function calls buried in the code that at run-time will go to a peer must only go to the port, which has an indirection mechanism of some kind.

Programming languages encourage all outgoing function or method calls to refer directly to the destination, or the destination’s interface, so you have to make an effort to avoid doing this.

A port is not an artefact of programming languages (yet) so they must be implemented logically somehow as normal code. To code a logical port, you need to do two things.

  1. The interface type of the port must not be owned by another peer abstraction. The interface type must be from a lower abstraction layer.

  2. The name of the port is the name of the field that accepts the interface.

A port can have multiple interfaces. In this case I make the names of the multiple fields contain the port name.

If you are using an asynchronous event driven design, the equivalent of a conventional outgoing function call is typically written something like this:

Send(Event, Receiver, Priority);

where the Event is something the receiver defines.

Again, we are sending the event directly to a peer abstraction using the peer abstraction’s interface (its event).

In ALA, sending an event should be self-oriented, so written something like this:

Send(Sender, SenderPort)

The sender just sends the event out, not knowing where it goes, and the port identifies the event (or you could have both port and event). This just tells the event framework who the sender and sender’s port was. The event framework gets information from the application in the top layer to know what to do with the event. The application has the specific knowledge to know what an event from a given sender on a given port means, and therefore where it should go, and what the priority should be.

In general, classes, modules, components, functions should all have ports for both input and output. They should not own the interface types for these ports, whether they are incoming or outgoing.

An output port from an abstraction may say 'This has happened' or 'Here is my result', not 'do this next', or 'here is your input'.

There are multiple ways to implement the indirection inherent in ports for outgoing calls. They can be callbacks, signals & slots, dependency injection, or calls to a framework send function.

Note that inputs and outputs are not necessarily on different ports. We may want to wire both inputs and outputs between two instances or two abstractions with a single wiring operation. The general case is that a single wiring operation wires multiple interfaces that are logically one port. One contains methods going in one direction and the other contains methods going in the other.

6.3. Complexity

6.3.1. Philosophy of complexity

6.3.2. Dijkstra on complexity

"It has been suggested that there is some kind of law of nature telling us that the amount of intellectual effort needed grows with the square of program length. But, thank goodness, no one has been able to prove this law. And this is because it need not be true. We all know that the only mental tool by means of which a very finite piece of reasoning can cover a myriad cases is called “abstraction”; as a result the effective exploitation of his powers of abstraction must be regarded as one of the most vital activities of a competent programmer. In this connection it might be worth-while to point out that the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise. Of course I have tried to find a fundamental cause that would prevent our abstraction mechanisms from being sufficiently effective. But no matter how hard I tried, I did not find such a cause. As a result I tend to the assumption —up till now not disproved by experience— that by suitable application of our powers of abstraction, the intellectual effort needed to conceive or to understand a program need not grow more than proportional to program length."

The "conceive" part I agree with, if by that we mean the development. However, the "intellectual effort to understand" part needs further insight. We shouldn’t have to read an entire program to understand a part of it. We ought to be able to understand any one part of it in isolation. The effort to read any one part should be approximately constant. In Chapter One of this article there was a quality graph of complexity here.

complexity curve.png

These graphs are qualitative in nature, based on experience. But now that we have a better understanding of ALA structure, we can explain how it manages to keep complexity from increasing.

In ALA, the design-time view of the system is nothing more than a static view of instances of abstractions composed together. In a typical application, there will be of the order of fifty different domain abstractions - not a difficult number to familiarize yourself with in a new domain.

Abstractions have no relationship with one another. Each is a standalone entity like a standalone program. If every abstraction contains say 500 lines of code, and the system itself contains 500 lines (instances of abstractions wired together) then the most complex the software gets is that of 500 lines of code.

Even if one abstraction is overly complex internally, say it conceals a piece of legacy code using a facade pattern, that doesn’t affect the complexity of any other part of the system.

ALA is based on the realization that abstraction is fundamentally the only mechanism available to us to achieve this constant complexity.

When doing this for the first time in a domain, it’s not easy to invent the abstractions. but the alternative is always runaway complexity.

The goal of software architecture should be to keep complexity constant.

6.4. No Loose Coupling

Here we meet the first meme from our list of software engineering topics that we must throw out. To many, this will seem a surprising one. Yes, I am saying 'loose coupling' is undesirable.

6.4.1. A common argument

An argument is sometimes stated along these lines: "There must be at least some coupling, otherwise the system wouldn’t do anything." Hence we have the common meme about "loose coupling and high cohesion". In this section we show how this argument is false and resolve the apparent dilemma. We will eliminate all forms of design-time coupling except one. That one remaining one is anything but loose and very desirable.

6.4.2. Classifying coupling

Think of some instances of dependencies you know of in a system and try to classify them into these three types by asking when the system would fail without it.

For example, let’s say that data flows from an ADC (analog to digital converter) to a display as part of a digital thermometer. At run-time, both must exist. At compile-time both must have the same method signature:

diagram 01.png

Or the display may tell the ADC when to do the conversion. At run-time there is temporal coupling.

diagram 02.png

In this one there is an association from a Customer class to an Account class to facilitate communication between them. At run-time there is coupling. At compile-time there is coupling too - the type of the Account class must be exactly the same as expected by the Customer class:

diagram 04.png

In all the above diagrams, relationships shown in red indicate they are disallowed by the ALA constraints. Green is for desirable relationships, of which there is only one. When we disallow all these types of coupling, the modules, components, functions and classes can now be abstractions.

6.4.3. Run-time, Compile-time and Design-time

A few times already in the article, I have sneaked in a magic qualifier, 'design-time'. You know how we sometimes talk about run-time and compile-time with reference to binding. In ALA we recognise that understandability, complexity, etc, are all happening at design-time. By design-time I mean any time you are reading code, writing code, or changing code.

At run-time, the CPU processes data. At compile-time, the compiler processes code. At design-time the brain is processing abstractions.

In conventional code, it is common for all forms of coupling, run-time, compile-time, and design-time, to appear as coupling between modules or classes.

You can work out what type of dependency you have by when it first breaks. A run-time dependency doesn’t break until the program runs. The program can still be compiled and it can still be understood.

A compile-time dependency first breaks at compile-time. At design-time the code can still be understandable.

A design-time dependency prevents code from even being understood. The code loses its meaning.

6.4.4. Layers

In everyday design, knowledge dependencies are not normally shown as lines. You simply use the abstraction by its name. But in this article, just so we can explain the meta-architecture, we will sometimes draw knowledge dependencies like this (always downward).

diagram 05.png

This represents that the implementation of abstraction C knows about abstraction A. A is more abstract than C. C and A cannot therefore be peers, as was the case with the components above. Peer abstractions cannot have any coupling with one another.

6.4.5. Whole-Part pattern

If you are familiar with the Whole-Part pattern, ALA uses it extensively. But there is a constraint. The Whole-Part pattern is only used with knowledge dependencies (since that is the only relationship you are allowed). It may of course be used in other forms inside an abstraction, provided it is completely contained in a single abstraction.

A real world example of the Whole-Part Pattern with knowledge dependencies is Molecules and Atoms. A water molecule, for example, is the whole.

diagram 06.png

Oxygen and hydrogen are the parts. Note that oxygen and hydrogen are abstractions, and they are more abstract than water because they are more ubiquitous, more reusable and more stable (as a concept) than any specific molecule. We could make a different molecule but still use exactly the same oxygen and hydrogen as parts to compose the new molecule.

When we use the word 'ubiquitous', it refers to the number of times the abstraction is used in a Whole-Part pattern to make other abstractions. It doesn’t refer to the number of abstractions that are instantiated. So just because there is a lot of water, that doesn’t make the abstraction ubiquitous. In comparing the abstraction levels of Oxygen and Hydrogen with water, Oxygen and Hydrogen are more ubiquitous because they are used to make more abstractions than water is.

The molecules and atoms analogy with ALA is very close, and we will return to it when we come to explain in more detail how run-time and compiler-time dependencies are moved inside a single abstraction.

For now we just need to remember that we are using the whole-part pattern with knowledge dependencies only. At design-time, the whole is explained and reasoned about in terms of the parts, just as the water molecule is in terms of the oxygen and hydrogen.

6.4.6. Run-time/design-time congruence

A software program can be temporally confusing. Everything that happens at design-time is in preparation for what will happen at run-time. Our low-level imperative languages tend to keep the two congruent. The statements in the program at design-time follow in the same order as they will execute at run-time. The only difference between the two is a time shift and the speeding up of the clock.

When we want the knowledge of run-time dependencies to be moved inside another abstraction, this congruence between design-time and run-time must be broken. Unfortunately, developers start out by learning a low-level imperative language, so it becomes unnatural to them to architect their programs without this congruence. Indeed, breaking this congruence needs a pattern to be learned, and then carefully protected from the temptations of our imperative languages. I call it the Ẃiring pattern'.

Before going into the pattern, we need to round out the most important aspects of ALA.

6.5. Wiring pattern - Part one

We now introduce the pattern that both solves the congruence problem just discussed in the previous section, and provides the alternative to all those disallowed coupling types discussed earlier. This pattern is usually an important part of ALA.

Note: The wiring pattern is not necessarily a part of an ALA architecture. For example, if your whole problem is just an algorithm, and therefore suits a functional programming style, then you can still compose abstractions with function abstractions, provided all function calls are knowledge dependencies, and not say, just passing data or events.

If you are using monads, especially I/O monads, or RX (reactive extensions), especially with hot observables, you are already using the wiring pattern. The pipes and filter pattern is also an example of the wiring pattern. Labview or Node-Red can use the wiring pattern. There are many other examples of the wiring pattern. Most support a data-flow programming paradigm. Here we generalize the pattern to support any programming paradigm.

The wiring pattern may be the same as the "Component Pattern" in some literature if used with what is referred to as 'configurable modularity' or 'abstracted interactions'.

The wiring pattern allows lines on your application diagram to mean any programming paradigm you want that express your requirements. It also allows you to implement multiple programming paradigms together in the same diagram.

If you are using dependency injection with explicit code for the wiring (not auto-wiring), then you are half way there.

The wiring pattern separates design-time/run-time congruence. It works by having a 'wiring-time' that is separated from run-time. 'Wiring-time' can happen any time before run-time. It can happen immediately before it, as for instance in LINQ statements or RX with a cold observable. It becomes powerful when we make wiring-time congruent with design-time. Usually the wiring code will actually run at initialization time, when the program first starts running. That initialization code becomes the architectural design.

Let’s suppose you have designed your system with two modules, A and B. There will be one of each in your system.

diagram 07.png

At run-time we know that A will talk to B. So we design A to have an association with B. The association may first appear on a UML model, or it may go straight into the code something like this:

static component A
static component B
   public B_method() { }

A and B may be implemented as non-static, with only one instance of each. The association is still there.

component A
   private var b = new B();
component B
   public method() { }

A may create B itself, which is a composition relationship, as above. Or A may have a local variable of type B passed in by some kind of dependency injection, which is still an association relationship.

component A
   B b;
   public setter(B _b) {b = _b}

Note that although dependency injection was used, it only eliminated part of the dependency, that of which particular subtype of B it is going to talk to, but A still knows the general type B, which is not allowed in ALA. (Part of the problem here is that A and B were probably arrived at by decomposition, and so they have subtle knowledge of each other, for example of how they collaborate.)

If A and B are collaborating, they are not abstractions. Their knowledge of each other at design-time (to enable their relationship at run-time) binds them to each other so that neither can be reused in any other way. And if they can’t be reused, they can’t be abstract.

Let’s revisit the water molecule analogy we discussed earlier for the Whole-Part pattern, and develop it further to be clearer how these dependencies affect abstractions. Let’s say we have decomposed water into two components, Oxygen and Hydrogen. Oxygen will talk to Hydrogen to get an electron, so we write:

component Oxygen
   var h1 = new Hydrogen();
   var h2 = new Hydrogen();

The diagram for that looks like this:

diagram 08.png

In the real world, oxygen is a very useful abstraction for making other molecules. In writing code this way to make water, we have tied it to hydrogen. Oxygen can’t be used anywhere else, at least not without bringing with it two hydrogens, rendering it useless. By implementing the Oxygen-Hydrogen relationship needed to make water in oxygen, we have destroyed the oxygen abstraction. We never even made the water abstraction. To understand water, we would have to read the code inside oxygen, where the parts about water have become entangled with the inner workings of oxygen, protons and neutrons and all that stuff. Oxygen is also used to make caffeine. We could never make coffee!

Caffeine molecule
Figure 28. caffeine - oxygen atoms are red

Abstractions are fragile and get destroyed easily, so we have to take care to protect them. What we needed to do was to put the knowledge about the relationship between oxygen and hydrogen to make water in a new abstraction called Water.

diagram 09.png

In general, to break coupling between peer modules A and B, we move the knowledge of the coupling to a higher level abstraction (less abstract level) where it belongs. Let’s call it C. C is a more specific abstraction. The knowledge is encapsulated there - it never appears as a dependency of any kind. And it is cohesive with other knowledge that may be contained inside abstraction C.

diagram 10.png


diagram 11.png

The diagram above is only to show the ALA knowledge dependency relationships between the three abstractions. It doesn’t yet show explicitly that an instance of Abstraction A will be wired to an instance of Abstraction B. In practice we never actually draw knowledge dependencies. We are just doing so here to show how ALA works. We would draw it in this way instead:

diagram 12.png
diagram 13.png

Now we have the explicit wiring. It looks a lot like the original diagram where we had no C. But where the knowledge is coded is very different. Because it is C and not A that has the knowledge of the relationship between A and B, Abstractions A and B do not change. They continue to know nothing of the connection. They remain abstractions. They remain re-usable.

It may seem at first that adding the extra entity C is a cost, but in fact C is an asset. It shows the structure of the system. It shows it explicitly. It shows it in one small understandable place. And it is executable - it is not a model.

The original abstractions were left below C to show that they still exist as free abstractions to be used elsewhere. They are not contained by C in any way as modules from a decomposition process would be. The A and B inside C are only instances. We wouldn’t normally bother to draw the abstractions below. So we just draw this:

diagram 14.png

C must achieve the connection between A and B either at compile-time or run-time. With current languages, the easiest time to do this is at initialization time, when the program first starts running. This is similar to dependency injection, except that we are not going to inject the instance of B into A.

This is what the code inside C might look like:

Abstraction C
   var a = new A();
   var b = new B();

Typically we will write the code using the fluent pattern, with the wireTo method always returning the object on which it is called, or the wireIn method always returning the object wired to. The constructor already returns the created object by default.

Abstraction C
   new A().wireTo(new B());

If A and B are static modules, this produces something like:

Abstraction C

6.6. Wiring pattern - part two

We are half-way through explaining the wiring pattern. Now we turn our attention to how A and B can communicate without knowing anything about each other.

This part of the pattern is also called "Abstract Interactions"

Of course, one way is that C acts as the intermediary. This way is less preferred because it adds to C’s responsibilities. But it is sometimes necessary if there are some abstractions brought in from outside. Such abstractions will 'own' their own interfaces or may come with a contract which C will have to know about. C will usually have to wire in an adapter, or handle the communications between the two abstractions itself.

A better way, because it leads to an architectural property of composability, is that A and B know about a 4th abstraction that is more abstract than either of them. This is legal because it is a design-time knowledge dependency. Let’s call it I.

diagram 15.png

I is an interface of some kind. It may or may not be an actual artefact. What it must be is knowledge that is more abstract than A and B and therefore knows nothing of A and B. It is more ubiquitous and more reusable than A and B are. In other words we can’t just design I to meet the particular communication needs of A and B. That would cause A and B to have some form of coupling or collaboration with each other, and again destroy them as abstractions.

I is so abstract, ubiquitous and reusable, that it corresponds to the concept of a programming paradigm. We will cover programming paradigm abstractions in following sections because they are a critically important part of ALA. We will see that ALA is polyglot with respect to programming paradigms.

Figure 29. In an electronic schematic, the components are abstractions that are composed using two paradigm interfaces - live analog signals and live digital signals

Returning to a software example, let’s choose a single simple programming paradigm: activity flow. This programming paradigm is the same as the UML Activity diagram. When we wire A to B and they use this paradigm, it means that B starts after A finishes. If A and B accept and provide this interface respectively, then wiring them together by drawing an arrow will have that meaning, and cause that to happen at run-time.

diagram 16.png

It is easy to create an interface for the activity-flow programming paradigm. It has a single method, let’s call it 'start'. Many abstractions at the level of A and B can either provide or accept this paradigm interface. Then instances of them can be wired up in any order and they will follow in sequence just like an Activity diagram.

Note that the Activity Diagram is not necessarily imperative in that any Activity can take an amount of time to complete that is not congruent with the actual CPU execution of code. In other words activities can be asynchronous with the underlying code execution, and for example, delay themselves during their execution, or wait for something else to finish, etc.

The code in Abstraction A could look something like this. Don’t take too much notice of the exact method used to accomplish the wiring. There are many ways to do this using only knowledge dependencies. The important thing is that A continues to know nothing about its peers, continues to be an abstraction, and yet can be wired with its peers to take part in any specific activity flow sequence:

 Abstraction A : IActivity
    private IActivity next = null;

    public IActivity wireTo(IActivity _next)
        next = _next;
        return _next;

        // start work

    // code that runs when work is finished.
    // may be called from the end of start, or any time later
    private finishedWork()
        if (next!=null) next.start();

Abstraction A both provides and accepts the interface. This allows it to be wired before or after any of its peer abstractions. In ALA we use the word 'accepts' rather than 'requires' because there is often an end to a chain of abstraction instances wired together. If no next interface is wired in, the activity flow ends.

Abstraction B would be written in the same way, as it also knows about the Activity flow interface:

 Abstraction B : IActivity
    private IActivity next = null;

    public IActivity wireTo(IActivity _next)
        next = _next;
        return _next;

        // start work

    // code that runs when work is finished.
    // may be called from the end of start, or asychronously later
    private finishedWork()
        if (next!=null) next.start();
As an aside, in C# projects, we wrote wireTo as an extension method for all objects. It used reflection to look at the private interface variables in the source class and the interfaces provided by the destination class. It would then match up the interface types and do the wiring automatically. It could even use port names to explicitly wire ports of the same types.

Now let’s revisit the molecule analogy. By now we would know to put the knowledge that Oxygen is bonded to two Hydrogens inside the water abstraction where it belongs.

diagram 17.png

In terms of knowledge dependencies it means this:

diagram 18.png

The programming paradigm here is a polar bond. It is more abstract (more ubiquitous and reusable) than any particular atom. We could have a second programming paradigm, a covalent bond, as well. Again, the important thing here is not what the code does - that is arbitrary (and not actually correct chemistry) but how the atoms can be made to interact while retaining their abstract properties with only design-time knowledge dependencies:

Abstraction PolarBond
 Abstraction Oxygen
    private PolarBond hole1 = null;
    private PolarBond hole2 = null;

    public Oxygen wireIn(PolarBond _pb)
        if (hole1==null) hole1 = _pb; else
        if (hole2==null) hole2 = _pb;
        return this;

    public Initialize()
        if (hole1!=null) { hole1.getElectron(); BecomeNegativelyCharged(); }
        if (hole2!=null) { hole2.getElectron(); BecomeNegativelyCharged(); }
 Abstraction Hydrogen : PolarBond
 Abstraction Water
    new Oxygen()
        .wireTo(new Hydrogen())
        .wireTo(new Hydrogen())

Let’s do one more example, this time with a Data-flow programming paradigm. I have found that data-flow is the most useful programming paradigm in practice. It is useful in a a large range of problems.

Let’s construct a thermometer. Assume we already have in our domain several useful abstractions: an ADC (Analog Digital Converter) that knows how to read data from the outside world, a Thermistor abstraction that knows how to linearise a thermistor, a Scale abstraction that knows how to offset and scale data, a filter abstraction that knows how to smooth data, and a display abstraction that knows how to display data.

All these domain abstractions will use the Data-flow programming paradigm. Note that none of them know anything about a Thermometer, nor the meaning of the data they process.

So we can go ahead and create a Thermometer application just by doing this:

diagram 19.png

Note that we configure all the abstraction instances for use in the Thermometer by adding configuration information into rows on the instances.

When we manually compile the diagram (assuming we don’t have automated code generation), it might look something like this (again using fluent coding style):

Abstraction Thermometer
   new ADC(Port2, Pin3)
       .wireTo(new Thermister().setType('K').setInputRange(20,1023)
           .wireTo(new Scale(32,0.013)
The configuration setters and the WireTo extension method return the object on which the call is made to support the fluent coding style.

The diagram is the requirements, the solution and the architecture of the application, and is executable. The diagram has all the cohesive knowledge that is a thermometer, and no other knowledge.

The diagram can be read stand-alone, because all the dependencies in it are knowledge dependencies on abstractions we would already know in the domain.

Let’s say when the Thermometer runs, there is a performance issue in that the ADC is producing data at 1kHz, and we don’t need the display to be showing Temperatures at that rate. Also the temperature readings are noisy (jumping around). Let’s make a modification to the Thermometer by adding a filter to reduce the rate and the noise:

diagram 20.png

If the domain abstractions are not already implemented, we have got the architecture to the point where we can ask any developer to implement them, provided we first give them knowledge of ALA and of the programming paradigm(s) being used.

But let’s look how the data-flow paradigm might work.

If you are familiar with RX (Reactive extensions) with a hot observable source (which is an example of the wiring pattern), this is similar in concept although RX tries to have duality with for-loops iterating through the data. The data-flow paradigm we set up here will just be a stream of data. The IDataFlow interface corresponds to IObserver, and the wireTo method corresponds to the Subscribe method.
The ideal would be a language where we don’t have to decide if the data-flow will be push or pull, synchronous or asynchronous, buffered or unbuffered or other characteristics of communications. The abstractions would not need to know these things - they would just have logical I/O ports, and the type of communications could be binded in at compile-time as part of the performance configuration of the system.
Later we will introduce an asynchronous (event driven) execution model. It is preferable to do the data-flow paradigm interface using that because it allows better performance of other parts of the system without resorting to threads.

For simplicity, we will just implement a synchronous push system. Again, don’t worry about the filter itself. The code is just there to see how the LowPassFilter fits in with the Data-flow programming paradigm, and how simple doing that can be.

Interface IDataFlow<T>
   push(T data);
 /// LowPassFilter is a Data-Flow paradigm decorator to be used in an ALA archtecture.
 /// 1. Decimates the incoming data rate down by the setCutoff configuration
 /// 2. Smooths the data with a single pole filter with cutoff frequency equall to the input frequency divided by the cutoff. T must be a numeric type.
 /// Normal checks and exceptions removed to simplify
 Class LowPassFilter<T> : IDataFlow<T>
    private Dataflow next;

    // This is normally done by a general extension method
    public IDataflow wireTo(IDataflow _next)
        next = _next;
        return _next;

    integer cutoff;

    setCutoff(integer _cutoff)
        cutoff = _cutoff;

    int count = 0;
    T filterState = NAN;

    IDataFlow.push(T newData)
        if (filterState==NAN) filterState = newData * cutoff;
        filterState = filterState - filterState/cutoff + newData;
        if (count==cutoff)
            count = 0;
            if (next!=null) next.push(filterState/cutoff);

You will notice that both the Domain abstraction, Filter, and the Programming Paradigm abstract interface, IDataFlow, use a parameterised type. This makes sense because only the application, the Thermometer, knows the actual types it needs to use.

6.7. Expression of requirements

One of the fundamental aspects of ALA is that the abstraction level of the application is fixed and defined by:

The succinct description of requirements

This is a similar concept to a DSL (but not quite the same). If the abstraction level were more specific, we wouldn’t have the versatility to describe changing requirements or new applications in the domain (too expressive). If it were were more general, we would have to write more code to describe the requirements (not expressive enough).

I noticed during 40 years of code bases written at our company, two did not deteriorate under maintenance. They always remained as easy to maintain as they were in the beginning, if not easier. All others deteriorated badly. Some deteriorated so badly that they could no longer be maintained at all. At the time we din’t know why and could not predict which way it would go. It seemed as if you just got lucky or unlucky.

Perhaps it was the type of changes that came along? But the two code bases that were easy to maintain seemed to be easy for any kinds of change. And the ones that were hard were hard for any change. This continued to hold for years on end. Of course, most changes were changes to requirements, but often enough, changes would be for performance or other reasons. These also seemed easy in these two code bases, but hard everywhere else.

I began to look at the structure and style of the easy and hard code. The easy code was not complicated while the hard code had degenerated well into the complex. The two easy code bases were doing very different things in very different ways, so there was apparently not a common structure or style. But they did have one thing in common. The code that represented the knowledge of the requirements was separated out. That code only described requirements, and it was expressed in terms of other things that were relatively independent, reusable and easy to understand (what we call abstractions).

This is what first gave rise to one of the core tenets in ALA. The first separation is not along the lines of functional or physical parts of the system, such as UI, Business logic, and Data model. The first separation is code that just describes requirements.

Of course this has a strong parallel with how DSLs work. Is ALA just DSLs? There are several differences. Firstly in ALA we don’t try to create a sandbox language for a domain expert to maintain applications. We don’t go as far as an external DSL. It’s for the developer and we don’t want to cut him off from the power he already has when it is needed. We just give him a way to organise the code and a process to get him there - describe the requirements knowledge in terms of abstractions and then trust that those abstractions, when written, will make it work.

6.8. No two modules know the meaning of data or a message.

The two modules will have collaborative knowledge. We reason that the sender must know the meaning to formulate the message, and the receiver must know the meaning to interpret the message. So how can it be avoided? The answer is to make the sender and receiver in same abstraction. They both know the same knowledge, so they are cohesive, so they should be together. In the logical view of the system, they are two instances of the one abstraction. We let the physical view fact that the sender and receiver will be deployed in different places drive them to be different modules.

6.9. Expressiveness

Requirements are usually understated initially in terms of abnormal conditions. However, they are usually communicated quite quickly relative to the time to write the code. In ALA, they are separately represented. The precise expression of the requirements using the right programming paradigms should take about the same amount of information as the English explanation of them.

In general, ALA probably requires about the same amount of total code. But once the requirements are represented, the domain abstractions are known and they are independent small programs with dependencies only on the programming paradigm interfaces used. This independence should make them much easier to write. As the system matures, the effort to modify gets less as more domain abstractions come on line as tested, mature and useful building blocks. The final cost of maintenance should be much less than an equivalent ball of mud architecture.

6.10. No models

Leave out details only inside abstractions

It is generally accepted that a software architecture must, by necessity, leave out some details. Somehow we need to find a satisfactory architecture without considering all the details. Often models are used to represent the architecture. Like its metaphor in the real world, a model leaves out details. The problem is they can leave out arbitrary details. We can’t be sure that some omitted detail won’t turn out to be important to the architectural design.

ALA therefore does not use the model metaphor. Instead, it uses diagrams (if not plain old text). Of course, this distinction comes down to semantics. I define a diagram as different from a model in that it does not leave out details arbitrarily. The only way to leave out details in an ALA diagram is inside the boxes, in other words inside abstractions. Because abstractions already have the required meaning when used in the diagram, the details omitted can’t be important to the diagram, and can’t affect the architectural design.

6.10.1. Executable architecture

Your architecture should be executable

The distinction between diagrams and models explained in the previous section gives rise to an interesting property of the ALA architecture. Diagrams are executable. Therefore the architecture itself will be executable. When the implementation of the abstractions is complete, there will be no work left to do to make the architecture execute (apart from practical considerations of bugs, misinterpretations of the requirements, performance issues, improvements to the initially conceived set of domain abstractions, and the like).

There should be two aspects of an architecture, the meta-architecture and the specific architecture. If using ALA, ALA itself is the meta-architecture and the top level application diagram is the specific architecture.

If your specific architecture is executable, it is also code. There is no separate documentation or model trying to act as a second source of truth.

6.10.2. Granularity

The final architecture of your software will consist only of abstractions. These abstractions will need to be independently readable and understandable. To meet this need, all of the abstractions will be small, even the 'system level' ones.

Conversely, none should be too small. We want them small enough to allow the human brain to understand them, but there is no need for them to be smaller, or we will just end up with an inordinate number of them. This inordinate number will tax the brain in a different way, by causing it to have to learn more abstractions than necessary in a given domain.

The ideal abstraction size is probably in the range of 50 to 500 lines of code.

6.10.3. Modules, Components, Layers

The common terms, modules, components, or layers often result from a decomposition process and therefore are parts of a specific system. The system may have only one of each type. The parts have a lower abstraction level than the system because they are just specific parts of it. In ALA we want to reverse this so that parts are more abstract than the system.

But say you do end up with some single use abstractions and implement it in a static way, it is important to still see these entities as two aspects in one: an abstraction and an instance.

6.11. Abstraction Layers

6.11.1. Layers pattern

With only design-time knowledge dependencies to deal with, layers are used for organising these dependencies so that there are no circular dependencies, and that they all go toward more abstract, more stable abstractions. As the name "Abstraction Layered Architecture" suggests, layers are crucially important to ALA.

In the section on the wiring pattern we ended with three layers:

diagram 21.png

There is a Layers pattern that also controls dependencies, but since most systems have numerous run-time dependencies between elements represented as design-time dependencies, these layers are used for the run-time dependencies. It is usually explained that each layer is built on services provided by the layer below it.

One example is the UI/Business Logic/Data model. Another example is the OSI communications model, where the layers are Application, Presentation, Session, Transport, Network, Data link, and Physical. In ALA, each of these ends up being turned 90 degrees. Metaphorically they become chains. In ALA each component wouldn’t know about the components next to it. That applies symmetrically, to the left and to the right. Data goes in both directions. At run-time, everything must exist for the system to work. It doesn’t really make sense to use a asymmetrical layers metaphor.

The design pattern for layers does have one or two examples of layering used by knowledge dependencies. The term ‘layer’ is therefore an overloaded term in software engineering. When used for knowledge dependencies, the English term 'layer' is a better metaphor. If a lower layer of a wall were to be removed, the layers above would literally collapse, and that’s exactly what would happen in knowledge dependency layering. The layers above literally need the knowledge of abstractions in lower layers to make any sense.

ALA’s ripple effects are already under control because the only dependencies are on abstractions, which are inherently stable, and furthermore, those abstraction must be more abstract. However, to make these dependencies even less likely to cause an issue during maintenance, we try to make the abstraction layers discrete, and separated by approximately an order of magnitude. In other words each layer is approximately an order of magnitude more abstract than the one above it. More abstract means more ubiquitous, so the layers contain abstractions which have greater scope, and greater potential reuse as you go down the layers.

We won’t need many layers. If you think about abstraction layers in the real world, we can get from atoms to the human brain in four layers. Remember the creativity cycle early in this article. We only need to go around the cycle four times to make a brain: Atoms, Molecules such as proteins, Cells such as neurons, neural nets, and finally the brain itself.

6.11.2. The four layers

We start with four layers. They have increasing scope as you go down. This type of layering was described by Meiler Page-Jones. Meiler Page-Jones’ names for the four layers are: "Application domain", "Business domain", "Architecture domain", and "Foundation domain".

Layers diagram
Figure 30. Four ALA layers

ALA uses slightly different names: Application layer, Domain Abstractions layer, Programming Paradigms layer, and Language layer.

Application layer

The top layer has knowledge specific to the application, and nothing but knowledge specific to the application, i.e. representing your requirements.

A simple Application might wire a grid directly to a table. When Business logic is needed, any number of decorators (that do validation, constraints, calculations, filtering, sorting, etc.) can be inserted in between the grid and the table by changing the wiring of the application.

Domain abstractions layer

Knowledge specific to the domain goes in this layer. A domain might correspond to a company or a department. As such, teams can collaborate on the set of abstractions to be provided there.

Applications have knowledge dependencies reaching into this layer.

Programming Paradigms layer

All knowledge specific to the types of computing problems you are solving, such as execution models, programming paradigm interfaces and any frameworks to support these, is in this layer.

The Programming Paradigms layer will abstract away how the processor is managed to execute different pieces of code at the right time. Execution models are covered in detail in chapter four.

This layer is also where we arrange for our domain abstractions to have common simple connections instead of having a specific language for each pair of modules that communicate. The Programming Paradigms layer abstracts away ubiquitous communications languages (which we have been referring to as programming paradigms in this article.)

Let’s use the clock as a real world example. (This is the same clock example we used in section 2.9 when introducing the role abstractions play in the creative process.) One of the the domain abstractions for clocks is a cog wheel. Cog wheels communicate with one another. But they don’t do it with communications languages specific to each pair, even though each pair must have the correct diameters and tooth sizes to mesh correctly. The cog abstraction just knows about the common paradigm of meshing teeth, a more abstract language in this lower layer. This language is analogous to a programming paradigm. With it, the clock abstraction (which is in the highest layer) can then instantiate two cogs and configure them to mesh. The concept of cog thus remains an abstraction and instances of it are composable. The clock, which already knows that two instances of cogs are needed, also knows where they will be fitted and what their diameters must be. The knowledge in the clock abstraction is cohesive.

Language layer

The language layer is included to show what is below the other three layers. It is not hardware as you would find in many other layering schemes, nor is it a database, because it is not run-time dependencies we are layering. The lowest layer has the remaining knowledge you need to understand your code, that of the languages, libraries and any very generic APIs you may use.

The hardware and database do have a place, but we will cover it later. Being a run-time dependency, it will be well off to one side and slightly higher up.

Domain Abstractions API

The boundary between the application layer and the domain abstractions layer is an API that supports the solution space of your requirements (within the constraints of your domain).

The scope of the Domain Abstractions layer defines the expressiveness available to the application. The greater the scope (or bigger the domain), the more applications are able to do. The cost is expressiveness. The applications will have to be longer to specify what is to be done. Conversely, a smaller domain allows less versatility in the applications, but there is greater expressiveness, which means you write less code.

Possible extra layers

The domain is an approximation of all the potential applications and all the modifications you are likely to make. If the domain is large because it is enterprise wide, you could have an additional layer for small domains. The enterprise domain would include enterprise wide abstractions such as a person identity, and the smaller domains would add additional, more specific abstractions, such as a customer (by composition).

If the applications are large and themselves need to be composed of features, an additional layer that supports plug-in style abstractions may work well. Plug-in abstractions may actually be instances of domain abstractions, such as a settings Menu, or a customer Table. A feature can then add settings to the menu, or columns to the table that remain unknown to any other features.

Programming Paradigms API

The boundary between all higher layers and the Programming Paradigms layer is another API. It separates the domain knowledge from the programming paradigm implementation knowledge. It almost always takes care of the ‘execution flow’, the way the computer CPU itself will be controlled to execute all the various parts of the code and when, often using a framework. On the other hand, the Programming Paradigms layer doesn’t necessarily have any code at all. Remember that the layers are ‘knowledge dependencies’, not run-time dependencies, so the paradigm could be a ‘computational model’ that just provides the knowledge of patterns of how to constracut the code in higher layers. The decisions about use of the patterns and about the way the code is executed have already been made and exist in the Programming Paradigms layer.

Rate of change of knowledge

The knowledge in each of the four layers has different change rates.

  • The Language layer contains knowledge that will likely change only a few times in your career.

  • The Programming Paradigms layer knowledge changes when you move to different computing problems types, or discover different approaches to solving a broad range of problems. For example, if you have not yet used an event driven execution model or state machines in your career, and you move into the embedded systems space, you will very likely need to have those skills.

  • The Domain Abstractions layer has knowledge that changes when you change the company you work for. It will change at the rate that the company’s domain is changing, or is becoming better understood. If your company uses lean principles, one of the things you want to do is capture knowledge for reuse. This is the whole point of the Domain Abstractions layer, it is a set of artefacts that capture the company’s reusable knowledge.

  • The Application layer has the fastest changing knowledge, the knowledge that changes at the rate that an application gets maintained.

6.12. ALA is a logical view

If the system is deployed on multiple machines (this is the subject of the physical view), the ALA abstractions, layers and diagrams all remain identical. A simple application diagram connecting a temperature sensor to a display field does not change if the sensor happens to be on a Mars Rover and the display field is at JPL.

Ideally, the performance view also does not affect the ALA logical view. This is a many faceted problem that we will return to later.

ALA usually works very well with aspects of the development view as discussed elsewhere. For example, the fact that domain abstractions have zero coupling greatly helps the allocation of teams. The teams need only cooperate on a common understanding of the programming paradigms used.

6.13. No separation of UI

In ALA we don’t separate the UI unless there is a reason to do so. The amount of knowledge in the UI that comes from a particular application’s requirements is usually quite small and that knowledge is usually quite cohesive and coupled with the business logic of the feature it belongs with. For example, the layout of the UI is a small amount of information, and the bindings of the UI elements to data are a small amount of information. So all that cohesive knowledge is kept together, encapsulated inside a feature. Instead, the UI is composed from Domain UI abstractions. Being domain specific, these abstractions have a little more knowledge to them than generic widgets. For example, their domain knowledge may include style, functionality and suitability to their domain context. For example, a softkey or menu item will have an appearance, functionality and suitability to the way UIs are designed in the domain. Using one in a specific application only requires a label and a binding to an action. They will also provide consistency in the domain.

If there is an actual requirement to have different UIs, say a command based UI and a GUI based UI, then you just abstract the UI abstractions further until they can be used either way. The UI abstractions still remain an integral part of the application.

In the example project for this chapter, we will for the first time use multiple programming paradigms, a usual thing in real ALA projects.

6.14. Features

You may have noticed throughout this article the word 'features' being used quite often instead of 'Application'. When the application is large, we can think of it as a composition of feature abstractions. This is exactly what happens in natural language in the domain when describing requirements. 'Features' is just the word we give the natural abstractions in the requirements, without even realizing it. Just go with this in the software itself.

6.15. Horizontal domain partitions

Say you are implementing a particularly large domain abstraction such as a 'Table', or are implementing a complicated programming paradigm. We would like to break these up into smaller components. Do we introduce a fractal type of structure to deal with this? Should we have hierarchical layers within layers contained completely inside the Table abstraction?

The astute reader will have noticed the non-ALA thinking in the statement "break these up into smaller components". In ALA we don’t decompose a large abstraction into components, we compose it from abstractions, which if necessary we invent as we go. These new abstractions will have a scope or level of ubiquity, stability and reuse that corresponds to one of the existing layers. So there should be no hierarchical or fractal structures in ALA.

However, the domain that these new abstractions are in won’t be the same domain as the one that provides for the writing of Application requirements. For example, the implementation of the Table abstraction will need to be connected to another abstraction in the domain of databases. One of the abstractions in that domain will know about a particular database, say SQL Lite. A polymorphic interface should exist between the two. That interface, being more abstract than either the Table or the SQL Lite abstractions, will be in the next layer down, where both the Table and the MySQL abstractions can have a knowledge dependency on it. Of course the SQL abstraction will actually be further composed of an adapter and a real database.

Some application domain abstractions are complicated. Examples of these are abstractions requiring a connection to an actual database, actual hardware, the Internet, etc. Implementing these will typically wire out horizontally into other technical domains. You can visualise them going in multiple directions, which is exactly the idea of Alistair Cockburn’s hexagonal architecture.

diagram 22.png

A communications domain using a OSI model may end up with a whole chain of communications domain abstractions going sideways:

diagram 23.png

The technicalities may be incorrect but the diagram gives the idea of how the OSI 'layers', which are just run-time dependencies, would fit into the ALA layers.

6.16. No hierarchical design

ALA does not use any form of hierarchical structure. Instead it uses abstraction layers, together with "Horizontal domain partitions" discussed earlier.

6.17. Product owner perspective


6.18. Reuse


6.19. Documentation


6.20. Symbolic indirection

Avoid use of symbolic indirection without abstraction

When we start assembling requirements from abstractions, a topic that we will cover in coming sections, we will be using symbolic indirection, such as function calls or the new keyword with a class name. Unless a symbolic indirection is to an abstraction, they are for the compiler to follow at compile-time, not for the code reader to follow at design-time. Understanding the code relies on allowing the reader to read a small cohesive block of code. The reader should never have to follow the indirection somewhere else. If you don’t achieve this, and abstraction is the only way you can, then any decoupled architecture will be more difficult to read.

Abstraction allows indirection while allowing the reader to continue reading on to the next line. The importance of this property cannot be overstated. As soon as we start thinking in mere programming language terms of modules, components, interfaces, classes, or functions, the abstraction will start to be lost. These other artefacts may have benefits at compile-time (the compiler can understand them), but that is not useful at design-time unless they are also good abstractions.

It would be nice if your compiler could tell you that you have a missing abstraction, just as it does for a missing semicolon, but alas, they are not capable of understanding abstractions yet. So it is still entirely up to you.

Abstraction is almost a black and white type of property. It’s either there or it isn’t. If the reader of your code does not have to follow the indirection, you have it.

Footnote: When the reader of your code meets your abstraction for the first time (usually a domain abstraction in a domain they have recently come into), ideally their IDE will give them the meaning in a little pop-up paragraph as their mouse hovers over any of its uses. Depending on the quality of the abstraction, after a single exposure, their brain will have the insight, like a light coming on, illuminating a meaning. The brain will form a new neuron to represent the concept. Since the reader will hopefully remain in the domain for some time, this overhead to readability shouldn’t be large.

6.21. Everything through interfaces

A class, in contrast to an abstraction, has an interface comprising all the public members. In ALA we only want this interface to be used by the application when it instantiates and configures an instance of an abstraction. All other inputs and outputs that are used at run-time are done through interfaces (abstract interfaces).

6.22. What do you know about?

Whenever I have only two minutes to give advice on software architecture, I use this quick tip. The tip is ALA reduced to its most basic driving principle.

Ask your modules, classes and functions:

What do you know about?

The answer should always be "I just know about…​".

The anthropomorphization helps the brain to see them as abstractions. The word 'knows' is carefully chosen to imply a 'design-time' perspective.

  1. It’s a restatement of the SRP (Single Responsibility Principle). Every element should know about one thing, one coherent thing. Furthermore, no other elements should know about this one thing.

  2. An element may know about a single hardware device.

  3. An element may know about a user story.

  4. An element may know about a protocol.

  5. An element may know an algorithm.

  6. An element may know how to do an operation on some data, or the meaning of some data, but not both.

  7. An element may know a composition of other elements.

  8. An element may know where data flows between other elements.

  9. No element should know the source or destination of its inputs and outputs.

6.23. Example project - Game scoreboard

For the example project for this chapter, we return to the ten-pin bowling and tennis scoring engines that we used in Chapter two, and add a scoreboard feature (well a simple ASCII scoreboard in a console application rather than real hardware).

As the requirement, say we want a console application that displays ASCII scoreboards that look like these examples:


|   1 |   2 |   3 |   4 |   5 |   6 |   7 |   8 |   9 |    10  |
| 1| 4| 4| 5| 6| /| 5| /|  | X| -| 1| 7| /| 6| /|  | X| 2| /| 6|
+  +--+  +--+  +--+  +--+  +--+  +--+  +--+  +--+  +--+  +--+--+
|   5 |  14 |  29 |  49 |  60 |  61 |  77 |  97 | 117 |   133  |

|   1 ||  4 |  6 |  5 |    |    ||    30  |
|   2 ||  6 |  4 |  7 |    |    ||  love  |

As usual in ALA, our methodology begins with expressing those requirements directly, and inventing abstractions to do so. So, we invent a 'Scorecard' abstraction. It will take a configuration which is an ASCII template. Here are the ascii templates that would be used for ten-pin and tennis:

|   1   |   2   |   3   |   4   |   5   |   6   |   7   |   8   |   9   |     10    |
+   +---+   +---+   +---+   +---+   +---+   +---+   +---+   +---+   +---+   +---+---+
|  T0-  |  T1-  |  T2-  |  T3-  |  T4-  |  T5-  |  T6-  |  T7-  |  T8-  |    T9-    |
| M0  ||S00 |S10 |S20 |S30 |S40 || G0---  |
| M1  ||S01 |S11 |S21 |S31 |S41 || G1---  |

The scorecard ASCII template has letter place-holders for the scores. (A single letter is used so it doesn’t take up much space on the template design.) Different letters are used for different types of scores. Digits are used to specify where multiple scores of the same type are arranged on the scoreboard. They are like indexes. Either 1-dimensional or 2-dimensional indexes can be used in the scoreboard template. For example, the frame scores in ten-pin bowling have scores for each ball for each frame, F00, F01 etc, as shown in the example above.

The scorecard abstraction needs functions it can use to get the actual scores. The functions are configured into little 'binding' objects that we then wire to the scoreboard. The binding objects are configured with the letter that they return the score for.

6.23.1. Ten-pin

Having invented the Scorecard and Binding abstractions, we can now do the ten-pin application diagram:

diagram bowling 3.png

An abstraction we didn’t mention yet is the ConsoleGameRunner. Its job is to prompt for a score from each play, display the ASCII scoreboard, and repeat until the game completes.

The 'game' instance of the Frame abstraction on the right of the diagrams is the scoring engine we developed in Chapter Two. Together with this engine, we now have a complete application.

The rounded boxes in the diagram are instances of domain abstractions as usual for ALA diagrams. The sharp corner boxes are instances of Application layer abstractions. They are the mentioned functions for the Bindings. That code is application specific so goes in the application layer. They just do a simple query on the scoring engine.

Now tranlate the diagram into code. Here is the entire application layer code for ten-pin:

consolerunner = new ConsoleGameRunner("Enter number pins:", (pins, engine) => engine.Ball(0, pins))
.WireTo(new Scorecard(
"-------------------------------------------------------------------------------------\n" +
"|F00|F01|F10|F11|F20|F21|F30|F31|F40|F41|F50|F51|F60|F61|F70|F71|F80|F81|F90|F91|F92|\n" +
"|    ---+    ---+    ---+    ---+    ---+    ---+    ---+    ---+    ---+    ---+----\n" +
"|  T0-  |  T1-  |  T2-  |  T3-  |  T4-  |  T5-  |  T6-  |  T7-  |  T8-  |    T9-    |\n" +
.WireTo(new ScoreBinding<List<List<string>>>("F",
    () => TranslateFrameScores(
        game.GetSubFrames().Select(f => f.GetSubFrames().Select(b => b.GetScore()[0]).ToList()).ToList())))
.WireTo(new ScoreBinding<List<int>>("T",
    () => game.GetSubFrames().Select(sf => sf.GetScore()[0]).Accumulate().ToList()))

If you compare this code with the diagram, you will see a pretty direct correspondence. Remember 'game' is the reference to the scoring engine project in the previous chapter.

That’s pretty much all the code in the application. Oh there is the 'translate' function, but it is pretty straight forward once you know the way a ten-pin scorecard works. For completeness here it is.

/// <summary>
/// Translate a ten-pin frame score such as 0,10 to X, / and - e.g. "-","X".
/// </summary>
/// <example>
/// 7,2 -> "7","2"
/// 7,0 -> "7","-"
/// -,3 -> "-","7"
/// 7,3 -> "7","/"
/// 10,0 -> "",X
/// 0,10 -> "-","/"
/// additional ninth frame translations:
/// 10,0 -> "X","-"
/// 7,3,2 -> "7","/","2"
/// 10,7,3 -> "X","7","/"
/// 0,10,10 -> "-","/","X"
/// 10,10,10 -> "X","X","X"
/// </example>
/// <param name="frames">
/// The parameter, frames, is a list of frames, each with a list of integers between 0 and 10 for the numbers of pins.
/// </param>
/// <returns>
/// return value will be exactly the same structure as the parameter but with strings instead of ints
/// </returns>
/// <remarks>
/// This function is an abstraction  (does not refer to local variables or have side effects)
/// </remarks>
private List<List<string>> TranslateFrameScores(List<List<int>> frames)
    // This function looks a bit daunting but actually it just methodically makes the above example tranlations of the frame pin scores
    List<List<string>> rv = new List<List<string>>();
    int frameNumber = 0;
    foreach (List<int> frame in frames)
        var frameScoring = new List<string>();
        if (frame.Count > 0)
            // The first 9 frames position the X in the second box on a real scorecard - handle this case separately
            if (frameNumber<9 && frame[0] == 10)
                int ballNumber = 0;
                foreach (int pins in frame)
                    if (pins == 0)
                    if (ballNumber>0 && frame[ballNumber]+frame[ballNumber-1] == 10)
                    if (pins == 10)

    return rv;

6.23.2. Tennis

So now that we have these domain abstractions for doing console game scoring applications, lets do tennis:

diagram tennis 3.png

I left the code out of the GetGameOrTieBreakScore box as it is a little big for the diagram here. It is similar to the other queries but it must first determine if a tie break is in progress and get that if so. Also it translates game scores from like 1,0 to "15","love".

And here is the code for the Tennis diagram:

consolerunner = new ConsoleGameRunner("Enter winner 0 or 1", (winner, engine) => engine.Ball(winner, 1))
.WireTo(new Scorecard(
        "--------------------------------------------\n" +
        "| M0  |S00|S10|S20|S30|S40|S50|S60|  G0--- |\n" +
        "| M1  |S01|S11|S21|S31|S41|S51|S61|  G1--- |\n" +
    .WireTo(new ScoreBinding<int[]>("M", () => match.GetScore()))
    .WireTo(new ScoreBinding<List<int[]>>("S", () =>
            .Select(sf => sf.GetSubFrames().First())
            .Select(s => s.GetScore())
    .WireTo(new ScoreBinding<string[]>("G", () => GetGameOrTiebreakScore(match)))

If you compare this code with the diagram, you can see a pretty direct correspondence. match comes from the scoring engine project in Chapter two.

6.23.3. Concluding notes

Although the diagrams must be turned into text code to actually execute, it is important in ALA to do these architecture design diagrams first. They not only give you the application, they give you the architectural design by giving you the domain abstractions and programming paradigms as well. If you try to design an ALA structure in your head while you write it directly in code, you will get terribly confused and make a mess. Using UML class diagrams will make it even worse. Code at different abstraction levels will end up everywhere, and run-time dependencies will abound. Our programming languages, and the UML Class diagram, are just not designed to support abstraction layered thinking - it is too easy to add bad dependencies (function calls or 'new' keywords) into code in the wrong places.

Note that at run-time, not all data-flows have to go directly between wired up instances of domain abstractions. The data can come up into the application layer code, and then back down. This was the case when we did the functional composition example in Chapter One. In this application we are doing that with the code in the square boxes that get the score from the engine. The important thing is that all the code in the application is specific to the application requirements.

That completes our discussion of the console applications for ten-pin and tennis. The full project code can be viewed or downloaded here:

7. Chapter seven - ALA compared with…​

7.1. Physical boundaries

I was listening to a talk by Eric Evans where he said that Microservices works because it provides boundaries that are harder to cross. We have been trying to build logical boundaries for 60 years, he said, and failed. So now we use tools like Docker that force us to use say REST style interfaces in oder to have physical boundaries. I have also heard it suggested that using multiple MCUs in an embedded system is a good thing because it provides physical boundaries for our software components. And I think, really? Is that the only way we can be create a logical boundary? I can tell you that multiple MCUs for this reason is not a good idea if only because all those MCUs will need updating, and the mechanisms and infrastructure needed to do that make it not worth it. Unless there is a good reason, such as to make different parts of your code independently deployable, the extra infrastructure required for physical boundaries that are just logical boundaries is not necessary. Furthermore, physical boundaries, like modules do not necessarily make good abstractions. The only boundary that works at design-time is a good abstraction. So ALA achieves it’s design-time boundaries by using abstractions.

7.2. Test Driven Development

It is said that TDD’s main advantage is not so much the testing, but the improvement in the design. In other words, making modules independently testable makes better abstractions. This is probably true, but in my experience, TDD doesn’t create good abstractions nearly as well as pursuing that goal directly. The architecture resulting from TDD is better but still not great.

7.3. Observer pattern and dependency inversion


7.4. Hexagonal Architecture (ports and adapters)

In the previous section we intimated that the sideways chains of interfaces going out in horizontal directions were the same as hexagonal architecture. While ALA shares this aspect of hexagonal architecture, there is still an important difference.

ALA retains domain abstractions of the UI, Database, communication and so on. For instance, in our XR5000 example, we had a domain abstraction for a persistent Table. We had domain abstractions for UI elements such as Page, Softkey etc. We don’t just have a port to the persistence adapter, we have an abstraction of persistence. We don’t just have a port for the UI to bind to, we have abstractions of the UI elements. The implementation of these abstractions will then use ports to connect to these external system components. Why is it important that we have domain abstractions of these external components?

  1. The Database and the UI will have a lot of application specific knowledge given them as configuration. Remember the creativity cycle. After instantiation of an abstraction comes configuration. The database will need a schema, and the knowledge for that schema is in the application. The Softkey UI elements will need labels, and that knowledge is in the application. By making domain abstractions for persistence and UI, the application can configure them like any other domain abstraction as it instantiates and wires up the application. To the application, these particular domain abstractions look like wrappers of the actual database and UI implementations, but they are more like proxies in that they just pass on the work.

    The Persistence abstraction then passes this configuration information, via the port interface to the actual database. The Softkey abstraction then passes its label, via the port interface, to the softkeys. Otherwise the Application would have to know about actual databases and actual softkeys.

    If you need a design where the UI can change, you just make the UI domain abstractions more abstract. A softkey may be a command abstraction. It is still configured with a label. But it may be connected to a softkey, a menu item, a CLI command, a web page button, or a Web API command.

  2. From the point of view of a DSL, it makes sense to have concepts of UI and persistence and communications in the DSL language. The application is cohesive knowledge of requirements. The UI and the need for persistence are part of the requirements. In fact, for product owners communicating requirements, the UI tends to be their view of requirements. They talk about them in terms of the UI. Many of the product owners I have worked with actually design the UI as part of the requirements (with the backing of their managers, who are easily convinced that software engineers can’t design UIs. PO can’t either, but that is another story.). The point here is that the UI layout, navigation, and connection to business logic is all highly cohesive. We explicitly do not want to separate that knowledge.

    As a restatement of an earlier tenet of ALA, it is much better to compose the application with abstractions of Business logic, UI and persistence than to decompose the application into UI, persistence and business logic.

  3. We want the application to have the property of composability. We have previously discussed how that means using programming paradigm interfaces for wiring up domain abstractions. By using domain abstractions to represent external components, the abstractions can implement the paradigm interfaces and then be composable with other domain abstractions. For example, the Table domain abstraction which represents persistence may need to be connected directly to a grid, or to other domain abstractions that map or reduce it. Indeed, the Table abstraction itself can be instantiated multiple times for different tables and be composed to form a schema using a schema programming paradigm interface. I have even had a table instance’s configuration interface wired to a another Table instance. (So its columns can be configured by the user of the application.)

  4. The fourth reason why it is important for the application to not directly have ports for external components of the system is that we don’t want the logical view of the architecture to become just one part of the physical view. If there is a communications port that goes to a different physical machine where there is more application logic, the application’s logical view should not know about that. It may be presented as an annotation on the application (lines) connecting certain instances, but it shouldn’t split the application up. At the application level, the collaboration between parts instantiated on different machines is still cohesive knowledge and belongs inside one place - the application.

7.5. Layer patterns

7.5.1. MVC


7.5.2. Application, Services, Drivers, Hardware


7.6. Factory method pattern

Let’s say you have a nice Table domain abstraction that is perfect for the requirements you have in your domain. At run-time, the Table abstraction must be wired via a polymorphic interface to a particular database, and the database must be instantiated. We don’t want the Application, the part that composes domain abstractions, to know anything about all this. We want the Application to be able to just use a Table as if it is a self-contained abstraction.

Similarly, in the partitioned off 'Database' domain, there will usually be some knowledge in the top layer to configure a particular database. This knowledge knows nothing of the Application and is not responsible for creating the Tables for it.

Table knows about the polymorphic interface, but doesn’t know it’s for a connection to a real database. This interface could have a method that the Table instance calls when it is instantiated. Table only knows that it has to call this method. However, there is not yet any provider of this interface in place, so we cannot get the database connected up this way. But we do want to wait until the first Table abstraction is instantiated, otherwise we would not need to spin up a database at all.

Instead, Table knows to instantiate a Factory design pattern object. To the Table, this action it must do is logically just part of the same polymorphic interface. It doesn’t even know it is a factory. If the interface is called 'IOutsideConnection', the factory could be called something like OutsideConnection and the method inside it called getAnOutsideConnection. It could be a static class or a singleton. Table just saves the object that it gets in the interface reference.

IOutsideConnection outside = (new OutsideConnection()).getAnOutsideConnection();

If it is the first time the factory object is used, it could, for example, invoke an object in the top layer (via dependency inversion, the object has already registered itself with the factory) that contains the configuration knowledge about the database. The object in the top layer instantiates a database domain abstraction to give to the Table.

7.7. Interface segregation principle


7.8. Open Closed Principle and decorators


7.9. Bridge pattern


7.10. Architecture styles

I am not an expert at these so called 'Architectural styles'. Any feedback about the accuracy of the following comparisons would be appreciated.

7.10.1. Presentation, Business, Services, Persistence, Database


7.10.2. Presentation, Application, Domain, Infrastructure

The middle two layers appear to be the same as ALA’s. The Presentation (UI) only has run-time dependencies on the Application, and the Domain layer only has run-time dependencies on the Infrastructure (Persistence etc), so these layers are not present in ALA.

Instead Presentation is done in the same way as the rest of the application, by composing and configuring abstractions in the domain. The meaning of composition for UI elements (typically layout and navigation-flow) is different from the meaning of composition in the use-cases (typically work-flow or data-flow).

In ALA, the foundation layer is also done in the same way as the rest of the application, at least a little. Domain abstractions that represent say a persistent table are in the Domain layer. The composition and configuration of them again goes in the Application layer. This time the meaning of composition is, for example, columns for the tables and schema relations.

If the implementation of any domain abstraction is not small (as is the case with the persistent Table abstraction mentioned above, which will need to be connected to a real database), it will be using other abstract interfaces (in the Programming Paradigms layer) connected to its runtime support abstractions in a technical domain, the same as in Hexagonal Architecture.

7.10.3. Object Oriented Programming

From my reading, it seems that the most characteristic feature of OOP is that when data and operations are cohesive, they are brought together in an object. Others may see it as enabling reuse, inheritance, and still others may see it as polymorphism. New graduates seem to be introduced to polymorphism in inheritance and not be introduced to interfaces at all, which is a shame because the concept of interfaces is much more important.

I have never been an expert at Object Oriented Design as I found the choice of classes difficult and the resulting designs only mediocre. But I think the most fundamental and important characterising feature of OOP is under-rated. That is the separation of the concepts of classes and objects. This separation is not so clearly marked when we use the terms modules or components. The separation is fundamentally important because it’s what allows us to remove all dependencies except knowledge dependencies. In the way described earlier in this article, you can represent the knowledge of most dependencies as a relationship between instances completely inside another abstraction. What OOP should have done is represent relationships between objects completely inside another class. The problem is that OOP doesn’t take advantage of this opportunity. Instead, it puts these relationships between objects inside those objects' classes, as associations or inheritance, thereby turning them into design-time dependencies, and destroying the abstract qualities of the classes. Abstractions, unlike classes, retain their zero coupling with one another.

ALA addresses the problem by calling classes abstractions and objects instances. Abstractions differ from classes by giving us a way to have logical zero coupling, as if they were on different physical platforms. Instances differ from objects by having ports because their classes give them no fixed relationships with other objects.

Of course, when you are writing ALA code, abstractions are implemented using classes, but you are not allowed associations or inheritance. Instances are implemented as objects but with ports for their connections. A port is a pair of interfaces that allow methods in both directions. The interfaces are defined in a lower layer.

In ALA, the UML class diagram completely loses relevance. Because classes have no relationships with each other, bar knowledge dependencies, a UML diagram in ALA would just be a lot of boxes in free space, like a pallet of things you can use. You could show them in their layers and you could even draw the downward composition relationships that represent the knowledge dependencies, but there would be no point to this except in explaining the concepts of ALA. When you are designing an actual system, the real diagram is the one inside of an abstraction, especially the uppermost one, the application. It shows boxes for instances of the abstractions it uses, with the name of the abstraction in the box, the configuration information for those instances, and of course the lines showing how they are wired together. The names inside the boxes would not even need to be underlined as in UML, because the boxes in such diagrams would always be instances.

Such a diagram is close to a UML object diagram. However, a UML object diagram is meant to be a snapshot of a dynamic system at one point in time. In ALA, any dynamic behaviour is captured in a static way by inventing a new abstraction to describe that dynamic behaviour. Thus the design-time view is always static. So the object diagram is static. The application class specifies a number of objects that must be instantiated, configured, and wired together to execute at run-time. Since the structure is always static, ideally this would be done by the compiler for best efficiency, but there is no such language yet. So, in the meantime, it is done at initialization time. The object diagram can be fairly elegantly turned into code using the fluent coding style shown in the XR5000 example.

7.11. DSLs

We briefly discussed ALA as a DSL in the structure chapter here

ALA includes the main idea of DSLs in that the fundamental method "represent[s] requirements by composition of domain abstractions". It shares the DSL property that you can implement a lot more requirements or user stories in a lot less code.

But ALA only tries to be a light-weight way of telling ordinary developers how to organise code written in your underlying language. Although the domain abstractions do form a language and the paradigm interfaces give it a grammar, ALA doesn’t pursue the idea of a language to the point of textural syntactic elegance. Instead, you end up with explicit wiring methods to combine domain entities, or plain old functional composition, or some other form of composition in the wider sense of the word. Often, the text form is only a result of hand translation of an executable diagram. ALA certainly doesn’t overlap with DSLs to the extent of an external DSL, nor does it try to sandbox you from the underlying language. It therefore does not require any parsing and doesn’t need a language workbench, things that may scare away 'plain old C' developers.

Like DSLs, ALA can be highly declarative depending on the paradigm interfaces being used to connect domain abstractions. It is better to have the properties of composition and composability in the your domain language even if they may not be in a perfectly elegant syntactic form. ALA may end up composing abstractions with calls to wireTo methods instead of spaces or dots. But often a diagram using lines is even better than spaces and dots.

In DSLs, it is important that different languages can be combined for different aspects of a problem. For example, a DSL that defines State machines (the state diagram) and a DSL for data organisation (Entity Relationship Diagram) may be needed in the same application. You don’t want to be stuck in one paradigm. ALA recognises this importance by having paradigm interfaces that are more abstract than the domain abstractions.

DSLs probably work by generating a lot of code from templates whereas ALA works by reusing code as instances of abstractions. Both of these methods are fine from the point of view of keeping application specific knowledge in its place, and domain knowledge in its place. Howver, the distinction between ALAs domain layer and programming paradigms layer is probably not so as clearly made in the implementation of the templates.

It is an advantage of DSLs that they can sandbox when needed. An example from the wiring pattern earlier is that the ports of instances do not need to be wired. Therefore, all abstractions need to check if there is something wired to a port before making a call on it. Enforcing this is a problem I have not yet addressed.

A possible solution, albeit inferior to a real DSL that would tell you at design-time, might be that when there are tools that generate wiring code from diagrams, they automatically put stubs on all unwired ports. These stubs either throw an exception at run-time, or just behave inertly.

ALA is different from external DSLs. ALA is just about helping programmers organise their code in a better way. It doesn’t try to make a syntactically elegant language, as a DSL does. Certainly an external DSL will end up representing requirements in a more elegant syntax. But that is not the most important thing in ALA. The most important thing is the separation of code that has knowledge of the requirements, which will cause the invention of abstractions that have zero coupling (because the coupling was really in each requirement - that is why a requirement is cohesive). ALA also avoids taking the average imperative language programmer out of their comfort zone. It does not require a language workbench and does not sandbox you from the underlying language.

ALA probably does fit into the broadest definition of an internal DSL. However, again, it does not target syntactic convenience in the expression of requirements so much as just separating the code that knows about those requirements from the code that implements them. An internal DSL usually aims to have a mini-language that is a subset of the host language, or it tries to extend the host language through clever meta-programming to look as if it has new features. ALA is about abstraction layering. It is about this design-time view of knowledge dependencies: what abstractions in lower layers are needed to understand a given piece of code.

7.12. Dependency injection

7.12.1. Similarities

In ALA you inject run-time required objects via setters.

7.12.2. Differences

ALA uses explicit wiring, never automatic wiring. For one thing, the wiring is required to compose from a pallet of domain abstractions. But secondly, and more importantly, you do not want the knowledge that the wiring represents to disappear into the abstractions themselves, not even as meta-data. That would destroy the abstractions.

In ALA, the explicit wiring can’t be XML or JSON, even if it can be modified at run-time. Usually, because a network structure will be required, the explicit wiring must be a diagram. However, it can be a projection editor, so that the structure is entered in text form (preferably not XML or JSON) and live viewed in graphical form.

In ALA, abstraction pairs don’t have their own interfaces for their instances to communicate. So we don’t have the situation where class A has a dependency on class B, and so an object of class B (or one of its subclasses) is injected into class A. Similarly, we wouldn’t have the situation where class A requires an interface that is implemented by class B.

In ALA the dependencies can only be on paradigm interfaces, which are a whole abstraction layer more abstract. So we need to be thinking that if class A accepts or implements a certain paradigm interface, there could be any number of other abstraction instances that could be attached. Furthermore, we could build arbitrarily large assemblies - composability. Or we don’t have to connect an instance at all. So it doesn’t really make sense to call what we are injecting 'dependencies'. We just think of it as wiring things up, like electronic components.

7.13. Component Based Software Engineering

ALA uses many of the same methods found in component based engineering or the Components and Connector architectural style.

  • Components are Abstractions.

  • Reusable software artefacts.

  • Connection ports for I/O.

  • Composability

  • Both instantiate components, specialize them by configuration, and compose them together to make a specific system.

  • ALA’s 3rd layer has interfaces used to wire abstractions in the 2nd layer, so at a lower level (more abstract) level. They represent something more like programming paradigms. The equivalent pattern in components engineering is "Abstract Interactions".

  • The architecture itself is composed of a generic part and a specific part. The general part is the ALA reference architecture itself and the components or the connectors architectural style. The specific part is the wiring diagram of the full system.

  • Component based engineering technologies such as CORBA primarily solve for platform and language interoperability in distributed system whereas ALA brings some of the resulting concepts and properties to everyday small-scale, non distributed development as well, where the only separation is logical.

  • In ALA there is perhaps more particular emphasis on making components clearly more abstract than the systems they are used in, and making the interfaces clearly more abstract than the components. The components are pushed down a layer and the interfaces down to a layer below that. Then all dependencies must be strictly downwards in these layers. In component based engineering, this structure is not necessarily enforced. If the components are just a decomposition of the system, then the system, components and interfaces may all be at the same level of abstraction, making the system as a whole complex.

  • ALA depends on the 'abstractness property' of components to get logical separation, and so calls them 'Abstractions' and not components to help them retain that property. Even if there will only be one use and one instance, it is still called an abstraction. This keeps them zero coupled and not collaborating with other abstractions they will be wired to.

  • ALA layers are knowledge dependency layers. Components may still be arranged in layers according to run-time dependencies, such as communication stacks. In ALA run-time dependencies are always implemented as explicit wiring inside another higher layer component.

  • ALA’s top layer must be a straight representation of the requirements, whereas components may tend to be decomposed pieces of the system.

  • ALA’s 2nd layer of components are designed for expressiveness of user stories or requirements, and provide DSL-like properties. ALA puts emphasis on the 2nd layer of components having the scope of a domain as the means of explicitly controlling the expressiveness of the pallet of components.

  • ALA is not fractal. In ALA the components of components are abstractions that become more abstract and thus ubiquitous and reusable. ALA therefore uses abstraction layers rather than hierarchies.

  • ALA forces decisions about which abstraction layers the software artefacts go into, and then controls knowledge (semantic) dependencies accordingly.

  • ALA tries to make the abstraction layers discrete and separated by a good margin.

  • ALA puts greater emphasis on wiring being able to represent any programming paradigm that suits the expression of requirements, and the use of many different paradigms in the same wiring diagram.

  • ALA emphasises the cohesion of functional parts of a system such as UI, logic and Data, by bringing them all together in one small diagram using domain level components

  • Instead of 'required' interfaces, in ALA they are called 'accepts' interfaces. This is because the abstractions are more abstract and composable, so, as with Lego blocks, there isn’t necessarily a connection to another instance.

7.13.3. Domain Driven Design

Domain Driven Design’s "Bounded Contexts" and ALA’s Domain Abstractions layer have the same goal, that of encapsulation of the domain specific knowledge.

Domain driven design appears to concentrate on common languages to allow pairs of elements to communicate, which ALA explicitly avoids. ALA tries to abstract the languages so that they are more abstract and fundamental than the domain, and more like programming paradigms.

7.14. Microservices


7.15. Hexagonal Architecture (Ports and Adapters)

ALA includes the basic idea of hexagonal architecture, but with modification using the Bridge Pattern to keep cohesive knowledge belonging to the application from being split. This was explained in an earlier section of this article. [ComparisonHexagonal]

7.16. Architecture evaluation methods

Methods such as ATAM tell us how to evaluate an architecture for quality attributes such as maintainability, for instance by giving it modification scenarios to test how difficult the modifications would be to implement. There are several scenarios based methods to do this such as ATAM. Using this we could, theoretically, iteratively search over the entire architecture design space to find a satisfactory solution. It’s a bit analogous to numerically solving for the maxima of a complex algebraic formula. In contrast, ALA is analogous to an 'algebraic solution'. If the desired quality attributes, and all the software engineering topics listed above are the equations, ALA is the algebraic solution. It simplifies them down into a parameterised template architecture, ready for you to go ahead and express your requirements.

7.17. Monads

In this article, we talked about monads a few times because they are an important example of composition in software engineering. Also, like ALA, they use the concept of separating (in time) composition from execution. You can bind monads together, and it builds a structure that you can later execute. You can wire instances of domain abstractions together, and it builds a structure that you can later execute. In this respect they are the similar.

When you later execute a monad structure (by calling a function on the last monad you binded with), it (usually) returns its value (or values). When you later execute a wired ALA structure, it (usually) starts running continuously, (usually) waiting for events that can appear at various places in the structure and acting on them. Monads can run continuously as well, as for example in reactive extensions with hot observables. Each monad binding is restricted to a data-flow of a single type, and in a fixed direction. Each ALA wiring is arbitrary in its meaning, according to whatever is most useful to describe requirements. A single wired connection can carry data as needed in both, or the composition may be about something other than data-flow.

Often when monads are used, the execution is done immediately following the binding. So the deferred nature of the execution is not always obvious. I found that the separation between composition and execution of monads to be an important aspect to understand when comparing with ALA composition. In ALA all composition takes place at initialization time. There is a very clear separation between that and run-time.

This much separation is not so common with monads. Monads use the separation primarily as a way to do composition with pure functions, and have all the dirty work contained and abstracted away in well tested reusable classes.

Where you might compose (bind) IObservable or Task monads for almost immediate execution following, in ALA you would tend to compose (wire up) data streams or event sources at initialization time that can then execute many-times thereafter.

Another difference is syntax. Monads are composed using a dot operator, a method call, and configured with lambda function passed to the method:


This code filters out values from the source that are negative and then calculates the squareroots. In ALA, because composition is generalized, the syntax would look like this:

source.WireIn(new Filter(x=>x>=0)).WireIn(new Select(x=>sqrt(x))

But usually this code is generated from a diagram.

In functional programming, the binding code that builds the structure is pure functions. When you ask the structure to 'execute' all the dirty code is contained inside these reusable abstractions called Monads. The code that constrauts a particular application is clean and free of side effects. ALA makes use of this same property of reuable abstractions, and its wiring code is pure functional.

7.17.1. Understanding monads

Monads are notoriously hard to learn, but they are nice simple insight once you get there. Monads actually seem to have this property that you cannot understand any explanation of them until you first understand them. Thus it is a bootstrapping problem. Here is my experience of going through that bootstrapping process in case it is useful. I am not going to try to explain monads myself, because, even it was possible, others would do that far better than I would.

  1. First understand that Monads are like physics. Physicists explain that you never really understand physics, you just get used it. Unless you are a mathematician or otherwise gifted, the same is true for monads.

  2. The way to get used to new concepts is to read multiple web-sites on the topic. Read each one until you get lost then swap to another one. Keep going like this. For average concepts like design patterns I use this technique and it requires maybe five websites. For monads it took me maybe ten. You will need to return to some of them iteratively to get further each time.

  3. If you don’t know Haskell, prefer the web sites that explain them in the language you already know.

  4. The common essential ideas in those websites will start to embed themselves in your brain.

  5. Eventually, and fairly suddenly, the simple insight that is monads will happen.

I thought few of the web-sites that I used adequately emphasised the monad property of separation (in time) of composition and execution. They did use examples of it such as IEnumerable and Task. They represent what they can do in the future, without actually doing it now. That’s why the binding functions are called bind in the functional world, because it doesn’t (necessarily) do anything except build a structure that can later be executed to actually do the work.

7.18. Reactive Extensions

In ALA, when you wire together

7.19. WPF’s XAML


7.20. Functional programming


7.21. Functional programming with monads


7.22. Functional Reactive Programming


7.23. Multi-tier Architecture


7.24. Onion Architecture


7.25. Clean Architecture


8. Chapter eight - Surrounding Topics

8.1. Recursive abstractions

ALA enforces a strictly layered (non-circular) knowledge dependency structure. It encourages a small number of abstraction layers at discrete well separated levels of ubiquity as a framework for the knowledge dependencies. This would appear to exclude the possibility of the powerful abstraction composition technique of recursion, where the meaning of an abstraction is defined in terms of itself. (Or an abstraction implementation may need knowledge of another abstraction, which in its turn has an implementation that needs knowledge of the first abstraction. This appears to require circular knowledge dependencies.

Circular knowledge dependencies happen all the time in functional programming where recursion replaces iteration. This is generally when a function needs to call itself or a class needs to use 'new' on itself. For example, a recursive descent compiler will have a function, 'statement', which will implement specific statements such as 'compound statement', 'if statement' and so on, and in those there will be a recursive call to the function, 'statement'. The following Syntax diagram represents part of the implementation of function 'statement'.

diagram 24.png
Figure 31. Syntax Diagram showing implementation of statement using recursion

In ALA, we want to preserve the idea of clear layers defining what knowledge is needed to understand what. Resolving this dilemma could get a bit philosophical. Since abstractions are the first class artefacts of the human brain, it may be best to think about how the brain does it. The brain must actually have two abstractions with the same name but at different levels. The first is analogous to a 'forward declaration' in a language to allow a compiler to know about something that will be referred to in a more abstract way before it finds out about it in a more specific way.

By this analogy, ALA sometimes requires the concept of a forward-declared-abstraction, something that is clearly more abstract than the concrete implementations. Therefore, we can put this forward declaration in the next layer down, just as we would a paradigm interface. In the recursive descent compiler example, we would first have the abstract concept of a statement, meaning a unit of executable code as an interface in a lower layer. Then the specific abstractions, compound statement, if statement and so on are in a higher layer. They both provide and accept the interface.

Another language example is that an expression is composed of terms, a term is composed of factors, and a factor can be composed of expressions (enclosed in round brackets). If we model these compositions as direct knowledge dependencies, we would have too many layers - and they would not be becoming more abstract as we go down. The existence of the recursion at the end reinforces that. It seems that all three, expressions, terms and factors, should have abstract interface versions at a lower level.

Not all cases of recursion would require these interfaces. If, for example, in your old way of doing things there is a long chain of function calls, with the last one calling the first one, all of them are probably run-time dependencies, not knowledge dependencies at all. So in ALA, they should all be changed to be wirable, and wired together by an abstraction in a higher layer. The paradigm interface that is used to allow them to be wired may be,for example, data-flow. So recursion does not necessarily require different interfaces for each different abstraction involved in the circular dependency.

8.2. Abstraction of Port I/O properties

This is an advanced topic that allows abstractions to be written without knowing details of the implementation of the communications. The idea is for the language to support logical or abstracted I/O ports that work for any type of technical communication properties such as described in the sections below. If we allow these properties to be binded late, say at compile-time, they can be changed independently of the domain abstractions. This allows tuning of performance or physical deployment of the abstractions to different processes or hardware.

I have been looking into how this could be accomplished using a conventional language, but it seems quite hard.

8.2.1. Push or Pull

Say an abstraction has a single logical input that can be wired to and a single logical output that can be wired from. Both the input and the output could be used in either a push or a pull manner.

For the input, push means we will be called with the data. Pull means we will call out to get the data.

For the output, push means we will call out with the data, and pull means that something will call us to get the data.

There are four combinations possible:

  • push push : push through

  • pull pull : pull through

  • push pull : internally buffer the input or output

  • pull push : active object

Let’s imagine we have a function that processes the data inside the abstraction.

The four combinations would require the function to run as a result of a function call from the input, or a function call from the output. The function result may be put into an internal buffered or be pushed out. The function may need to receive its input from an input buffer or by pulling. The function may need to run via a 3rd input that is polled or called by a timer.

We could conceivably write an abstract I/O class with an output interface and an input interface and a configuration interface that allows it to be configured late on how to do the I/O. This abstract I/O object would call the function to do the work at the right time according to its configuration.

8.2.2. Synchronous or Asynchronous

8.2.3. Buffered or unbuffered

8.2.4. Shared memory or messaging

8.2.5. Exposed state plus notification

8.2.6. Synchronous Request/Response

8.3. Working with legacy code

In old (non-ALA) legacy code, abstractions, if they ever existed, have usually been destroyed by coupling. If there is no model left by the original designer, or it is out of date, I first create one. I usually have to 'reverse engineer' the model by doing many 'all files searches' and trying to build a mental picture of how everything fits together. It can quickly become mentally taxing if the user story is non-trivial. So I build a UML class diagram from the searches (their one useful application) as the background (using light lines), and a tree of method calls for the specific user story on top of it (using heavier lines). These diagrams can end up looking pretty horrific, because the knowledge of the user-story has become so scattered, especially when inheritance is involved. The tree of method calls will come into the base class but leave from a subclass method.

This process can take several hours to a day for a single user story. Once the code for the single user story is understood, some acceptance tests are put in place for it, by putting in insertion points as close as practical to the inputs and outputs for the user story.

The next step is to factor out the method call tree for the user story into a new abstraction. This typically contains a sequence of new abstract activities or data transformations. These new abstractions are pitched at the domain level. Sometimes, if in C#, I will use Reactive Extensions. The user story may become a single RX sequence. The abstractions are then implemented, with tests, by copying and pasting useful code snippets from the original classes into the new abstractions. The old classes are marked for deprecation.

Conversion of user stories takes place iteratively.

8.4. Writing tests architected in ALA


8.5. Debugging ALA programs

Because in ALA you can get multiple instances of the same class used in multiple places, and multiple implementations of the same interface used in different places, debugging is easier if the instances are able to identify themselves. For this reason I tend to have a local property in every class called Name. The property is immutable and set by the constructor.

8.6. ALA language features

One of my first hobby programming projects was a compiler for a C-like high level language for embedded systems. At the time I had lots of energy to write the compiler and optimize the object code (written in itself, the performance of both compiling and of object code execution beat the first C compilers to later appear by around a factor of ten) but I lacked a lifetime of experience to design a language. Forty years later, I feel as if it’s partially the other way around, at least for language feature that would support good architecture. The language I should have implemented way back then should have been an ALA language - one that supported ALA architecture by having the needed constraints.

It would have had Abstractions and Instances as first class elements. The name Abstraction is to reinforce the obvious use of the only type of element that the brain uses at design-time for any kind of separation of concerns.

It would support a single type of relationship - a knowledge dependency. You would have to define your four layers, and keep them in separate folders so you would be forced to decide at what abstraction level any given piece of design-time knowledge would go. Of course, it would only allow knowledge dependency relationships from one layer to a lower layer. If you wanted to add an extra layer to the chain of dependencies, that would be a bad design decision. For example, if your application is getting too large, you could create a layer between it and the domain abstractions layer called 'plug-ins'.

Instances would work like components in that they would have ports for I/O. Like interfaces, ports are defined in a lower layer. The only way of instantiating abstractions and connecting them together is inside an abstraction in a higher layer.

Abstractions would support multiple ports of the same interface. Current languages have the difficulty that you can only implement one interface of a given type, which we had to workaround by having connector objects.

Ports would support late configuration of all communication properties such as push, pull, asynchronous, synchronous (explained above) without changing the Abstraction.

Such a language would overcome many of the problems of current languages that encourage non-ALA compliant practices. But the invention of good abstractions in the first sprint of any green-field project would still be a skilled phase requiring an innate ability to abstract.


Any feedback about this article is welcomed. Please send to johnspray274<at>gmail<dot>com