Skip to main content

Design Concept 🎬

doc-header

Welcome to Maatuska Workshop 🪆 🪆 🪆

The main idea behind this design concept is building the image through a flow of processes, with each process given its own parameters, and (re)using artifacts from previous steps as well as generating its own. In this concept, the modules are named after the real life process of building a matryoshka set.

Concept

design-concept

  • WORKSHOP : The cluster of processes building the image and its artifacts. It depends on the implementation, but a cluster can be an OTP cluster, Kubernetes cluster, running locally, distributed .. etc.

  • WORKBENCHES: The different processes which consume/produce artifacts based on input parameters. Workbenches are processes that would typically run tasks and system commands while generating artifacts, things like: builders, installers, writers .. etc.

  • INVENTORY: A special process offering access to a storage on which both intermediate artifacts as well as the final image(s) are stored. Stored image(s) can be cached and accessed using a key based on their unique parameters, given that when a process is given the same parameters it would generate the exact same image.

  • planner: The process supervising the main flow of the build. It is responsible for supervising the execution of an input plan and firing up concurrent processes (workbenches).

  • plan: The input declarative configuration of the image.

Elixir

An implementation in Elixir can have these modules implemented as follows:

ModulePossible Implementation
WORKSHOPA K3s cluster. K3s offers minimum memory/CPU footprint, and can run on embedded devices, yet it provides compatibility with Kubernetes API. Using Kubernetes API is more familiar when provisioning and scaling clusters of processes/services.
WORKBENCHA GenServer consuming/producing artifacts and takes input parameters when started.
INVENTORYA special GenServer/Agent which provides storage API to workbenches (and probably supports caching, different persistence and retention options). The storage back-end can be some DBMS like PostgreSQL, Mnesia .. etc, or some object/blob store like LeoFS, SeaweedFS .. etc, or even Maatuska's own storage implementation.
plannerThe main cluster Supervisor. It may use libcluster (in addition to other libraries as well such as Swarm, Horde) to automatically detect joining nodes and their workbenches, and distribute tasks on them. Also it would orchestrate the main image building using Flow.
planA configuration similar to Mix config files, which defines different components and parameters of the final image.