~ read.
Monorepo, scaling projects

Monorepo, scaling projects

Reading PouchDB modularization plans I learnt about a scaffolding pattern used in big code projects as Babel, React, Meteor, Ember, Lodash or Async projects.

The fact is that I read about that in The Majestic Monolith, an article written by @dhh that I felt very agree.

Now I decided write this post to collect all that I know.

What is it?

Monorepo is a unified source code repository used by an organisation to host as much of its code as possible.

At first instance, when you think about this concept looks weird (like java waterfall, right 😜).

Also if you think about the concept and associate it with a big project don't look a good idea.

But mongorepo actually means:

  • Single lint, build, test and release process.
  • Easy to coordinate changes across modules.
  • Single place to report issues.
  • Easier to setup a development environment.
  • Tests across modules are ran together which finds bugs that touch multiple modules easier.

Then the idea under mongorepo is modularizes code centralizing in one place.

Why use it

Because code is broke all the time.

Monolithic repositories increase developer productivity.

When you are building and project and you separate the project in a set of components, you are be agreeing with:

  • Have a separate and unique scaffold with each core submodule dependency.
  • Have multiple issues trackers associated with each submodule.
  • Replication of effort in the patch flow.
  • Confuse collaboration flow for the casual open source user.

(not exahustive list)

The idea under mongorepo is have the advantages the modularization but having a simple scaffold an issue track, avoiding replication of effort.

How to

First of all, each project is different, so don't try to do exactly the same. Just thing about how to avoid and simplify core collaboration.

A set of examples could be:

  • Lodash have a main unique repository and it uses build script to fragment the code and ship each core functionality individually. This is because each functional method of the API need to be installed individually without fetch all the core code.

  • Webtorrent are dependency modularized and use the main repository to control the version and connect between them. This is because each dependency is linked with a Bitorrent specific feature implementation and they need to life out of the box.

  • Async is modularized, but API is not registered indiviually in the NPM registry, because the core size is small and you can reference each API functionality individually.

I feel a gap between what a dependency and core submodule is, but in all example the idea is the same: avoid effort replication.

Tooling

It is moment to talk about tooling, because the idea looks cute, but you need a specific tool for do that. I want to expose a set of tooling that are useful for me.

zelda

zelda is little tool created by feross to link all project dependencies together and make easy setup local environment. Useful when you core modularization need to be expose as individual packages.

eachdir

eachdir is a Bash script to apply an action across a group of folders. It complement zelda perfectly.

lerna

lerna is the tool created by Babel core collaborators.

It is similar to zelda, but while zelda just link the dependencies, lerna provides you a default scaffolding for the monorepo and a CLI commands to control the project.

bumped

Bumped is a tool that I created this to make easy release software.

It is a task manager for releasing and you can use the rest of the tooling with it.

Basically you can define the actions to take when you release a new version of your project and abort the release process if the preconditions are not ok for the release.

I use it in my projects because be simple release a version without effort.