In the world of react and redux, there is no shortage of tutorials, to-do apps, and how-to guides for small web applications. There's a rather steep learning curve when trying to deploy a modern web application and when researching how to scale and maintain a large one, I found very little discussion on the subject.
Contrary to what people think, react is not a framework; it's a view library. That is its strength and also its weakness. For people looking for a batteries-included web framework to build a single-page application, react only satisfies the V in MVC. For small, contained applications this is an incredible ally. React and redux don't make any assumptions about how a codebase is organized.
There is no standard for how to organize a react redux application. We cannot even settle on a side-effects middleware for it. This has left the react redux ecosystem fragmented. From ducks to rails-style layer organization, there is no official recommendation. This lack of standardization is not because the problem has been ignored, in fact, the official redux site states that it ultimately doesn't matter how you lay out your code on disk. In this article is to show how I like to build large accplications using react and redux.
Inspiration #
There really are not a lot of large and open codebases to gain inspiration from. The most notable examples I have found are Automattic's calypso and most recently Keybase's client.
Uncle Bob's Clean Architecture argues that architecture should describe intent and not implementation. The top-level source code of a project should not look the same for every project. Jaysoo's Organizing Redux Application goes into the details of how to implement a react/redux application using a feature-based folder organization.
Code Organization #
Monorepo #
On a recent project I was responsible for multiple platforms which include but are not limited to: web (all major browsers), desktop (windows, mac, linux), outlook plugin, chrome extension, and a salesforce app.
We decided that all that code should live under one repository. The most important reason was for code sharing. I also felt it unnecessary and unmaintainable to build seven separate repositories.
A quick overview #
I leveraged yarn workspaces to
do all the installation. Every package was located under the packages folder.
Each platform had its own folder for customization under the platform folder.
Platform specific packages would also be located under the packages folder.
Although if desired it would be easy to move platform specific packages under
each platform folder respectively. This made initial setup easier to handle
because all packages lived in one place.
1plaforms/
2 web/
3 webpack/
4 index.js
5 store.js
6 packages.js
7 cli/ # same structure as web
8 salesforce/ # same structure as web
9 desktop/ # same structure as web
10 chrome/ # same structure as web
11 outlook/ # same structure as web
12packages/
13 login/
14 packages.json
15 index.js
16 action-creators.js
17 action-types.js
18 effects.js
19 sagas.js
20 reducers.js
21 selectors.js
22 logout/ # same structure as login
23 messages/ # same structure as login
24 web-login/ # same structure as login
25 cli-login/ # same structure as login
26packages.json
Feature-based folder organization #
There are two predominate ways to organize code: layer-based and feature-based folder organization. When building an application, the top level source code should not look the same for every single application. The rails-style MVC folder structure (layer-based) muddles each feature together into one application instead of treating them as their own entities. Building a new feature in isolation is more difficult when each component of a feature needs to join the other features. Using a feature-based approach the new feature can be built in isolation, away from everything else and then "hooked up" later when it's finished.
Layer-based
1src/
2 models/
3 login.js
4 logout.js
5 views/
6 login.js
7 logout.js
8 controllers/
9 login.js
10 logout.js
Feature-based
1src/
2 login/
3 model.js
4 view.js
5 controller.js
6 logout/
7 model.js
8 view.js
9 controller.js
Every feature is an npm package #
This was a recent development that has been successful for us. We leveraged yarn workspaces to manage dependencies between features. By developing each feature as a package, it allowed us to think of each feature as its own individual unit. It really helps decouple a feature from a particular application or platform. Using a layer-based approach, it's really easy to lose site that these features are discrete contributions to an application.
Absolute imports #
It was a nightmare moving code around when using relative imports for all of our internal dependencies. The weight of each file being moved multiplies by the number of thing depending on it. Absolute imports were a really great feature to leverage. The larger the app, the more common it is to see absolute imports.
Lint rules around inter-dependencies #
One of the best things about absolute imports was the lint tooling that could be
built. We used a namespace @company/<package> for our imports so it was
relatively easy to build lint rules around that consistent naming.
Strict package boundaries #
This was another key to scaling a codebase. Each package had to subscribe to a consistent API structure. It forces the developer to think about how packages are interacting with each other and creates an environment where there is only one API that each package is required to maintain.
For example, if we allowed any package to import another package, it's difficult
to understand what happens when a developer decides to move files, folders
around. For example when building a package, let's say we want to change the
file utils to helpers. By allowing a package to import utils directly, we
inadvertantly broke the API. Another example is when a package is really simple
and could be encapsulated inside one file. As long as the package has an
index.js file and it exports all of the components that another package needs,
it doesn't matter how the package is actually organized. It's important for a
large codebase to have some sort of internal consistency, however, I found
having some flexbility allows to fit an organization that matches the needs of
the feature.
Another reason why strict module boundaries is important is to simplify the dependency tree. When reaching into a package to grab a submodule, the dependency graph treats that submodule as a full-blown package. When creating module boundaries and a pacakge imports another package, it imports the entire package. This simplifies the dependency graph and makes it easier to understand. Here's an article on the important of dependency graph.
Each package exports the following:
1{
2 reducers: Object,
3 sagas: Object,
4 actionCreators: Object,
5 actionTypes: Object,
6 selectors: Object,
7 utils: Object,
8}
Creating this consistent API provided opportunities ripe for tooling.
One of the most important rules was the module-boundary lint rule. This
prohibited any package from importing a sibling package's submodules directly.
They must always use the index.js file to get what they want.
For example:
1// bad and a lint rule will prevent this
2import { fetchNewsArticle } from "@company/news/action-creators";
3
4// good
5import { actionCreators } from "@company/news";
6const { fetchNewsArticle } = actionCreators;
This setup came at a cost. Import statements became more verbose as a result of this change.
Probably one of the greatest benefits to this structure was circular
dependencies. I know that sounds insane, who would actually want circular
dependencies in their codebase? Especially since every circular dependency that
was introduced caused an ominous runtime error: cannot find X of undefined.
I'll go into more details about why these errors were favorable later.
A package is a package is a package #
Another huge benefit to our "feature-based, everything is an npm package" setup was the fact that every package was setup the same way. When I onboard new developers, I usually ask them to add a new feature. What this means is they get to build their own package that does something new. This made them understand exactly how a package works and they have plenty of examples on how to build them. It really reduced the barrier to entry into a massive codebase and was a great ally when trying to introduce people into a large codebase. With this architecture, I created a scalable system that anyone can understand.
Support tools #
Because of how tedious it can be to maintain a list of internal dependencies for
each package, not to mention creating package.json files for each feature, I
outsourced it to tooling. This was a lot easier than I originally thought.
I leveraged a javascript AST to detect all import statements that matched
@company/<package>. This built the list I needed for each package. Then all I
did was hook that script up to our test runner and it would fail a) anytime a
dependency was not inside the package.json or b) whenever there was a dependency
inside the package.json that was no longer detected in the code. I then built an
automatic fixer to update those package.json files that have changed.
Another huge benefit to having internal dependencies within each package was the
ability to quickly look at a package.json file and see all of its
dependencies. This allowed us to reflect on the dependency graph on a
per-package basis.
Making our packages npm install-able was easy after this and I don't have to do anything to maintain those package.json files. Easy!
I wrote the support tools into a CLI lint-workspaces
Package loader #
Since I had a consistent API for all of our packages, each platform was able to
load whatever dependencies it needed upfront. Each package exported a reducers
object and a sagas object. Each platform then simply had to use one of our
helper functions to automatically load our reducers and sagas.
So inside each platform was a packages.js file which loaded all reducers and
sagas that were required by the platform and the packages it wanted to use.
By registering the packages, it made it very clear in each platform what kind of state shape they required and what kind of sagas would be triggered.
1// packages.js
2import use from "redux-package-loader";
3import sagaCreator from "redux-saga-creator";
4
5const packages = use([
6 require("@company/auth"),
7 require("@company/news"),
8 require("@company/payment"),
9]); // `use` simply combines all package objects into one large object
10
11const rootReducer = combineReducers(packages.reducers);
12const rootSaga = sagaCreator(packages.sagas);
13export { rootReducer, rootSaga };
1// store.js
2import { applyMiddleware, createStore } from "redux";
3import createSagaMiddleware from "redux-saga";
4
5export default ({ initState, rootReducer, rootSaga }) => {
6 const sagaMiddleware = createSagaMiddleware();
7 const store = createStore(
8 rootReducer,
9 initState,
10 applyMiddleware(sagaMiddleware),
11 );
12 sagaMiddleware.run(rootSaga);
13
14 return store;
15};
1// index.js
2import { Provider } from 'react-redux';
3import { render } from 'react-dom';
4
5import createState from './store';
6import { rootReducer, rootSaga } from './packages';
7import App from './components/app';
8
9const store = createState({ rootReducer, rootSaga });
10
11render(
12 <Provider store={store}>
13 <App />
14 </Prodiver>,
15 document.body,
16);
I have extracted the package loader code and moved it into its own npm package redux-package-loader.
I also wrote a saga creator helper redux-saga-creator
Circular dependencies #
Circular dependencies were a very important signal when developing. Whenever I came across a circular dependency, some feature was organized improperly. It was a code smell, something I need to get around not by ignoring it, not by trying to force the build system handle these nefarious errors, but by facing it head on from an organizational point of view.
One of the :key: topics I learned about along the way was Directed acyclic graph
I'll explain by example, give the following packages:
packages/
mailbox/
thread/
message/
I would regularly run into situations where pieces of code within the mailbox
package would want to access functionality inside the thread package. This
would usually cause a circular dependency. Why? Mailboxes shouldn't need the
concept of a thread to function. However, thread needs to understand the
concept of a mailbox to function. This is where DAG came into play. I needed to
ensure that any piece of code inside mailbox that needed thread actually
didn't belong inside mailbox at all. A lot of the time what it really meant
was I should simply move that functionality into thread. Most of the time
making this change made a lot of sense from a dependency point of view, but also
an organizational one. When moving functionality into thread did not work or
make any sense, a third package was built that used both mailbox and thread.
Cannot find X of undefined #
For whatever reason, the build system (webpack, babel) had no problem resolving
circular dependencies even though at runtime I would get this terribly vague
error cannot find X of 'undefined'. I would spend hours trying to track down
what was wrong because it was clear that this was a circular dependency issue.
Even when I knew it was a dependency issue, I didn't know what caused it. It was
a terrible developer experience and almost made me give up completely on strict
package boundary setup.
Tools to help detect them #
Originally the tool that helped detect circular dependency was madge. It was a script that I would run and it would normally indicate what would be the dependency issue.
Once I moved to yarn workspaces however, this tool failed to work properly.
Thankfully, because every package had an up-to-date package.json file with all
inter-dependencies mapped out, it was trivial for to traverse those dependencies
to detect circular issues.
An open example #
The project codebase is not publicly accessible but if you want to see some
version of it, you can go to my personal project
youhood. It is not a 1:1 clone of the
setup, primarily because I am using TypeScript for my personal project and yarn
workspaces was not necessary to accomplish what I wanted, but it organizes the
code in the exact same way by leveraging redux-package-loader.
It's not perfect #
There are a few issues when developing an application like this.
- Importing a package brings everything with it
- Import statements are more verbose
In a follow up blog article I will go into more detail about these issues.
This code organization could build multiple platforms using most of the same code. As with most things in life, this was not a silver bullet. They :key: take-aways were:
- Feature-based organization scaled really well
- A consistent package interface allowed for tooling
- Force developers to think about dependency graph