On autoloading

November 1, 2021 on Eric Bower's blog

Autoloading is a paradigm in some programming languages or frameworks that allows code to be automatically imported into any developer’s source code file. This has a few benefits, like removing the need to have the header of a file contain a series of import statements.

Autoloading appears to be the most popular in scripting languages like Ruby (via Rails) and PHP.

At Aptible, a significant portion of our infrastructure is a set of services written using Rails. After spending the better part of a couple years working with rails, I would argue that autoloading negatively impacts the developer experience as well as the productivity of our engineers.

Autoloading is the worst feature in rails and getting rid of it entirely would make the framework much more enjoyable. Even the creator of ruby wrote a post about how he strongly discourages the use of autoloading. Granted their reasons are unrelated to what I’m interested in discussing and later in the post they rescinded their stance.

For this post, I’d like to focus on how autoloading impacts the developer experience of a novice.

As someone who was first getting introduced to ruby and rails, I found it extremely difficult to figure out where classes or functions were defined in the codebase. Since there are no import statements at the top of the file, it was a guessing game to figure out where source code was located. Is SomeClass a class we created in one of our repos? Is it a third-party class? I have no clue just by looking at how the class is being used. This makes onboarding to a codebase frustrating and confusing. There are conventions that help you correctly guess, but I found the best way to find the source code was to grep for it.

Here was my basic search algorithm for finding the source to SomeClass:

It was not uncommon for me to give up entirely. Autoloading made it more difficult to figure out how something worked and left me feeling defeated.

Even after searching for the right way to do this and asking colleagues, it still feels like there’s no great solution to this problem that virtually every other modern programming language avoids by having import statements.

There are IDEs like RubyMine or an LSP that make finding the source code one click away but solargraph – the LSP I use with neovim – still comes up empty sometimes. And at least for solargraph it doesn’t even bother searching the source from my installed gems. There’s also the source_location method which we can use at runtime to find the source, but it’s also not full-proof and a little awkward. My immediate reaction is: Do I really need to run rails c in order to figure out where the source code is for a piece of functionality?

Coming from python and typescript, the entire experience feels foreign, backwards, and confusing.

I think when people say that rails feels “magical,” autoloading is a significant part of the magic.

fxn – the creator of the current autoloading implementation in rails called Zeitwerkcommented about the reasons why he likes autoloading.

I’ll focus on the arguments they claim result in a better developer experience:

  1. Being able to reload code is handy in web applications development. That is replacing the objects stored in the autoloaded constants, not reopening classes by reevaluating the files.

Every other development web server I’ve used solves this by watching files and reloading the server. It is rarely, if ever, an issue. This argument doesn’t convince me in the slightest.

  1. In any non-trivial project, getting the require calls right is difficult, you always forget some and gives load order bugs.

The error shows up immediately, you fix it, and then move on with your life. Again, this is not a very convincing argument as I would much rather fix an easily traceable error once than perform my crude search algorithm every single time I want to inspect a definition.

  1. If you structure your project in a conventional manner in which file paths match constant paths, the requires don’t feel DRY. You are repeating something all the time that could be automated.

I have another rant about DRY that deserves its own post. I’ll keep it brief here: the obsession with DRY in the rails community is a virus that has infected the minds of engineers. I know DRY wasn’t invented with rails but it certainly popularized it. All aspects of structuring code need to be evaluated based on their merits and DRY is not always the correct decision. I would also argue in some cases it produces far less readable and maintainable code.

  1. Being able to work as if all your classes and modules are just available everywhere (as in Rails) is a great user experience.

Fair enough, I do see this as a positive and I’m not blind to the benefits of having all modules, classes, and functions automatically loaded. Once you’ve memorized them you can really cut down on the preamble of writing code. I have also spent a significant amount of time refactoring JS code to fix imports because I wanted to move some code to another location.

I remember when I first started to learn python and stumbled across this stackoverflow post about auto importing modules in python. I think it provides some interesting arguments for having import statements:

  1. They serve as a sort of declaration of intent.
  2. Imports serve as a proxy for the complexity of a module.

So much information and complexity is hidden by removing the import statements.

If we want rails to feel less magical, we should start by looking at removing autoloading.

Articles from blogs I read

Generated by openring

How to help improve SourceHut's design

SourceHut is a software development forge and it is designed with the software engineer’s needs first and foremost. The design prioritizes things like page speed, minimal distractions, and information-forward layouts. It does not prioritize aesthetics, and p…

via Blogs on Sourcehut October 13, 2022

In praise of ffmpeg

My last “In praise of” article covered qemu, a project founded by Fabrice Bellard, and today I want to take a look at another work by Bellard: ffmpeg. Bellard has a knack for building high-quality software which solves a problem so well that every other solu…

via Drew DeVault's blog October 12, 2022

Site Update: HLS support

Hi blog readers! It's time for a regular update on stuff I've gained expertise in frantically googled and hacked together so I can put it in front of your faces. I use YouTube as a video hosting service because it's largely predictable and it'…

via Xe's Blog October 11, 2022