Why Most Rails Apps Get Background Jobs Wrong

Most web applications start simple. A user clicks a button and the server does something and a new page loads. For a while this works perfectly. The database is small and the work the server does is quick. Everything feels snappy.

Then you add features. When a user signs up you need to send a welcome email. When they upload a photo you need to resize it into a few different versions. When they connect their account to an external service you need to make an API call.

Suddenly your app feels slow. A user signs up and has to wait five seconds staring at a spinner before they can use the app. The problem is not your database queries. The problem is you are trying to do too much work inside the web request.

The obvious solution is a background job. You take the slow work like sending an email or resizing an image and move it out of the request. The controller’s job is just to tell the background system “please do this work later” and then immediately render a response to the user. The app feels fast again. This is where most Rails developers get to and they stop. And that is the problem. Using background jobs is easy. Using them correctly is not.

The Default Trap

Rails provides a wonderful abstraction for background jobs called Active Job. It lets you write your jobs once and then choose which backend to run them on. In development the default backend is :inline. This means the job isn’t sent to a background system at all. It’s run right there as soon as it’s enqueued. Synchronously.

This is a sensible default for making development simple but it creates a dangerous blind spot. You write your code thinking it will be asynchronous in production but you can’t actually see how it behaves asynchronously in development. The welcome email job that takes two seconds still freezes your local server for two seconds. You might be tempted to ignore it because it’s “just development” but you’re missing the point. You are not building resilience for a world where the job and the web request are completely disconnected.

Moving to a real background processor in development is the first step. You need to feel the disconnect. You need to see what happens when the job fails and the web request has already finished.

The Real Backends

To run jobs in the background you need a backend adapter and a running process to execute them. For years the default choice has been Sidekiq. It uses Redis to store jobs and is exceptionally fast and reliable. It is the workhorse of the Rails world for a reason. Its main downside is that it requires you to run and manage a separate Redis server which adds a piece of infrastructure.

A more recent option I find interesting is Good Job. It uses your existing PostgreSQL database to store and manage jobs. For many applications this is a huge simplification. Your database is already there it’s already being backed up and you don’t need to add a new dependency. Unless you are processing thousands of jobs per second Good Job is often powerful enough and much simpler to manage.

The mistake is not choosing one. The mistake is sticking with :inline for too long and not understanding that a real background system is a required part of your application’s architecture not an optional add on.

The Most Important Trait Idempotency

If you only take one thing away from this essay it should be this make your jobs idempotent. Idempotent means that running the same job with the same arguments multiple times has the same effect as running it just once.

Why is this so important? Because background jobs will fail. It’s not a matter of if but when. Your server will be restarted during a deploy while a job is running. The third party API you are calling will be down. A network blip will interrupt a connection. In all these cases the background processing system will likely retry your job. If your job is not idempotent a retry can be catastrophic.

Imagine a job to charge a customer.

ChargeCustomerJob.perform_later(user, 1000)

If this job starts running charges the customer’s credit card and then fails before it can mark itself as complete the system will run it again. The customer gets charged twice. This is the kind of bug that destroys trust.

An idempotent version looks different. You introduce a unique key for each attempt.

ChargeCustomerJob.perform_later(user, 1000, charge_attempt_id)

Inside the job the first step is to check if a charge has already been recorded for that charge_attempt_id. If it has the job does nothing and exits successfully. If it hasn’t it proceeds to charge the card and record that the ID has been processed. Now you can run it a hundred times and the customer will only be charged once. This pattern is fundamental to building a robust system.

Getting a Response

Another common mistake is thinking background jobs are only for tasks where you don’t care about the outcome. Fire and forget. But often you do care. The user wants to know when their video has finished processing or when their report is ready to download.

The solution is for the job to update the state of a model in your database. The user uploads a video and you create a Video record with a status of processing. Your ProcessVideoJob runs in the background. When it’s done it updates the video’s status to complete and stores the URL of the processed file.

The user’s browser in the meantime can poll a simple API endpoint every few seconds. /videos/123/status. This endpoint just returns the status from the database. When it sees complete it can reload the page or display a download link. This avoids tying up a web server process for minutes and gives the user a responsive experience.

Getting background jobs right isn’t about finding a magic gem. It’s about a shift in mindset. You have to start thinking of your application as a distributed system even if it’s all running on one server. The web process and the job process are two different worlds. Design your jobs to be resilient independent and idempotent and you will have built an application that can not only start fast but stay fast as it grows.

— Rishi Banerjee
September 2025