Durable Orchestration on Rails: Turning Active Job Continuations into Workflows
Ruby on Rails
Vladimir Dementyev leads a backend team at Evil Martians and has contributed to Rails for about a decade. In this talk he looks at how teams coordinate multi-job processes today: pipelines, batches, timers, and human-in-the-loop approvals. He argues that Active Job continuations, introduced about a year earlier, have quietly become an orchestration primitive. They are still too primitive for most workflows, though. His proposal is to extend continuations, in a Rails-native way, into "durable continuations" so that an entire workflow can live in a single job class.
From thin facade to a feature of its own
Dementyev starts with a short history of Active Job from Rails 4.2 to 8.2. When it was announced in 2014, the same year he shipped his first production Rails app, he thought it was nice that Rails finally had its own background job interface. He saw no reason to migrate, though, because it seemed like a thin compatibility layer over existing gems that added nothing and lacked a lot. That app is still alive as a large monolith and still doesn't use Active Job. He says he has seen many apps like it, and he urges teams still using, for example, Sidekiq workers directly to reconsider. In his telling, Active Job has gradually picked up solid ideas of its own. A year ago it gained something that backend engines themselves didn't provide: continuations.
What continuations were built for
Dementyev describes continuations as a resilience concept by design. They make long-running jobs interruptible and resumable by introducing steps and cursors that act as checkpoints. He points to the original pull request, where the stated motivation was surviving deployments.
He illustrates this with a webhook dispatch job. Without continuations, a deploy in the middle of an iteration drops the job. The backend retries it from the beginning, so the same webhook event can be sent twice. The common workaround has been to split the work into an orchestrating job that enqueues one atomic job per record. That approach is robust but more verbose and less cohesive. The continuation version keeps the same guarantees with the whole workflow in one file. Dementyev prefers this, saying it is "more resilient to human or agent errors."
Looking ahead, he highlights a set of recent Active Job pull requests, some merged and some not. About half of them relate to continuations, and they come from community members rather than the core team. He reads this as rapid adoption, but by users whose goals differ from the original intent. They use continuations to orchestrate long-running multi-step operations with state or fairness guarantees, not to survive deploys. He shows a snippet he uses often: splitting a long LLM-related job into isolated steps so work is distributed more fairly across workers. His conclusion is that a feature "born as a workaround for infrastructure quirks" has become an orchestration primitive. It is still too primitive to be a building block for common orchestration scenarios, and the rest of the talk explains why.
Pattern 1: Pipelines
The first pattern is the pipeline: a sequence of jobs, common in file processing, data imports, and syncing. Each job does one step and enqueues the next on success. Chains can grow very long and even branch. Dementyev says this is typical of large Rails apps, and that months later nobody knows where the chain starts, which way it goes, what its outcomes are, or how it fails.
Refactoring the chain into continuation steps puts the workflow in one place. He notes this has nothing to do with interruptibility; it is purely about organizing code. Some awkwardness remains. Step execution still depends on state saved on the model, which he calls "not the best way," but there is no other state to use.
A second pipeline example comes from Evil Martians' AnyCable Plus project. A single job is re-enqueued for each step, and a separate state machine controls it. No job class even appears in the code, because the team uses the Active Job Performs plugin, which turns a method into a background job. The problem is that too many entities are involved in orchestrating the work. The process could be written as a continuation-based state machine. The new attributes API helps keep state during the workflow, but that state lives in the job payload and can't be seen from outside.
From these examples Dementyev lists what is missing:
- No step-level control-flow hooks. There are no step callbacks and no per-step rescue handlers. In his example every step repeats a broadcast update that an
after_stepcallback could handle. - No observability. With no persistent state, you can't tell which step a job is on, how long it has been running, or what its step history looks like.
- No uniqueness. Long-running workflows usually shouldn't be enqueued again while one is running, or the running one should be cancelled and replaced. Some providers offer this, but Active Job has no framework-level concept for it.
Existing third-party options
Dementyev reviews the tools available today. His team has used Acidic Jobs on a couple of projects. It stores the workflow's state in the database so it can be restored and observed, and it has built-in uniqueness. What matters most to him is that it keeps the Active Job interface: callers still see an Active Job, and durability stays hidden inside the workflow class.
He also mentions Geneva Drive, a more recent project that he says has almost everything he would want in a workflow implementation, and he returns to it several times. His main objection is that "it's not a Rails thing." It has its own DSL and its own entity, isn't fully compatible with Active Job, and requires refactoring everything to adopt it. He wants something framework-friendly with the same guarantees.
Pattern 2: Batches
Batching means one job does setup, many jobs do their own tasks, and something runs on completion. Sequential batches can already be written with continuations. He shows an example from Spree: begin processing, create a job per CSV row, then finalize. Batches usually imply parallel execution, though, and that requires a coordinator. Typical coordinators are counters, often in Redis, or polling jobs that check every few seconds whether everyone is done.
Teams have usually reached for third-party gems here. Dementyev notes that Solid Queue (he cites version 1.7) has just gained batch support, thanks to JP Camara with significant help from Rosa. He recommends reading the story of how it got there. In his view it shows how much effort an outside contributor needs to ship a major feature to a cornerstone library. In Solid Queue's design everything is a job, and completion handlers are jobs enqueued on success or failure. The interface is inspired by GoodJob, which has had batches for a while.
Many libraries offer batching, each with its own interface. Dementyev calls this a problem, though a manageable one. He singles out Doximity's Simplekiq. It is Sidekiq-specific, but its interface closely resembles continuations, with steps and also parallel steps, which continuations lack. To him this suggests continuations could adopt parallel steps. The blocker is that Active Job itself has no batch interface yet. He says there have been discussions about adding one, and once it lands, continuations could become batch-aware. Until then, "we need to wait."
Pattern 3: Timers
The next group covers reminders, expirations, notifications, and similar time-based tasks. His AnyCable example is a license-expiration workflow made of recurring polling jobs that run every ten minutes or every hour. Each job is an entry point that enqueues more jobs, which adds many concepts for a single workflow, and the structure must be repeated for each new workflow. Continuations help somewhat: iteration within steps and cursors can collapse everything into one job class. The scheduling layer is still needed.
The other time-based approach is to schedule a job in the future with wait or wait_until. Dementyev says this has many problems. Active Job has no framework-level API to cancel a scheduled job, so teams usually check whether the job is still relevant when it runs. Provider-specific cancellation exists but tends to be a lot of work, and he finds the relevance check simpler and more reliable. The approach sounds good conceptually but is hard to get right, so teams usually fall back to cron-driven sweeps.
He surveys tools that make this easier. One relatively new project extends Active Job with DSL-style method decorations, including a wait that marks where the job stops, records itself in a waiting table, and gets re-enqueued to continue when the time comes. Geneva Drive offers timers as a step option. It also links workflows to Active Record models through what it calls "heroes," so you can find and cancel the workflow tied to a given record. Dementyev calls this very useful. Third-party services such as Temporal go further. In an appointment-reminder example, when the date changes, you update the reminder's deadline as part of the workflow's state instead of cancelling and restarting the workflow.
Pattern 4: Signals and human-in-the-loop
Dementyev treats approval and signal-driven workflows as a broader version of timers. The workflow waits for something other than time, such as a user action. In code, a signal typically splits the workflow into "before" and "after" halves in different places. His example from Mastodon is one process that should be read as a single flow with a pause in the middle, but Rails offers no way to write it that way. Because a signal may never arrive, the app also needs ad hoc cleanup for tasks that were never signaled.
Webhooks are another common kind of signal. In a real app he worked on, a withdrawal workflow was spread across three files: a job that requests the withdrawal, a webhook handler that receives confirmation or rejection, and another job that processes the result. Using the signaling feature of a gem he calls ChronoForge (via a step he describes as having the non-obvious name continue_if), the flow could be a single perform with three steps: run the first step, wait for the payment receipt, run the credit step.
The team behind that project chose Temporal instead, and Dementyev says he isn't happy with the result. In his view, something you can describe in a few English words shouldn't cost hundreds of lines of code, dozens of Ruby objects, and "voodoo magic" to work. The general problem, as he sees it, is that more sophisticated workflows push teams toward more sophisticated tools. He doesn't think that is the right direction for Rails. He wants "more batteries included."
The proposal: durable continuations
Dementyev says he first floated the idea at a conference about a year ago. Rather than investing in custom gems or SDKs, the community could build on continuations and turn them into something bigger. In this talk he presents the first implementation.
He defines "durable" plainly and says the word is overhyped. The core difference is that the job's execution is a record, or several records, in a database or other store, rather than just a payload in the queue. Other properties, such as surviving killed processes, follow from carefully committing that state and making it restorable and replayable.
His main design requirement, which he says he applies to every library he builds, is staying as close to the Rails way as possible. It should feel like using the framework and be an extension rather than a replacement, something that could eventually be merged into Rails. He argues that these style and taste constraints usually lead to a better implementation.
Persisted state
His walkthrough uses a card-generation job from one of the team's projects, which turns user photos into cute characters. It has three steps: process the original image, moderate it so inappropriate images aren't shown to the community, and generate the final content. In the durable version, a table records each run, its state, and its steps. A record appears when the job is enqueued. The state changes when a worker picks it up, and each step updates as it progresses. Job attributes are persisted too, so they can be observed from outside and used by other features. A query API lets you inspect, cancel, or retry runs. Unlike Geneva Drive, it doesn't use "heroes." It infers identifiers from the job's arguments or from identifiers defined explicitly on the job.
He then revisits the webhook example. With plain continuations, a hard crash or uncaught exception can leave the cursor uncommitted, so the job starts over with no memory of progress within a step. With durable continuations, which take a one-word change to the class, the cursor is committed (configurably, not necessarily on every iteration). He says this gives stronger reliability guarantees against various failures.
Lifecycle features
Durable continuations add step callbacks such as after_step, which removes the repeated inline code from step bodies. Dementyev says this makes a workflow look like a typical Rails object with the conveniences of models and controllers, "no longer just a tiny wrapper over background job execution." A halt_on rescue handler ends the workflow after a particular step. It differs from discard_on because the job can be restarted once the problem is fixed, since some errors are recoverable. Uniqueness comes almost for free because runs are stored in a database.
Timers
For timers, the license-expiration workflow that previously needed a scheduler and many jobs becomes a single job whose steps take wait and wait_until. With uniqueness, renewing a license just means enqueuing the job again: the new run replaces the previous workflow and continues. Another example shows an Active Record integration, a has_one_performed-style association that gives easy access to a model's associated runs.
Signals
Running short on time, Dementyev moves to what he calls his favorite feature. He converts the Temporal-based withdrawal example, and it becomes a single Ruby class with a few configuration lines. All the logic sits in one place that can be observed and controlled. When someone asks whether it reads better than English, he replies that it's Ruby, which "has structure and clear meaning."
He then upgrades the card-generation example. If the moderation score for a photo is borderline, a human is asked whether the image may be posted. The current implementation tracks this with enum states on the model and checks them from the job. He lists the problems with that: possible concurrent executions, no history, and no review deadlines. With durable continuations this becomes a single wait step, a special step with an empty body that parks the job. It can be released by a timeout or by a signal sent from a controller or anywhere else. He says this is the whole API, built on the same primitives as timers.
As a bonus, he mentions that RubyLLM 2.0 was recently released with example snippets that use continuations. He argues they would be simpler with durable continuations. An agent loop could live in one job that halts and resumes after the user approves or rejects an action. RubyLLM already stores chat messages durably, so durable data isn't really needed here. What is needed is durable control-flow state.
Availability and next steps
Dementyev announced that he would open source the library right after the talk as a gem named AJDC (Active Job Durable Continuation, "nothing related to music"). It includes everything shown in the talk, with more features planned. It also ships with AI agent skills, which he tied to a Rails project he called "hyperdrive," so you can install the gem and ask an agent to configure it, find jobs worth migrating, and migrate them. He added that he doesn't recommend skipping a read of the code afterward.
The goal, as he puts it, is to bring just enough orchestration into Active Job to stay on the Rails way, without reaching for external services or "cryptic gems." Batches and parallel steps remain open until Active Job gains a native batch interface. He asked the audience to pay more attention to continuations, look for these orchestration patterns in their own code, consider extracting them into workflow-like structures, and try durable workflows.
All right, everyone, glad to be on the RailsConf stage again. Thanks to Amanda, the Rails Foundation, and all the organizers for making this conference a once-in-a-lifetime or twice-in-a-lifetime for someone, maybe last-in-a-lifetime event and experience.
My name is Vladimir. And I've been building with Ruby on Rails for more than 10 years, and about the same amount of time I've been contributing to Rails. And over the years I managed to publish a lot of gems. You probably know a few of them. Some of them were mentioned. Even wrote a book you probably read, or you haven't read. Ask it. By the way, is it good?
And most recently my Rails contributions were all around Action Cable. And with the next release, it's been re-architected to support different server implementations without hacks, like Async Cable, the most interesting one. And on top of that architecture, some feature fixes and improvements have been already prepared, and also expecting to land in the future releases.
But that's not what I'm going to talk about today, because that proposal hasn't been selected. So, that's it. Next topic. Sorry. I'm first time using this clicker.
A few words about my job. I'm leading a backend team full of brilliant Ruby and not only Ruby engineers at Evil Martians, makers on my team, sorry. And we're helping startups and companies at scale to do things and do that with higher quality. And we also help Rails Foundation since recently. You probably heard about this Rails AI benchmarks project. And that's been my primary focus for the last month. And now my nightmares are full of LLM sessions transcripts full of shitty reasoning, so I can sometimes recall them when I'm awake. Don't be afraid.
But I'm not talking about my job today, luckily. I'm talking about your jobs. More specifically, your background jobs. And that's the topic of my talk today. And I'd like to start with a little bit of retrospective.
I wanted to show you how the role of Active Job has changed over the years from 4.2 to 8.2. It all started in 2014, the same year I shipped my first production Rails application. And I clearly remember the moment Active Job has been announced. My thoughts were like, "Oh, wow, that's really cool. Finally, Rails shipped a background job kind of implementation on its own. But I'm not going to migrate to it because what's the point? It's just a thin compatibility layer on top of the existing gems and doesn't add anything and missed a lot of stuff."
The app I worked on back then, by the way, is still alive and now it's a large Rails monolith and it still doesn't use Active Job. And I've seen many apps like that in my life, unfortunately. But I encourage you, if you're still using Sidekiq workers directly, please consider migrating to Active Job because today Active Job is no longer just a common interface. Over the years, gradually, slowly, it's been evolving.
It was bringing new solid ideas until we got to the point where a year ago a new concept not present in background job provider engines appeared in Active Job itself. So, the framework finally started bringing something specific on top of other works. And let's talk about continuation for a moment. What is it and why it's important?
Continuation is by design a resilience concept. It was added to make long-running jobs interruptible and resumable by introducing steps and cursors that act as checkpoints. And the original motivation for that was surviving deployments. And you can find it in the original PR. That's the actual reason why the feature was brought to Rails.
And what does it mean, surviving deployments? Let's imagine the same webhook dispatch job from the previous slide, but without continuation. It looks pretty similar, same shape, same logic, but different guarantees. Whenever a deploy happens in the middle of the iteration, the job is dropped and it's retried by the background processing engine, but it starts over the iteration from the first step and could result in sending the same webhook event twice. So, it's not cool for your application.
That's why for many years we've been using some workarounds like this one. That's the most popular: having several jobs, one is just orchestrating and queuing atomic jobs per record. So, making this approach robust, reliable, maybe a bit more verbose and less cohesive than a single job version, but it worked.
Now, let's again compare it to the continuation version. We have everything in one place. We have the same guarantees. And we have the whole workflow, let's call it workflow, defined in a single file. And to my taste, it's much better than having it scattered across files. It's more resilient to human or agent errors, whatever you prefer. So, this is a canonical example of using Active Job, and this is what it was created for.
Now, back to the timeline. As I already said, Active Job started as a tiny facade over other runtimes. But over the years, it evolved into a foundation of its own. And together with Solid Queue, which is an important component of the ecosystem today, they lead the way for everyone on how to implement background jobs and maintain them on Rails. But what's coming next? What awaits us in the next version?
I picked some PRs to Active Job, some merged, some not yet, but the most interesting because they bring in new stuff to Active Job. And half of those PRs are continuation-related. And what's more important, they're contributed by the community members, not by the core team or by the core company team members. And that's an interesting fact because continuation is just a year old, but it's being adopted quickly.
But the way regular users, like not the ones who contributed it originally, use continuation differs from the original intention. It's no longer about surviving deployments. It's about orchestrating long-running multi-step operations with their state or with their kind of fairness guarantees. That's the snippet I use a lot these days to split a long-running LLM-related job into multiple steps and run them in isolation so we can have more fair distribution of work across workers.
And that means that the feature that was born as a workaround for infrastructure quirks actually became an orchestration primitive. But this primitive is still too primitive to be a building block for the common job orchestration scenarios. Let's look at some examples of how we coordinate multiple background jobs today and what are the common patterns and tools to implement them, and how we can use continuation for that, maybe a little bit upgraded version of it.
Let's start with pipelines or job sequences. This is a pattern that's common for processing files or importing data, syncing, whatever. There are a lot of use cases, but they have similar shape. One of the shapes is a chain of jobs.
We have one job performing the first step of the operation and then on the successful result we enqueue another job. We can terminate the process by failing it somehow, but then there could be another job and the number of such jobs in a chain and possibly branching could be enormous and it's very typical for large Rails applications. And the problem with that is that months after implementing that, no one knows where the start of the chain is, how it goes, what is the direction, what are the outcomes, what are the failure modes, and so on.
We can refactor this chain of jobs into a chain of steps with continuation. We can encapsulate the whole workflow in a single place and this is no longer about interruptibility and resumability, right? This is just about organizing code in the application. And there are some quirks still left because we rely on the states saved on the model to control the execution of the steps, which is not the best way to do that, but we don't have any other state to rely upon. So, it's still just a background job running by Active Job.
Another example of the kind of pipeline, same idea, multiple jobs running one after each other, but now we have actually the single job which is being re-queued to perform the next step controlled by a separate state machine. This is an example from our AnyCable Plus project. There is not even a background job class visible, right? Because we use Active Job Performs plugin that just turns a method into a background job, very convenient. But the problem here is that we have too many entities to orchestrate the work.
And moreover, that's how it works. And it could be also written into a kind of state machine to make it more clear what the order of the steps is, what are the guards for each step. And it could be done today, especially with the addition of this new attributes API that allows us to keep some state during the execution of the workflow, but this state is not accessible anywhere outside because it lives in the job's payload. There are some limitations.
Another problem is that unlike with many things in Rails, like performing actions and whatever, we do not have control flow hooks for steps. There are no step callbacks, no per-step rescue handlers, and so on. So, every step has to deal with it manually. Like here, we do broadcast update in every step because we need to do that. If we had after-step broadcast, that would be a much better experience and much better readability of the whole process. But that's something that's missing today.
We mentioned a couple things and I slightly mentioned that we don't have observability for the continuation process. We don't know which step the job is currently in, how long has it been running, what is the history of steps, just because there is no persistent state. And the rules and steps and retries I already talked about.
One other thing related to such workflows is uniqueness constraint. For long-running workflows, we typically don't want them to be re-enqueued multiple times when the previous one is still running. Or if it's still running, it's better to cancel it and re-enqueue. But there is no such API in Active Job natively. Some providers have something to do with it, but it's not a framework-level concept. And we also need it for many workflows and pipelines.
What about the third-party solutions? They exist. They've been existing for a while. There's, for example, Acidic Jobs. We used them in a couple of projects. It introduces durable state, so everything happening with the workflow is stored in the database. We can restore it, we can observe it. There is built-in uniqueness, and so on. And what's important for me, at least, is that it has an Active Job interface. So, from the outside caller, the job is still Active Job. Everything you can use with Active Job works. And the whole durability of the workflow is hidden within the workflow class.
Another more recent development is Julik's Geneva Drive project. Actually, it comes with almost everything I wish a workflow implementation had. And we're going to refer to it a few times in the talk. But the biggest problem to me, it's not a Rails thing. It's a custom DSL, and it's a custom entity. It's not surely compatible with Active Job, so you need to refactor everything to use it. And I wish we had something more kind of framework-friendly, but with the same guarantees.
Okay, next pattern I want to touch a bit is batching. So, when we have one job performing the setup, many jobs performing their own task, and then something running on completion. And batches could be actually executed sequentially, and we can say that we can use continuation for that, and that's again an example from the wild world from Spree. We begin processing, we create a job per CSV file row, and then we do some finalization.
But actually, whenever we talk about batches, we usually imply parallel execution, right? And that's a problem because whenever we start parallel and we need to coordinate the end of the execution of many jobs, we need some coordinator. And typical coordinators are counters, possibly stored in Redis, and maybe polling like in this example. We just have a job that periodically checks, "Oh, everyone's ready? No, let's try again in 5 seconds."
And for this use case, we typically use third-party gems. But the good news is that since very recently, we can use a third-party gem. And Solid Queue 1.7 just got Solid Queue batch support thanks to JP Camara and Rosa, of course, who helped a lot with that. And it's been a long and winding road to batches in Solid Queue, and I highly recommend reading the story because it demonstrates what amount of effort is required to ship a major feature to a cornerstone library like Solid Queue when you are not a core contributor, an outside contributor. That's massive work and thanks everyone involved in that. That's cool.
That's just a quick review of how it looks. Everything is a job, so whenever we want to run something on completion, we create a job on success, on failure, and so on. This interface is inspired by GoodJob that had batches for a while. And we had many more examples of batching, with everyone coming with their own interface. That's a problem. But still, the problem is kind of solved. It just needs your choice on which library to choose.
And I want to highlight the one from Doximity, Simplekiq. It's Sidekiq-specific, but what's interesting here is that the interface is very similar to continuation. We have steps, called runs. We have parallel steps, something that's missing in continuation, but it hints, at least to me, that we can reuse the same idea of parallel steps, probably, in continuation. But there is just one blocker.
And what's the blocker? We don't have it in Active Job, and we'll probably have it at some point because there were some discussions about it. And as soon as the interface lands in Active Job, we can bring it to continuation and make our workflow batches-aware. But we need to wait.
And speaking of waiting, that's another pattern, another group of orchestration scenarios, background tasks that involve all kinds of timers, like reminders, expirations, notifications, and so on. For example, again from our AnyCable project, this is a typical flow. We have a bunch of recurring tasks, like polling tasks that run every 10 minutes or every hour or whatever depending on the scenario.
For the same process actually, this is a license expiration workflow, but it's split across multiple background jobs and each background job is actually an entry point to enqueue more background jobs and it introduces a lot of code concepts just to implement a single workflow. We need to repeat it for every workflow.
With continuation, it could be simplified a bit because we don't need this nesting. We can use iteration within steps and cursors and limit the amount of job classes to one. Still we need the scheduling thing, but that already looks better than without it.
Another pattern related to time is parking a job someday in the future with wait or wait until and then executing there. But the problem with this approach, actually there are a lot of problems with this approach. The first problem is that we usually need to validate if this job is still relevant because there is no, at least Active Job-level, API to cancel a job that is scheduled for the future. Yes, you can reach out for the provider-specific API, but usually that's just a lot of work and validation is simpler and more reliable. There are many questions related to this approach, but conceptually it sounds good, but technically it's hard to implement and we typically fall back to the cron-driven sweeps.
Let's take a look at the gems that implement something that makes this easier. This is another relatively new project called ChronoForge and it extends actually Active Job with custom, let's say, DSL method decorations including wait. So, whenever you enqueue a job, it reaches the wait statement and says, "Oh, I'm stopped here. I add myself to the waiting
queue or table, and then I'm being re-enqueued when time comes and continue the execution.
Geneva Drive also has this concept of timers. It's just an option for the step, and it also has this beautifully designed mechanism of associating workflows with models through heroes. So, you can easily find a workflow associated with some Active Record model and cancel it when needed. That's very useful. So, there is an API for that.
And another option is actually using third-party services, like Temporal, a pretty popular one recently. It has a lot of stuff. And what's interesting here is that that's kind of the same example with appointment reminder, but instead of canceling the workflow and restarting it from scratch when the date has changed, for example, we can just update the deadline for the reminder. So, it's also part of the state of the workflow when we want to trigger it.
So, kind of related to timers is a more generic group of workflows or features: approval workflows or signal-driven workflows or human-in-the-loop scenarios, right?
So, a signal is something that a workflow waits for that isn't time. It may be a user action. And typically in code, signal splits the implementation of the workflow into before and after parts, living in different places in code. Like here, it's from a Mastodon example. It's the same process. It should be read as one process with a break in the middle, but we don't have a way to implement that in Rails.
And given that signal is not guaranteed, we need to take care of cleaning up a task that hasn't been signaled yet. That adds a lot of ad hoc logic to your application.
Another example, another type of signal, very popular, is a webhook event. This is a real example from one of the apps I worked on, and we see that workflow is scattered across three files. So, we have the one job that requests the withdrawal, then we have a webhook that sends the confirmation or rejection, and then finally have another background job that processes the result.
If we migrate, for example, to ChronoForge, because it also has this signaling feature with continue_if step—not an obvious naming, but anyway—we could rewrite the workflow, this one, into just a single perform method with three steps. Run third step, wait for the payment receipt, run credit step. And it's much easier to reason about than finding all the logic in between multiple files.
But unfortunately, the project I borrowed this example from picked another solution to implement this workflow. Sorry, it's loading. Yeah. So, they picked Temporal, and I can't say I'm happy with that. So, I don't think implementing such things that could be expressed in English in a few words should cost us hundreds of lines of code and dozens of Ruby objects and scary voodoo magic to make them all work.
So, that's the problem. The more complicated, the more sophisticated your workflow becomes, the more sophisticated tools you have to use. And I don't think it's the right way of going into the future with Rails, right? We need more batteries included. We are missing some of them, clearly. And why not adding them?
About a year ago at a SF Ruby conference, which is happening this year again in November, by the way, don't miss out, I already shared this idea that now we have a cool concept for building more than just single jobs in Active Job, so we can build something bigger, some workflows. And let's not invest into building custom gems or ugly SDKs. Let's just build on top. Let's turn continuation into something bigger.
And I've been keeping this idea in mind for almost a year. And today, I want to present the first implementation of it. So, let me introduce the durable continuation as part of the job.
All right. What is durable, by the way? What do we mean by that, right? Everyone's talking about that, especially since the bureau coined this term and started using it everywhere, trying to make it look like it's something very special. It's nothing special. The main difference is that, basically, a job, the execution process, is a record or multiple records in a database or some other store, and not just a payload in the queuing system. That's the primary difference. Everything else is a consequence of it. So, like surviving SIGKILLs and so on and so forth. They're just the properties we get if we carefully commit the state and make it restorable and replayable.
So, to turn Active Job continuation into durable continuation, we need to implement a few things, the ones I've been already talking about in the previous section, right? That's missing. But the most important requirement, which I always add to anything I work on, any library, is that it must be as close to the Rails way as possible. It must feel like using the framework, not a third-party library. So, it should be an extension, not a replacement. Something that could be merged into Rails in the future. If you build your library like that, then you get some kind of style and taste constraints by default, and they usually help you to get a better implementation.
So, let's move to the durable continuation proposal. And let's start with the state. So, back to one of the examples. This is also from a separate via our cards generation job, the previous version, the one that generates cute characters based on the photos.
And it has multiple steps because we need to process the original image first, we need to moderate it. We do not allow any type of images downloaded and shown to everyone in the community. So, we have some checks there, moderation step. And finally, we generate the final content. So, that's how it looked in the original version. It's not durable yet.
What does it mean to make it durable? And what changed? So, we have a table with the execution runs and their state and step state and whatever. And we can see how it's being populated and updated as the execution progresses. So, first, we enqueue the job, and we got a record on the table. So, the process hasn't started yet, but we already know that it's being enqueued.
Then, as soon as we hit the worker, we change the state to—then we update each step as we progress. We keep the attributes in the state as well, so they're also durable. So, you can observe them from the outside, so you can build some features that use these attributes as data for your other operations.
And finally, you have an API to query this data easily, kind of similar to what Geneva Drive has, but we don't have the concept of heroes. We just infer identifiers from arguments or specifically defined identifiers on the job. And you can see what's happening with the job. You can cancel it if you want, or retry, or whatever.
Let's see how this state, durable state that we add, affects our resumability properties. This is the original webhook example using continuation. Imagine that we run the job, it triggers webhooks, but then it crashes. And the job restarts, and the cursor value hasn't been committed because it was a hard crash or some exception that hasn't been caught, and we start over because we don't have any memory of what's been in between the steps and within the step.
When we switch to durable continuation, just one word changed, right? Nothing changed in the class, just a single module. We have a different picture. We have cursor committed, not every step, actually, but you can configure it however you want. And that gives you better guarantees in case of different problems that could occur to your app. So, it just improves the reliability of the operation.
Now let's talk about the life cycle and what else we add to make workflow more useful. Remember I mentioned that while we don't have callbacks, we have to inline everything in our step bodies. And now, with durable execution, we also ship callbacks, after_step callbacks. That makes your workflow look like a typical Rails object with all the functionality typically available for models, controllers, whatever. It's no longer just a tiny wrapper over background job execution.
Similarly, we can fix the problem with terminating the workflow after a particular step, not just the whole thing, not just discarding everything. And there is a halt_on rescue handler. And it differs from discard_on because it allows you to restart the job when the condition is fixed, because some errors are recoverable. There is also uniqueness built in. Just because we have a database of runs, we can easily achieve uniqueness.
And let me show you how timers evolve. So, previously we had a scheduler and many jobs. Now we can have a single job with steps, and now steps have wait and wait_until, and they describe the whole process. And again, we have uniqueness here, and when we want to restart the life cycle workflow for a license because it's been renewed, we just re-enqueue the job and it replaces the previous workflow and continues working.
All right. Yeah, that's another example with timers, and it has Active Record integration, has_one_performed. So, you can easily access associated runs. And we're a bit out of time. I expected that.
So, we just have a couple of minutes left to show the other features, and I think the most favorite one. So, this is a previous Temporal example with signals. And when we convert it into durable continuation, it becomes just this. It's just a single file with a few configuration things, and that's basically it. The whole logic lives in a single Rails Ruby class, and you can observe it, you can control it. I don't know. It looks much better. And it looks better than English, I mean. It's Ruby. It has structure and clear meaning.
So, yeah, a couple of examples with signals. Again, the generation example, but now an upgraded version when we decide whether the photo is good enough. If it's not, it has a border score, we add a human to the loop and ask, "Do you allow posting this image on ads?" And now we do that by keeping a lot of states in the enum on the model, and just checking them and iterating in the job. But there are problems with that because there could be concurrent executions. We don't have history, we don't have deadlines for the review.
With durable continuation, we can change it with just a single line. It's a wait step. It's just a special kind of step that parks the job for a given period of time. It could be released after timeout, and then we can send the signal, say, from the controller or wherever we want to continue the execution. That's the whole API. And it works on top of the same primitives as timers, with just steps being empty bodies.
So, one bonus example. Recently RubyLLM 2.0 was released, and it has a lot of example snippets with continuations involved, but they were not as good as if they were using durable continuation, because now the whole loop could be implemented in a single job with restarting after approval, because we just have this halt method and you can call resume whenever you approve or not the action, and that also simplifies this workflow. We don't need durable state for real here because RubyLLM has its own durable state as chat messages, but still we need the control flow durable state to exist to implement it this way.
All right. So, that was a quick overview of the features we need to add to continuation to make it durable and more useful and drop all the stuff you're using for that. And it's available today. In a minute I'm going to open-source it. It's not yet available, actually, as a gem, AJDC, Active Job Durable Continuation, nothing related to music. And, yeah, it comes with everything I mentioned in the talk and more features coming.
It comes with AI skills backed by the Rails Hyperdrive project. So, all we need is to install the gem and tell your agents, "Configure durable continuation, find a job that would be nice to migrate and do the migration," and then that's it. Probably you'd like to read the code afterwards. I don't recommend not reading code.
And the idea behind it is to bring just enough orchestration to Active Job to stay on the Rails way, to not reach out for services or cryptic gems to implement these features. We can do that in Rails, and then you can see that it could be much better than using any of those. So, yeah, more features coming. I mentioned that.
That's the plan for you. Pay more attention to continuation. Try to see if you have these patterns in your code and they're worth extracting into something looking more like workflows, and give durable workflows a try. That's it. Thank you for your attention.
[applause]
[music]
Article published · Updated