Code Is Cheap, Developers Still Aren't: Joanna Wang on Rails, Workflow Orchestration, and Going Fully Agentic at Sixfold
Ruby on Rails
In this episode of On Rails, host Robby Russell talks with Joanna Wang, a senior software engineer at Sixfold, which builds AI underwriting tools for insurance teams. Wang came to Rails from Java and other languages and recently joined a team that has, by Wang's account, stopped writing code by hand altogether. The conversation covers why a homegrown workflow system built on state machines and callbacks became brittle, what the team gained by moving to Hatchet, the costs of relying on an unmaintained gem, and why agentic coding tools may make Rails' "magic" easier to live with. Wang's central claim is that code is now cheap, but the value of a software developer lies in architecture and problem-solving rather than in writing code.
Coming to Rails from Java
Asked what keeps Wang on Rails, Wang points to the maturity of the ecosystem. It has many maintainers and a large community and developer base, so whatever problem you hit, you are probably not the first to encounter it.
Rails was new to Wang at Sixfold. Wang studied computer science, started with Java in college, spent about a year on Python, and then worked in Groovy on Grails, which Wang describes as essentially another JVM language stack. Russell asked whether Wang had ever used JRuby. Wang hadn't. Russell explained it as a Java-based Ruby interpreter that let many organizations 15 to 20 years ago "sneak Rails into their Java environment."
What Sixfold Does
Wang describes Sixfold as "an AI underwriter." Insurance underwriters gather many data points, review documents sent by brokers, and then judge how risky a company or person is to insure. Sixfold replaces much of that human process. A case might start with just a name and a few documents, or with an email forwarded to the system. Sixfold then carries it through to a final assessment of how much risk the company represents.
On the AI side, the team uses different models for different evaluations. They evaluate against ground truth to confirm that questions are answered correctly and that a given model performs well, and they tune prompts through evaluation to make results as accurate as possible.
Where Rails Sits in the Stack
The main application runs on Rails. At the time of recording the team was in transition: workflows had been managed with a state machine gem in Ruby, and they were moving to Hatchet, a workflow orchestration tool.
Wang explained the difference in approach. The old system orchestrated workflows through code: the state machine plus Rails callback hooks such as on-create. With Hatchet, the team defines a DAG (a directed acyclic graph) of steps. It specifies when one workflow triggers another, when a workflow ends, and what conditions let the next step run. Hatchet then acts as the engine that executes the orchestration.
Engineering has roughly 20 to 30 people, and a significant share work in Python. Wang says the Rails and Python sides work "very closely." The Rails app serves as the web application, the worker that orchestrates jobs, and the only entry point to the database. The Python codebase is effectively a microservice for AI interactions. It sends requests to OpenAI or Anthropic and reports results back to the Rails app. No other microservice touches the database directly. Rails also serves the REST API. The frontend is built in React, not Rails views.
Writing Less Code, and Living With the Magic
Wang says Rails' biggest benefit was its lack of verbosity: you can write very little code. Wang called this "a double-sided sword" because the same property that saves effort had historically caused problems too.
Coming from Java, Wang experienced Rails as "a lot of magic." Early on, understanding code meant jumping from line to line, sometimes into library code, to discover that a helper existed under the hood even though nothing in the app defined it. Only then would it become clear where a method call came from.
The effect on colleagues was uneven. Some knew the internals well. Wang says the Python-focused engineers struggled whenever they touched the Rails side, at least before agentic coding, because "almost nothing is explicitly defined."
Wang still appreciates Ruby's dynamism. For example, you can construct a class dynamically and then call a class method on it, and conveniences like calling compact on an array do the work for you. Wang again described this as two-sided. It helped the team a lot, but it also caused confusion and frustration for people unfamiliar with the codebase or with Rails.
For the company's growth, Wang says writing less code mattered most. With very few people, and in an era when "you actually had to handwrite code," the team could stand up an initial app quickly and start selling AI underwriting to customers. Not having to list every database column or spell out every frontend response was "actually really like acceleration."
When Homegrown Orchestration Got Brittle
Before Hatchet, the team used the state machine gem together with ActiveJob and Faktory, which Wang described as basically Sidekiq but usable from other languages. That mattered because the team also started using Faktory in Python for job handling.
Wang described two separate problems. The first was Faktory itself: under heavy load, the team had trouble scaling. Queues would back up, and when they did it was hard to see what was happening or to fix it quickly.
The second, and more serious, problem was the in-house orchestration built on state machines and callbacks such as after-create hooks. As workflows grew more complicated, the system became very brittle. State management was scattered. Transitions depended on database transactions completing properly, and each transition triggered further side effects to start the next workflow. Wang says this setup created race conditions. The team added more logic to handle them, which scattered the logic even further.
Fan Out, Fan In, and Race Conditions Everywhere
Russell asked for a concrete example, and Wang described web ingestion. For a given company, the system searches for information online, scrapes the relevant websites, turns them into documents, and processes the web content.
The team found that scraping often returned uncertain results. The same company may go by different names, and unrelated companies may share the same name. So they added a step: after scraping and ingesting documents, they check whether each document is actually relevant. That check fans out across documents and then fans back in. Meanwhile, user-uploaded documents for the same company may be processed in parallel.
The fragile moment came when everything had to fan back in, so the system could declare document or web ingestion complete and move on to analysis. Jobs were spread out, fanning out, calling the Python microservice to judge relevance, and then fanning back in. Wang called it "a nightmare," with a race condition "at every possible place."
Buy Versus Build, and Piloting Hatchet
Russell asked whether the team had hesitated to hand this problem to an outside platform. Wang said both instincts were present. The team generally prefers buying over building when a better tool exists. It also seriously discussed rewriting the whole system, partly because, as Wang acknowledged, the team may not have thought the architecture through well as it added complexity. A rewrite of these workflows had been under discussion for a while, even before Hatchet was available.
Adoption was gradual. Some other internal tooling used Hatchet first as a pilot to see how well it handled workflows. The team then moved smaller pieces onto it to address specific problems. Wang says it proved possibly more efficient and easier to maintain, partly because of its dashboard, where you can view and cancel jobs. Wang said the decision drew on both considerations: the long-standing desire to rewrite and the availability of a better tool.
JSON API Resources: Empty Controllers and Rigid Errors
Another key piece of the stack is the JSON API Resources gem. It produces JSON:API-spec responses for REST endpoints and connects "almost directly" from the database to the controller layer. Wang says the team's controllers are "almost 100% empty." The files have to exist, but the gem handles the rest. For a new model, the team barely writes anything and gets a standard JSON:API response back, with no serializers, queries, or ORM calls to write.
Authorization sits alongside the gem. The team uses Pundit, defines read and write roles, and checks for the right access. JSON API Resources also lets them declare which operations a resource supports, such as create, read, or index.
The downside, in Wang's words: "With the magic comes the rigidness." Error handling is the clearest example. The team can define ActiveRecord validation errors as usual, but the gem has its own fixed list of error types and its own way of populating them. That makes it hard to surface errors the way the team wants, and customization is painful. On top of that, the gem is no longer maintained, so its issues won't be fixed upstream. The team keeps its own fork with some fixes. Russell checked the gem's GitHub during the conversation and noted that the most recent update, about three months earlier, fixed a Ruby 2.6 logger compatibility issue for Rails 6.0.
Dependencies and Supply Chain Security
Russell asked whether the team's approach to external dependencies had changed with agentic development. Wang said the philosophy is largely unchanged. Hatchet illustrates how they treat paid vendors, and for open-source gems the team still looks first for existing tools. As Wang put it, the company's value lies not in building small tools but in building the overall system for insurance underwriting.
Russell raised security, mentioning that upcoming versions of Bundler and RubyGems were announced to include a cooldown feature to avoid upgrading too quickly into a compromised package. Wang said security had been a real concern, prompted not by Ruby incidents but by the npm attacks, since the frontend uses TypeScript. The team runs automated tools that monitor for new vulnerabilities and prioritizes fixing them quickly. For major upgrades, such as Rails itself or a major ActiveRecord update, they skip routine Dependabot upgrades and do a proper migration with a person verifying that everything works, "even in the age of AI."
Testing and Observability
For testing, the team uses RSpec, FactoryBot for test data, and Capybara for some end-to-end tests. Datadog handles essentially all monitoring.
Going Fully Agentic
Asked about AI tools in daily work, Wang said the team uses "basically every MCP." At one point the organization deliberately worked to bring everyone up to speed on agentic coding, and Wang considers the migration to fully agentic coding now complete.
Wang named two frustrations with the tools. First, they sometimes don't do what you ask, or you have to phrase the request very specifically to get what you need. Second, they can be "overly defensive." Wang's example: Claude might add around 20 extra lines to validate that a string isn't empty and contains no odd characters, even though the string comes from upstream code where that case can never occur.
When Russell asked what "fully agentic" means in practice, Wang was direct: "we don't write code anymore." Practices vary in reading code. Some people still read what agents produce, while others have reached the point where "if it works then it's good" and no longer care to read it. Across the organization, nobody writes code, and people mostly don't review each other's PRs either, because agents review PRs too. Much of the engineering work has shifted to building harnesses for agents.
Harnesses for Agents
Wang described two parts of that harness. First, CodeRabbit reviews every PR. The team has tuned it and given it guidelines about what to watch for. Second, everyone is asked to write documentation, including an agent.md file. It records the pitfalls encountered, how a feature is designed, the general architecture, and the patterns in use, so an agent extending a feature later has guidelines to follow.
What the Job Becomes When Code Is Cheap
Wang is excited about the shift: "what a time to be alive," and a remarkable time to work in the industry, because it has "completely changed what it means to be a software developer." Writing code once carried an almost elitist status, Wang said, but "nowadays code is cheap."
Wang doesn't think this erases the developer's value. It relocates it. The value now lies in thinking through a solution to a problem, designing an architecture that is robust rather than brittle, and knowing how to scale it. Wang added that they never personally believed the job was about writing code, since syntax and library documentation were always available online. What's different now is that this has become explicit.
Why Agentic Tools Pair Well With Rails Magic
Wang argues agentic coding is especially useful for Rails because of the magic. In more explicit languages, a global search finds a method's definition. In Rails, the method may be constructed dynamically or come from a non-obvious framework helper. With Claude Code, Wang says, that is no longer a problem, because you can ask it to trace where things come from.
This has changed how the Python engineers work. Previously, they would describe what they needed on the Rails side and hand it to Rails engineers. Now, Wang says, "they can actually take it to done" themselves.
CLI Tooling and Claude Skills for Dev Setup
For developer experience, the team has long had clear instructions and commands for starting a dev environment. It has also built CLI commands for production-specific tasks, so engineers don't have to open the Rails console. Many originated in customer support requests, such as a particular kind of export. Early on these were written with Thor. When a customer request kept recurring, the team would turn it into a CLI command.
The team later built a separate hosted internal tool to handle most customer support requests. Wang described it as a kind of playground for customer support, and it goes through Rails rather than connecting to the database directly. More recently the team has added a set of Claude skills, including one that sets up the dev environment for you.
Russell mentioned that other teams he has spoken with build Claude skills that let customer support staff run natural-language requests against a protected Rails console. He also recalled a recent Rails addition, whose name he couldn't remember, that wraps Rails runner functionality to generate queries.
Multi-Tenancy, Compliance, and Regional Data Residency
Sixfold is multi-tenant. Wang cited the requirements of the insurance domain: HIPAA compliance, SOC 2 compliance, and some clients whose regional rules require their data to stay in a specific region. Combined with a heavy data-processing load, Wang says, a multi-tenant setup made more sense.
Infrastructure is split by region. Each region that needs it, such as the EU, gets its own hosted instance of the components, including Hatchet. The team can deploy to regions selectively but generally deploys to all of them at once.
Feature Flags and Moving QA to an Agent
Feature flags tie into multi-tenancy, because different clients want different features enabled. The team started with PostHog, then moved to its own approach: environment variables plus an internal tool for flipping them. Most feature flags live on the Rails side. Features specific to the AI side, such as a particular prompt, are controlled by an environment variable on that microservice. The internal platform can also apply a feature to a defined set of customers.
The team doesn't do percentage rollouts, such as enabling a feature for 50% of customers. Instead it rolls out 100% to a group of alpha or beta customers, or to UAT and sandbox environments first, and then to production.
QA is partly automated through frontend tests and Capybara. Full QA was once entirely human, and the team at one point had a dedicated QA person. After a few iterations, they now use QA Wolf.
Advice on the Way Out, and an Open Question
Russell asked what Wang would tell a successor in five minutes on a last day. The answer was short. The current workflows have a lot of race conditions, so watch out. The goal is to move things to a better state. Most things are documented somewhere in Notion, and if you have Claude, you'll be fine.
For a book recommendation, Wang said they have historically recommended Clean Code and Clean Architecture by Robert C. Martin ("Uncle Bob"). Wang then questioned whether that still holds in the age of agentic coding: "maybe messy code is okay as long as it works."
Russell left the question open. He described the tension between knowing when to look under the hood and when to rely on agents, and the need to steer agents toward coherent, maintainable code. Otherwise, he suggested, teams may burn tokens in the future the way they once burned human hours on messy codebases, and he noted that the industry is currently in "a slightly subsidized token era." Neither offered a definitive answer on whether clean-code discipline still matters when agents write the code.
Welcome to On Rails, the podcast where we dig into the technical decisions behind building and maintaining production Ruby on Rails apps. I'm your host, Robby Russell, and I run Planet Argon. For over 21 years, we've helped teams maintain and evolve their long-lived Rails apps, so I tend to approach these conversations through that lens. In this episode, I'm joined by Joanna Wang, senior software engineer at Sixfold AI. Sixfold helps insurance teams analyze complex cases using a mix of document processing, automated workflows, and AI-powered tools. Joanna has worked across Java, Go, Python, Node, and Ruby throughout her career, which gives her a unique perspective on what Rails does well, and where teams can run into challenges as their applications grow.
We'll talk about what Rails has enabled for her team, the realities of managing increasingly complex workflows, and whether agentic coding tools are changing how developers think about Rails' famous magic. Joanna joins us from New York City. All right, check for your belongings. All aboard.
Joanna Wang, welcome to On Rails.
Thank you. Thank you. It's great to be here.
I'm so glad that we were able to finally make this happen. I think we had a conversation maybe three or four months ago and we're like, "Oh yeah, this would be a great conversation." So, we're here. Let's do this. All right. So, Joanna, let's start with this. What keeps you on Rails?
Well, Rails has just a very almost complete ecosystem. There are just a lot of maintainers and it just has a really big community and dev base, so that if you have any problem with it or anything, you're probably not the only one, or you know, you're not the first one to encounter almost anything. So, I think that's a really great thing.
Now, we'll talk about Sixfold in a moment, but if I recall, you didn't work with Rails prior to your current position. Is that correct? Correct.
Just to set the scene a little bit, what technology, programming languages, stacks were you kind of working with prior to Ruby on Rails?
I started with Java in college. I was a computer science major, and then I worked on Python for about a year or so, and then I was back on Groovy on Grails, which is actually basically also a Java JVM language suite.
Did you ever, out of curiosity, work with JRuby at all?
No. Okay. Do you know what that is? No. Okay. Well, it's actually a way that you can run your Rails app within the JVM. So, it's a Java-based Ruby interpreter.
Interesting.
It was a way that a lot of organizations back maybe 15, 20 years ago were able to kind of sneak Rails into their Java environment and deploy it. I don't fully understand how it all works, but it's still a thing, for those listening out there. But all right. Well, great. So, you have set the scene: to work with Java, some Python and things like that. So for our listeners who might not be familiar with Sixfold, can you give us a sense of what the company does and the kinds of problems you're helping solve as an organization?
Yeah, we are an AI underwriter. So basically when insurance underwriters have to underwrite cases, they need to do a lot of, you know, basically data points gathering. They look at the documents that are sent to them by brokers. Then they make a decision of what the risk of this insurance, this company or person, is. We kind of replace a lot of the human process in there. We basically streamline it from, you know, you have a case with a name, with maybe some documents. It could be from an email. You can forward us an email. Then we'll basically just take it all the way to the end where we give you, okay, this is how much risk we think there is for this company.
With that workflow, it sounds like you're ingesting a lot of data, kind of a lot of complex things to try to make sense of. And I would imagine with insurance, I'm still trying to wrap my head around a little bit around the AI and how to describe this to, I'm air quoting, non-technical people, in the sense of non-deterministic output, and being able to say what's the risk level here. What sort of technology and tooling have you been able to use there to make that kind of happen? And then maybe tell us about how Rails then kind of fits into that.
Yeah. So the main app right now is actually on Rails. We're in this transitional phase where we were using the state machine, which is actually a gem on Ruby, and then we are moving to Hatchet for basically workflow management. And then on the AI side, for every kind of evaluation we use different models. We do evaluation with ground truths to make sure your questions are being answered properly, like this model is actually good. We do tune our prompts with evaluation to try to give the most accurate result possible.
Can you tell us a little bit about what's Hatchet?
It's a workflow orchestration tool.
And so is that basically a way to move your state machine type logic into a different platform? Is that— Yeah. Now, is that some sort of WYSIWYG, map it all out that way, or do you write code to orchestrate that in their tool, but then you're kind of letting them manage that workflow?
Let me just give you some background on how we did it. We have our code that orchestrates all the workflows with the state machine, the Rails callback hooks like on create and stuff like that. Now we're moving to Hatchet, where you just get to define, you know, this DAG, step after step, and then when does this trigger another workflow, when does this end, what's the condition under which a certain next step runs. And then Hatchet itself will be the engine that actually orchestrates it.
I see. Out of curiosity, what does the engineering organization look like today? Roughly how large is the team on the engineering side?
I think we have about 20 to 30 people on engineering right now.
I think you mentioned in our prior conversation that there's a lot of Python developers there, I believe, working on Python things, and there's some folks maintaining and working on the Rails app and stuff. So how much crossover has historically needed to happen between the teams?
Quite a lot. We actually work very closely, because basically the Rails app is the web application, the worker for orchestrating the jobs, for database access, whereas the Python codebase is basically the microservice for any AI interaction. So they're in charge of making a request to OpenAI, to Anthropic. It will then report back to the Rails app.
What do you think Rails has enabled your team to do particularly well?
It's very... what's the opposite of verbose, I guess? So it's really not verbose, and you can write very little code, which is really nice. It's a double-edged sword. It has its benefits, which is, you know, yes, we write a lot less code, but then I think historically it had also created some problems, maybe.
When you started working with Ruby on Rails, people throw this word around a little bit, maybe a little overused, but did it feel like there was a little too much magic at all? How would you have described it at the time, when you speak to the verboseness aspect of it, or the lack of it?
It was a lot of magic. I mean, to me it was a lot of magic because I came from Java, which is a very verbose language. So for a little while it was, I wouldn't say difficult necessarily, but it was a lot of, oh, what is happening? I have to keep jumping from this line to that line, jump all the way to maybe the library code to see, oh okay, so then there is a helper, because, you know, you don't actually need to define it, it's just under the hood, it's already defined. And then you kind of see that, then you understand where this specific method call is coming from.
Yeah. What about your peers? Did many of them have a Ruby or Rails background prior to—
Some. I think they expect or they kind of know a lot of the under-the-hood stuff. But I also think, especially for the team that works mostly on Python, when they ever have to touch the Rails side of code, they have a lot of trouble, because a lot of it, or at least before agentic coding, you know, we can't avoid that now, but prior to it, they were definitely having a lot of trouble understanding what is going on, because almost nothing is explicitly defined.
What parts of Ruby on Rails do you feel have been most beneficial, or what did you appreciate the most, relative to other platforms and things you'd worked with in the past?
I do like how dynamic it can be. As an example, one thing is with Ruby, you could literally dynamically construct a class and then make a class method call. That is really, really cool. And a lot of things like that, and then you can just be like, oh, do compact on an array, and then it does everything for you. The ease of using it is really great, and then the ease of making things dynamic is also really great. But again, I feel like it has two sides to it. I think it benefited us a lot, but it also creates some confusion and frustration when people are not so familiar with the codebase or with Ruby on Rails.
So maybe for folks that are kind of just jumping in and out of it and aren't working in it on a regular basis, do you feel like it's one of those things where it's, where is this defined in the codebase, and it's not super obvious or something?
Yeah.
I can appreciate that. So, what has Rails allowed your team to accomplish during this important phase of your company's growth over the last few years?
I think not having to write as much code was very important. Being able to make things more dynamic accelerates the early setup of an initial app where we can sell, where we can actually enable our customers to do the AI underwriting. That was very important because we had very few people. It wasn't the agentic coding era. You actually had to handwrite code. Not having to, for example, list out every column in your database, not having to list out every response to the front end, that was actually really an acceleration.
Now, has Rails been used for a lot of the customer-facing views and things like that, or is it more of the behind-the-scenes tool?
Yeah, so we use it for the REST API as well. We don't use it necessarily for views. We do use React for the front end, but the REST API responses, all of that is still handled by Rails.
What about from a database perspective? What have you been running that in? And then is everything historically running through Rails, or does everything touch their own databases?
Yeah, the Rails app is also the entrance to the database, the only entrance to the database. Other microservices do not actually touch the database.
Interesting. Can you give us a little bit more background on the migration to using something like Hatchet? Historically, you mentioned you're using state machines. So if you were going to be migrating to a new platform, could you tell us a little bit about some of the pain points that you found? And out of curiosity, which gem were you using? Was it just an access state machine type of thing, or—
Yeah, it was mostly just state machine. I mean, we were also using Active Job. We were using Faktory, which is basically Sidekiq but can be used in other languages, because then we also started using Faktory in Python for the job handling.
Interesting. What were some of the limitations of using the gem? Was it just that it was too consolidated within the Rails app, or did you want to leverage that elsewhere? Or did you feel like, maybe from an architecture perspective, you were relying on feeding too many jobs to the Rails app to then have it do something, when maybe the microservices could be just dispatching their own jobs, so to speak?
Yeah, a couple things. One is on the Faktory side, I think under severe load we were having problems with scaling. It gets backed up, but it's hard to see or to quickly resolve it when it's backed up. So that's one problem that's more or less on Faktory. But then on the other hand, we built our in-house orchestration with the state machine and the callbacks, for, you know, after create, stuff like that. So that becomes very brittle as the workflow becomes more and more complicated. It's very, very scattered how we manage states. It's very reliant on, you know, your database transaction actually being completed, all of that working well, to be able to transition. Then with that transition itself, we have some other side effects that come with the transition so that, you know, we can push for the next workflow to start. But then that also makes it very brittle, because sometimes you create a lot of race issues, and then to handle those you add more logic, then making the logic even more scattered.
Out of curiosity, off the top of your head, are you able to walk us through an example of some of that at a high level, of what the orchestration might look like for an actual use case, just for our listeners?
Yeah, for example, we do web ingestion for a company, which means we would go search for information about the company online. Then we scrape those websites, create those documents, and then process that web content. But the problem came as we realized sometimes when we were scraping the web, it's very hard to say this is actually the correct website, because there are sometimes different names for the same company, or same-name companies that are actually not really related. So we started doing this thing, which is after we scrape the web, we ingest the documents, then we're going to check if every one of them is relevant. That means there is a fan out, then there is a fan back in, while we could also have user-uploaded documents for this company. So these are all running in parallel. So now by the time you want to fan all of them back in to say, okay, now we're done with this document ingestion or web ingestion, we can move on to the analysis. That step became very brittle. We see so much racing with it, because they're all spread out. They are fanning out, we are starting a job on the microservice that's doing, you know, the AI stuff to decide if it's relevant, then it fans back in. It became a nightmare. At every possible place there is a race condition.
That sounds like a fun challenge. Reaching out for a platform to maybe assist with that, was part of that, do you feel, just outside of the capacity of what your team could try to solve and keep figuring out amongst yourselves without needing to bring in some more external or additional help for a while? Because I know there's companies that are doing really complex things within their apps from time to time. But also, it's an interesting thing to be like, maybe we should reach out for a platform that might be able to assist with that and let them figure out some of that stuff. I would imagine it doesn't solve all of the problems, right? But maybe it addresses enough of it that it becomes, oh, let's experiment with that. Do you remember some of the conversations, or was there any hesitation, like, why would we pass this off to someone else, we can figure this out ourselves type of thing? Or does your team have a culture of, no, we're going to use the best tool for the job, and we'll move off and try something else if that's going to
solve the problem for us?
I think both are true for us. One being, I think we do try to buy over build. If there is a better tool out there, we would try to use the better tool. On the other hand, we did have the whole conversation about maybe we just rewrite this whole thing because, you know, to some extent it was also because as we add more complexity, we maybe didn't think through the architecture that well. So there was also the side, you know, maybe we don't have to do this. We just rewrite.
To be fair, the rewrite of these workflows has been on the table for a while, even before we even knew, or even before Hatchet became available at all. But at the time it was also like, okay, there is Hatchet, let's maybe just try. We did have some other internal tooling that started using Hatchet as a pilot to see how well they do with workflows, and then we started moving smaller things onto Hatchet to solve some of the problems, and then it's proven to be possibly more efficient and easier to maintain because, you know, they give you this whole dashboard where you can cancel jobs, look at jobs, and, you know, that decision was made on both ends.
Nice.
This episode of On Rails is brought to you by ApplicationController, where all roads lead and no logic escapes. Do you need to share a method across all controllers? Add it here. Want to run a callback before every action? You know where to put it. Do you need a place for auth logic, flash helpers, and a little panic code? It's waiting for you. ApplicationController is the one file that's holding your app together
and also maybe holding it a little hostage.
Side effects may include confusion, long scroll sessions, and blaming something in here for half of your bugs.
ApplicationController, because if you don't know where it goes, it probably goes here.
When we were preparing for this conversation, you mentioned, I think it was JSON API resources. Could you tell us a little bit about that and what type of problems has that been helping your team solve since it was introduced?
It's a gem that gives you a JSON API spec response for your REST endpoints. It hooks almost directly to your database, to your controller layer. It's all the way. So you don't need to write anything that's like a create endpoint. Our controllers are almost 100% empty. The files are there and you still need to create those files. But those magic
Yes, it's a lot of magic. It's great. It's great in the sense that whenever you have a new model, you have a new entity or data model, you barely need to write anything and it already gives you back a very, you know, it's a JSON API standard response.
Interesting.
You don't need to write serializers either. So
Is it everything within Active Record and Active Model where your business logic can kind of live and kind of protect everything, or are there some interesting layers where you do have to... is there a DSL that it has to say only these things are exposed, so you can't update certain columns or attributes or something like that?
So we do use Action Controller. We do use Pundit for authentication and some of the authorization. We do define roles of read/write, and then we do check, you know, if they have the read/write access, and then with the JSON API resource you also get to define if you want a create, you want a read, you know, index, list, anything like that.
Was that introduced prior to you joining the organization, or was that... Did you run into any interesting challenges with that, or has that in general actually done its job pretty well?
With the magic comes the rigidness of it. It's very rigid in what you can do with it, how the errors can be handled. For example, if you want specific errors when you're doing, for example, a create, and then you do model checks, right? So generally with JSON API resource, you don't need to do anything. You don't even write a query. You don't write the ORM query either. You don't write none of that. But you can still define, you know, my error classes with Active Record, right? Like the validation errors. But then with JSON API resource, those errors are very hard to then surface in the way you want it, because it has its own very defined list and way of how it populates errors.
I see.
So then, yeah, whenever you want customization to it, it's a little bit of a pain because you try to add to it. On the other hand, the other thing is this gem is now not maintained anymore. So issues that come with it just aren't going to be solved. So we kind of have our own forked version of it where we solve some issues.
What's your team's approach to picking and deciding when you'll bring in an external dependency? Do you feel like your team's philosophy has changed much in the last six months or so, now that presumably LLM, maybe agentic development, is a little bit more prevalent in teams now?
The introduction of Hatchet is actually kind of an example of how we deal with vendors. I think we generally would like to find, especially if, for example, Hatchet is a bigger tool, right, you got to pay, but if there are open source gems that do things for us, we would definitely try to, you know, look for tools that are already out there. I guess the mentality is that our value is not in creating those smaller tools, but in creating this overall system for insurance underwriting.
The reason I asked about that is I feel like I've talked to some people where they feel like their teams are revisiting that approach a little bit, especially when it comes to bringing in, let's say, a Ruby gem or some open source library. And you used that example of JSON API resources, maybe not as actively maintained, and I just looked at their GitHub; it was updated three months ago, but it was to fix a Ruby 2.6 logger compatibility thing, and for Rails 6.0. So yeah, it's a little behind. Anyhow,
but when it comes to security in particular, it has been a thing where we're pulling in external dependencies. In particular, there's a lot of other attack surface level things that everybody's being a little bit more mindful of. Like RubyGems, they just announced that the new version of Bundler and RubyGems will have a cooldown feature so that we're not automatically upgrading things too quickly and bringing in a potential security risk, because maybe an LLM just installed the gem or what have you. Hopefully people are more thoughtful and catch that stuff. But yeah, I was just curious if that has been at all a discussion there about how your team's thinking about that, even maybe beyond just your Rails app.
For sure, the security issue has been a problem. I mean, not on Ruby or Rails, but, you know, the npm attacks, because we also use TypeScript for the front end. So the npm attacks have definitely raised our awareness of security vulnerabilities. So we do have a lot of automatic tools that are monitoring, you know, new security vulnerabilities, and we put a lot of emphasis on fixing them as soon as possible. And then with bigger, larger updates, especially, for example, Rails or anything Rails-related, like Active Record got a major update or anything like that, we actually don't do the general Dependabot upgrade. We make sure we do a proper upgrade or migration, having someone, even in the age of AI, we're having someone make sure that everything works.
Nice. Well, out of curiosity, what sort of testing tooling has your team historically been using from the Rails side?
We use RSpec and then FactoryBot for factory items, and then we also use Capybara for some end-to-end testing.
Are there any other interesting tooling? How about APMs or observability tools that your team tends to rely on?
Yeah, we use Datadog for basically all monitoring.
Has your team been able to leverage some of the LLM tools with Datadog at this point? And out of curiosity, are you using much LLM tooling day-to-day in your coding activities?
Yeah, so much. We have basically every MCP. We are migrating to, well, at this point I think the migration to fully agentic coding is done, but at one point we were just really being aware of this change and really trying to bring everyone in the organization up to speed with agentic coding.
What were some of the challenges that you found through that process, regardless of the Rails part of it? I'm just kind of curious about that.
Yeah, sometimes it doesn't do what you ask it to do, or rather you have to ask it in a very specific way for it to do what you need it to do. And I do see it also sometimes being overly defensive when it's coding.
Tell me more. What do you mean by that?
Sometimes there are things where it's like, oh, we need to prevent this string being empty or having any weird characters, but we know if that string is fed from upstream code
it's never going to happen.
But then Claude would sometimes be overdefensive and just try to be like, hey, writing, you know, just 20 extra lines just to be like, oh, let's just make sure that we catch every possibility of the format of this string.
I can appreciate that. And when you mentioned you're fully agentic at this point, what does that mean for you right now at Sixfold?
We almost, well, not almost, we pretty much don't write code anymore. We don't write code anymore, basically.
Like in the 80, 90% plus perspective of most of the code, or even more so, is almost entirely, right? Like, are you orchestrating a bunch of agents right now at this point in parallel, or is that all happening kind of in your local environment? I know by the time we record this, it might change by the time it gets published, but just kind of getting a sense of where your folks are at at the moment.
Yeah, we are fully agentic, or we are basically not writing any code. I think we're at a point where some people are still reading the code, but some people are fully at the point where, if it works, then it's good. I don't care to read the code anymore. I guess this is where the org is at, but in general nobody's writing code anymore. We don't review other people's PRs mostly. We do have agents also review the PRs. So a lot of the work is actually putting in the harnesses for the agents instead of just writing code.
Can you tell us a little bit about what some of those harnesses look like?
Yeah, so one, we have CodeRabbit that is reviewing every PR. We do tune CodeRabbit. We do give guidelines of, you know, what you need to be aware of, and then we do ask everyone to basically have documentation, have an agent.md file, which is, you know, these are the pitfalls I've had, this is how this is designed, this is the general architecture, this is a pattern, so that when an agent is coming to make more additions to a feature, it has all these guidelines that it needs to model.
How has that been for you as a software developer? Are you embracing it? You're excited, optimistic about this, or has it been kind of an
I think it's an exciting time. To me it's like, oh wow, what a time to be alive. And what a time to be working in this industry, actually, because I think it completely changed what it means to be a software developer. In the past, writing code was, you know, some people might say it's an elitist thing almost. But nowadays code is cheap.
I don't think the value of a software developer is gone. I actually think it's more like, you know, the value of you is not about writing code. It's about thinking through a solution to a problem. It's about how you architect something so that it is not brittle, that it is robust and you are solving a problem. You know how to scale it. I think that's really where the value of a software engineer is at now. It's not so much about code. To be honest, I myself never thought it was about writing code, because syntax is always online, documentation was always online, your library is always there. But I think now it's very explicit in that way, such that, you know, code is cheap. It's easy to write.
Do you think that AI-assisted development changes how teams might be thinking about those Rails abstractions? Do you feel like you've noticed much with how using AI to help maintain the Rails app in particular has been a positive thing you've seen, and like, oh, it seems to actually work really well with Rails? Or what's your experience like?
Yeah, I think it's especially useful for Rails because of the magic. You know, in the past you had to go through the code, you had to go very deep in the lines, because with other robust languages you see it, you do a global search, you see that method, you see the definition. But with Rails, a lot of times you don't see it. Sometimes it's maybe constructed. Sometimes it may be just a Rails helper method that's not obvious. But with Claude Code that's not a problem anymore. You know, it knows. You ask it, it can actually trace for you.
That's interesting. You mentioned earlier that there are people that kind of jump in and out of Rails with other tool sets that you're working with there, that agentic development has been able to make that a little bit easier for them because they didn't understand the magic.
For sure. For sure. They can actually do the Rails stuff. In the past it was more like, okay, we kind of have this ask of how we want the Rails side to be, and then they kind of hand it off to, you know, the engineers that are working on the Rails side. But now they can actually take it to done.
Out of curiosity, with the Rails app, has your team built any interesting tooling to help historically with onboarding or developer experience? Any interesting little tools that are very, very specific to ramping up things? For example, do you have a pretty healthy seed system, or how does your team debug production-level type things in a local environment with the Rails application part of your platform, or is that kind of a challenging thing for you?
I think we did have some tooling for just getting things up. We did have, you know, clear instructions and commands that you just run to start your dev environment, and then we also did a lot of CLI commands to be like, okay, you know, it's production, there are some specific things you need to do, you can just run this, you don't need to necessarily go into the Rails console. And then we sometimes have customer support requests where they need a specific type of export or something like that. So we had a lot of CLI commands that were built for this reason. Now we also kind of have a set of Claude skills, especially for setting up dev. There is a whole Claude set of things that you can just run, and then it'll take care of setting up the dev environment for you.
That's cool. Does your Rails codebase have a bunch of dead pieces of rake tasks? You mentioned you're doing customer support and needing to generate some weird reports. How does your team typically manage that type of work historically, and do you feel like that's a little bit different now in your codebase?
So in the very beginning we used Thor for writing CLI commands. So most of the time, you know, if a customer request keeps coming, we then write a CLI command you just type
in what you need then it, you know, like it does what you need to do for it. Then we actually started this internal tool completely for handling like the mass majority of like the customer requests, which is just completely like its own, it's like hosted, and it does a lot of like explores, like the playground for the customer support.
Interesting.
Yeah.
And then is that just connecting back straight to the same database then, kind of bypassing Rails, or does it go through Rails at all?
It actually just goes through Rails.
Okay, that's cool. Yeah, I was talking to a couple people recently and they were talking about how they've been building, I don't know if your team, your organization done that, but they've been building tools for like Claude skills that allow their customer support people to actually run Claude skills against the Rails, like Rails console, and they have like a protected version of the Rails console, other things like that, so they can kind of like let it be a more natural language and stuff like that. Seems like there's a lot of teams trying to figure out how to like take advantage of some stuff. And I think Rails recently came out with a, was it Rails query or something? Forget the name of it, but it basically, kind of a little bit of a wrapper for like the Rails Runner things. So you can pass, it'll generate queries for you or something like that. Like that's kind of neat and interesting.
Right now I'm curious, when it comes to like AI and the state of things at Sixfold, when it comes to, you know, the Rails app, and you talked, you shared a little bit about the orchestration and the underwriting process, out of curiosity, is there anything about the data that Sixfold manages that's, is there any interesting challenges you've had to do with like keeping, segmenting data, or do you have to do like a multi-tenant type thing, or is like the data that's being shared into the system very, very important that you need to compartmentalize it from different customers and different clients, and that beyond just being like here's a row in a database with a column? Or tell us a little bit more about that.
Yeah, we are multi-tenant because we deal with insurance. We have to be HIPAA compliant. We have to be SOC 2 compliant. Our clients sometimes, if they're in a specific region, they actually have compliance where like your data can only be retained in that specific region.
So with all of that it just made more sense, and also we do have a pretty big load in terms of like data processing. So with all of that I think like the multi-tenant just makes more sense.
Is the infrastructure then distributed across several regions then at that point, to make that work?
That's interesting. Okay, like how does that then work with something like Hatchet? Is that then also needing to be isolated in different regions as well?
Yeah, we have specific, like basically for every, like for example EU, like for every region like that where it needs to be, there is like one for every, like one of those hosted things.
And is the app basically then deployed out to each of those regions in parallel, and is it the same deploy, or do you phase deployments out to different regions at different points and it's kind of like a one by one, or you can be selective about that approach?
We could be selective, but like in general, we just like deploy all at once. Yeah.
Okay. Do you lean on a lot of like feature flags or anything?
Yeah, that's actually the other thing, is like a lot of like our different clients actually want to use different features. So that also plays into the multi-tenancy, is like, you know, we get to turn only certain things on for specific clients.
And what sort of tooling do you use for that?
We started with PostHog.
Yeah.
But then we actually started doing our own internal. We're just like env vars, and then we have this internal tool for flipping them.
Have you found like a good pattern for how to do that? Is that across your different stacks as well? So it's not just like something in your Rails app, but if you're using environmental variables and flipping switches somewhere, like what's the architecture of that kind of look like a little bit, if you don't mind sharing?
Generally, most of the features are maintained on the Rails side. If it's a feature, you know, that we want to turn on for some clients but not the others, for example, maybe like a prompt or like some specific things are on the AI side, then it would be an env var on that microservice. We do have an internal platform that handles it. No, maybe a set of customers have a specific feature. So we also have that to be able to apply to just a set.
Do you use any of that tooling to like roll out new feature ideas to a subset of clients before you roll them out to everybody, or to test things out before you fully roll it out to everybody?
Yeah, we don't do the, you know, like 50% of customers on a place. We don't really do that. We generally roll it out 100% to a group of like maybe alpha or beta customers, or we roll it out into like the UAT and sandbox environments first, and then we would roll out to like the actual production.
You mentioned like things like Capybara, but does your team have like a QA type team there that is always some humans doing that, or is that somewhat automated at this point?
It's somewhat automated. We do have frontend testing as well. And then, but then the full, really, like, you know, QA testing used to be completely human. We went through a couple iterations. I guess we had like a dedicated QA person. We're now using QA Wolf to do the QA for us.
Nice. So Joanna, I want to ask you something. So let's pretend, we're do a little exercise here. Let's pretend it's your last day at Sixfold and someone else is going to come join the organization, and you had five minutes with them and you needed to quickly say, "Hey, this is a typical Rails app." Well, you mentioned, you touched on the JSON API resources. You don't need to go too deep into that, but let's say here's two or three things you should probably know about this Ruby on Rails application that might be a little atypical from other Rails apps. What would you tell them as you walk out the door and say, "Good luck"?
We have a lot of race conditions in the current state of workflows. Things are moving to a better state is our goal. Be on the watch out. And then most things are documented somewhere in Notion. If you have your cl... will be good. I think that's it.
Okay. All right. That's fair. All right, Joanna, is there a technical programming-esque type book that you've historically found yourself recommending to your colleagues?
Yeah, historically I would usually recommend Clean Code and Clean Architecture by Uncle Bob.
Okay.
But I don't know if it's still relevant as of today with the agentic coding, you know, like maybe messy code is okay as long as it works. Do you think there's still some value in trying to corral the agents into
It's interesting because it is so much about, I think it's an interesting thing about like when are we opening up and having to look under the hood and do these things ourselves versus relying on the agents to sort these things out, but also needing to know how to corral them in cohesive ways so that we can keep things maintainable, right? So otherwise we'll be potentially burning a lot more tokens in the future to do things, in the same way we'd burn human hours on being like, this is a mess, this is taking too long to do this, and we're still paying for this stuff supposedly. And we're also living in this slightly subsidized token era of development. So see what that looks like. All right. But great recommendations. I'll include links to both of those in the show notes. Joanna, and with that, I want to thank you so much for stopping by to talk shop with us on On Rails.
Thank you for having me.
Yeah, it's been a delight talking with you as well. Thanks so much.
That's it for this episode of On Rails. This podcast is produced by the Rails Foundation with support from its core and contributing members. If you enjoyed the ride, leave a quick review on Apple Podcasts, Spotify, or YouTube. It helps more folks find the show. Again, I'm Robby Russell. Thanks for riding along. See you next time.
Article published