Docassemble editing, simplified thanks to LLMs

I’ve been working on something. It’s at an early stage, but I’m going to take the chance and build in the open.

Jonathan Pyle’s Docassemble document automation platform has made it an astonishing 10 years. Announced at the Legal Services Corporation’s TIG conference in 2015, it has gone from an experiment to the engine behind the document automation of thousands of law firms and nonprofits. It has been the juice that boosted startups from Gavel and Hello, Divorce to AfterPattern (now part of Netdocuments).

I have no doubt that the secret of that success is due to Jonathan’s choice to keep it a largely code-based system, giving it infinite customizability. That is why I chose it to write MADE in, 8 years ago. Complex systems, like most legal forms more than a few pages long, need the complexity of code. Docassemble has some niceties; the Playground puts all of your editing needs at your fingertips, including help inserting variables. But Docassemble authors have always had to learn a mix of programming languages, from Python, to YAML, and Mako and Jinja2, not to mention Docassemble’s own layer of functions and syntax expectations. The power of text has also helped Docassemble in the era of LLMs; increasingly, the newest AI models can quickly edit and even generate new Docassemble code out of the box.

But the power of code has also turned some folks away from Docassemble. I hope that this new project will give more folks on the fence about trying Docassemble the nudge they need to use it.

Code is over, if you want it?

Enter what I’m calling the next-generation Weaver. The first Weaver (originally called Assembly Line Generator) simply helped an author get the first draft of a guided interview without writing any code. The new version will help an author through the whole process of building an interview with a visual editor experience.

But without the tradeoffs

The design of the new Weaver is simple. Docassemble code is already organized into different kinds of blocks; question blocks, code blocks, and two dozen other specialized block types that provide different options, like reusable text templates. Rather than hide that from the user, the Weaver gives you a visual representation of each block, preserving its location in the Docassemble file. The new visual editors for various block types help new authors avoid simple syntax errors, and help experienced developers quickly preview changes without having to upload and run the full interview.

The visual mode doesn’t compete with the power of code. You can use the visual editor when it’s helpful, and switch to code on a block-by-block basis when that is faster–for example, when you want to copy and paste a large section of code.

Gavel and AfterPattern (the original Docassemble editing interfaces) both give you a simple editing experience at the cost of hiding most of Docassemble’s functionality. For example: Docassemble’s powerful object features can save a tremendous amount of repetitive code, and integrations are limited by your imagination instead of a product roadmap. The new Weaver will give you a similar editing experience to those systems for simple interviews, without preventing you from authoring the complex, multi-document, hundreds of fields interviews where Docassemble undeniably shines.

What it looks like right now

1. A simple text search allows filtering by block type and keywords for help quickly locating a block.
2. Block types can be filtered with a single click.
3. An outline view for quick navigation to a particular block.
4. Special block editor views for blocks like the Question block, which enforce field types and other aspects of the question format.
5. An easy way to add another block in any spot in the interview file.
6. The ability to connect to an existing Docassemble server and to have a two-way sync of the interview contents.

This is much better than the first draft, but I still have thoughts on whitespace, padding, layout, etc. Overall though, this looks nicer than I would have produced on my own a few years ago, which is great.

So, is it better?

This tool is currently just in my hands–no Docassemble authors have given it a try yet, other than a few members of the internal team. I’ll keep this series going to share what we learn. My plan is to report back whether it succeeds or fails.

My hope is that the Weaver is a good middle ground between raw code and the extremely limited interface of existing tools. At worst, I expect it to be a good learning tool for totally new developers, some of whom do not have a choice about using Docassemble (like my students). But it might turn out to be a Frankenstein’s monster that isn’t good for either population.

I’ll let you know what we find out!

An automated labeling tool, too

We also have a small working tool to automatically add Docassemble syntax to a DOCX file, which is the starting point of a Docassemble project and previously a major pain point.

Roadmap

• Integration with the existing Weaver, for one-click interview drafts
• New AI features to generate individual questions
• Advanced visual editing of interview flow and order

An experimental build method, too

I have to be honest: this concept has been floating around in my head for years, but I’ve never acted on it, due to always having a more pressing task and the daunting user interface work this project requires. So I decided to take a chance and build the tool with the help of AI, using an approach that is sometimes called vibe coding. I’m an experienced developer, and I have strong opinions on the the code architecture, layout, and interfaces. My instructions to the AI tools reflect those opinions. When I run into a bug or error, I’m also not helpless. I can read the code and make fixes without hoping the AI can fix its own mistakes. Finally, as I go, I can massage the code into the direction that I want with the benefit of experience from building real-world, large code bases by hand. Despite that, there are some downsides to using AI, which I will cover below.

My experimentation has been rewarding, so far. I think AI has a lot of potential to unblock procrastination by letting you go from blank page to something in a few minutes. For me, it’s provided a few specific advantages:

• Quickly iterating on user interface choices, a weak point for me.
  • I was able to prototype a few potential layouts in a few prompts, and then choose the direction that seemed most promising, without having to write a ton of CSS code by hand. Backend is my comfort zone. AI levels that playing field for me a bit.
• Generating boilerplate code in React, which is certainly the best frontend framework for us to use, but one I’m only slightly familiar with (having used it on CourtFormsOnline.org)
• Helping improve my test coverage by making writing tests quick and painless instead of tedious and boring (how I usually feel about them).

The good, the bad and the ugly of AI-assisted coding

You might have seen the study that shows that AI tools can slow down some developers, even while offering perceived speed boosts. I don’t want to pretend that AI-assisted coding doesn’t have tradeoffs; it does. It still takes a lot of time to build tools with the help of AI (a single prompt can take 5-20 minutes to run), and sometime the result isn’t what you want and you need to try again. Code can be much larger than required, with more repetitions and convoluted solutions.

But: the best code is the one someone uses. Code is by definition oriented at producing something that exists in the world; not a museum. I feel strongly that the most important goal in software development is to release something that works. I believe that this project, so far, shows that AI was key to making that happen. This project wouldn’t exist without the reduced starting cost that AI is providing.

Overall, I am satisfied that the tradeoff–getting something passable to work after spending 5 years procrastinating working on it–is in favor of using AI.

A note about time to develop

Where AI has helped me is not in saving development time. Coding probably takes a bit more time than writing it by hand. But it frees me up to plan and architect without getting stuck. It creates a pretty fun flow state when all is going well. To me this raises questions about AI, in general, lowering development costs.

My stack

I make use of 4 different AI coding tools:

• ChatGPT in the browser (I have the $20/month plan)
• VS Code, with the following plugins:
  • GitHub Copilot (the $10/month plan, which is free for University faculty)
  • Google Gemini CLI, embedded into VS Code (the free tier only)
  • OpenAI Codex, which leverages my $20/month ChatGPT subscription

I tend to use ChatGPT mostly for planning and thinking through options. The full-screen view is nicer than the cramped chat window in VS Code. Conversely, it’s awkward to copy code back and forth from ChatGPT.

It’s not hard to use up the quota for either Codex (which has hourly and weekly limits) or Github Copilot (300 “premium” requests/month). So I often see what I can get out of the free Gemini CLI or the included, unlimited “basic” requests from Github Copilot before trying a premium model. However, as of this moment (November 4th, 2025), Google’s flagship 2.5 pro model in the Gemini plugin still isn’t as good as the best alternatives.

My brief experience suggests that the best tool right now is OpenAI Codex, embedded in VS Code. I had previously tried it as a command line tool and didn’t really enjoy it that much, but in the editor, it is more pleasant to interact with, quickly can bootstrap a whole project, and often produces easy to read code without errors.

There are some strong advantages to Github Copilot in agent mode which might outweigh the code quality from Codex, at least if you always use the premium models and pay accordingly. The premium model that I have had the most success with is Claude Sonnet 4.5, and the “basic” model that I use as my first pass is gpt-5-mini. I like that Copilot has a “keep” button with arrows that lets me inspect each change one at a time, that it pauses to ask questions when it needs to, and that it tends to “know” its role as a coding agent better than the other plugins, which can drift back and forth between summarizing your question in the abstract and actually reviewing the code and making changes.

In comparison, both Gemini and Codex offer a more confusing and hard to navigate diff view to review changes to code.

Do not be afraid to revert bad changes

Work with small, iterative changes. Each feature should be in its own commit, or even its own branch. For a big feature, I would make a new branch, try several small steps towards my ultimate goal, and be ready to discard the branch if it gets too bogged down in dead ends. Always make a commit after a working change so you can easily restore to that working point.

Reading all that code

Writing code can be satisfying. Reading code that someone else wrote: not as much. It takes effort to parse through it and understand what it does. AI tools today tend to add comments to help you decipher the code, but the comments are sometimes pretty useless: obvious notes that don’t explain why a choice was made, just literally describe what the code does.

By using AI coding, you’re committing to doing much more reading code than writing, which takes out some of the fun. AI can be helpful here–especially using a secondary AI tool to explain the code written by the first one. But you will still need to cross reference it against the documentation.

Code quality and maintenance

Another unknown with AI code is whether the code will be easily maintained. One flaw with AI agents is that they can sometimes rewrite working code, breaking it in the process. They do not have memory or true understanding, so unless you find a way to provide it enough context, they may reinvent or duplicate existing interfaces instead of re-using them when a human author would avoid a similar mistake.

You need to decide if these tradeoffs are acceptable. This project has been more hands off for me than past AI-assisted coding, in that the AI truly wrote most of the code. Based on my past experience, I am sure that some of the code is much longer and more repetitive than necessary. I have run into a handful of bugs, but not necessarily more than I do in a project I write by hand. Often I write code, run it, and then discover bugs and fix them. I think this is common for developers around the world. I tend to have fewer errors, by this measure, with AI written code, because the agents that I use tend to run some interactive tests before saying they are finished. It just takes more effort to spot what is causing the error.

Brittle code

There’s something magic in getting something up and running almost instantly. A common problem with AI systems, though, is their failure to account for edge cases and failure scenarios. The “happy” path is all well and good, but systems can fail in the real world. For what it’s worth, humans can fail in this way too. But because of this common gap, it’s important to:

• Run manual tests regularly, covering different real-world scenarios
• Write tests for the edge cases you can think of (and read the code to be sure the tests actually work and test what you want them to)
• Ask about edge cases (maybe with a different AI tool)
• Read the code and reason through failure patterns and make sure exceptions are handled

When you do ask AI to write its own patches for edge cases, look over the solutions carefully. I’ve often seen very complex and overengineered patterns for failure modes. You need to apply some judgment. Ask it for alternatives and evaluate independently. You should be prepared to manually rewrite some of the code.

Security and privacy

Paid AI systems are safer here, but notice that I included the free Gemini CLI in my stack. By using that system, I am agreeing to let Google improve its coding tools on my code. That doesn’t bother me, as the Weaver is being released for free under the permissive MIT license, but it might not work for you.

The paid tools that I use do not have the same flaw.

Security is also a concern when using AI coding agents. Some of the tools do better than others, but its up to you to audit any places where user information or credentials are stored to ensure that they stored safely. You need to actually read to code to know that it is safe. Steer the AI towards standard libraries and frameworks for tasks that involve user data and review the documentation for best practices. Do not let the AI invent its own cryptographic methods, for example!

Running into dead ends

When I run into a dead end with the coding tool (i.e., a bug), there are a few different options:

• Try asking a different tool for a solution. The AI tool that broke your system might not be able to fix it, for the simple reason that the failed approach is stored in its context and will influence the retry attempt.
• Read the code and the documentation. You can at least ask the AI tool where the problematic code is. Slow down, read it, and see if you can spot the issue. Experiment with small changes to resolve the problem. If the thing you built is a web application, you can try editing the HTML in the browser inspect window for almost instant feedback.
• Narrow the scope of the code to read by sticking with small changes at a time, small enough that you can read the difference between the working version and the broken one.
• Importantly, use GitHub branches so you can discard broken approaches and start over. Starting over shouldn’t normally be needed, but if you are fighting the AI more than it is helping you, trying fresh is a strong option. You could even try in a new project and ask an AI to help you figure out what worked about the new approach and instruct the AI in the original project to apply the working solution.

What does the future look like?

I haven’t reached the finish line in this project yet. I promise to keep adding more to this story. But: what we have so far works and is useful. I think the size and scope is small enough to be something that we can continue to build using the existing approach of having AI do most of the work.

It raises interesting existential questions:

1. What does “good” code look like when a person almost never needs to read it? Will we get to the point where looking at Python code with human eyes is the equivalent of trying to speed up a program by handwriting assembly code?
2. What does it mean to be a programmer when the code is written by AI? Most programmers enjoy writing code. Managing a team of junior developers who never learn, by contrast, sounds less fun.
3. Is it worth giving up control over aspects like look and feel as well as architectural questions, given that controlling the AI on those choices can be difficult?