Project Plans Changed the Problem Again

Products have specs. Project plans have steps. That turned out to be a very different problem.

Up to this point, almost everything the website agent knew about was a product, a policy page or a manual. Then I started adding project plans: the kind of thing someone reads when they want to actually build something. A list of materials. A cut list. Step-by-step directions.

I started with a small batch, just to see what would happen. I'm glad I did.

A plan isn't a product

With a product, most questions are about facts. What does it do? What size is it? What comes in the box? If the agent finds the right product, it can usually find the right answer.

A plan is different. A plan is a sequence. Step 3 only makes sense after step 2. The quantity in the materials list matters. The measurement in the cut list matters. And a single step, pulled out on its own, can be misleading or even unsafe if it's missing what came before it.

So the questions changed too. People don't just ask "what is this?" They ask "what do I do next?" and "how many of these do I need?" and "what size screws does it use?"

"Which one?"

The conversation work from the last post helped here, but it also needed to go a step further.

Imagine a conversation like this:

Show me a chair project.

Then:

What size screws does it use?

With products, "it" meant a product. Here, "it" means that specific plan. Not plans in general. Not some other project that also uses screws. The one we just looked at.

The answer has to come from that plan, and only that plan. With products, context had been about what we were talking about. With plans, it was about which one.

The steps didn't come back right

Then I looked closely at how the directions were coming back, and that's when the real problem showed up.

The steps weren't always staying the way they were written. Sometimes two steps got merged into one, so an instruction that should have been two separate actions read like a single run-on sentence. Sometimes steps came back out of order. And sometimes a step showed up twice.

None of that is a big deal in a paragraph about what a product does. In a set of directions, it's a very big deal. If you're halfway through building something and the instructions skip ahead, or repeat themselves, or quietly combine two steps, you're going to have a bad afternoon.

It was the PDF lesson all over again in a different form. The content was right. The way it got broken apart and put back together wasn't respecting how it was meant to be read.

Keeping steps whole and in order

The fix meant treating directions differently from everything else.

Up until now, content was mostly just text to split into chunks and search. Directions couldn't be handled that way. A step needed to stay whole. Steps needed to stay in the order they were written. And each step needed to stay attached to the plan it came from.

So instead of treating a plan as one long block of text, I broke every plan down step by step, one at a time, and stored it as structured JSON. Each step got its own number and its own text, inside the plan it belonged to. Something like this:

{
  "plan": "Chair project",
  "steps": [
    { "step": 1, "text": "Cut all parts to length using the cut list." },
    { "step": 2, "text": "Drill holes in the seat rails." },
    { "step": 3, "text": "Assemble the front and back leg frames." }
  ]
}

(That's a simplified, made-up example, but the shape is the idea.)

Once the steps were stored that way, there was nothing to guess about. Step 2 was step 2. It couldn't quietly merge with step 3, jump ahead of step 1 or show up twice, because it was its own piece with its own place in the list.

It sounds obvious written down. It took a fair amount of testing to get right, and the testing mattered more than the fix. The only way to be sure the steps came back correctly was to look at them, plan by plan, and check them against the original.

Context isn't just what we're talking about. It's which one.

The next problem

With a small batch of plans, it mostly worked. Steps stayed in order. Follow-up questions stayed on the plan you were looking at.

But I'd been thinking about this one plan at a time. There's a large variety of plans, and a lot of them are similar. Similar materials. Similar steps. Similar names.

I hadn't really thought about what would happen when two plans were so similar that the agent mixed them up, taking a step from one and a measurement from the other.