My First Real Introduction to Embeddings

I knew how search worked.

You type some words. The database looks for those words. If the words match, you get a result. I'd worked with that kind of search for years, and I understood all its little tricks: partial matches, ignoring capital letters, ranking results that mention a word more often.

Embeddings broke that whole idea, in the best possible way.

People don't use the website's words

The problem with normal search showed up as soon as I thought about real questions.

A customer doesn't know what we called the page. They don't know that the answer to "how do I send this back?" lives on a page titled "Returns Policy." They don't know that "it stopped working" is a warranty question. They just ask the way people ask.

Keyword search is great when people use the same words you did. Most of the time, they don't.

That gap was the whole reason I wanted a website agent in the first place. A visitor should be able to ask a question in their own words and still find the right answer.

Searching by meaning

This is where embeddings come in.

An embedding is a way of turning a piece of text into a long list of numbers that represents what it means. Not the exact words, the meaning. Text that means similar things ends up with similar numbers, even if it uses completely different words.

I used OpenAI's text-embedding-3-small model to create them. Every chunk of text I'd collected, from product pages to policy pages to manuals, got its own list of numbers. Those went into PostgreSQL, right next to the text, thanks to pgvector.

Then, when a question comes in, it gets turned into numbers the same way. And instead of asking the database "which text contains these words?" you ask "which text means something close to this?"

That was the moment it clicked for me. I wasn't asking the database to find the same words anymore. I was asking it to find things that meant roughly the same thing.

The first time it worked

I remember testing it with a question that didn't share any important words with the page I hoped it would find.

I decided to start with something simple: "I need to send something back." No mention of returns, refunds, or policies. And there it was. The system found the returns page and gave me a pretty decent summary of how to return an item. It wasn't perfect, but it understood what I was asking without me using the exact words from the website. That was pretty cool.

It's hard to explain how strange that feels the first time if you've spent years building keyword search. The database found the right thing without being given the right words. It felt a little bit like magic, and I knew it wasn't, which somehow made it more interesting.

Then I noticed the near misses

The more I tested, the more I noticed something else.

Searching by meaning doesn't just find the right thing. It finds things that are close. Most of the time, close is exactly what you want. But sometimes close is just... close.

Ask about registering a product, and alongside the registration page, you might get something about signing up for emails. They're not the same thing at all. But to a system that's measuring meaning, "sign up for this" and "register that" are neighbors.

(That's an illustrative example, but the pattern was very real.)

With keyword search, a bad result usually looks obviously bad. It matched a word, but you can see right away it's the wrong page. With embeddings, a bad result can look reasonable. It's in the same neighborhood. It sounds related. And if a website agent is going to read it and answer from it, sounding related isn't good enough.

I filed that away as interesting. I didn't know yet how much it would matter later.

Finding things by meaning is incredibly useful. It also creates an entirely new way to get the wrong answer.

Ready for the next part

By the end of April, I had something that felt like real progress. The website's content had been crawled, cleaned up as well as I could manage, chunked, turned into embeddings and stored in a database I understood. I could ask a question in plain language and get back pieces of content that actually related to it.

But I still hadn't done the part everyone thinks of when they hear "AI."

I hadn't handed any of it to a model and asked it to answer a question.

That was next. And the first time it worked, it really did feel like this could actually work.