The tacos aren’t gonna find themselves

I don’t know why I landed on a chatbot. I thought it’d be cool to set one up with geospatial restaurant data, and I’d learn a lot. I wasn’t wrong. There are 15,000+ restaurants in the 5 NYC boroughs. How do you figure out where to eat? How does anybody figure out where to eat? (top 10 unsolved mysteries of all time)

What can this bot do? 🤖

Source: hungry.city

With this chatbot, you get restaurant recommendations in the five boroughs of NYC. If you’re in NYC, you can share your location and it’d pull up recommendations around you.

How it works

Magic. Mostly AWS. A lot of magic though.

A bird’s eye view:

Source

1. chat prompt comes in to fargate container,
2. it is forwarded to the AgentCore runtime,
3. Runtime then invokes the bedrock model, runs inference / tool execution
4. As the text is generated, the output is sent back to the client in the same path through SSE streams.

Why Fargate middleman?

There’s a re:post AWS discussion that you could follow, and get rid of a chat lambda / fargate entirely.

But I wanted a service that could fetch chat history from a datastore, for persistent chat data. I also found it easier to invoke the AgentCore runtime from fargate than doing what this discussion said. YMMV

Why Fargate over Lambdas? Aren’t Lambdas cheaper?

Lambdas are cheaper for when you have no traffic. You pay for a lambda’s execution time. They get more expensive compared to paying for a lightweight fargate task that’s always on. Fargate also supports application auto-scaling, so the number of containers you have running goes up depending on the number of requests / CPU / mem utilization.

Source

Why AgentCore?

You could just use the Anthropic API, and set up a chatbot instead of all this. But the value prop of AgentCore is the managed execution environment for your agent logic. Under high traffic, you’re not babysitting servers, scaling policies, or container orchestration. Your focus is just building the agent.

I’ve handed my Strands Agent off to AgentCore to be able to productionize it.

Why supabase?

When I started this project, I wanted a database that supported geospatial indexing, and makes location queries super fast. Supabase was one of them with a free tier that gave me a lot of potential to improve my data quality / size.

Some more design choices I explored.

What could I have done better?

If you think about it, how often do people open up older chats they’ve started? How often do you go back to them? It’s actually not incredibly read-heavy.

You have people writing a lot, and chatting with your agent a lot more. So the write path is a lot higher traffic. Having a separate read path would add a performance boost to my write path.

What’d I learn?

Building hungry.city myself, I’ve rediscovered the passion I once had for building things - before AI. Now I’ve got some free GCP credits to spend in the next few months heh, I’ll probably build something new.

Thanks for reading!

Show some ❤️ by ⭐ing hungry.city on github!