Skip to content
Updates
Prompts as the new PRDs
Sean Horgan
Over the last 6 months at Google I’ve gone from using AI about 2-3x every week to 2-3x every hour. Fueling this growth is the confluence of some exciting AI capabilities:
You can see how this accelerates the classic Build => Measure => Learn loop and here’s a one-shot image that captures it (along with AI’s challenges in getting anything to 100% with arrows all over the place).
Simplifying AI infrastructure
I’ll start with a practical example where I’m trying to simplify the work that application developers (using Google Cloud) need to do to figure out which high-powered AI accelerator chips to use for their inference applications. This isn’t trivial because these chips are expensive, hard to come by and even harder to configure and tune for their specific inference workloads which can vary from biotechs processing proteins to next-gen AI companies generating new video avatars. Oh and layer on the complexity of kubernetes as that’s the only way to manage any deployment at scale.
We’ve tried many approaches at Google to make this simpler, from writing some amazing documentation, developing traditional UIs and CLIs, and hiring world-class customer engineers to guide our customers through the whole process. While these approaches have helped, it’s unrealistic to think that they can keep up with the rapidly changing (swirling!) ecosystem of tools and players in the AI space.
AI to the rescue (of AI)
So we’ve started to see if AI can help us AI our way through this AI world.
I started playing around with Gemini CLI as it had some built-in support for Google Cloud and I found that it got a lot right, right out of the box, helping me work more naturally with gcloud’s complex command line patterns and parameters.
A team in GKE created an MCP server that included some tools that made it easier to work with kubernetes clusters, grab benchmarks we’ve run and generate deployment specs & manifests.
Here are the tools we started with:
Here are some instructions on how to get this running on your own system. I have all this set up on my mac:
I started to play around with MCP and putting aside the challenges with MCP discovery and conflict, it’s a powerful tool that can guide AI agents to solving specific problems with less variability.
An engineer on the team implemented a tool that clarified how to use a specific gcloud CLI to generate kubernetes manifests uing GKE Inference Quickstart (GIQ):
which tells Gemini to use this tool instead of gcloud for specific use cases:
While MCP tools are pretty easy to set up, it feels like another layer of code on top of more code. Through some trial-and-error we’ve learned that Gemini CLI also responds well to direct system prompts which wrap existing CLIs in more natural language terms. We started to build this out:
It was pretty easy to see some good results:
With that, you don’t have to look too far into the future to sense the challenges with building & maintaining system prompts at scale.
My hypothesis is that the key point in , “that general methods that leverage computation are ultimately the most effective, and by a large margin”, should encourage us to build in a modular way on top of current state-of-the-art models.
The design of system prompts
To give AI the context it needs to help users, I dug into the world of system prompts by following Simon Willison’s recent work to reverse engineer them:
I was especially interested in this which confirm his findings and highlights the importance of applying foundational design principles to enabling AI:
hacker news exchange
I found this system prompt while prototyping an app with bolt.new
For all designs I ask you to make, have them be beautiful, not cookie cutter. Make webpages that are fully featured and worthy for production.
By default, this template supports JSX syntax with Tailwind CSS classes, React hooks, and Lucide React for icons. Do not install other packages for UI themes, icons, etc unless absolutely necessary or I request them.
Use icons from lucide-react for logos.
Back to basics
Knowing the importance of designing the right context for AI, I went back to JTBD to ensure that I was grounding my guidance in outcomes that mattered to users and covering all the activities needed to get them there.
I’ve worked with many frameworks into my over the years but I’ve always found JTBD () to have real sticking power when it’s used correctly. For me that means keeping it simple and anchored in the expected outcome for the user, e.g. :
Word of caution: there is a whole cottage industry surrounding JTBD that blows it up at scale, e.g. . Keep it simple!
I used notebookLM with source material for customer notes and escalations to draft a list of the top 10 JTBD we need to focus on using this template
I then used Google AI Studio to help me create instructions for any AI agent to use for that JTBD using the prompt template below. I didn’t include the full version as I’m still iterating on it but it’s quite lengthy as it covers a lot of ground.
I’m also focused on setting up clear evaluation criteria and a process for doing this at scale.
Want to print your doc?
This is not the way.
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.