Building a RAG Pipeline with Next.js and Pinecone
Retrieval-Augmented Generation (RAG) has become the standard approach for grounding LLM responses in your own data. In this post, I'll walk through the exact pipeline we use at Web Intel Agent for our ARIA intake agent.
Architecture Overview
The pipeline has four discrete stages:
- Ingestion — chunk documents, embed them, upsert to Pinecone
- Retrieval — embed the query, run ANN search, pull top-k chunks
- Augmentation — inject retrieved context into the system prompt
- Generation — stream the final response via the Vercel AI SDK
Setting Up the Vector Store
First, install the dependencies:
npm install @pinecone-database/pinecone @ai-sdk/google ai
Create a typed Pinecone client wrapper:
// lib/pinecone.ts
import { Pinecone } from '@pinecone-database/pinecone';
const pinecone = new Pinecone({
apiKey: process.env.PINECONE_API_KEY!,
});
export const index = pinecone.index(process.env.PINECONE_INDEX_NAME!);
Embedding and Upsert
We use Gemini's text-embedding-004 model to generate 768-dimensional vectors:
// lib/embed.ts
import { GoogleGenerativeAI } from '@google/generative-ai';
const genAI = new GoogleGenerativeAI(process.env.GOOGLE_API_KEY!);
export async function embedText(text: string): Promise<number[]> {
const model = genAI.getGenerativeModel({ model: 'text-embedding-004' });
const result = await model.embedContent(text);
return result.embedding.values;
}
The Retrieval Route
// app/api/agent/route.ts
import { streamText } from 'ai';
import { google } from '@ai-sdk/google';
import { index } from '@/lib/pinecone';
import { embedText } from '@/lib/embed';
export async function POST(req: Request) {
const { messages } = await req.json();
const lastMessage = messages[messages.length - 1].content;
// 1. Embed the user query
const queryEmbedding = await embedText(lastMessage);
// 2. Retrieve top-5 relevant chunks
const results = await index.query({
vector: queryEmbedding,
topK: 5,
includeMetadata: true,
});
const context = results.matches
.map((m) => m.metadata?.text)
.filter(Boolean)
.join('\n\n---\n\n');
// 3. Stream augmented response
const result = streamText({
model: google('gemini-1.5-pro'),
system: `You are a helpful assistant. Use the following context to answer:\n\n${context}`,
messages,
});
return result.toDataStreamResponse();
}
Key Takeaways
- Always namespace your Pinecone indexes by tenant/environment to avoid data bleed
- Chunk size matters — 512 tokens with 10% overlap is a solid default
- Cache embeddings for static documents; re-embed only when content changes
Tip
For production workloads, consider using a job queue (BullMQ + Redis) for the ingestion phase to handle large document sets without blocking.