The instinct with any AI coding assistant is to paste in more. More of the file, more of the surrounding module, more of the README, on the theory that more information can only help. It is a reasonable instinct, and it is wrong often enough to matter. Past a certain point, adding context does not make the assistant smarter about your codebase. It makes the signal you actually care about harder to find inside a pile of tokens the model has to weigh evenly against everything else you handed it.
Every context window has a hard ceiling, and every request against a hosted model has a cost attached to every token inside that ceiling, whether the model used that token productively or not. Treating context as an unlimited resource you top up "just in case" is how teams end up with slow, expensive, and surprisingly unreliable AI-assisted workflows, and it is avoidable once you start treating context as a budget instead of a dumping ground.
Photo by Myburgh Roux on Pexels
Why More Context Isn't Automatically Better
Large language models do not read context the way a person skims a file, picking out what matters and setting the rest aside for later. Every token in the window competes for the model's attention during generation, and research on long-context performance has repeatedly shown that models handle information in the middle of a long context worse than information near the start or end. Bury the one function signature that actually matters in the middle of a 4,000-line file dump, and there's a real chance the model treats it as background noise.
There's a practical cost on top of the attention cost. More context means slower responses, because the model has to process every token before it produces the first one back. On a hosted API such as Anthropic's, it also means a bigger bill, since providers price by token, not by task complexity. A one-line fix that pulls in an entire unrelated service directory as context can cost more, and take longer, than the actual code change it produces.
What Actually Consumes the Token Budget
Most teams underestimate how fast context fills up because they think in terms of "the file I'm editing" and forget everything else riding along with it. A typical AI coding assistant request usually includes the system prompt defining its behavior, the conversation history from earlier turns, any tool or function definitions available to it, retrieved snippets or open files, and finally your actual instruction. By the time your real question shows up, a meaningful chunk of the budget may already be spent on scaffolding you never see.
Conversation history is the sneakiest offender. Long-running chat sessions with a coding assistant accumulate old messages, old file contents, and old tool outputs that stay in the context on every subsequent turn unless something actively prunes them. A session that started as a quick question about one function can quietly balloon into thousands of tokens of stale context by the twentieth message, most of it no longer relevant to what you're currently asking.
Sizing the Context Window to the Task
The fix starts with a question most people skip: what does this specific task actually need to see? Renaming a variable needs the file it's in and maybe its call sites. Debugging a race condition needs the relevant concurrency logic and any shared state it touches, not the entire service. Writing a new feature from scratch needs the patterns it should follow, not every file in the repository that happens to share a directory.
Framing the task narrowly before you assemble context is more effective than any trimming technique applied after the fact. If you can describe in one sentence what the assistant needs to accomplish, you can usually reason directly about what evidence it needs to accomplish it, and everything else becomes an easy thing to leave out rather than a hard thing to cut later.
Picking What to Include When You Can't Include Everything
When a task genuinely spans multiple files, prioritize ruthlessly instead of including everything at moderate detail. Full contents of the two or three files doing the actual work will usually outperform a shallow skim of fifteen files, because the model gets complete, unambiguous information about the part that matters instead of partial information about everything adjacent to it.
Interfaces and signatures often matter more than implementations. If the assistant needs to call a function correctly, its signature, docstring, and a usage example are frequently enough. Including the full implementation only helps when the task requires understanding internal behavior, like fixing a bug inside that function, and dragging it in reflexively for every task that merely calls the function is wasted budget more often than not.
Photo by ᛟᛞᚨᛚᚹ ᚨᚱᚲᛟᚾᛊᚲᛁ on Pexels
Techniques for Trimming Without Losing the Important Parts
A few habits consistently free up budget without costing accuracy. Summarizing older turns of a long conversation instead of carrying the raw history forward keeps the gist of earlier decisions without the token cost of the original exchange. Stripping generated boilerplate, vendor code, and auto-generated files out of anything you paste in removes tokens that were never going to inform the answer. Referencing a file by path and asking the assistant to request it if needed, rather than including it preemptively, works well for tools that support that kind of on-demand retrieval.
It also helps to separate "context the model needs to understand the codebase" from "context the model needs to complete this one task." The first category changes slowly and is worth investing effort in curating well once. The second category should be assembled fresh, and narrowly, every single time, because reusing yesterday's broad context for today's specific question is one of the most common ways teams quietly drift into bloated requests.
"The biggest context-budget mistake we see isn't teams including too little. It's teams reusing the same broad context bundle for every task because building a narrow one feels like extra work. Narrow context is more work upfront and less work everywhere else." - Dennis Traina, founder of 137Foundry
When to Reach for Retrieval Instead of Stuffing
Past a certain codebase size, manually deciding what to include stops scaling, and that's the point where retrieval-augmented approaches earn their complexity. Rather than stuffing in everything that might be relevant, a retrieval layer indexes the codebase and pulls in only the pieces that match the current query, keeping the context window focused on what's actually related to the task at hand instead of everything that happens to live nearby in the file tree.
This isn't an all-or-nothing decision. Plenty of AI automation work we do at 137Foundry for client codebases combines both: a lightweight retrieval step narrows a large repository down to a handful of relevant files, and then those files get included in full rather than summarized, so the model still gets complete information about the narrow slice that actually matters.
Photo by https://kaboompics.com/ on Pexels
Signs Your Context Strategy Is Failing
A few symptoms show up reliably when context management has gone wrong. Responses get slower over the course of a session without the task getting harder, which usually means conversation history is accumulating unchecked. The assistant starts referencing details from an earlier, unrelated part of the conversation as if they still apply to the current question, which means old context is bleeding into new requests. Costs climb noticeably even though the team isn't shipping more code, which almost always traces back to context bloat rather than genuinely larger tasks. This holds regardless of which provider a team is on, whether that's OpenAI or a competing hosted model, since the token-based pricing model behaves the same way everywhere.
The fix in every one of those cases is the same: stop and ask what the current request actually needs, then rebuild the context from that answer instead of continuing to carry forward whatever accumulated before. It feels like it slows things down in the moment. It is consistently faster over a full working session than letting context grow unmanaged.
Photo by BYB BYB on Pexels
Building This Into Team Workflow, Not Just Prompts
Individual habits around context help, but they don't stick without something structural behind them. Teams that get this right usually build small conventions: a standard way of starting a fresh session for a new task instead of extending an old one indefinitely, a shared understanding of which files are "always relevant" versus "relevant to this specific area," and a habit of periodically summarizing and archiving long conversations instead of letting them run indefinitely.
None of this requires exotic tooling. It requires treating context assembly as a deliberate step in the workflow, the same way a team treats code review or test coverage as a deliberate step, rather than something that happens automatically as a side effect of however the conversation unfolded.
Where We've Seen This Go Right
The clearest wins we've observed on client projects through 137Foundry's web development work come from teams that stopped treating the assistant's context window as a scratchpad for the entire codebase and started treating it as a curated brief. Response quality went up, response time went down, and the token bill dropped enough that engineers stopped worrying about it entirely, which let them use the assistant more, not less.
That last part surprises people. Trimming context sounds like it should mean using the tool less. In practice, tighter budgets tend to produce more confident, more accurate answers, which means engineers trust the output more and reach for the assistant more often, not less.
The Habit Worth Building
Context management for an AI coding assistant is not a one-time setup task. It's an ongoing discipline, closer to how a team manages technical debt than how it configures a linter once and forgets about it. The teams that treat it that way get consistently better results from the same underlying model than the teams throwing everything at the window and hoping the important part survives.
If you're evaluating how AI-assisted development could fit your own team's workflow, the starting question is rarely which model to use. It's whether you have a deliberate answer to what belongs in the window for a given task, and what doesn't. Everything else follows from getting that answer right.
