Agents burn cash. Every token counts. Tura claims 80% reduction. You can achieve similar results by tightening your agent loop. Most developers send too much data to the LLM. We fix that today. Stop sending raw logs. Stop keeping full history.
Step 1: Audit Your System Prompt
Your system prompt runs on every call. Keep it under 100 tokens. Remove polite instructions. LLMs do not need to be told to be helpful. Define the task strictly. Use fewer examples. One shot is enough.
Replace "You are a helpful assistant who..." with "Extract entities. Output JSON." Measure the token count before deployment. Use a tokenizer library locally. If you see 500 tokens here, cut it down. Hardcode constraints in your code, not the prompt. Remove requests for chain-of-thought unless debugging. It adds tokens without value in production.
Step 2: Compress Conversation History
Do not send the full chat history. Summarize past turns. Keep only the last three messages raw. Convert older messages into a single summary paragraph. This reduces context window pressure. It speeds up inference.
Implement a sliding window. Every five turns, call a cheaper model to summarize the exchange. Use gpt-4o-mini for this task. Store the summary in your state store. Pass only the summary and recent messages to the main agent. This keeps context linear, not exponential. Your costs drop immediately.
Step 3: Truncate Tool Outputs
Tools return large JSON blobs. Do not feed them all to the model. Parse the output locally. Extract only the relevant fields. If a search returns 10 results, send only the top 2. Filter before the API call.
Write middleware for your tools. If a tool returns HTML, strip tags. If it returns a list, limit to 5 items. The agent cannot read 10,000 tokens of data anyway. It will hallucinate. Give it exactly what it needs to make the next decision. Cap tool output at 500 tokens per call.
Common pitfalls
Over-compression loses nuance. The agent might forget user constraints. Test with complex queries. Ensure the summary retains intent. Do not truncate tool errors. The agent needs to know why a tool failed to recover.
Another risk is context switching. If you summarize too aggressively, the agent loses the thread. Monitor success rates. If accuracy drops, increase the window size. Balance cost against performance. Cheap agents are useless if they fail. Watch your latency too. Summarization adds a step.
Next step
Explore the Tura repository on GitHub to see their specific compression algorithms and implement similar logic in your LangChain or LlamaIndex stack.