Context Compression & Token Gating

Context Compression & Token Gating (Gemini API Tuning)

The Problem

Web scraping of prop firm rules and news articles resulted in payload sizes exceeding 15,000 characters per scan cycle, driving high API costs and context-window bloat on the Gemini scanner.

The Solution (Context Compression)

Implemented keyword-extraction filters in prop_guardian.py to compress raw rules into under 4,000 characters—a 73% payload reduction with zero loss in validation accuracy.

# prop_guardian.py - Context Compression
def compress_rules_context(raw_text: str) -> str:
    keywords = ["drawdown", "loss limit", "leverage", "consistency", "restricted"]
    lines = raw_text.split("\n")
    filtered_lines = [line for line in lines if any(k in line.lower() for k in keywords)]
    return "\n".join(filtered_lines)[:4000]

Output Gating

Applied strict max_output_tokens limits across Gemini calls:

• Visual Bias Checks: Gated at exactly 10 tokens (binary/short response).
• Audit Engine Reports: Clamped to 300 - 800 tokens to prevent verbose, narrative responses.

Result: Reduced Gemini API monthly credit consumption by over 60%.