> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hystersis.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Compression Engine

> How Hystersis achieves 80-85% token reduction at 97% accuracy using ProMem extraction and spreading activation retrieval

# Compression Engine

Hystersis's proprietary compression engine achieves 80-85% token reduction at 97% accuracy using ProMem-style extraction and spreading activation retrieval, outperforming Mem0's baseline compression.

## Overview

The compression engine processes memories through a multi-stage pipeline:

1. **LLM Router** — Routes tasks to fast (GPT-4o-mini) or verify (Claude) providers based on complexity
2. **ProMem Extractor** — Extracts facts via self-questioning, verification, and gap detection
3. **Async Pipeline** — Non-blocking compression with under 5ms write latency impact
4. **Tiered Storage** — Working, Hot, Cold, and Archive tiers optimize access cost

## Architecture

```
Content → LLM Router → Fast/Verify Path → ProMem Extraction
                                               ↓
         Spreading Activation ← Compressed Output ← Verification
                                               ↓
                                       Tiered Storage
```

## Compression Modes

| Mode         | Description                           | Token Reduction | Accuracy |
| ------------ | ------------------------------------- | --------------- | -------- |
| `extract`    | Fact extraction with verification     | 80-85%          | 97%+     |
| `balanced`   | Balanced extraction and summarization | 75-80%          | 95%+     |
| `aggressive` | Maximum compression                   | 85-90%          | 90%+     |

## ProMem Extraction

The extraction process follows four phases:

1. **Self-Question Generation** — Ask "what does this memory mean?"
2. **Answer Verification** — Validate answers against original memory
3. **Gap Detection** — Identify missing critical information
4. **Active Extraction** — Pull key facts, not just summarize

### Configuration

```bash theme={null}
COMPRESSION_ENABLED=true
COMPRESSION_LLM_FAST_PROVIDER=openai
COMPRESSION_LLM_FAST_MODEL=gpt-4o-mini
COMPRESSION_LLM_VERIFY_PROVIDER=anthropic
COMPRESSION_LLM_VERIFY_MODEL=claude-3-5-sonnet
COMPRESSION_COMPLEXITY_THRESHOLD=0.6
COMPRESSION_MODE=extract
```

### API Usage

```python theme={null}
from hystersis import Hystersis

client = Hystersis(api_key="your-api-key")

# Create memory with compression
memory = client.create_memory(
    content="User prefers dark mode and works late hours from 9 PM to 5 AM",
    compression_mode="extract"
)

# Get compression stats
stats = client.get_compression_stats()
print(f"Accuracy retention: {stats.accuracy_retention}")
print(f"Token reduction: {stats.token_reduction}")
```

## Tiered Memory

Memories automatically move between storage tiers based on access patterns:

| Tier    | Storage        | Latency     | Retention         |
| ------- | -------------- | ----------- | ----------------- |
| Working | In-memory      | under 5ms   | Current session   |
| Hot     | Redis          | under 20ms  | 7 days (balanced) |
| Cold    | Neo4j + Qdrant | under 100ms | 90 days           |
| Archive | S3/GCS         | >1s         | Configurable      |

### Tier Policies

```bash theme={null}
# Aggressive: 1-day hot retention
TIER_POLICY=aggressive

# Balanced: 7-day hot retention (default)
TIER_POLICY=balanced

# Conservative: 30-day hot retention
TIER_POLICY=conservative
```

## Benchmark Targets

| Metric              | Target         | Mem0 Baseline | Advantage    |
| ------------------- | -------------- | ------------- | ------------ |
| Accuracy Retention  | ≥97%           | 91%           | +6%          |
| Token Reduction     | 80-85%         | 80%           | +5%          |
| Multi-hop Reasoning | +23% vs vector | baseline      | +23%         |
| P95 Latency         | under 200ms    | \~400ms       | 2x faster    |
| Write Impact        | under 5ms      | N/A           | Non-blocking |

## See Also

* [Spreading Activation](/features/spreading-activation) for retrieval details
* [Compression API Reference](/api-reference/compression) for API endpoints
* [Performance Tuning](/performance-tuning) for optimization