- Add proactive_cron, proactive_prompt, last_proactive_at fields to Discord models
- Add /proactive command handler for configuring check-in schedules
- Add evaluate_proactive_checkins task (runs every minute via celery beat)
- Add execute_proactive_checkin task that evaluates interest and sends messages
- Smart bot selection finds the correct bot for each server
- Channel selection defaults to "general" text channel for servers
- Add database migration for new fields
- Add comprehensive tests for commands and tasks
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Query analysis now extracts recalled_content (titles/essays the LLM
recognizes as relevant, e.g., "predetermined conclusions" → "The Bottom Line")
- Added title-based chunk fetching to ensure recalled content appears in
results even when BM25/embedding search doesn't rank it highly
- Fixed BM25 to run separate searches for each query variant and merge
results (was concatenating into one AND query)
- Added RECALLED_TITLE_BOOST (0.05) for sources matching recalled titles
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Query analysis and HyDE are both LLM-based operations that run in
parallel via asyncio.gather, so enabling query analysis adds no
extra latency when HyDE is also enabled.
Query analysis provides:
- Modality detection from query content
- Query cleaning and reformulation
- Query variants for better recall
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Initialize checkboxes to match server defaults (BM25, HyDE, Reranking
enabled; Query Analysis disabled) so the UI accurately reflects what
the server is actually doing. Previously showed all as unchecked which
was confusing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add TTL-based cache for query embeddings (5 min TTL, 100 entries max)
- Cache hit avoids VoyageAI API call entirely
- First call: ~1s, subsequent calls: ~0.09s
- Only caches single-query embeddings (the common search case)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Run BM25 and embedding searches in parallel with asyncio.gather
- Fix word filter in both search.py and bm25.py: >= 2 chars instead of > 2
(allows important 2-letter terms like "AI", "ML" to be included)
- Better error handling for parallel search failures
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Include 2-letter terms (AI, ML) in query term extraction (was > 2, now >= 2)
- Add guard for empty data before accessing data[0].data[0] in scorer
- Preserve chunks without content in reranking instead of silently dropping
- Remove legacy wrapper functions (apply_title_boost, apply_popularity_boost)
- Update tests to use apply_source_boosts directly
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add useModalityDetection config flag to detect content type hints
from natural language queries (e.g., "on lesswrong" → forum,
"comic about" → comic)
- Strip meta-language noise from queries ("there was something about")
- Add family term expansion (father ↔ son, parent ↔ child, etc.)
- Modality detection is off by default, configurable per-request
TODO: Replace regex-based detection with LLM-based query analysis
(Haiku) that can run in parallel with HyDE for better accuracy.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Allow callers to enable/disable BM25, HyDE, reranking, and query
expansion on a per-request basis via SearchConfig. When not specified,
falls back to global settings from environment variables.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add tests for extract_query_terms (stopword filtering, short words)
- Add tests for apply_query_term_boost (boost calculations, edge cases)
- Add tests for deduplicate_by_source (keeps highest per source)
- Add tests for apply_title_boost (title matching with mocked DB)
- Add tests for fuse_scores_rrf (RRF score fusion, ranking behavior)
- Add tests for rerank module (VoyageAI reranker mocking)
Uses pytest.mark.parametrize for concise, data-driven tests.
77 tests total covering all new search functionality.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
HyDE generates a hypothetical document passage that would answer the user's
query, then embeds that alongside the original query. This bridges the gap
between how users describe what they're looking for and the actual document
terminology.
Changes:
- Add hyde.py with expand_query_hyde() function
- Integrate HyDE into search_chunks() pipeline
- Add ENABLE_HYDE_EXPANSION and HYDE_TIMEOUT settings
- Only expand queries with 4+ words (short queries are specific enough)
- Simple in-memory cache to avoid re-generating for repeated queries
Example:
- Query: "saying what you mean not using specific words"
- HyDE generates: "Clear communication requires expressing your thoughts
directly and honestly, even when you lack the precise terminology..."
- This finds articles about word meaning and clear communication
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Major changes:
- Replace OOM-causing in-memory BM25 with PostgreSQL full-text search
- Add tsvector column and GIN index for fast keyword search
- Implement hybrid score fusion (70% embedding + 30% FTS + 15% bonus)
- Add CANDIDATE_MULTIPLIER (5x) to search more candidates before fusion
- Add stopword filtering to FTS queries for less strict matching
- Make search limit configurable (default 20, max 100)
- Propagate relevance scores through the search pipeline
Search improvements:
- "clowns iconoclasts" → finds target at rank 1 (score 0.815)
- "replacing words with definitions" → finds target at rank 1
- Vague queries now find results with limit=30 that were previously missed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Added 38 new issues from comprehensive security audit
- Documented 8 fixes applied in this pass
- Listed remaining issues requiring further investigation
- Updated fix count to 53+
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- BUG-023: Mark as acceptable design (SHA256 for exact dedup)
- BUG-026: Mark as acceptable design (BM25/embedding for recall, LLM for ranking)
These are architectural decisions, not bugs - semantic dedup and hybrid scoring would be feature enhancements.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Mark BUG-059 as N/A (MockRedis is sufficient for actual usage)
- Update improvement suggestions to reflect completed fixes:
- Retry logic (BUG-015)
- Health checks (BUG-043)
- Rate limiting (BUG-030)
- CSRF protection (BUG-049)
- Resource limits (BUG-067)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
BUG-030: Add rate limiting via slowapi middleware
- Added slowapi to requirements
- Configurable limits: 100/min default, 30/min search, 10/min auth
- Rate limit settings in settings.py
BUG-028: Fix filter validation in embeddings.py
- Unknown filter keys now logged and ignored instead of passed through
- Prevents potential filter injection
BUG-034: Fix timezone handling in oauth_provider.py
- Now uses timezone-aware UTC comparison for refresh tokens
BUG-050: Fix SQL injection in test database handling
- Added validate_db_identifier() function
- Validates database names contain only safe characters
Also:
- Updated tests for bcrypt password format
- Updated test for filter validation behavior
- Updated INVESTIGATION.md with fix status
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Root cause: BookSection._chunk_contents() called extract_text() without
specifying modality, which defaults to "text". This caused 9,370 book
chunks to be stored in the 'text' collection instead of 'book'.
Fix: Added modality="book" to all DataChunk creation in BookSection:
- extract_text() call for single-page sections
- Direct DataChunk creation for multi-page sections
Note: The original investigation reported 1,338 mail items, but current
analysis shows those are actually email attachments which correctly go
to text/doc/photo collections based on their content type.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
More bugs verified:
- BUG-021: Chunk validation exists via yield_spans ✅
- BUG-027: N/A - defaulting to 0.0 is reasonable fallback
- BUG-055: collection_model now returns None instead of "unknown" ✅🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Mark BUG-010 (MCP servers) as already fixed
- Mark BUG-011 (User ID type) as already fixed
- Document BUG-061 to BUG-068 fixes from commit 1c43f1a
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
tools/restore_databases.sh: Script to restore PostgreSQL and Qdrant
backups from encrypted backup files.
tools/restore_files.py: Python script to restore Fernet-encrypted
file backups.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Logic error: `boolean_env(...) or True` always evaluates to True,
making the environment variable useless.
Fixed by removing `or True`. Note: This setting is currently unused
in the codebase but the fix ensures correct behavior when it's used.
Also updates DISCORD_MODEL default to claude-haiku-4-5 for faster
and cheaper responses.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
