@factory/search
Search layer for AI Software Factory (docs/architecture/search-architecture.md).
OpenSearch full-text + vector hybrid search with RRF fusion, permission-filtered at
query time.
The single search core (Phase 8): pure, dependency-free logic over an injectable
fetch. Both writers consume it — the worker indexer
(apps/worker/src/processors/search-index/) builds the docs, and the query API
(apps/api/src/modules/search/) builds the viewers/queries. There is no second
client: the API and the worker import SearchClient from here.
Source-first main → ./src/index.ts (no build step).
The index (recorded layout)
- One index for all types + tenants:
factory-content-v{n}behind the aliasfactory-content(CONTENT_INDEX_ALIAS). Tenant isolation is the mandatoryworkspaceIdterms filter (tenantFilter), never a per-workspace index. - camelCase stored doc:
workspaceId,type,title,body,grants,archived,updatedAt,teamId,createdBy, … — the same names the mapping and the query builders use. - OpenSearch 2.x vector type:
dense_vector(cosine, HNSWm/ef_construction), default dimension 1536 (DEFAULT_VECTOR_DIMENSION, env-tunable at index-creation time by the worker; seecontentMapping({ vectorDimension })). - The Phase-0 scaffold's per-workspace
factory-search-<wid>indices are retired;indexName()no longer exists.
API
| Export | What |
|---|---|
SearchClient |
thin OpenSearch client over injectable fetch (bulk index upsert, doc delete, full-text & hybrid search, prefix, fuzzy); failures surface as ApiError SEARCH_UNAVAILABLE (503 for transport, upstream status preserved for 429/5xx). All index paths go through contentIndex(). |
CONTENT_INDEX_ALIAS / contentIndex() |
the factory-content alias; contentIndex() is the single place the physical index name may change (rollover) |
tenantFilter(viewer) / visibilityFilter(viewer) |
the two mandatory query clauses: tenant scoping (terms workspaceId ∈ viewer.workspaceIds) and the §5.2 permission filter (grants.public / grants.workspace / grants.users ∋ userId / grants.teams ∩ viewer.teamIds) |
baseFilters(query, viewer) |
tenant + visibility + the filter chips (type, teamId, createdBy, databaseId, updatedAfter/Before, archived — archived:false by default) |
buildFullTextBody / buildPrefixBody / buildFuzzyBody / buildHybridBody / buildHybridKnnOnlyBody |
OpenSearch body builders (title 3× / body 1× / creator 0.5× multi_match with <mark> highlights; prefix + fuzzy on title.keyword/id; kNN block with filter: bool.filter [tenant, visibility]) |
contentMapping({ vectorDimension? }) |
the §3 index mapping (camelCase, grants object, dense_vector, knn: true, 1 s refresh) |
vectorDimensionMatches(aliases) |
rollover guard: true when the live alias's dense_vector.dimension equals the requested one |
reciprocalRankFusion / fuseHybrid |
RRF, standard k = 60, score(d) = Σ 1/(k + rank_i(d)), 1-based ranks, ties broken by docId |
chunkText |
deterministic chunker: paragraph + fence aware, word-bounded hard splits ≤ maxChars (default 800), small-piece merging, tail overlap (default 120, clamped to maxChars/4), exact char offsets into the original |
buildIndexDocs / chunkDocId / dedupeToParent |
indexer helpers: doc expansion (page/block → one doc per chunk, parentDocId shared, single vector applies to chunk 0 only) and chunk→parent de-dup |
EmbeddingProvider / StubEmbeddingProvider / stubEmbed / cosineSimilarity |
provider contract + deterministic stub (sha256 char-trigram bag, unit-normalized, default 64-dim) for tests/dev |
encodeCursor / decodeCursor / withSearchAfter |
base64url search_after tuples (score, updatedAt, id); decodeCursor throws on garbage |
Hybrid search (recorded decisions)
- Two requests + client-side RRF instead of the combined
query + knnbody: the combined body's_scoreis BM25-dominated, so the kNN rank list is not recoverable from it. Pass 1 = BM25 (sized toknnK, default 50, per search §4.5), pass 2 = kNN-only (k: 50,num_candidates: 200), fused withfuseHybrid(k = 60). - Result
sourceis"bm25" | "vec" | "bm25+vec"; hybrid hits present in both lists carry the fused RRF score (≈ 2/61 for a double #1). - Stub embeddings are not semantic — a char-trigram hash for pipeline tests.
Real providers live in
@factory/ai(Phase 12); the docvectorModelkeyword is the migration guard (search §7.3), never the mapping.
Verification
pnpm --filter @factory/search test (7 suites: rrf, chunking, embeddings, queries,
indexer, mapping, client-over-fake-fetch — 72 tests) and
pnpm --filter @factory/search typecheck (clean under the root strict config,
including noUncheckedIndexedAccess + exactOptionalPropertyTypes).