Skip to content
KnowFlow logoKnowFlow

Architecture

The overall flow

Section titled “The overall flow”
Input             Deterministic         Any coding agent            You
(URL / tweet / ──► capture into ──► synthesize per contract ──► review,
 note)             raw/ (no AI)      docs/agents/synthesize.md   one word
                                                      │            pending → reviewed
                          ┌───────────────────────────┤
                          ▼                           ▼
                    knowledge graph           optional semantic search
                    (graph.html/json)         (vector index + query)


Pending pages stay out of the graph and the index until reviewed.

Three boundaries define the design:

  1. Capture is deterministic. ingest fetches and stores source material as Markdown under raw/. It never calls an LLM and never writes wiki pages.
  2. Synthesis is delegated, review is human. Any coding agent — ZCode, Codex CLI, Claude Code, OpenCode — turns raw/ into structured pages by following the written contract at docs/agents/synthesize.md. Every synthesized page arrives as status: pending in its final directory and joins the wiki only when you flip it to reviewed.
  3. Unreviewed content is gated. graph and index build strip frontmatter and skip pending pages; health checks pending pages fully for broken links and size but exempts them from the orphan rule; fix never touches them.

Repository layout

Section titled “Repository layout”
knowflow/
├── bin/
│   └── knowflow.js          # CLI entry point and command routing
├── scripts/
│   ├── ingest.sh            # single URL/text capture (fetch → raw/)
│   ├── graph_builder.py     # build the knowledge graph from wiki pages
│   ├── wiki-health.sh       # health checks (broken links, small files, orphans)
│   ├── wiki-auto-fix.sh     # repairs behind `knowflow fix`
│   ├── tags-builder.mjs     # tag hub pages behind `knowflow tags`
│   ├── vector-store.mjs     # vector index (embedding + storage) and query
│   ├── graph_relation_labeler.py  # LLM edge-type labeling (optional)
│   ├── bookmark_sync.sh     # X/Twitter bookmark sync
│   └── wechat_sync.sh       # WeChat official-account article sync
├── templates/               # editable page templates (entity/concept/comparison/source)
├── docs/                    # architecture, methodology, reference docs
└── package.json

The core commands (init, ingest, compose, check, status, health, fix, graph, tags, index, query, ask) are stable and tested. The agent workflow from raw capture to reviewed pages is live: the synthesis contract ships at docs/agents/synthesize.md, and the review gates keep unreviewed pages out of the graph and the index.

Life of a captured URL

Section titled “Life of a captured URL”
  1. Capture. knowflow ingest <url> detects the source type, fetches the full text (Jina Reader for web pages; yt-dlp may be needed for some platforms), and stores it under raw/ with metadata. Nothing else is written.
  2. Synthesize. Any coding agent reads the raw file and follows docs/agents/synthesize.md: it drafts a status: pending source page (plus minimal entity pages) straight into its final directory, with created_from provenance and (EXTRACTED)/(INFERRED) confidence markers on key points. knowflow compose --list shows what is still unsynthesized.
  3. Review. You flip pending to reviewed — one word — or delete the file to reject. knowflow check validates the eight-condition page anatomy before review; knowflow status lists everything still awaiting you.
  4. Graph. knowflow graph strips frontmatter, skips pending pages, resolves [[wikilink]] paths over the reviewed pages, and emits graph.json plus the interactive graph.html viewer.
  5. Search (optional). With an embedding-provider key (Zhipu by default; OpenAI and custom OpenAI-compatible endpoints supported), pages can be embedded into a local vector index built with knowflow index build — pending pages are skipped — then queried with knowflow query and knowflow ask.

Why Markdown instead of a database?

Section titled “Why Markdown instead of a database?”
  • Human-readable — open any page in an editor and it just works
  • Version-control friendly — Git tracks every change
  • Agent-friendly — LLMs are naturally good at reading and writing Markdown
  • Portable — no database service, no lock-in
Section titled “Why both a graph and vector search?”
Capability Knowledge graph Vector retrieval
Exact lookup ✅ by entity/relation ❌
Semantic search ❌ ✅ “similar to X”
Discovering connections ✅ A→B→C paths ❌
Fuzzy matching ❌ ✅

The two are complementary. The graph runs locally with zero dependencies; semantic search is optional and off by default.

Tech stack

Section titled “Tech stack”
Component Technology Why
CLI runtime Node.js 18+ npm ecosystem, familiar to developers
Graph builder Python 3.10+ mature graph tooling, vis-network rendering
Synthesis any coding agent + the written contract (docs/agents/synthesize.md) no LLM inside the CLI; the agent you already use does the drafting
Vector index (optional) pluggable embeddings (Zhipu default, OpenAI, custom) local index file, no external service to run
Storage plain files (Markdown/JSON) zero dependencies, Git-friendly