Skip to content

Upgrades — Evolving Artesian Without Losing Memory

Artesian treats the OKF markdown bundle as the durable record of truth. Vector databases are derived indexes. If an embedding model, dimension, distance metric, or payload schema changes, rebuild the index from OKF instead of mixing vectors in place.

Compatibility Guard

Each vector collection stores Artesian compatibility metadata:

  • Artesian crate version
  • OKF version
  • embedding model id
  • embedding dimensions
  • distance metric

VectorMemoryBackend refuses reads and writes when the collection metadata does not match the configured model or dimension. This prevents silent vector mixing. A mismatch means: migrate first.

Safe Migration Procedure

  1. Verify the OKF bundle:
artesian okf verify .artesian/memory
  1. Rebuild into a new collection under the current model/dimension/schema:
artesian migrate .artesian/memory --config artesian.toml --retention-days 30

  1. Artesian backfills by content hash, so re-running the command resumes and skips duplicates.

  2. When the new collection verifies, Artesian swaps the Qdrant alias to the new collection and keeps the old collection for the configured retention window. Rollback is then an alias swap back to the retained collection.

  3. Keep the old collection for at least one backup cycle before deleting it manually.

For a first Qdrant deployment, configure memory.collection as an alias target. Atomic alias swap requires an alias; it cannot safely shadow a concrete collection with the same name.

Snapshots and OKF Exports

Use both backups:

  • Qdrant snapshot for fast restore of the derived index.
  • OKF export for the durable source of truth.

Example schedule:

# daily index snapshot
artesian snapshot --config artesian.toml --output-dir /path/to/qdrant-snapshots

# daily source-of-truth export
artesian okf export .artesian/memory /path/to/okf-backups/$(date +%Y-%m-%d)
artesian okf verify /path/to/okf-backups/$(date +%Y-%m-%d)

Keep backup paths environment-specific and out of the repository. On default ports, one Qdrant URL is enough: :6333 derives gRPC :6334, and :6334 derives REST :6333. If your Qdrant REST API is not the default sibling of the configured gRPC endpoint, set qdrant_rest_url in artesian.toml or QDRANT_REST_URL for snapshot download and alias swap.

Recovery Pattern

  1. Restore or copy the OKF bundle.
  2. Run artesian okf verify.
  3. Run artesian migrate to rebuild the current vector index.
  4. Point the alias at the rebuilt collection.
  5. Keep the old collection until the new run has been verified in normal use.