82 — Building a sandbox to develop against

GuideRendered from 82-demo-sandbox-cookbook.md in the public repository. Edits belong there.

This page states how to create a sandbox of datum documents for developing and testing tools and lenses, and the rules a sandbox's documents must satisfy.

Use the provisioner

fnd_app/scripts/provision_instance.py reads an instance.yaml manifest and creates what it declares: farm-shaped and channel-shaped sandboxes, their anchors, local domains, and the registrar rows they read as sources.

$ python -m fnd_app.scripts.provision_instance --manifest instance.yaml --db <store>          # dry run
$ python -m fnd_app.scripts.provision_instance --manifest instance.yaml --db <store> --apply

Without --apply it writes nothing.
With --apply it backs the store up with VACUUM INTO before the first write (--backup-dir sets where).
It refuses to run as root.
Every step is idempotent and returns a status. A step that is not OK fails the run and the process exits non-zero.

fnd_app/tests/fixtures/test_instance.yaml is a complete synthetic manifest. fnd_app/tests/_provisioned_instance.py builds a whole test instance from it in a temporary directory: sandboxes, registrar entities, a taxonomy, a channel with a catalogue, a signing key and a fiat datum. Tests should use it instead of reading a live store.

For a single document in a unit test, document() and anchor_document() in fnd_app/tests/_store_fixtures.py build a canonical document from rows.

The document model

A datum document is an AuthoritativeDatumDocument (micyte/core/datum_documents.py). The fields a builder sets:

Field	Meaning
`document_id`	Canonical id, computed (below).
`source_kind`	`sandbox_source` for a sandbox document; the only other accepted value is `system_anthology`.
`document_name`, `canonical_name`	The document's name within its sandbox.
`is_anchor`	`True` for the anchor document.
`document_metadata`	JSON object. `datum_template_archetype`, `samras_family`, `schema` and `datum_template_schema` are read as archetype tokens.
`rows`	Tuple of `AuthoritativeDatumDocumentRow`, each a `datum_address` (`0-0-1`, `4-2-7`) and a `raw` cell list.

Documents are stored only in the instance's MOS authority database, never as JSON files.

Canonical ids

A sandbox document id is lv.<msn_id>.<sandbox>.<name>.<version_hash>, matched by _LV_RE and built by format_canonical_document_id in micyte/core/document_naming/__init__.py. <version_hash> is 64 lowercase hex characters. The sandbox token uses underscores (agro_erp, not agro-erp).

The hash covers the document, and the id contains the hash, so a builder makes the document twice:

Build it with a placeholder hash of 64 zeros.
Call compute_mss_hash(document) (micyte/core/mss/datum_identity.py). It returns policy, version_hash and canonical_payload. Strip the sha256: prefix from version_hash.
Build the id again with the real hash and rebuild the document with it.

The hash is deterministic over the rows and metadata, so the same input gives the same id.

The anchor

An anchor holds the rows that a sandbox's other documents reference, in layers by address:

Addresses	Layer	Example
`0-0-K`	Primitive units	`[["0-0-9", "~", "0-0-0"], ["fiat-currency-unit"]]`
`1-1-K`	Magnitudes over a primitive	`1-1-1`, a SAMRAS bitstream labelled `txa-SAMRAS`
`2-0-K`, `2-1-K`	Abstraction spaces over a magnitude
`3-1-K`	Babelettes, the targets source rows reference as `rf.3-1-K`

Every anchor carries its own copy of the primitive rows. fnd_app/scripts/bootstrap_agro_erp_anchor.py builds all four layers (_build_anchor_rows) and its taxonomy rows (_build_txa_rows: one 4-2-N row per taxon and a 5-0-1 collection).

Do not hand-add a price row. The fiat datum is a three-row chain found by its structure (find_fiat_chain); fnd_app/scripts/provision_fiat_datum.py creates it. A guessed 3-1-N row makes the provisioner refuse that address.

Writing documents

SqliteSystemDatumStoreAdapter (micyte/adapters/sql/datum_store.py), constructed with allow_legacy_writes=False:

Method	Use
`replace_single_document_efficient`	Replace one document by its prior id, or append when the prior id is `None`. Re-encodes only that document.
`replace_documents_efficient`	Replace several documents in one transaction.
`delete_single_document_efficient`	Delete one document.
`store_authoritative_catalog`	Rewrite the whole catalog. Re-encodes every document.

Every write path raises NonCanonicalDocumentIdError for an id that does not parse, and NonCatalogPrefixError for an id that is not lv.. Writes clear the in-memory catalog caches.

A builder reads the current catalog, finds the prior document with the same sandbox and name, and replaces it. Running it twice with the same input changes nothing.

Making tools eligible

A document is offered a tool when the tool's applies_to_archetype matches one of the document's archetype tokens or its applies_to_source_kind matches the document's source_kind. Archetype tokens come from the metadata keys above and from the document's shape: micyte/tools/_archetype.py recognizes hops_geospatial_filament (a 4-* row with at least three coordinate markers) and samras_taxonomy (a 4-2-* row defining titled nodes). To make a test document reach a tool, set the matching metadata key when building it. See 80 and 81.