MiCyte20 — L2 persistence
BuiltRendered from 20-l2-surface-persistence.md in the public repository. Edits belong there.
The datum store port, its SQLite adapter, and how a document is read from and written to the MOS database.
The MOS database is the canonical datum store. Filesystem JSON is used only for development, tests and one-way bootstrap. Writes are canonical-only: one document per id, ids in MSS form.
The port
micyte/ports/datum_store/contracts.py defines runtime-checkable Protocols:
| Protocol | Methods |
|---|---|
SystemDatumStorePort | read_system_resource_workbench |
AuthoritativeDatumDocumentPort | read_authoritative_datum_documents, read_document_index, read_authoritative_document |
AuthoritativeDatumDocumentMutationPort | adds read_document_version_identity, replace_authoritative_document, delete_authoritative_document |
PublicationTenantSummaryPort | read_publication_tenant_summary |
PublicationProfileBasicsWritePort | write_publication_profile_basics |
The value objects are in micyte/core/datum_documents.py: AuthoritativeDatumDocument (document_id, source_kind, rows, anchor_rows, tool_id, document_metadata), AuthoritativeDatumDocumentRow (datum_address, raw), and AuthoritativeDatumDocumentCatalogResult. They validate their fields on construction.
The SQLite adapter
SqliteSystemDatumStoreAdapter in micyte/adapters/sql/datum_store.py implements every port above and is the only writer. Construct it through open_mos_store in micyte/adapters/sql/mos.py, which fixes the write posture to canonical-only (allow_legacy_writes=False) and can cache one adapter per database.
| Method | Effect |
|---|---|
replace_documents_efficient | Writes one or more documents in one transaction: semantics, row semantics, the documents row (its integer id kept), provenance, binary identity, bitstream and catalog index entry. Changed rows are checked against the engine's invariants first. A prior id the store does not hold is refused |
replace_single_document_efficient | The single-document form |
replace_authoritative_document, apply_document_insert / _delete / _move | Port writes; each writes the one document it names through replace_documents_efficient, then returns the catalog |
delete_single_document_efficient, delete_authoritative_document | Delete one document; the port form also returns the catalog |
store_authoritative_catalog | Rewrites the whole tenant's catalog and semantics. Used for bootstrap, not for edits |
bootstrap_from_filesystem | One-way seed from a filesystem catalog into SQLite |
read_authoritative_datum_documents | Cached catalog read |
A write raises NonCanonicalDocumentIdError when a document_id does not parse as canonical, and IdentityPolicyMismatch when its version hash was computed under a policy other than the store's (10).
Documents created or appended to since the last catalog snapshot are recorded in document_creates and document_row_appends and assembled on read, so creating or appending to one document does not rewrite the snapshot.
The workbook executor execute_migration in micyte/adapters/sql/datum_workbook_apply.py writes every sheet a migration touched in one replace_documents_efficient call, so a cascade is one transaction. It backs up first and restores if verification fails after the write.
Schema
micyte/adapters/sql/mos_schema.py holds the MOS tables; micyte/adapters/sql/_sqlite.py holds the audit, portal authority and directive-context tables and opens connections with WAL and foreign_keys=ON.
| Table | Holds |
|---|---|
documents | One row per document: document_id, prefix (CHECK (prefix IN ('lv','stl','cptr','art'))), msn_id, sandbox, name, version_hash, is_anchor, origin (local or foreign) |
authoritative_catalog_snapshots, authoritative_catalog_index, authoritative_catalog_facts | The catalog snapshot and its index |
datum_document_semantics, datum_row_semantics | Version identity and per-row hyphae and semantic hashes |
datum_document_identity, datum_document_binary, datum_document_bitstream | Identity records, the binary (v3) identity and the stored transport bitstream (61) |
datum_document_provenance | Where each document came from |
document_row_appends, document_creates | Changes since the snapshot |
Document ids
Ids are composed and validated only in micyte/core/document_naming/__init__.py:
| Prefix | Form | Meaning |
|---|---|---|
lv | lv.<msn_id>.<sandbox>.<name>.<hash> | A datum document in a sandbox |
stl | stl.<msn_id>.<name>.<hash> | A stored payload |
cptr | cptr.<msn_id>.<name>.<hash> | A cache pointer |
art | art.<msn_id>.<name>.<hash> | An artifact: a file whose rows are its bytes |
<hash> is the 64-character version hash.
Sandboxes
A sandbox has no object of its own. Its identity is the <sandbox> segment of lv. ids, stored in documents.sandbox. The packages fnd_app/packages/sandboxes/system/ and fnd_app/packages/sandboxes/orchestration/ are empty scaffolds. A sandbox's anchor document defines the namespace its fields are numbered in (41).
The database file
Each instance has one SQLite authority database. The host reads its path from MYCITE_V2_PORTAL_AUTHORITY_DB. test_no_disk_datum_authorities.py forbids datum documents on disk under a live instance and any second SQLite database; test_no_filesystem_datum_authority_in_runtime.py forbids runtime code from reading the filesystem adapter.
The filesystem adapter, FilesystemSystemDatumStoreAdapter in micyte/adapters/filesystem_datum/live_system_datum_store.py, is read-only. It walks data/system/sources/*.json and data/sandbox/<tool>/sources/*.json and emits legacy ids that derive_canonical_id_from_legacy converts. Nothing writes the database back to disk.
Not built
- An authority backend other than SQLite.