# Persistence in Polycentric Core

Polycentric implements a layered storage architecture that provides consistent interfaces while supporting different storage backends across platforms. Polycentric requires persistence for most use cases in order to collate data from multiple sources and cache it for fast access. For information on how data is actually queried, see the Storage and Querying section.

# Meta Store

The Meta Store manages store lifecycle and metadata, acting as a registry for all stores in the system.

# Store Information

Each store contains metadata including:

interface StoreInfo {
    system: PublicKey;
    version: number;
    ready: boolean;
}

# Meta Store Interface

The Meta Store provides operations for managing stores:

interface IMetaStore {
    openStore: (system: PublicKey, version: number) => Promise<BinaryAbstractLevel>;
    deleteStore: (system: PublicKey, version: number) => Promise<void>;
    listStores: () => Promise<StoreInfo[]>;
    setStoreReady: (system: PublicKey, version: number) => Promise<void>;
    setActiveStore: (system: PublicKey, version: number) => Promise<void>;
    unsetActiveStore: () => Promise<void>;
    getActiveStore: () => Promise<StoreInfo | undefined>;
}

# Persistence Drivers

Polycentric abstracts storage operations through persistence drivers, allowing different implementations for various platforms.

# Core Interface

interface IPersistenceDriver {
    getImplementationName: () => string;
    openStore: (path: string) => Promise<BinaryAbstractLevel>;
    estimateStorage: () => Promise<StorageEstimate>;
    persisted: () => Promise<boolean>;
    destroyStore: (path: string) => Promise<void>;
}

# Platform-Specific Implementations

Browser (IndexedDB) In the browser, we use IndexedDB as the persistence driver. In order to ensure persistence, we use whatever browser vendors need to do to ensure persistence. On Chrome, this is handled by asking for notifications. On Firefox, this is handled by asking for permission to use storage. On Safari, this is handled by requiring installing as a PWA.
Desktop and Mobile (LevelDB) We use LevelDB as the persistence driver on the desktop and mobile. LevelDB is a fast, persistent key-value store that is well-suited for the desktop and mobile environments. For mobile, we maintain our own Capacitor plugin for LevelDB.

# Store Structure

The Store class manages various indexes and data structures.

# Key Features

Binary Storage All data is stored as binary (Uint8Array) for consistency:

type BinaryAbstractLevel = AbstractLevel.AbstractLevel<
    Uint8Array,
    Uint8Array,
    Uint8Array
>;

Transcoders Data is consistently encoded/decoded using transcoders:

function deepCopyTranscoder(): LevelTranscoder.IEncoding<
    Uint8Array,
    Uint8Array,
    Uint8Array
> {
    return {
        name: 'deepCopyTranscoder',
        format: 'buffer',
        encode: (input: Uint8Array): Uint8Array => {
            const outputBuffer = new ArrayBuffer(input.length);
            const output = new Uint8Array(outputBuffer);
            output.set(input);
            return output;
        },
        decode: (buffer: Uint8Array): Uint8Array => {
            return buffer;
        },
    };
}

# Usage Examples

# Creating a Store

// Create persistence driver
const driver = createPersistenceDriverMemory();

// Create meta store
const metaStore = await createMetaStore(driver);

// Open a store
const store = await metaStore.openStore(system, version);

# Managing Store Lifecycle

// Set store as ready
await metaStore.setStoreReady(system, version);

// Set as active store
await metaStore.setActiveStore(system, version);

// List all stores
const stores = await metaStore.listStores();

// Delete store
await metaStore.deleteStore(system, version);

# Best Practices

Error Handling

Always use tryLoadKey instead of direct get operations
Handle missing stores and invalid states gracefully
Properly manage store lifecycle states

Store Management

Keep track of active stores
Clean up unused stores
Maintain store versioning

Platform Considerations

Browser: Handle storage quotas and persistence
Desktop: Manage file system access and permissions
Memory: Monitor memory usage

Data Consistency

Use proper encoding/decoding for all data
Maintain atomic operations where needed
Handle version migrations

# Testing

The storage system includes comprehensive tests. These tests verify:

Store creation and deletion
Active store management
Data persistence
Error handling

# Platform-Specific Notes

# Browser (IndexedDB)

Handles storage quotas
Manages persistence permissions
Provides storage estimates

# Desktop (LevelDB)

File-based storage
Persistent by default
Handles file system operations

# Memory Storage

Non-persistent
Useful for testing
Limited by available memory