@firebase-backup-platform/backup-core

The backup-core package provides the core orchestration layer for FireBackup. It coordinates the entire backup and restore process, integrating with Firebase, compression, encryption, and storage connectors.

Installation

npm install @firebase-backup-platform/backup-core
# or
yarn add @firebase-backup-platform/backup-core

Overview

The backup-core package is the heart of FireBackup's backup engine. It handles:

• Firebase Firestore data export
• Firebase Authentication user export
• Compression and encryption orchestration
• Storage upload coordination
• Progress tracking and event emission
• Restore operations with document mapping

---

Quick Start

Basic Backup

import { BackupCore } from '@firebase-backup-platform/backup-core';
import { createStorageConnector } from '@firebase-backup-platform/storage-connectors';

// Configure the backup core
const backupCore = new BackupCore({
  firebase: {
    projectId: 'my-firebase-project',
    credentials: require('./service-account.json')
  },
  storage: createStorageConnector({
    type: 's3',
    bucket: 'my-backups',
    region: 'us-east-1',
    credentials: {
      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
    }
  }),
  encryption: {
    enabled: true,
    key: process.env.ENCRYPTION_KEY
  },
  compression: {
    enabled: true,
    algorithm: 'brotli'
  }
});

// Run backup
const result = await backupCore.backup({
  collections: ['users', 'orders', 'products'],
  includeAuth: true,
  backupPath: 'backups/2024-01-15'
});

console.log('Backup completed:', result);

Quick Backup Helper

For simple use cases, use the quickBackup helper:

import { quickBackup } from '@firebase-backup-platform/backup-core';

const result = await quickBackup({
  firebase: {
    projectId: 'my-project',
    credentials: serviceAccount
  },
  storage: {
    type: 's3',
    bucket: 'my-backups',
    region: 'us-east-1'
  }
}, {
  collections: ['users'],
  backupPath: 'backups/quick'
});

---

API Reference

BackupCore Class

The main class for orchestrating backup operations.

Constructor

new BackupCore(config: BackupCoreConfig)

BackupCoreConfig:

Property	Type	Required	Description
firebase	FirebaseConfig	Yes	Firebase connection configuration
storage	StorageConnector	Yes	Storage connector instance
encryption	EncryptionConfig	No	Encryption settings
compression	CompressionConfig	No	Compression settings
logger	Logger	No	Custom logger instance

Methods

backup(options)

Creates a backup of specified collections.

async backup(options: BackupOptions): Promise<BackupResult>

BackupOptions:

Property	Type	Default	Description
collections	string[]	All	Collections to backup
excludeCollections	string[]	[]	Collections to exclude
includeAuth	boolean	false	Include Firebase Auth users
includeSubcollections	boolean	true	Include subcollections
backupPath	string	Auto-generated	Storage path for backup
metadata	object	{}	Custom metadata to include
onProgress	function	-	Progress callback

BackupResult:

interface BackupResult {
  id: string;
  path: string;
  size: number;
  sizeFormatted: string;
  collections: string[];
  documentCount: number;
  authUserCount?: number;
  duration: number;
  durationFormatted: string;
  checksum: string;
  createdAt: Date;
  metadata: object;
}

Example:

const result = await backupCore.backup({
  collections: ['users', 'orders'],
  includeAuth: true,
  backupPath: 'backups/2024-01-15/full',
  onProgress: (progress) => {
    console.log(`Progress: ${progress.percent}%`);
    console.log(`Processing: ${progress.currentCollection}`);
    console.log(`Documents: ${progress.documentsProcessed}`);
  }
});

restore(options)

Restores data from a backup.

async restore(options: RestoreOptions): Promise<RestoreResult>

RestoreOptions:

Property	Type	Default	Description
backupPath	string	Required	Path to backup file
collections	string[]	All	Collections to restore
restoreAuth	boolean	false	Restore Auth users
mode	RestoreMode	'merge'	How to handle existing docs
dryRun	boolean	false	Preview without writing
onProgress	function	-	Progress callback

RestoreMode:

Mode	Description
'merge'	Merge with existing documents
'overwrite'	Replace existing documents
'skip'	Skip existing documents

Example:

const result = await backupCore.restore({
  backupPath: 'backups/2024-01-15/full',
  collections: ['users'],
  mode: 'merge',
  dryRun: true,  // Preview first
  onProgress: (progress) => {
    console.log(`Restored: ${progress.documentsRestored}`);
  }
});

// If dry run looks good, run actual restore
if (result.preview) {
  console.log('Preview:', result.preview);
  // Run without dryRun to actually restore
}

verify(options)

Verifies backup integrity.

async verify(options: VerifyOptions): Promise<VerifyResult>

Example:

const verification = await backupCore.verify({
  backupPath: 'backups/2024-01-15/full'
});

console.log('Valid:', verification.valid);
console.log('Checksum match:', verification.checksumMatch);
console.log('Document count:', verification.documentCount);

list(options)

Lists available backups.

async list(options?: ListOptions): Promise<BackupInfo[]>

Example:

const backups = await backupCore.list({
  prefix: 'backups/2024-01',
  limit: 10
});

for (const backup of backups) {
  console.log(`${backup.path} - ${backup.sizeFormatted} - ${backup.createdAt}`);
}

delete(path)

Deletes a backup.

async delete(path: string): Promise<void>

Example:

await backupCore.delete('backups/2024-01-15/full');
console.log('Backup deleted');

---

Configuration

FirebaseConfig

interface FirebaseConfig {
  projectId: string;
  credentials: ServiceAccount | string;  // Object or path to file
  databaseURL?: string;
}

Using service account object:

const config = {
  firebase: {
    projectId: 'my-project',
    credentials: {
      type: 'service_account',
      project_id: 'my-project',
      private_key_id: '...',
      private_key: '-----BEGIN PRIVATE KEY-----\n...',
      client_email: 'firebase-adminsdk@my-project.iam.gserviceaccount.com',
      // ... rest of service account
    }
  }
};

Using file path:

const config = {
  firebase: {
    projectId: 'my-project',
    credentials: './path/to/service-account.json'
  }
};

Using environment variable:

const config = {
  firebase: {
    projectId: process.env.FIREBASE_PROJECT_ID,
    credentials: JSON.parse(process.env.FIREBASE_CREDENTIALS)
  }
};

EncryptionConfig

interface EncryptionConfig {
  enabled: boolean;
  algorithm?: 'aes-256-gcm';  // Currently only supported algorithm
  key: string | Buffer;       // 32-byte key for AES-256
  keyDerivation?: {
    enabled: boolean;
    salt?: string;
    iterations?: number;
  };
}

Example:

const config = {
  encryption: {
    enabled: true,
    key: process.env.ENCRYPTION_KEY,  // Base64-encoded 32-byte key
    keyDerivation: {
      enabled: true,
      salt: process.env.KEY_SALT,
      iterations: 100000
    }
  }
};

CompressionConfig

interface CompressionConfig {
  enabled: boolean;
  algorithm?: 'brotli' | 'gzip' | 'none';
  level?: number;  // 1-9 for gzip, 1-11 for brotli
}

Example:

const config = {
  compression: {
    enabled: true,
    algorithm: 'brotli',
    level: 6  // Good balance of speed/ratio
  }
};

---

Events

BackupCore emits events for monitoring progress:

import { BackupCore } from '@firebase-backup-platform/backup-core';

const backupCore = new BackupCore(config);

// Backup events
backupCore.on('backup:start', (data) => {
  console.log('Backup started:', data.backupId);
});

backupCore.on('backup:progress', (data) => {
  console.log(`Progress: ${data.percent}%`);
  console.log(`Collection: ${data.currentCollection}`);
  console.log(`Documents: ${data.documentsProcessed}/${data.totalDocuments}`);
});

backupCore.on('backup:collection:complete', (data) => {
  console.log(`Completed collection: ${data.collection}`);
  console.log(`Documents: ${data.documentCount}`);
});

backupCore.on('backup:complete', (data) => {
  console.log('Backup complete:', data.result);
});

backupCore.on('backup:error', (error) => {
  console.error('Backup failed:', error);
});

// Restore events
backupCore.on('restore:start', (data) => {
  console.log('Restore started');
});

backupCore.on('restore:progress', (data) => {
  console.log(`Restored: ${data.documentsRestored}`);
});

backupCore.on('restore:complete', (data) => {
  console.log('Restore complete:', data.result);
});

---

Advanced Usage

Incremental Backups

// First, create a full backup
const fullBackup = await backupCore.backup({
  collections: ['users', 'orders'],
  backupPath: 'backups/full/2024-01-15'
});

// Later, create an incremental backup
const incrementalBackup = await backupCore.backup({
  collections: ['users', 'orders'],
  backupPath: 'backups/incremental/2024-01-16',
  incremental: {
    enabled: true,
    basePath: 'backups/full/2024-01-15',
    since: new Date('2024-01-15T00:00:00Z')
  }
});

Parallel Collection Processing

const result = await backupCore.backup({
  collections: ['users', 'orders', 'products', 'reviews'],
  parallel: {
    enabled: true,
    maxConcurrency: 4
  }
});

Custom Document Filtering

const result = await backupCore.backup({
  collections: ['users'],
  filter: {
    users: {
      // Only backup active users
      where: [
        { field: 'status', operator: '==', value: 'active' }
      ]
    }
  }
});

Stream-Based Processing

For very large collections, use streaming:

const stream = backupCore.createBackupStream({
  collections: ['large_collection'],
  chunkSize: 10000  // Process 10k documents at a time
});

stream.on('data', (chunk) => {
  console.log(`Processed chunk: ${chunk.documentCount} documents`);
});

stream.on('end', () => {
  console.log('Stream complete');
});

Custom Transformers

Transform documents during backup:

const result = await backupCore.backup({
  collections: ['users'],
  transformers: {
    users: (doc) => {
      // Remove sensitive fields
      const { password, ssn, ...safeDoc } = doc;
      return safeDoc;
    }
  }
});

---

Error Handling

import {
  BackupCore,
  BackupError,
  FirebaseConnectionError,
  StorageError,
  EncryptionError,
  ValidationError
} from '@firebase-backup-platform/backup-core';

try {
  await backupCore.backup(options);
} catch (error) {
  if (error instanceof FirebaseConnectionError) {
    console.error('Firebase connection failed:', error.message);
    // Handle reconnection
  } else if (error instanceof StorageError) {
    console.error('Storage operation failed:', error.message);
    // Handle storage issues
  } else if (error instanceof EncryptionError) {
    console.error('Encryption failed:', error.message);
    // Handle encryption issues
  } else if (error instanceof ValidationError) {
    console.error('Validation failed:', error.message);
    // Handle validation errors
  } else if (error instanceof BackupError) {
    console.error('Backup error:', error.message);
    console.error('Code:', error.code);
    console.error('Details:', error.details);
  }
}

---

Best Practices

Performance Optimization

// Use batch processing for large collections
const result = await backupCore.backup({
  collections: ['large_collection'],
  batchSize: 5000,  // Process 5000 docs per batch
  parallel: {
    enabled: true,
    maxConcurrency: 2
  }
});

Memory Management

// For very large backups, use streaming mode
const result = await backupCore.backup({
  collections: ['huge_collection'],
  streaming: {
    enabled: true,
    chunkSize: 10000,
    flushInterval: 5000  // Flush every 5 seconds
  }
});

Retry Configuration

const backupCore = new BackupCore({
  ...config,
  retry: {
    maxAttempts: 3,
    initialDelay: 1000,
    maxDelay: 30000,
    backoffMultiplier: 2
  }
});

---

TypeScript Support

Full TypeScript support with comprehensive type definitions:

import {
  BackupCore,
  BackupCoreConfig,
  BackupOptions,
  BackupResult,
  RestoreOptions,
  RestoreResult,
  VerifyResult,
  BackupInfo,
  ProgressCallback,
  BackupError
} from '@firebase-backup-platform/backup-core';

// Fully typed configuration
const config: BackupCoreConfig = {
  firebase: {
    projectId: 'my-project',
    credentials: serviceAccount
  },
  storage: connector,
  encryption: {
    enabled: true,
    key: encryptionKey
  }
};

// Typed options and results
const options: BackupOptions = {
  collections: ['users'],
  includeAuth: true
};

const result: BackupResult = await backupCore.backup(options);

---

Related Packages

• storage-connectors - Multi-cloud storage
• encryption - Encryption utilities
• compression - Compression utilities
• firebase-oauth - OAuth connector