Firecube: Plugin-driven framework for turning Earth Observation products into analysis-ready data cubes.

Earth observation workflows for producing analysis-ready data products must balance three competing requirements for operational readiness: consistency, cloud-scale processing, and a simple user experience. In practice, these goals often conflict. Systems focused on operational control can become rigid, while those optimised for extensibility can become inconsistent, non-standard, or difficult to operate.

Here, we introduce Firecube, an open-source ingestion framework for building analysis-ready data cubes in cloud-native environments. Firecube is designed around a stateless CLI that integrates with external orchestrators and exposes modular runtime and plugin interfaces. It provides a storage abstraction across local disk and S3-compatible object storage, and supports cloud-native formats including Zarr and Parquet. Firecube is designed for batch-oriented workflows and provides shared ingestion semantics, configurable parallel batch execution, observability, and a write-ahead log (WAL) backed chunk-management control plane. Rather than bundling product logic itself, Firecube provides the runtime, control-plane tooling, and plugin interface that allow users to bring their own product-specific ingestors while preserving consistent behaviour for batching, appends, partial writes, resume, and write-domain coordination. This keeps semantics consistent across plugins even when they target different EO products, ingestion patterns, and cloud-native formats.

Firecube relies on two key design choices. The first is ChunkManager, the public facade for chunk state. Rather than relying on a shared mutable manifest, Firecube uses a product-local control-plane root in which the write-ahead log is authoritative, while snapshots provide an efficient current-state view, and claims represent write-domain coordination state. This supports safe re-ingests, cleanup, lifecycle tracking, and operational tooling. The second is that Firecube remains orchestrator-agnostic. Firecube acts as an ingestion layer rather than a workflow engine and relies on external cloud-native orchestrators, such as Kubeflow and Argo Workflows, for scheduling, workflow orchestration, dependencies, retries, and resource allocation. This separation creates a portable ingestion layer without coupling product logic to a single scheduler.

We illustrate this with practical examples showing how Firecube can transform Earth observation products into analysis-ready datasets and manage their lifecycle in operational environments.