Christine Smit - Free, public, standardized Zarr stores of geospatial data in the cloud for all!

NASA is making Zarr stores of geospatial data publicly available in AWS US-West-2 region for free access

The Zarr stores feature standardized dimensions (latitude, longitude, time), units, and bounds to simplify algorithm development and data analysis

Each Zarr store contains a single measurement variable to keep the data structure clean and simple, rather than bundling multiple variables together

Data is chunked strategically to optimize both time series and spatial data access patterns, avoiding the need to load entire datasets

The system allows for easy updates as new observations come in, though versioning and continuous updates are still being developed using tools like LakeFS

Performance improvements over previous systems - calculations that took 40 seconds now complete in 1-2 seconds

Data discovery is currently somewhat limited, but improvements are planned for metadata and discoverability through Earth Data Search

Direct S3 access is only available from US-West-2 to avoid egress charges, though there are discussions about letting users pay for egress themselves

The stores feature aggressive data normalization and standardization to simplify algorithm development and ensure consistency

Documentation and support are available through help desk staff and detailed tutorials for accessing and working with the data using tools like xarray