Can you tell me where I can get more detailed, exactly technical information on storage of weather and archive data in the form of time series? How it is organized in details. Do you use your own structures or are they based on other research. Thanks
Thank you for the great open-meteo work.
Can you tell me where I can get more detailed, exactly technical information on storage of weather and archive data in the form of time series? How it is organized in details. Do you use your own structures or are they based on other research. Thanks
Hi, the file format is inspired by others like HDF5 or zarr. However it is drastically simplified and consists of a header with meta data, a look-up table for chunks and a data region for compressed blocks. You can find the implementation here: https://github.com/open-meteo/open-meteo/blob/741a82ccee48eb7dfe45eb828d345cdaf7b763d5/Sources/SwiftPFor2D/SwiftPFor2D.swift#L286. In future versions I would like to extend format to work well with cloud native deployments (S3 with range queries).
Thank you