diff options
Diffstat (limited to 'filesystems/py-kerchunk')
| -rw-r--r-- | filesystems/py-kerchunk/Makefile | 43 | ||||
| -rw-r--r-- | filesystems/py-kerchunk/distinfo | 3 | ||||
| -rw-r--r-- | filesystems/py-kerchunk/pkg-descr | 28 |
3 files changed, 74 insertions, 0 deletions
diff --git a/filesystems/py-kerchunk/Makefile b/filesystems/py-kerchunk/Makefile new file mode 100644 index 000000000000..4feaee6115a7 --- /dev/null +++ b/filesystems/py-kerchunk/Makefile @@ -0,0 +1,43 @@ +PORTNAME= kerchunk +PORTVERSION= 0.2.7 +CATEGORIES= filesystems python +MASTER_SITES= PYPI +PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX} + +MAINTAINER= sunpoet@FreeBSD.org +COMMENT= Functions to make reference descriptions for ReferenceFileSystem +WWW= https://fsspec.github.io/kerchunk/ \ + https://github.com/fsspec/kerchunk + +LICENSE= MIT +LICENSE_FILE= ${WRKSRC}/LICENSE + +BUILD_DEPENDS= ${PYTHON_PKGNAMEPREFIX}setuptools>=42:devel/py-setuptools@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}setuptools-scm>=7:devel/py-setuptools-scm@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}wheel>=0:devel/py-wheel@${PY_FLAVOR} +RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}fsspec>=0:filesystems/py-fsspec@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}numcodecs>=0:misc/py-numcodecs@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}numpy>=0,1:math/py-numpy@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}ujson>=0:devel/py-ujson@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}zarr>=0.1<3,1:devel/py-zarr@${PY_FLAVOR} + +USES= python +USE_PYTHON= autoplist concurrent pep517 + +NO_ARCH= yes + +OPTIONS_DEFINE= CFTIME FITS GRIB2 HDF NETCDF3 +CFTIME_DESC= Time handling support +FITS_DESC= FITS file format support +GRIB2_DESC= GRIB2 file format support +HDF_DESC= HDF file format support +NETCDF3_DESC= netCDF 4 file format support + +CFTIME_RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}cftime>=0:devel/py-cftime@${PY_FLAVOR} +FITS_RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}xarray>=0:devel/py-xarray@${PY_FLAVOR} +GRIB2_RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}cfgrib>=0:science/py-cfgrib@${PY_FLAVOR} +HDF_RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}h5py>=0:science/py-h5py@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}xarray>=0:devel/py-xarray@${PY_FLAVOR} +NETCDF3_RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}scipy>=0,1:science/py-scipy@${PY_FLAVOR} + +.include <bsd.port.mk> diff --git a/filesystems/py-kerchunk/distinfo b/filesystems/py-kerchunk/distinfo new file mode 100644 index 000000000000..45262a4ccc15 --- /dev/null +++ b/filesystems/py-kerchunk/distinfo @@ -0,0 +1,3 @@ +TIMESTAMP = 1748107898 +SHA256 (kerchunk-0.2.7.tar.gz) = 0425aa0fbf56f898053ee4c4dd40b35cea12d2fc986e036086e99a4ad16bd4e6 +SIZE (kerchunk-0.2.7.tar.gz) = 709052 diff --git a/filesystems/py-kerchunk/pkg-descr b/filesystems/py-kerchunk/pkg-descr new file mode 100644 index 000000000000..a351e30943fe --- /dev/null +++ b/filesystems/py-kerchunk/pkg-descr @@ -0,0 +1,28 @@ +Kerchunk is a library that provides a unified way to represent a variety of +chunked, compressed data formats (e.g. NetCDF, HDF5, GRIB), allowing efficient +access to the data from traditional file systems or cloud object storage. It +also provides a flexible way to create virtual datasets from multiple files. It +does this by extracting the byte ranges, compression information and other +information about the data and storing this metadata in a new, separate object. +This means that you can create a virtual aggregate dataset over potentially many +source files, for efficient, parallel and cloud-friendly in-situ access without +having to copy or translate the originals. It is a gateway to in-the-cloud +massive data processing while the data providers still insist on using legacy +formats for archival storage. + +We provide the following things: +- completely serverless architecture +- metadata consolidation, so you can understand a many-file dataset (metadata + plus physical storage) in a single read +- read from all of the storage backends supported by fsspec, including object + storage (s3, gcs, abfs, alibaba), http, cloud user storage (dropbox, gdrive) + and network protocols (ftp, ssh, hdfs, smb...) +- loading of various file types (currently netcdf4/HDF, grib2, tiff, fits, + zarr), potentially heterogeneous within a single dataset, without a need to go + via the specific driver (e.g., no need for h5py) +- asynchronous concurrent fetch of many data chunks in one go, amortizing the + cost of latency +- parallel access with a library like zarr without any locks +- logical datasets viewing many (>~millions) data files, and direct + access/subselection to them via coordinate indexing across an arbitrary number + of dimensions |