{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ ".. _tutorials voxel images:\n", "\n", "# Introduction to Voxel Images \n", "\n", "Voxel images are the heart of Digital Rock Petrophysics workflows in RockVerse. The ``VoxelImage``\n", "class builds upon\n", "[Zarr arrays](https://zarr.readthedocs.io/en/stable/_autoapi/zarr.core.Array.html#zarr.core.Array)\n", "by adding attributes and methods specifically optimized for digital rock petrophysics workflows in \n", "a high performance parallel computing environment, seamlessly enabling advanced computational \n", "capabilities in a user-friendly API. \n", "\n", "If you're unfamiliar with Zarr, we recommend exploring the \n", "[Zarr tutorial](https://zarr.readthedocs.io/en/stable/tutorial.html)\n", "to gain a deeper understanding of its fundamentals.\n", "\n", "## Chunked storage\n", "\n", "VoxelImage leverages the power of Zarr arrays for storing voxel data in a chunked format. \n", "This means that the large 3D image is divided into smaller, manageable chunks for efficient \n", "storage, retrieval, and parallel processing. \n", "\n", "Chunked storage is a powerful feature in Zarr arrays. Chunking divides large 3D datasets \n", "into smaller, manageable blocks (chunks), allowing efficient storage, access, and parallel\n", "processing of voxel data. It also supports compression within chunks, significantly reducing \n", "file size without compromising data integrity. This is especially useful for the massive \n", "datasets usually encountered in digital rock petrophysics. \n", "\n", "Key advantages of chunked data:\n", "\n", "- Efficient storage: Chunks can be compressed individually, reducing file size.\n", "- Random access: Specific chunks can be accessed and processed without loading the entire dataset.\n", "- Parallel processing: Different chunks can be processed simultaneously, speeding up computations.\n", "\n", "
\n", " \"Chunked\n", "
\n", " Structure of a chunked array. The large dataset is divided into small chunks.\n", " [(Originalimage)](https://www.unidata.ucar.edu/software/netcdf/workshops/2012/nc4chunking/WhatIsChunking.html)\n", "
\n", "
\n", "\n", "# Internal Structure\n", "Internally, Zarr organizes chunked data into a structured directory format. Each chunk is \n", "stored as a separate binary file, while metadata about the dataset is saved in JSON files. \n", "This modular design ensures flexibility and scalability.\n", "\n", "The key components of a Zarr array are:\n", "\n", "- ``.zarray`` file: contains metadata such as array shape, chunk size, and data type.\n", "- ``.zattrs`` file: stores user-defined metadata. In ``VoxelImage`` it store the image metadata such as voxel lengths and unit.\n", "- Chunk Files: Individual binary files containing the data for each chunk.\n", "\n", "
\n", " \"Structure\n", "
\n", " Structure in memory of a Zarr array \n", " [(Originalimage)](https://aws.amazon.com/pt/blogs/publicsector/decrease-geospatial-query-latency-minutes-seconds-using-zarr-amazon-s3/)\n", "
\n", "
\n", "\n", "\n", "# Creating a voxel image\n", "\n", "There are several options to create a ``VoxelImage`` object (see the documentation for the [voxel_image_module](../../api/voxel_image.rst)." ] } ], "metadata": { "kernelspec": { "display_name": "rockverse", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.7" } }, "nbformat": 4, "nbformat_minor": 2 }