HDF5

Preprocessing

group hdf5_const

Defines

MOCCA_HDF5_CHUNK

Set the chunk size for the HDF5 files. This is only applicable for MOCCA objects. Default: 1M entries.

MOCCA_HDF5_COMPRESSION

Set the compression level of the HDF5 file. This is only applicable for MOCCA objects. Default: 5.

MOCCA_HDF5_CACHE

Set the cache size when reading MOCCA objects from HDF5 files. Default: 16 MB.

MOCCA_HDF5_CACHE_SLOTS

Set the number of cache slots when reading MOCCA objects from HDF5 files. This should be a prime number. Default: 127.

Flags

group hdf5_flags

Variables

static const hid_t default_flags = H5P_DEFAULT

Default HDF5 flags.

static const hid_t invalid_id = H5I_INVALID_HID

Invalid HDF5 id.

static const uint truncate = H5F_ACC_TRUNC

Truncate the file, i.e., if the file already exists, overwrite the contents of the file.

static const uint exclusive = H5F_ACC_EXCL

If the file already exists, returns an error.

static const uint read_only = H5F_ACC_RDONLY

Open the file in read-only mode.

static const uint read_write = H5F_ACC_RDWR

Open the file in read-write mode.

static const uint swmr_writer = H5F_ACC_SWMR_WRITE | read_write

Open the file that have a single-writer and multiple-readers (SWMR). Use this flag for the writer.

static const uint swmr_reader = H5F_ACC_SWMR_READ | read_only

Open the file that have a single-writer and multiple-readers (SWMR). Use this flag for the readers.

static const H5FD_mpio_xfer_t mpio_collective = H5FD_MPIO_COLLECTIVE

Set the MPI IO driver to use collective IO operations.

static const H5FD_mpio_xfer_t mpio_independent = H5FD_MPIO_INDEPENDENT

Set the MPI IO driver to use independent operations.

High-Level Routines

group hdf5_high

Functions

template<typename ...Args>
void read_hdf5(const hdf5::Group &group, Args&&... args)

Imports data from a HDF5 file.

Usage: 

Matrix<int> A;
Vector<double> v;
io::hdf5::File file("test.h5");
file.open();
io::Group group = file.root().open("G1");
io::read_hdf5(group, "mat", A, "vec", v);

This code will read “mat” and “vec” from the group “G1” in the file “test.h5”.

Parameters:

  • group – [in] a HDF5 group

  • args – [in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to store the data

template<typename ...Args>
void read_hdf5(hdf5::File &file, Args&&... args)

Imports data from a HDF5 file.

Usage: 

Matrix<int> A;
Vector<double> v
io::hdf5::File file("test.h5");
io::read_hdf5(file, "mat", A, "G1/vec", v);

This code will read “mat” from the file “test.h5” and store into A, and then read “vec” from the group “G1” in the file and store into v.

Parameters:

  • file – [in] a HDF5 file

  • args – [in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to store the data

template<typename ...Args>
void read_hdf5(std::string path, hdf5::FileProperties properties, Args&&... args)
template<typename ...Args>
void read_hdf5(std::string path, Args&&... args)

Imports data from a HDF5 file.

Usage: 

Matrix<int> A;
Vector<double> v;
io::read_hdf5("path/to/test.h5", "mat", A, "G1/vec", v);

This code will read “mat” from the file “test.h5” and store into A, and then read “vec” from the group “G1” in the file and store into v.

Parameters:

  • path – [in] path to the HDF5 file

  • properties – [in] file properties when opening the files (optional). See hdf5::FileProperties for more information.

  • args – [in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to store the data

template<typename ...Args>
void write_hdf5(const hdf5::Group &group, Args&&... args)

Writes MOCCA objects to a HDF5 file.

Usage: 

Matrix<int> A;
Vector<double> v;
io::hdf5::File file("test.h5");
file.create_group();
io::Group group = file.root().create_group("G1");
io::write_hdf5(group, "mat", A, "vec", v);

This code will create_group a file “test.h5” and a group “G1”, and then write A and v as “mat” and “vec” in the group “G1”, respectively.

Parameters:

  • group – [in] a HDF5 group

  • args – [in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to use as data source

template<typename ...Args>
void write_hdf5(hdf5::File &file, Args&&... args)

Writes MOCCA objects to a HDF5 file.

Usage: 

Matrix<int> A;
Vector<double> v;
io::hdf5::File file("test.h5");
io::write_hdf5(file, "mat", A, "G1/vec", v);

This code will create_group a file “test.h5” and then write A as “mat” and v as “vec” in the group “G1”.

Parameters:

  • file – [in] a HDF5 file

  • args – [in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to use as data source

template<typename ...Args>
void write_hdf5(std::string path, hdf5::FileProperties properties, Args&&... args)
template<typename ...Args>
void write_hdf5(std::string path, Args&&... args)

Writes MOCCA objects to a HDF5 file.

Usage: 

Matrix<int> A;
Vector<double> v;
io::write_hdf5("/path/to/test.h5", "mat", A, "G1/vec", v);

This code will create_group a file “test.h5” and then write A as “mat” and v as “vec” in the group “G1”.

Parameters:

  • path – [in] path to the HDF5 file

  • properties – [in] file properties when opening the files (optional). See hdf5::FileProperties for more information.

  • args – [in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to use as data source

File Properties

Warning

doxygenclass: Cannot find class “mocca::io::hdf5::FileProperties” in doxygen xml output for project “mocca” from directory: doxygen/xml/

File

class File

A object representing a HDF5 file. This object is only movable, but not copyable to avoid multiple objects pointing to the same file.

All objects (groups, datasets) within the file share the same properties (e.g., access flags, mpi support, etc.) as specifying by the FileProperties after creating or opening the file. See FileProperties for more information.

Public Functions

inline File(std::string path = "a.h5", FileProperties properties = FileProperties())

Construct a new File stream.

Parameters:

  • path – [in] path to the HDF5 file (optional)

  • properties – [in] file properties. See FileProperties for all options (optional).

inline File(File &&other)

Move constructor.

inline File &operator=(File &&other)

Move assignment.

inline virtual ~File()

Closes the associated file and deletes the File.

inline hid_t id() const
Returns:

the HDF5 file identifier.

inline std::string_view path() const
Returns:

the path to the the HDF5 file.

inline bool is_open() const
Returns:

true if the file is already opened.

inline void set_properties(FileProperties properties)

Set the file properties.

Parameters:

properties – [in] new file properties

inline FileProperties get_properties()
Returns:

the file properties.

inline void create(std::string path, uint flags)

Creates a new HDF5 file.

Parameters:

  • path – [in] path to the file (optional)

  • flags – [in] file creation flags (optional). This will overwrite the existing flags in the file properties. Options: exclusive or truncate.

inline void open(std::string path, uint flags)

Opens a HDF5 file.

Parameters:

  • path – [in] path to the file (optional)

  • flags – [in] file access flags (optional). This will overwrite the existing flags in the file properties. Options: read_write, read_only, swmr_writer or swmr_reader.

inline void close()

Closes the HDF5 file.

inline Group root() const
Returns:

the “root” of the file.

template<typename T>
inline void write_attribute(std::string name, const T &val) const

Set the attribute to be equal to val.

Parameters:

  • name – [in] name of the attribute

  • val – [in] value of the attribute

template<typename T>
inline T read_attribute(std::string name) const
Parameters:

name – [in] name of the attribute

Returns:

the value of the attribute.

inline void swap(File &other)

Swaps the content between this object and other.

Group

class Group

A object representing a HDF5 Group within a file or another group. This object is only movable, but not copyable to avoid multiple objects pointing to the same group.

Warning

Group objects should not be constructed explicitly. Instead, use root() method in the File to access the root group. From there, a subgroup can then be created or opened using the create_group() and open_group methods from the “parent” group, e.g., file.root().open_group("A").

Public Functions

inline Group &operator=(Group &&other)

Move Assignment.

inline virtual ~Group()

Closes the associated group and deletes the HDF5Group.

inline void close()

Closes the HDF5 group.

inline hid_t id() const
Returns:

the HDF5 group identifier.

inline std::string_view path() const
Returns:

the path to the HDF5 group.

inline bool is_open() const
Returns:

true if the file is already opened.

inline bool group_exist(std::string group_name) const

Checks if the group exists.

inline Group open_group(std::string path) const

Opens the group specified by the path.

inline Group create_group(std::string path) const

Creates a new group specified by the path. The group must not exist in the path.

template<typename T>
inline DenseDataObject<T> create_dense(std::string name, index_t size) const

Creates a new DenseDataObject that represents a dense datatype (e.g., a Vector or Matrix).

Parameters:

  • name – [in] name of the object

  • size – [in] size of the object

template<typename T>
inline SparseDataObject<T> create_sparse(std::string name) const

Creates a new SparseDataObject that represents a sparse datatype (e.g., a CSRMatrix or SparseVector).

Parameters:

name – [in] name of the object

template<typename T>
inline DenseDataObject<T> open_dense(std::string name) const

Opens a DenseDataObject that represents a dense datatype (e.g., a Vector or Matrix).

Parameters:

name – [in] name of the object

template<typename T>
inline SparseDataObject<T> open_sparse(std::string name) const

Opens a SparseDataObject that represents a sparse datatype (e.g., a CSRMatrix or SparseVector).

Parameters:

name – [in] name of the object

template<typename T>
inline void read(std::string name, Matrix<T> &mat) const

Reads a Matrix object from a HDF5 file.

Parameters:

  • name – [in] name of the object

  • mat – [out] Matrix with the contents of the file

template<typename T, internal::TypeLayout L>
inline void read(std::string name, Vector<T, L> &vec) const

Reads a Vector object from a HDF5 file.

Parameters:

  • name – [in] name of the object

  • vec – [out] Vector with the contents of the file

template<typename T>
inline void read(std::string name, CSRMatrix<T> &csr_mat) const

Reads a CSRMatrix object from a HDF5 file.

Parameters:

  • name – [in] name of the object

  • csr_mat – [out] CSRMatrix with the contents of the file

template<typename T, internal::TypeLayout L>
inline void read(std::string name, SparseVector<T, L> &vec) const

Reads a SparseVector object from a HDF5 file.

Parameters:

  • name – [in] name of the object

  • vec – [out] SparseVector with the contents of the file

template<typename T>
inline void write(std::string name, Matrix<T> &mat) const

Writes a Matrix object to a HDF5 file.

Parameters:

  • name – [in] name of the object

  • mat – [in] Matrix to use as data source

template<typename T, internal::TypeLayout L>
inline void write(std::string name, Vector<T, L> &vec) const

Writes a Vector object to a HDF5 file.

Parameters:

  • name – [in] name of the object

  • vec – [in] Vector to use as data source

template<typename T>
inline void write(std::string name, CSRMatrix<T> &csr_mat) const

Writes a CSRMatrix object to a HDF5 file.

Parameters:

  • name – [in] name of the object

  • csr_mat – [in] CSRMatrix to use as data source

template<typename T, internal::TypeLayout L>
inline void write(std::string name, SparseVector<T, L> &vec) const

Writes a SparseVector object to a HDF5 file.

Parameters:

  • name – [in] name of the object

  • csr_mat – [in] SparseVector to use as data source

template<typename T>
inline void write_attribute(std::string name, const T &val) const

Set the attribute to be equal to val.

Parameters:

  • name – [in] name of the attribute

  • val – [in] value of the attribute

template<typename T>
inline T read_attribute(std::string name) const
Parameters:

name – [in] name of the attribute

Returns:

the value of the attribute.

inline void swap(Group &other)

Swap the contents of Group with an other.

Parameters:

other – [in] another Group

Dense Data Objects

template<typename T>
class DenseDataObject

Object representing a dense datatype (Vector or Matrix) in the HDF5 file. This object contain all the necessary information to read and write from the corresponding file. This object is only movable, but not copyable.

Warning

DenseDataObject should not be constructed explicitly. Instead, use open_dense() and create_dense() methods from the “parent” group to open and create dense objects, respectively.

Public Functions

inline DenseDataObject &operator=(DenseDataObject &&other)

Move assignment.

inline virtual ~DenseDataObject()

Destructor.

inline bool is_open() const
Returns:

true if the file is already opened.

inline void open()

Opens the HDF5 dataset associated with this object.

inline void create(hsize_t size)

Creates a chunked and compressed HDF5 dataset and binds it to this object.

inline void close()

Closes the HDF5 dataset associated with this object.

inline index_t size() const
Returns:

the size of the DenseDataObject.

inline index_t rows() const
Returns:

the number of columns in the DenseDataObject.

inline index_t cols() const
Returns:

the number of columns in the DenseDataObject.

inline Vector<T> read_vec()

Reads a Vector object from a HDF5 file.

Parameters:

name – [in] name of the object

Returns:

a Vector with the contents of the file

template<internal::TypeLayout L>
inline void read(Vector<T, L> &vec)

Reads a Vector object from a HDF5 file.

Parameters:

vec – [out] Vector with the contents of the file

template<internal::TypeLayout L>
inline void read(Vector<T, L> &vec, index_t begin, index_t end)

Reads a data block from the Vector object from a HDF5 file as specified by the range [begin, end[. The size of output Vector will be equal to end - begin.

Parameters:

  • vec – [out] Vector with the contents of the file

  • begin – [in] index of the first element

  • end – [in] index of the last element (not included)

template<internal::TypeLayout L>
inline void write(Vector<T, L> &vec)

Writes a Vector object to a HDF5 file.

Parameters:

vec – [in] Vector to use as data source

inline Matrix<T> read_mat()

Reads a Matrix object from a HDF5 file.

Parameters:

name – [in] name of the object

Returns:

a Matrix with the contents of the file

inline void read(Matrix<T> &mat)

Reads a Matrix object from a HDF5 file.

Parameters:

mat – [out] Matrix with the contents of the file

inline void read(Matrix<T> &mat, index_t first_row, index_t last_row, index_t first_col, index_t last_col)

Reads a data block from the Matrix object from a HDF5 file as specified by the range [first_row, last_row[ and [first_col, last_col[. The number of rows and columns of output Matrix will be equal to last_row - first_row and last_col - first_col, respectively.

Parameters:

  • mat – [out] Matrix with the contents of the file

  • first_row – [in] row index of the first element in the submatrix

  • last_row – [in] row index of the last element in the submatrix (not included)

  • first_col – [in] column index of the first element in the submatrix

  • last_col – [in] column index of the last element in the submatrix (not included)

inline void write(Matrix<T> &mat)

Writes a Matrix object to a HDF5 file.

Parameters:

mat – [in] Matrix to use as data source

template<typename U>
inline void write_attribute(std::string name, const U &val) const

Sets the attribute to be equal to val.

Parameters:

  • name – [in] name of the attribute

  • val – [in] value of the attribute

template<typename U>
inline U read_attribute(std::string name) const
Parameters:

name – [in] name of the attribute

Returns:

the value of the attribute.

inline void swap(DenseDataObject &other)

Swaps the contents between this object and other.

Sparse Data Objects

template<typename T>
class SparseDataObject

Object representing a sparse datatype (CSRMatrix or SparseVector) in the HDF5 file. This object contain all the necessary information to read and write from the corresponding file. This object is only movable, but not copyable.

Warning

SparseDataObject should not be constructed explicitly. Instead, use open_sparse() and create_sparse() methods from the “parent” group to open and create sparse objects, respectively.

Public Functions

inline SparseDataObject &operator=(SparseDataObject &&other)

Move assignment.

inline virtual ~SparseDataObject()

Destructor.

inline bool is_open() const
Returns:

true if the file is already opened.

inline void open()

Opens the HDF5 group associated with this object.

inline void create()

Creates the HDF5 group associated with this object.

inline void close()

Closes the HDF5 group associated with this object.

inline index_t size() const
Returns:

the number of nonzeros in the SparseDataObject.

inline index_t rows() const
Returns:

the number of columns in the SparseDataObject.

inline index_t cols() const
Returns:

the number of columns in the SparseDataObject.

inline CSRMatrix<T> read_csrmat()

Reads a CSRMatrix object from a HDF5 file.

Returns:

a CSRMatrix with the contents of the file

inline void read(CSRMatrix<T> &mat)

Reads a CSRMatrix object from a HDF5 file.

Parameters:

mat – [out] CSRMatrix with the contents of the file

inline void read(CSRMatrix<T> &mat, index_t first_row, index_t last_row)

Reads a row block from a CSRMatrix object from a HDF5 file as specified by the range [first_row, last_row[. The nonzero entries will be placed in their original row.

Warning

Before calling this routine, set the CSRMatrix to the correct dimensions as this routine will not resize the CSRMatrix.

Parameters:

  • mat – [out] CSRMatrix with the contents of the file

  • first_row – [in] index of the first matrix row

  • last_row – [in] index of the last matrix row (not included)

inline void write(CSRMatrix<T> &mat)

Writes a CSRMatrix object to a HDF5 file.

Parameters:

mat – [in] CSRMatrix to use as data source

inline SparseVector<T> read_sparse_vec()

Reads a SparseVector object from a HDF5 file.

Returns:

a SparseVector with the contents of the file

template<internal::TypeLayout L>
inline void read(SparseVector<T, L> &vec)

Reads a SparseVector object from a HDF5 file.

Parameters:

vec – [out] SparseVector with the contents of the file

template<internal::TypeLayout L>
inline void write(SparseVector<T, L> &vec)

Writes a SparseVector object to a HDF5 file.

Parameters:

vec – [in] SparseVector to use as data source

template<typename U>
inline void write_attribute(std::string name, const U &val) const

Sets the attribute to be equal to val.

Parameters:

  • name – [in] name of the attribute

  • val – [in] value of the attribute

template<typename U>
inline U read_attribute(std::string name) const
Parameters:

name – [in] name of the attribute

Returns:

the value of the attribute.

inline void swap(SparseDataObject &other)

Swaps the contents between this object and other.