HDF5#

Preprocessing#

group hdf5_const

Defines

MOCCA_HDF5_CHUNK#

Set the chunk size for the HDF5 files. This is only applicable for MOCCA objects. Default: 1M entries.

MOCCA_HDF5_COMPRESSION#

Set the compression level of the HDF5 file. This is only applicable for MOCCA objects. Default: 5.

MOCCA_HDF5_CACHE#

Set the cache size when reading MOCCA objects from HDF5 files. Default: 16 MB.

MOCCA_HDF5_CACHE_SLOTS#

Set the number of cache slots when reading MOCCA objects from HDF5 files. This should be a prime number. Default: 127.

Flags#

group hdf5_flags

Variables

static const hid_t default_flags = H5P_DEFAULT#

Default HDF5 flags.

static const hid_t invalid_id = H5I_INVALID_HID#

Invalid HDF5 id.

static const uint truncate = H5F_ACC_TRUNC#

Truncate the file, i.e., if the file already exists, overwrite the contents of the file.

static const uint exclusive = H5F_ACC_EXCL#

If the file already exists, returns an error.

static const uint read_only = H5F_ACC_RDONLY#

Open the file in read-only mode.

static const uint read_write = H5F_ACC_RDWR#

Open the file in read-write mode.

static const uint swmr_writer = H5F_ACC_SWMR_WRITE | read_write#

Open the file that have a single-writer and multiple-readers (SWMR). Use this flag for the writer.

static const uint swmr_reader = H5F_ACC_SWMR_READ | read_only#

Open the file that have a single-writer and multiple-readers (SWMR). Use this flag for the readers.

static const H5FD_mpio_xfer_t mpio_collective = H5FD_MPIO_COLLECTIVE#

Set the MPI IO driver to use collective IO operations.

static const H5FD_mpio_xfer_t mpio_independent = H5FD_MPIO_INDEPENDENT#

Set the MPI IO driver to use independent operations.

High-Level Routines#

group hdf5_high

Functions

template<typename ...Args>
void read_hdf5(const hdf5::Group &group, Args&&... args)#

Imports data from a HDF5 file.

Usage:

Matrix<int> A;
Vector<double> v;
io::hdf5::File file("test.h5");
file.open();
io::Group group = file.root().open("G1");
io::read_hdf5(group, "mat", A, "vec", v);

This code will read “mat” and “vec” from the group “G1” in the file “test.h5”.

Parameters:
  • group[in] a HDF5 group

  • args[in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to store the data

template<typename ...Args>
void read_hdf5(hdf5::File &file, Args&&... args)#

Imports data from a HDF5 file.

Usage:

Matrix<int> A;
Vector<double> v
io::hdf5::File file("test.h5");
io::read_hdf5(file, "mat", A, "G1/vec", v);

This code will read “mat” from the file “test.h5” and store into A, and then read “vec” from the group “G1” in the file and store into v.

Parameters:
  • file[in] a HDF5 file

  • args[in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to store the data

template<typename ...Args>
void read_hdf5(std::string path, hdf5::FileProperties properties, Args&&... args)#
template<typename ...Args>
void read_hdf5(std::string path, Args&&... args)#

Imports data from a HDF5 file.

Usage:

Matrix<int> A;
Vector<double> v;
io::read_hdf5("path/to/test.h5", "mat", A, "G1/vec", v);

This code will read “mat” from the file “test.h5” and store into A, and then read “vec” from the group “G1” in the file and store into v.

Parameters:
  • path[in] path to the HDF5 file

  • properties[in] file properties when opening the files (optional). See hdf5::FileProperties for more information.

  • args[in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to store the data

template<typename ...Args>
void write_hdf5(const hdf5::Group &group, Args&&... args)#

Writes MOCCA objects to a HDF5 file.

Usage:

Matrix<int> A;
Vector<double> v;
io::hdf5::File file("test.h5");
file.create_group();
io::Group group = file.root().create_group("G1");
io::write_hdf5(group, "mat", A, "vec", v);

This code will create_group a file “test.h5” and a group “G1”, and then write A and v as “mat” and “vec” in the group “G1”, respectively.

Parameters:
  • group[in] a HDF5 group

  • args[in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to use as data source

template<typename ...Args>
void write_hdf5(hdf5::File &file, Args&&... args)#

Writes MOCCA objects to a HDF5 file.

Usage:

Matrix<int> A;
Vector<double> v;
io::hdf5::File file("test.h5");
io::write_hdf5(file, "mat", A, "G1/vec", v);

This code will create_group a file “test.h5” and then write A as “mat” and v as “vec” in the group “G1”.

Parameters:
  • file[in] a HDF5 file

  • args[in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to use as data source

template<typename ...Args>
void write_hdf5(std::string path, hdf5::FileProperties properties, Args&&... args)#
template<typename ...Args>
void write_hdf5(std::string path, Args&&... args)#

Writes MOCCA objects to a HDF5 file.

Usage:

Matrix<int> A;
Vector<double> v;
io::write_hdf5("/path/to/test.h5", "mat", A, "G1/vec", v);

This code will create_group a file “test.h5” and then write A as “mat” and v as “vec” in the group “G1”.

Parameters:
  • path[in] path to the HDF5 file

  • properties[in] file properties when opening the files (optional). See hdf5::FileProperties for more information.

  • args[in] pattern: (“name”, obj, …). Here name specify the name of the variable and obj, the object to use as data source

File Properties#

Warning

doxygenclass: Cannot find class “mocca::io::hdf5::FileProperties” in doxygen xml output for project “mocca” from directory: doxygen/xml/

File#

class File#

A object representing a HDF5 file. This object is only movable, but not copyable to avoid multiple objects pointing to the same file.

All objects (groups, datasets) within the file share the same properties (e.g., access flags, mpi support, etc.) as specifying by the FileProperties after creating or opening the file. See FileProperties for more information.

Public Functions

inline File(std::string path = "a.h5", FileProperties properties = FileProperties())#

Construct a new File stream.

Parameters:
  • path[in] path to the HDF5 file (optional)

  • properties[in] file properties. See FileProperties for all options (optional).

inline File(File &&other)#

Move constructor.

inline File &operator=(File &&other)#

Move assignment.

inline virtual ~File()#

Closes the associated file and deletes the File.

inline hid_t id() const#
Returns:

the HDF5 file identifier.

inline std::string_view path() const#
Returns:

the path to the the HDF5 file.

inline bool is_open() const#
Returns:

true if the file is already opened.

inline void set_properties(FileProperties properties)#

Set the file properties.

Parameters:

properties[in] new file properties

inline FileProperties get_properties()#
Returns:

the file properties.

inline void create(std::string path, uint flags)#

Creates a new HDF5 file.

Parameters:
  • path[in] path to the file (optional)

  • flags[in] file creation flags (optional). This will overwrite the existing flags in the file properties. Options: exclusive or truncate.

inline void open(std::string path, uint flags)#

Opens a HDF5 file.

Parameters:
  • path[in] path to the file (optional)

  • flags[in] file access flags (optional). This will overwrite the existing flags in the file properties. Options: read_write, read_only, swmr_writer or swmr_reader.

inline void close()#

Closes the HDF5 file.

inline Group root() const#
Returns:

the “root” of the file.

template<typename T>
inline void write_attribute(std::string name, const T &val) const#

Set the attribute to be equal to val.

Parameters:
  • name[in] name of the attribute

  • val[in] value of the attribute

template<typename T>
inline T read_attribute(std::string name) const#
Parameters:

name[in] name of the attribute

Returns:

the value of the attribute.

inline void swap(File &other)#

Swaps the content between this object and other.

Group#

class Group#

A object representing a HDF5 Group within a file or another group. This object is only movable, but not copyable to avoid multiple objects pointing to the same group.

Warning

Group objects should not be constructed explicitly. Instead, use root() method in the File to access the root group. From there, a subgroup can then be created or opened using the create_group() and open_group methods from the “parent” group, e.g., file.root().open_group("A").

Public Functions

inline Group &operator=(Group &&other)#

Move Assignment.

inline virtual ~Group()#

Closes the associated group and deletes the HDF5Group.

inline void close()#

Closes the HDF5 group.

inline hid_t id() const#
Returns:

the HDF5 group identifier.

inline std::string_view path() const#
Returns:

the path to the HDF5 group.

inline bool is_open() const#
Returns:

true if the file is already opened.

inline bool group_exist(std::string group_name) const#

Checks if the group exists.

inline Group open_group(std::string path) const#

Opens the group specified by the path.

inline Group create_group(std::string path) const#

Creates a new group specified by the path. The group must not exist in the path.

template<typename T>
inline DenseDataObject<T> create_dense(std::string name, index_t size) const#

Creates a new DenseDataObject that represents a dense datatype (e.g., a Vector or Matrix).

Parameters:
  • name[in] name of the object

  • size[in] size of the object

template<typename T>
inline SparseDataObject<T> create_sparse(std::string name) const#

Creates a new SparseDataObject that represents a sparse datatype (e.g., a CSRMatrix or SparseVector).

Parameters:

name[in] name of the object

template<typename T>
inline DenseDataObject<T> open_dense(std::string name) const#

Opens a DenseDataObject that represents a dense datatype (e.g., a Vector or Matrix).

Parameters:

name[in] name of the object

template<typename T>
inline SparseDataObject<T> open_sparse(std::string name) const#

Opens a SparseDataObject that represents a sparse datatype (e.g., a CSRMatrix or SparseVector).

Parameters:

name[in] name of the object

template<typename T>
inline void read(std::string name, Matrix<T> &mat) const#

Reads a Matrix object from a HDF5 file.

Parameters:
  • name[in] name of the object

  • mat[out] Matrix with the contents of the file

template<typename T, internal::TypeLayout L>
inline void read(std::string name, Vector<T, L> &vec) const#

Reads a Vector object from a HDF5 file.

Parameters:
  • name[in] name of the object

  • vec[out] Vector with the contents of the file

template<typename T>
inline void read(std::string name, CSRMatrix<T> &csr_mat) const#

Reads a CSRMatrix object from a HDF5 file.

Parameters:
  • name[in] name of the object

  • csr_mat[out] CSRMatrix with the contents of the file

template<typename T, internal::TypeLayout L>
inline void read(std::string name, SparseVector<T, L> &vec) const#

Reads a SparseVector object from a HDF5 file.

Parameters:
  • name[in] name of the object

  • vec[out] SparseVector with the contents of the file

template<typename T>
inline void write(std::string name, Matrix<T> &mat) const#

Writes a Matrix object to a HDF5 file.

Parameters:
  • name[in] name of the object

  • mat[in] Matrix to use as data source

template<typename T, internal::TypeLayout L>
inline void write(std::string name, Vector<T, L> &vec) const#

Writes a Vector object to a HDF5 file.

Parameters:
  • name[in] name of the object

  • vec[in] Vector to use as data source

template<typename T>
inline void write(std::string name, CSRMatrix<T> &csr_mat) const#

Writes a CSRMatrix object to a HDF5 file.

Parameters:
  • name[in] name of the object

  • csr_mat[in] CSRMatrix to use as data source

template<typename T, internal::TypeLayout L>
inline void write(std::string name, SparseVector<T, L> &vec) const#

Writes a SparseVector object to a HDF5 file.

Parameters:
  • name[in] name of the object

  • csr_mat[in] SparseVector to use as data source

template<typename T>
inline void write_attribute(std::string name, const T &val) const#

Set the attribute to be equal to val.

Parameters:
  • name[in] name of the attribute

  • val[in] value of the attribute

template<typename T>
inline T read_attribute(std::string name) const#
Parameters:

name[in] name of the attribute

Returns:

the value of the attribute.

inline void swap(Group &other)#

Swap the contents of Group with an other.

Parameters:

other[in] another Group

Dense Data Objects#

template<typename T>
class DenseDataObject#

Object representing a dense datatype (Vector or Matrix) in the HDF5 file. This object contain all the necessary information to read and write from the corresponding file. This object is only movable, but not copyable.

Warning

DenseDataObject should not be constructed explicitly. Instead, use open_dense() and create_dense() methods from the “parent” group to open and create dense objects, respectively.

Public Functions

inline DenseDataObject &operator=(DenseDataObject &&other)#

Move assignment.

inline virtual ~DenseDataObject()#

Destructor.

inline bool is_open() const#
Returns:

true if the file is already opened.

inline void open()#

Opens the HDF5 dataset associated with this object.

inline void create(hsize_t size)#

Creates a chunked and compressed HDF5 dataset and binds it to this object.

inline void close()#

Closes the HDF5 dataset associated with this object.

inline index_t size() const#
Returns:

the size of the DenseDataObject.

inline index_t rows() const#
Returns:

the number of columns in the DenseDataObject.

inline index_t cols() const#
Returns:

the number of columns in the DenseDataObject.

inline Vector<T> read_vec()#

Reads a Vector object from a HDF5 file.

Parameters:

name[in] name of the object

Returns:

a Vector with the contents of the file

template<internal::TypeLayout L>
inline void read(Vector<T, L> &vec)#

Reads a Vector object from a HDF5 file.

Parameters:

vec[out] Vector with the contents of the file

template<internal::TypeLayout L>
inline void read(Vector<T, L> &vec, index_t begin, index_t end)#

Reads a data block from the Vector object from a HDF5 file as specified by the range [begin, end[. The size of output Vector will be equal to end - begin.

Parameters:
  • vec[out] Vector with the contents of the file

  • begin[in] index of the first element

  • end[in] index of the last element (not included)

template<internal::TypeLayout L>
inline void write(Vector<T, L> &vec)#

Writes a Vector object to a HDF5 file.

Parameters:

vec[in] Vector to use as data source

inline Matrix<T> read_mat()#

Reads a Matrix object from a HDF5 file.

Parameters:

name[in] name of the object

Returns:

a Matrix with the contents of the file

inline void read(Matrix<T> &mat)#

Reads a Matrix object from a HDF5 file.

Parameters:

mat[out] Matrix with the contents of the file

inline void read(Matrix<T> &mat, index_t first_row, index_t last_row, index_t first_col, index_t last_col)#

Reads a data block from the Matrix object from a HDF5 file as specified by the range [first_row, last_row[ and [first_col, last_col[. The number of rows and columns of output Matrix will be equal to last_row - first_row and last_col - first_col, respectively.

Parameters:
  • mat[out] Matrix with the contents of the file

  • first_row[in] row index of the first element in the submatrix

  • last_row[in] row index of the last element in the submatrix (not included)

  • first_col[in] column index of the first element in the submatrix

  • last_col[in] column index of the last element in the submatrix (not included)

inline void write(Matrix<T> &mat)#

Writes a Matrix object to a HDF5 file.

Parameters:

mat[in] Matrix to use as data source

template<typename U>
inline void write_attribute(std::string name, const U &val) const#

Sets the attribute to be equal to val.

Parameters:
  • name[in] name of the attribute

  • val[in] value of the attribute

template<typename U>
inline U read_attribute(std::string name) const#
Parameters:

name[in] name of the attribute

Returns:

the value of the attribute.

inline void swap(DenseDataObject &other)#

Swaps the contents between this object and other.

Sparse Data Objects#

template<typename T>
class SparseDataObject#

Object representing a sparse datatype (CSRMatrix or SparseVector) in the HDF5 file. This object contain all the necessary information to read and write from the corresponding file. This object is only movable, but not copyable.

Warning

SparseDataObject should not be constructed explicitly. Instead, use open_sparse() and create_sparse() methods from the “parent” group to open and create sparse objects, respectively.

Public Functions

inline SparseDataObject &operator=(SparseDataObject &&other)#

Move assignment.

inline virtual ~SparseDataObject()#

Destructor.

inline bool is_open() const#
Returns:

true if the file is already opened.

inline void open()#

Opens the HDF5 group associated with this object.

inline void create()#

Creates the HDF5 group associated with this object.

inline void close()#

Closes the HDF5 group associated with this object.

inline index_t size() const#
Returns:

the number of nonzeros in the SparseDataObject.

inline index_t rows() const#
Returns:

the number of columns in the SparseDataObject.

inline index_t cols() const#
Returns:

the number of columns in the SparseDataObject.

inline CSRMatrix<T> read_csrmat()#

Reads a CSRMatrix object from a HDF5 file.

Returns:

a CSRMatrix with the contents of the file

inline void read(CSRMatrix<T> &mat)#

Reads a CSRMatrix object from a HDF5 file.

Parameters:

mat[out] CSRMatrix with the contents of the file

inline void read(CSRMatrix<T> &mat, index_t first_row, index_t last_row)#

Reads a row block from a CSRMatrix object from a HDF5 file as specified by the range [first_row, last_row[. The nonzero entries will be placed in their original row.

Warning

Before calling this routine, set the CSRMatrix to the correct dimensions as this routine will not resize the CSRMatrix.

Parameters:
  • mat[out] CSRMatrix with the contents of the file

  • first_row[in] index of the first matrix row

  • last_row[in] index of the last matrix row (not included)

inline void write(CSRMatrix<T> &mat)#

Writes a CSRMatrix object to a HDF5 file.

Parameters:

mat[in] CSRMatrix to use as data source

inline SparseVector<T> read_sparse_vec()#

Reads a SparseVector object from a HDF5 file.

Returns:

a SparseVector with the contents of the file

template<internal::TypeLayout L>
inline void read(SparseVector<T, L> &vec)#

Reads a SparseVector object from a HDF5 file.

Parameters:

vec[out] SparseVector with the contents of the file

template<internal::TypeLayout L>
inline void write(SparseVector<T, L> &vec)#

Writes a SparseVector object to a HDF5 file.

Parameters:

vec[in] SparseVector to use as data source

template<typename U>
inline void write_attribute(std::string name, const U &val) const#

Sets the attribute to be equal to val.

Parameters:
  • name[in] name of the attribute

  • val[in] value of the attribute

template<typename U>
inline U read_attribute(std::string name) const#
Parameters:

name[in] name of the attribute

Returns:

the value of the attribute.

inline void swap(SparseDataObject &other)#

Swaps the contents between this object and other.