将hdf5文件作为二进制Blob 字符串传递给h5py？

将hdf5文件作为二进制Blob /字符串传递给h5py？

如何绕过h5py中的磁盘I / O？

这是一个工作示例：

"""HDF5 in memory file reading example."""try:    import contextlib    import os    import tempfile    import h5py    hdf5_data = (        b'x89HDFrnx1anx00x00x00x00x00x08x08x00x04x00x10x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00xffxffxffxffxff'        b'xffxffxff|x05x00x00x00x00x00x00xffxffxffxffxffxff'        b'xffxffx00x00x00x00x00x00x00x00`x00x00x00x00x00x00'        b'x00x01x00x00x00x00x00x00x00x88x00x00x00x00x00x00x00'        b'xa8x02x00x00x00x00x00x00x01x00x01x00x01x00x00x00x18'        b'x00x00x00x00x00x00x00x11x00x10x00x00x00x00x00x88x00'        b'x00x00x00x00x00x00xa8x02x00x00x00x00x00x00TREEx00x00'        b'x01x00xffxffxffxffxffxffxffxffxffxffxffxffxffxffxff'        b'xffx00x00x00x00x00x00x00x000x04x00x00x00x00x00x00'        b'x08x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00HEAPx00x00x00x00X'        b'x00x00x00x00x00x00x00x10x00x00x00x00x00x00x00xc8x02'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00testx00x00'        b'x00x00x01x00x00x00x00x00x00x00Hx00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x01x00x06x00x01x00x00x00x00x01x00'        b'x00x00x00x00x00x01x00(x00x00x00x00x00x01x02x01x00'        b'x00x00x00x00x01x00x00x00x00x00x00x00x01x00x00x00x00'        b'x00x00x00x01x00x00x00x00x00x00x00x01x00x00x00x00x00'        b'x00x00x03x00x10x00x01x00x00x00x10x08x00x00x04x00x00'        b'x00x00x00 x00x00x00x00x00x05x00x08x00x01x00x00x00'        b'x02x02x02x01x00x00x00x00x08x00x18x00x01x00x00x00x03'        b'x01xx05x00x00x00x00x00x00x04x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x12x00x08x00x00x00x00x00x01x00x00'        b'x00xedxf3xa1Yx00x00px00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00SNODx01x00x01x00x08x00x00x00x00x00x00'        b'x00 x03x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'        b'x00x00x00x00x00x00x0090x00x00'    )    file_access_property_list = h5py.h5p.create(h5py.h5p.FILE_ACCESS)    file_access_property_list.set_fapl_core(backing_store=False)    file_access_property_list.set_file_image(hdf5_data)    file_id_args = {        'fapl': file_access_property_list,        'flags': h5py.h5f.ACC_RDONLY,        'name': next(tempfile._get_candidate_names()).enpre(),    }    h5_file_args = {'backing_store': False, 'driver': 'core', 'mode': 'r'}    with contextlib.closing(h5py.h5f.open(**file_id_args)) as file_id:        with h5py.File(file_id, **h5_file_args) as h5_file: assert h5_file['test'][0] == 12345    assert not os.path.exists(file_id_args['name'])except:    print('Something went wrong!')    raiseelse:    print('It works!! You can read HDF5 in memory!')

这似乎是一个hack，但这是我在

h5py

Github上挖掘时发现的。

从中可以看到

assert notos.path.exists(file_id_args['name'])

，尽管我传递了一个文件名（该文件名在文件系统中不应该存在），但实际上并未创建任何文件。

看到：

• https://github.com/h5py/h5py/blob/c2ad0b91f074b5b62d5c10b3970d39ae55b8ec1f/h5py/h5p.pyx#L1171
• https://github.com/h5py/h5py/pull/680#issue-135939862
• http://docs.h5py.org/en/latest/high/file.html#file-driver

如果有人知道这样做的技巧性较差，或者知道如何简化我的代码，请编辑此回复或发表评论。