How to overwrite array inside h5 file using h5py

I'm trying to overwrite a numpy array that's a small part of a pretty complicated h5 file.

I'm extracting an array, changing some values, then want to re-insert the array into the h5 file.

I have no problem extracting the array that's nested.

f1 = h5py.File(file_name,'r')
X1 = f1['meas/frame1/data'].value
f1.close()

My attempted code looks something like this with no success:

f1 = h5py.File(file_name,'r+')
dset = f1.create_dataset('meas/frame1/data', data=X1)
f1.close()

As a sanity check, I executed this in Matlab using the following code, and it worked with no problems.

h5write(file1, '/meas/frame1/data', X1);

Does anyone have any suggestions on how to do this successfully?

3 Answers

You want to assign values, not create a dataset:

f1 = h5py.File(file_name, 'r+') # open the file
data = f1['meas/frame1/data'] # load the data
data[...] = X1 # assign new values to data
f1.close() # close the file

To confirm the changes were properly made and saved:

f1 = h5py.File(file_name, 'r')
np.allclose(f1['meas/frame1/data'].value, X1)
#True

askewchan's answer describes the way to do it (you cannot create a dataset under a name that already exists, but you can of course modify the dataset's data). Note, however, that the dataset must have the same shape as the data (X1) you are writing to it. If you want to replace the dataset with some other dataset of different shape, you first have to delete it:

del f1['meas/frame1/data']
dset = f1.create_dataset('meas/frame1/data', data=X1)

Different scenarios:

1. Partial changes to dataset

with h5py.File(file_name,'r+') as ds: ds['meas/frame1/data'][5] = val # change index 5 to scalar "val" ds['meas/frame1/data'][3:7] = vals # change values of indices 3--6 to "vals"

2. Change each value of dataset (identical dataset sizes)

with h5py.File(file_name,'r+') as ds: ds['meas/frame1/data'][...] = X1 # change array values to those of "X1"

3. Overwrite dataset to one of different size

with h5py.File(file_name,'r+') as ds: del ds['meas/frame1/data'] # delete old, differently sized dataset ds.create_dataset('meas/frame1/data',data=X1) # implant new-shaped dataset "X1"

Since the File object is a context manager, using with statements is a nice way to package your code, and automatically close out of your dataset once you're done altering it. (You don't want to be in read/write mode if you only need to read off data!)