You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What is the recommended way to add a new tensor to an existing safetensors file?
For example, I load the existing safetensors file into dictionary d using the torch API. Then I add a new tensor with the key "test" into the dictionary. Can I save the modified dictionary of tensors using the same filename? Safetensors is memory-mapped so I assume it's not safe but I'm asking for clarification or suggestion as to what is the best way to do this kind of update.
from safetensors.torch import load_file, save_file
import torch
# Load
file_path = "embedding.safetensors"
d = load_file(file_path)
# Add entry
d['test'] = torch.tensor([0.0, 0.1, 0.2])
# Save to same filename ok?
save_file(d, file_path)
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
What is the recommended way to add a new tensor to an existing safetensors file?
For example, I load the existing safetensors file into dictionary
d
using the torch API. Then I add a new tensor with the key "test" into the dictionary. Can I save the modified dictionary of tensors using the same filename? Safetensors is memory-mapped so I assume it's not safe but I'm asking for clarification or suggestion as to what is the best way to do this kind of update.Beta Was this translation helpful? Give feedback.
All reactions