TransWikia.com

Export high precision data to HDF5

Mathematica Asked on May 20, 2021

Recently, I would like to use HDF5 instead of CSV because of times Import taking.
Especially, I have to manage large data with high precision (about 50 degits) and it takes very long time to export such data to CSV. Therefore, HDF5 seems nice to me.

However, when I do Import HDF5 file, I only get data with MachinePrecision.
On the other hand, CSV returns data with exact precision which I exported.

Here is a simple sample:

In[1]= data = N[Pi, 50];
In[2]= data // Precision
Out[2]= 50.

Then,

In[3]= Export["testing_precision_HDF5.h5", {data}];
In[4]= Export["testing_precision_csv.csv", data];

Finally I do Import and evaluate their Precision;

In[5]= Import["testing_precision_HDF5.h5", {"Datasets", 
    "/Dataset1"}][[1]] // Precision
Out[5]= MachinePrecision

On the other hand,

In[6]= Import["testing_precision_csv.csv"] // Precision
Out[6]= 49.4971

So, where does this difference come from? Is the reason why HDF5 takes short time to Import HDF5 cannot handle high precision data?

Please tell me how to manage high precision data with HDF5 & Mathematica.

Update: I don’t think it is a problem with ”’Import”’ because I checked the .h5 file with HDFview, but it was same result.

One Answer

I tried some solutions, finally I found the best way to do; using ToString at Export and ToExpression at Import. Furtheremore, use FullForm before type casting not to export with exponential notation (then we cannnot import such data in usual way). This enable us to handle with data with any precision.

Answered by Keyspire on May 20, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP