Mathematica Asked on March 31, 2021
I would like to give some context: I need to analyse many data sets stored in files using different kind of analyses. To keep track I like to tag each results of my analysis for example like result["analysis1"]["file1"]
. For this I am used to write code like this:
analyse[params_][file_]:=Module[{data},
data=accessdata[file];
result["analysis1"][file]=doanalysis1[params, data];
result["analysis2"][file]=doanalysis2[params, data];
]
(*analalyse many files*)
files={"file1", "file2", ...};
Map[analyse[params],files];
(* aggregate results e.g. in plots or histograms*)
ListPlot[result["analysis1"]/@files];
Histogram[result["analysis2"]/@files]
This works fine and gives me great flexibility. For example if I need to re-analise e.g. "file17", maybe with some other parameters, I can just call analyse[otherparam]["file17"]
and plot the updated results again. or I can save the results with Save["resultfile", result]
.
My problem arises, if I try to parallelize this.
ParallelMap[analyse[params],files];
does not work, because each kernel will have its own version of result
.
I tried:
SetSharedFunction[result];
ParallelMap[analyse[params],files];
But I got error messages from all kernels like:
Set::write: Tag Null in Null["file1"] is protected.
Also I tried:
SetSharedFunction[result["analysis1"]]
which apparently does not even get executed.
How can one share result
such that the code above works with ParallelMap
?
I found a workaround that avoids the use of tagged functions inside the ParallelMap
but allows its use outside.
analyse[params_][file_]:=Module[{data},
data=accessdata[file];
result["analysis1", file]=doanalysis1[params, data];
result["analysis2", file]=doanalysis2[params, data];
]
result[tag_][file_]:= result[tag, file]
SetSharedFunction[result];
ParallelMap[analyse[params],files];
ListPlot[result["analysis1"]/@files];
Still I would be interested in other ideas!
Answered by dnet on March 31, 2021
Get help from others!
Recent Answers
Recent Questions
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP