TransWikia.com

How to share multiple "tagged" functions (symbols) like f[a][b] in parallel kernels?

Mathematica Asked on March 31, 2021

I would like to give some context: I need to analyse many data sets stored in files using different kind of analyses. To keep track I like to tag each results of my analysis for example like result["analysis1"]["file1"]. For this I am used to write code like this:

analyse[params_][file_]:=Module[{data},
  data=accessdata[file];
  result["analysis1"][file]=doanalysis1[params, data];
  result["analysis2"][file]=doanalysis2[params, data];
]

(*analalyse many files*)
files={"file1", "file2", ...};
Map[analyse[params],files];

(* aggregate results e.g. in plots or histograms*)
ListPlot[result["analysis1"]/@files];
Histogram[result["analysis2"]/@files]

This works fine and gives me great flexibility. For example if I need to re-analise e.g. "file17", maybe with some other parameters, I can just call analyse[otherparam]["file17"] and plot the updated results again. or I can save the results with Save["resultfile", result].

My problem arises, if I try to parallelize this.

ParallelMap[analyse[params],files];

does not work, because each kernel will have its own version of result.
I tried:

SetSharedFunction[result];
ParallelMap[analyse[params],files];

But I got error messages from all kernels like:

Set::write: Tag Null in Null["file1"] is protected.

Also I tried:

SetSharedFunction[result["analysis1"]]

which apparently does not even get executed.
How can one share result such that the code above works with ParallelMap?

One Answer

I found a workaround that avoids the use of tagged functions inside the ParallelMapbut allows its use outside.

analyse[params_][file_]:=Module[{data},
  data=accessdata[file];
  result["analysis1", file]=doanalysis1[params, data];
  result["analysis2", file]=doanalysis2[params, data];
]
result[tag_][file_]:= result[tag, file]
SetSharedFunction[result];
ParallelMap[analyse[params],files];
ListPlot[result["analysis1"]/@files];

Still I would be interested in other ideas!

Answered by dnet on March 31, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP