Unix & Linux Asked by Roofkiller on December 24, 2020
I found all processes on my machine to only run on a single core and their core affinity set to 0.
Here is a small python script which reproduces this for me:
import multiprocessing
import numpy as np
def do_a_lot_of_compute(a):
for i in range(1000):
a = a * np.random.randn(123789)
return a
if __name__ == '__main__':
with multiprocessing.Pool() as pool:
pool.map(do_a_lot_of_compute, np.arange(10000))
htop looks like this:
And the core affinities are:
pid 15977's current affinity list: 0
pid 15978's current affinity list: 0
pid 15979's current affinity list: 0
pid 15980's current affinity list: 0
pid 15981's current affinity list: 0
pid 15982's current affinity list: 0
pid 15983's current affinity list: 0
pid 15984's current affinity list: 0
pid 15985's current affinity list: 0
So my question boils down to: Why are the core affinities all set to 0? There are no OMP or KMP environment variables set.
The problem was related to SLURM
and PBS
setting the core affinity based on the number of requested cores.
In SLURM
adding the following line enables the use of all cores:
#SBATCH --cpus-per-task=8
Correct answer by Roofkiller on December 24, 2020
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP