Bioinformatics Asked on July 16, 2021
I am setting up a HiChIPseq pipeline, but installing software necessary for the HiC side of things has been unbecoming.
Both my PI and I have tried to install Hi-C Pro onto our Mac Pro that we use as our compute server (though in a couple of weeks we hope to migrate to an EPYC 64 Core computer server of our own!).
There are three approaches that I have taken to install Hi-C Pro.
To install from source, you need to supply the paths to its dependencies. Believing we did a good job, we run it and find problems accessing bx-python.
No matter how many times we update pip and try to install bx-python and specify that we want it installed for python2.
This is the one I tried. From what I understand this creates a VM with the necessary dependencies installed, and allows you to use the program without building. However, it appears to abort when conda tries to install numpy.
+ conda install -y -c anaconda numpy=1.16.3
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): / Killed
ABORT: Aborting with RETVAL=255
Cleaning up...
I have seen the failed solving environment bug many times, but have no idea how to solve it. I have tried updating conda, but that doesn’t solve it.
Running conda.sh give me this:
Solving environment: failed
ResolvePackageNotFound:
<Lots of packages...>
Sorry if this is sloppy.
I could install HiC-Pro using conda when creating a separate environment (which I always do for side-projects), and then pull the conda build of HiC-Pro:
conda create --name HiCPro python=2.7
conda activate HiCPro
conda install -c davebx hicpro
Not sure what conda.sh
is that you describe, sometimes tricky to get random code from GitHub running. Give it a go with a standard conda install as I described.
Answered by ATpoint on July 16, 2021
If you're reading this, I can only assume that's because you had trouble installing HiC-Pro too. I was finally able to install from source and run the software, so here is how I did it, giving detail to the error that I was hit with and the clues I used to solve the puzzles and get through installing this program:
From Source To install from source, you need to supply the paths to its dependencies. Believing we did a good job, we run it and find problems accessing bx-python.
No matter how many times we update pip and try to install bx-python and specify that we want it installed for p
I downloaded from source once again using git clone
.
The I cat
ed the config file to find the programs I needed to find the path of, which I did using which
. I ensured that I was using python2.
I also used pip2
to install pysam, numpy, and scipy. I needed to use sudo to install pysam, though. I also installed something called bx-python.
Here is the error I get when I run make configure
Checking Python libraries ...
Traceback (most recent call last):
File "../scripts/install/check_pythonlib.py", line 24, in <module>
import bx.intervals
ModuleNotFoundError: No module named 'bx'
Traceback (most recent call last):
File "../scripts/install/check_pythonlib.py", line 28, in <module>
raise ImportError('Error - bx-python cannot be imported')
ImportError: Error - bx-python cannot be imported
Can not proceed without the required Python libraries, please install them and re-run
make[1]: *** [configure] Error 1
make: *** [configure] Error 2
Looking at this error for the fifth time, I realize that the first part, and not the last part, is the important one. It's yelling at me that I don't have the bx module installed. It also gives me the path to the file that threw the exception, so I read it and saw that this was what was being run:
## bx
try:
import bx.intervals
if vcmp(bx.__version__, '0.5.0') == -1:
raise ValueError('bx-python '+bx.__version__+' detected. Version >= 0.5.0 required')
except ImportError:
raise ImportError('Error - bx-python cannot be imported')
I notice that there's nothing fancy, the thing just needs to be imported, and its actual name is bx. Wonderful, I pip2 it. But... I hit another error in pip:
pyrsistent requires Python '>=3.5' but the running Python is 2.7.16
Apparently, pyrsistent dropped support for python2. Wonderful. I get around it by:
sudo pip2 install awsebcli pyrsistent==0.16.0
In the middle of this installation, Im reading the output and see:
bx-python 0.8.9 has requirement six>=1.13.0, but you'll have six 1.11.0 which is incompatible.
So:
sudo pip2 install six==1.13.0
At this point, I, without any more problems, sudo pip2 install bx
.
I run make configure
again, and I run into...
Checking Python libraries ...
Traceback (most recent call last):
File "../scripts/install/check_pythonlib.py", line 24, in <module>
import bx.intervals
ModuleNotFoundError: No module named 'bx'
Traceback (most recent call last):
File "../scripts/install/check_pythonlib.py", line 28, in <module>
raise ImportError('Error - bx-python cannot be imported')
ImportError: Error - bx-python cannot be imported
Can not proceed without the required Python libraries, please install them and re-run
make[1]: *** [configure] Error 1
make: *** [configure] Error 2
huh? Didn't I fix this?? Something seems fishy. For whatever reason, my mind then wanders to the Makefile to see if anything there pops out, cause... what else would be calling that python file? In the main Makefile I see:
make -f ./scripts/install/Makefile CONFIG_SYS=$(CONFIG_SYS)
So I head on over to ./scripts/install/Makefile and see:
./scripts/install/install_dependencies.sh -c $(CONFIG_SYS) -o $(realpath $(PREFIX))/HiC-Proo _$(VNUM) -q
Nothing like a goose chase. So I head on over to install_dependencies.sh, where I see:
# python
which python > /dev/null;
if [ $? != "0" ]; then
echo -e "$RED""Can not proceed without Python, please install and re-run""$NORMAL"
exit 1;
else
pver=`python --version 2>&1 | cut -d" " -f2` #<- THIS LINE
vercomp $pver "2.7.0"
if [[ $? == 2 ]]; then
echo -e "$RED""Python v2.7.0 or higher is needed [$pver detected].""$NORMAL"
exit 1;
fi
fi
I check that lines output, AND IT'S PYTHON3!! For some reason, it didn't give me that error message, it just kept going from that line.
So, I changed all instances of python
with python2
in that file.
I reran make configure
and got The required Python libraries appear to be already installed.
Bliss.
Next step was make install
, and guess what? A roadblock:
Traceback (most recent call last):
File "setup.py", line 2, in <module>
from numpy.distutils.core import setup
ModuleNotFoundError: No module named 'numpy'
make: *** [iced] Error 1
Hmm... well, it's probably the same thing. python3 being called where python2 should be. So I double checked the main Makefile and changed the setup.py call to use python2.
I run make install
and it runs for awhile, chugging along using the g++ compiler. Then I hit an odd error:
cp: /Users/bosslab/Downloads/HiC-Pro-master/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4: name too long (not copied)
I'm guessing that the Makefile has a recursive path somewhere. I check the appropriate section and find:
cp -Ri $(MK_PATH) $(INSTALL_PATH)
I simply replace that line with the actual paths, with the install_path being adjacent instead of inside the mk_path.
Then I get hit with:
HiC-Pro installed in /Users/bosslab/Downloads/HiC-Pro-master/HiC-Pro_2.11.4 !
So running $install_dir/HiC-Pro --help
returns the help. YAY!
I installed it, but how about when I actually run it?
Error: The 'ice' command is not in your path. Please check where the 'iced' python package has been installed and update your PATH !
Ok:
$ sudo pip2 install iced
Requirement already satisfied: iced in /Users/bosslab/Library/Python/2.7/lib/python/site-packages (0.4.2)
...so, I set up a test to see whether it actually is installed:
$ python2
Python 2.7.16 (default, Nov 9 2019, 05:55:08)
[GCC 4.2.1 Compatible Apple LLVM 11.0.0 (clang-1100.0.32.4) (-macos10.15-objc-s on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> iimport iced
>>> quit()
It's already installed. But then I had an epiphany:
$ python3
Python 3.8.5 (default, Jul 21 2020, 10:48:26)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ipomport iced
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'iced'
>>> quit()
Aha! Somewhere (i don't know where though) python3 is being called. So I just installed that module for python3 with sudo pip3 install iced
.
Then I had some trouble with pointing to the right bowtie index (just give the directory where the index exists) and then I hit my next error dealing with HiC-Pro itself:
Pairing of R1 and R2 tags ...
Logs: logs/dixon_2M/mergeSAM.log
make: *** [bowtie_pairing] Error 1
Checking the log file named in the error message:
/usr/local/bin/python /Users/bosslab/Downloads/HiC-Pro-master/HiC-Pro_2.11.4/scripts/mergeSAM. py -q 0 -t -v -f bowtie_results/bwt2/dixon_2M/SRR400264_00_R1_hg19.bwt2merged.bam -r bowtie_re sults/bwt2/dixon_2M/SRR400264_00_R2_hg19.bwt2merged.bam -o bowtie_results/bwt2/dixon_2M/SRR400 264_00_hg19.bwt2pairs.bam
File "/Users/bosslab/Downloads/HiC-Pro-master/HiC-Pro_2.11.4/scripts/mergeSAM.py", line 26
print "Usage : python mergeSAM.py"
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Usage : python mergeS AM.py")?
Missing parentheses? Oh my god, it's using python3 for some reason again. I grep the installation directory for a file that calls merge_sam.py, and find config-system.txt. I go in there and just need to change the python path from /usr/bin/local/
where python3 is installed to /usr/bin/
where python2 is installed.
Lastly, I get an error during the ICE normalization step that numpy is not installed. Using my big fat brain, I isntall numpy to my python3 and call it a day. Now it all works as intended.
Answered by Jeff on July 16, 2021
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP