Subset FASTA file by species name

Splitting into multiple files and changing the IDs can be easily done:

perl -pe 'if(/>/){/[(.*?)]s*$/; $_="> $1n"}' file.fa | 
    awk '(/^>/){name=$2} {print >> name".fa"}'

That assumes all your FASTA headers have [foo bar baz] as the last element of a line. It will create a file called foo.fa (the bacterium's name) with all sequences saved there.

Explanation

• perl -pe : run the script given by -e on each line of the input file, and print the resulting line.
• if(/>/) : if this line starts with a >.
• /[(.*?)]s*$/ : match an opening bracket ([), then capture (that's what the parentheses do, they capture a pattern so we can refer to it as $1) everything until the first ] (.*?])
• $_="> $1n" : the $_ special variable in Perl is (in this case) the current line. So, $_=foo means "make the current line read foo. Since the -p prints each input line, changing the value of $_ means the changed value will be printed. So here, we are printing >, whatever was in the square brackets ($1) and a newline character.

The output of the perl command alone on your example input file is:

$ perl -pe 'if(/>/){/[(.*)]s*$/; $_="> $1n"}' file.fa
> Arthrobacter phage Mudcat
MGLSNTATPLYYGQF...

> Achromobacter phage phiAxp-3
MSNVLLKQELDEWL...

> Delftia phage RG-2014
MSEPRKLVKKTLD...

So, we now pass it through awk which does:

• (/^>/){a=$2} : if this line starts with an >, save the second field (the bacterial species) as the variable name.
• {print >> name".fa"} : print each line into a file whose name is the current value of the variable name with a .fa. extension.

---

If you prefer python scripts to the one-liner approach, you can do the same thing with:

#!/usr/bin/env python
import re
outFile = None
with open("file.fa", "r") as inFile, open(species + ".fa", 'a') as outFile:
    for line in inFile:
        line = line.rstrip()
        if line.startswith('>'):
            regex = re.compile('.*[((.+?)s+.*?)].*')
            matches = regex.search(line)
            species = matches[2]
            outFile.write('>%sn' % matches[1])
        else:
            outFile.write("%sn" % line)

---

As for your script, you've got the right idea, but have a small bug. You forgot to remove the n from your input file, so it looks for Arthrobactern instead of Arthrobacter. The golden rule of debugging is "print all the things". If you add print("Searchterm: ",searchterm) you will see:

Searchterm:  (terminase large subunit AND viruses[Organism]) ANDArthrobacter  
AND refseq[Filter]
Searchterm:  (terminase large subunit AND viruses[Organism]) ANDAchromobacter  
AND refseq[Filter]
Searchterm:  (terminase large subunit AND viruses[Organism]) ANDDelftia  
AND refseq[Filter]

So, you need to remove the newline characters and add a space like so (I also made it a bit more "pythonic" and conforming to the Python syntax guidelines):

#!/usr/bin/env python
from Bio import Entrez
Entrez.email = "[email protected]"

with open("species2.txt", "r") as in_handle:
    for line in in_handle:
        line = line.rstrip()
        searchterm = ("(terminase large subunit AND viruses[Organism]) " +
                      "AND %s AND refseq[Filter]" % line)
        print("Searchterm: ", searchterm)
        searchResultHandle = Entrez.esearch(db="protein",
                                            term=searchterm, retmax=1000)
        searchResult = Entrez.read(searchResultHandle)
        ids = searchResult["IdList"]

        handle = Entrez.efetch(db="protein", id=ids,
                               rettype="fasta", retmode="text")
        record = handle.read()

        with open('terminase_large_' + str(line[:-1]) + '.fasta', 'w') as out_handle:
            out_handle.write(record.rstrip('n'))