Move specific files to another folder based on partial pattern in .txt file

Unix & Linux Asked by fidodido on August 17, 2020

I have a folder with various file types, but I am interested in the files with .img as extension, with the following pattern:


The only changing bit of the file names is the P_XXX part.

I have created two .txt files, stable.txt and recurring.txt, each containing a list of a subset of P_XXX that I would like to use to move the subsets to separate folders (stable and recurring, respectively). For example, P_76 and P_201 are listed in stable.txt, while P_104 and P_154 are listed in recurring.txt.

I tried a for loop to return the relevant P_XXX from the .txt file, so that I can use that to retrieve the matching .img in another for loop from the folder, which should then be moved to the stable folder:

for P in $(< stable.txt); do
     for f in *"$P"*.img; do
         echo mv - "$f" "./stable/$f"

It returns the correct number of P_XXX listed, but $f does not return the full filename (only the *P_XXX bit). Strangely enough, it does return the full filename for the last P_XXX in the .txt file (so ppi_noTD_d0_P_201_con_0001.img)

As there seems to be something going wrong with calling $f, I can’t move the files to their respective folders (stable and recurring).

How do I solve this?


This is the output that I’m getting:

*.img ./stable/*P_76
*.img ./stable/*P_86
*.img ./stable/*P_89
*.img ./stable/*P_90
*.img ./stable/*P_91
*.img ./stable/*P_99
*.img ./stable/*P_121
*.img ./stable/*P_128
*.img ./stable/*P_132
*.img ./stable/*P_136
*.img ./stable/*P_140
*.img ./stable/*P_144
*.img ./stable/*P_153
*.img ./stable/*P_156
*.img ./stable/*P_162
*.img ./stable/*P_180
*.img ./stable/*P_203
*.img ./stable/*P_205
*.img ./stable/*P_208
*.img ./stable/*P_211
*.img ./stable/*P_215
*.img ./stable/*P_229
*.img ./stable/*P_250
*.img ./stable/*P_256
mv - ppi_noTD_d0_P_257con_0001.img ./stable/ppi_noTD_d0_P_257con_0001.img

One Answer

Your stable.txt file was likely created or edited on a Windows system, where the newline is represented by the carriage rturn + line feed sequence (often referred to using the abbreviation CR LF or the escape sequence rn).

For instance, assuming this sample file:

printf '%srn' P_76 P_201 >stable.txt

After the first line is read by your script, the globbing expression *"$P"*.img matches nothing (unless your file names actually contain carriage return characters) and, if no nullglob (or equivalent) option is in effect, the value of f is *P_76r*.img. When mv - "$f" "./stable/$f" is echoed, the two CR characters cause the subsequent text to be inserted at the beginning of the line, overwriting what was already there.

You can check your files for CR LF newline sequences with

$ cat -v stable.txt


$ od -An -c stable.txt
   P   _   7   6  r  n   P   _   2   0   1  r  n


$ file stable.txt
stable.txt: ASCII text, with CRLF line terminators

And you can convert them to the Unix, LF-terminated format with (among other ways):

$ dos2unix stable.txt

See also:

Correct answer by fra-san on August 17, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP