Easily renaming multiple files.

Posted by Steve on Wed 1 Jun 2005 at 05:29

Tags: none.

Renaming multiple files seems to be a problem which many newcomers to shell scripting, or administration, have problems with. But once you've done it a few times the actual solutions are very simple.

There are many cases where you might have a large number of files to rename en masse, for example files which are output from a given script or tool. Or files that must be renamed to be uploaded to a web-host.

How you rename the files mostly depends on which tools you have available, and which shell you're using.

The way I tend to rename large numbers of files is the same way that I tend do any job which requires running the same command on a number of files - I use the looping facility within the bash shell.

For example lets assume you have several files in a directory a.JPG, b.JPG, c.JPG and d.JPG then you can work with each one as follows:

skx@lappy:~$ for i in *.JPG; do echo $i; done
a.JPG
b.JPG
c.JPG
d.JPG

This command has worked with each file which matches the pattern "*.JPG", and runs a command on each file - in this case it's "echo $i".

Obviously the command which we're running, here it is echo, can be replaced with other commands.

If we used to use this style of command to rename each of the files to lowercase their extensions we have two possible approaches:

  • Take advantage of bash's variable substituion.
  • Use the standard unix command basename.

The bash shell has a notion of text substitutions which is very powerful, and something that can be very useful to know about so we'll cover that first.

The simplest example I can think of is to strip the extension. For a filename which is stored in the shell variable $i you can strip a known extension with this:

skx@lappy:~$ i=a.JPG 
skx@lappy:~$ echo $i
a.JPG
skx@lappy:~$ echo ${i/.JPG/}
a

Here we've used the substitution operation to strip the ".JPG" from the end of our variable - we can plug this into the loop we showed earlier to force a rename:

skx@lappy:~$ ls
a.JPG  b.JPG  c.JPG  d.JPG
skx@lappy:~$ for i in *.JPG; do mv "$i" "${i/.JPG}".jpg; done
skx@lappy:~$ ls
a.jpg  b.jpg  c.jpg  d.jpg

(The "quotes" around the arguments to the mv command are useful if you're ever working with files which contain strange characters, or spaces, in their names).

This is the basic way that you can rename files with bash - use a loop to work on each file, then use the built-in substituion operations to strip or add to each name in turn.

The other approach is to use basename to manipulate filenames. basename is a standard command which allows you to strip suffixes from filenames:

skx@lappy:~$ basename c.jpg .jpg
c
skx@lappy:~$ basename c.jpg pg
c.j

Here you can see that the command just returns the filename you've specified with the second argument removed from it.

This can be plugged into our loop like this:

skx@lappy:~$ ls
a.jpg  b.jpg  c.jpg  d.jpg
skx@lappy:~$ for i in *.jpg; do mv "$i" "`basename $i .jpg`.JPG"; done
skx@lappy:~$ ls
a.JPG  b.JPG  c.JPG  d.JPG

Using Specialised Commands

Instead of remembering how to run these loops over files with bash, you can instead rename files en masse with specialised renaming commands.

A simple one is the rename command which is included with the Debian perl package and almost certainly available to you already.

The rename command allows you to rename files with full perl expressions, in a simple manner.

For example our preceeding example of changing the extension can be achieved with:

skx@lappy:~$ rename 's/\.JPG/\.jpg/' *.JPG

We can also, for example, strip spaces from filenames with this:

skx@lappy:~$ rename 's/ //' *.JPG

Renaming files from upper to lower case can be a simple job:

skx@lappy:~$ rename 'y/A-Z/a-z/' *

There are also other specialised commands which you can locate with the apt-cache command, such as mmv and mrename.

 

 


Posted by Arnaud (131.254.xx.xx) on Wed 1 Jun 2005 at 12:04
[ View Weblogs ]
Thanks for these little tips.
I was usually renaming files one by one, because I was too lazy to write bash scripts or one-liners to do that.

I knew there were commands to do that but I never really searched. Your article made me discover the "rename" command, which seems simple but also pretty powerfull thanks to the use of perl regexp. I have been using Debian for 4 years, but never used it before !

--
Arnaud

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Wed 1 Jun 2005 at 16:59
[ View Steve's Scratchpad | View Weblogs ]

Thanks - I'm glad you liked it.

I'm often unsure of the right level to pitch things at, so I figure some "simple" pieces and some "complex" pieces will please most people.

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by Anonymous (86.3.xx.xx) on Fri 19 Mar 2010 at 23:34
The mv command in the first example (variable substitution) could have been written even more simply as

for i in *.JPG; do mv $i ${i/JPG/jpg}; done;

-Mahesh

[ Parent | Reply to this comment ]

Posted by Anonymous (82.119.xx.xx) on Wed 1 Jun 2005 at 12:59
Those mentioned quotes around the argument won't prevent problems with spaces in filenames. The problem is in "for" command which will use the space as separator between its arguments, so if we have files "a.jpg" and "b b.jpg":
for i in *.jpg
is equivalent to
for i in a.jpg b b.jpg
so "echo $i" will be executed 3 times and for files which actually don't exist ("b" and "b.jpg")

Does anyone have a solution to this?

[ Parent | Reply to this comment ]

Posted by jaddle (66.11.xx.xx) on Wed 1 Jun 2005 at 13:28
Actually, I find that using "for i in *.jpg" works fine in this situation. The problem arises when you try to use the output of another command in the for.

i.e.:

for i in `ls *.jpg` will have the problem you describe, as will
for i in `find -name *.jpg`

When I ran into that problem, I ended up just using 'for i in *.jpg' without using a backticked command to work around it.

[ Parent | Reply to this comment ]

Posted by lindenle (68.77.xx.xx) on Wed 1 Jun 2005 at 13:53
[ View Weblogs ]
I usually have to convert as well as rename so i use something like this:

for file in $(ls *.ps); do newfile=$(echo $file | sed -e s/\.ps/\.jpg/); convert $file $newfile; done

of course for renaming you just change convert to mv....

just my $.02

[ Parent | Reply to this comment ]

Posted by Anonymous (80.58.xx.xx) on Sat 4 Jun 2005 at 01:11
You can optimize a bit :

for file in *.ps
do
convert $file ${file%%.ps}.jpg
done

#;-)

[ Parent | Reply to this comment ]

Posted by Swynndla (210.54.xx.xx) on Thu 2 Jun 2005 at 22:09
I have had trouble with for loops and spaces for other things, eg when cat'ing files, the following won't keep spaces between words:
for i in `cat file`; do blah blah "$i"; done

So I use the following instead:
cat file | while read i; do blah blah "$i"; done

And that works as "read" reads in a line at a time.

Now I tend to use this method for everything and hardly ever use "for".

[ Parent | Reply to this comment ]

Posted by Anonymous (199.8.xx.xx) on Thu 5 Apr 2007 at 15:37

I think cat is the problem on this.

For me, a better syntax for processing a file line by line is:

while read LINE 
do

  blah
  blah
  blah
	
done < $1 

Just specify the file on the command line, or you can replace the $1 with the filename or full path to the file.

--Jim
linuxactivist.blogspot.com

[ Parent | Reply to this comment ]

Posted by apeekaboo (85.226.xx.xx) on Thu 18 Aug 2005 at 17:01
In bash you can change the input field separator to handle filenames with blank space in them.
It's best used in a script like this:
IFS=$'\t\n'

This changes IFS to only trigger on tabs and newlines.

The default is to trigger on spaces as well, like this:
IFS=$' \t\n'

Just beware of commands that rely on IFS being in default state, if you notice any oddities.

[ Parent | Reply to this comment ]

Posted by Anonymous (24.200.xx.xx) on Mon 9 Jul 2012 at 05:10
Bravo ! This works best ;-)

[ Parent | Reply to this comment ]

Posted by yanchina12 (193.60.xx.xx) on Fri 14 Dec 2007 at 17:25
Hello there,

I've got several files with names like u90021_dw_v001.txt, u90021_dw_v011.hdr etc in a UNIX directory. I would like to rename them to u90021_dw_v1.txt and u90021_dw_v11.hdr respectively and copy them within a folder as a backup. So basically I would like to remove 'one/two zeros' immideately after 'v'. Your help will be highly appreciated!

[ Parent | Reply to this comment ]

Posted by Anonymous (198.133.xx.xx) on Wed 1 Jun 2005 at 15:36

This is the other thing I end up needing to do a lot, so I thought I'd throw it in.

Say I use a local domain of prentiss.house because I live on Prentiss street, but then I move to Massachusetts Avenue and want to change my domain accordingly in my local DNS.

perl -pi -e's/prentiss.house/massave.house/g' /etc/bind9/*

Awesome.

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Wed 1 Jun 2005 at 16:58
[ View Steve's Scratchpad | View Weblogs ]

I was saving that for my next piece!

(Although I usually use perl -pi.back -e "...." - to create a backup of the input file too).

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by Anonymous (198.133.xx.xx) on Wed 1 Jun 2005 at 17:31
Hehe...whoops! Although that backup thing is a good call...I had never thought really about what trouble I could get myself into that way.

[ Parent | Reply to this comment ]

Posted by stevenothing (84.12.xx.xx) on Wed 1 Jun 2005 at 22:19
Or sed -i.back -e's/fish/cake/g' *.txt for simple substitutions but without having to invoke perl, which is the approach I generally prefer for simple bash scripts.

[ Parent | Reply to this comment ]

Posted by Anonymous (199.8.xx.xx) on Thu 5 Apr 2007 at 15:48
I use this one as well. Sed is pretty amazing.

--Jim

[ Parent | Reply to this comment ]

Posted by Anonymous (199.8.xx.xx) on Thu 5 Apr 2007 at 15:58
I use this one as well. Sed is pretty amazing.

--Jim

[ Parent | Reply to this comment ]

Posted by pgquiles (81.202.xx.xx) on Wed 1 Jun 2005 at 21:05

When you have thousands of files to rename, remove or do anything with, the "for" approach is very slow. And if you try to run the shell command (i. e. rm *.JPG) will not able to deal with so much files.

I found a more convenient way (and faster, too) is to use ls ands xargs, as describes this article in Unix Review

[ Parent | Reply to this comment ]

Posted by Anonymous (199.8.xx.xx) on Thu 5 Apr 2007 at 16:01
Thanks for the tip. I will definitely be using that in the future.

--Jim

[ Parent | Reply to this comment ]

Posted by maurits (80.126.xx.xx) on Wed 1 Jun 2005 at 22:03
I'll add the renameutils package. Especially the 'qmv' command can be handy sometimes.

'qmv *' will open an editor with all files in two columns. You edit the second column to your liking and the files will then be renamed.

[ Parent | Reply to this comment ]

Posted by Anonymous (80.58.xx.xx) on Thu 2 Jun 2005 at 16:17
With mmv:

mmv -r '*JPG' '#1jpg'

[ Parent | Reply to this comment ]

Posted by denjer (82.241.xx.xx) on Mon 6 Jun 2005 at 15:10
To me the best approach by far, since 'mmv' handles filename collisions... Option '-v' shows what is being done.

[ Parent | Reply to this comment ]

Posted by Anonymous (150.216.xx.xx) on Thu 2 Jun 2005 at 21:42
If you use ZSH, man zshcontrib and search for 'zmv'.

[ Parent | Reply to this comment ]

Posted by Anonymous (82.238.xx.xx) on Mon 6 Jun 2005 at 12:05
I wrote a little Ruby script to handle renaming of multiple files 'cause I never remember regexp and things like that.

You can download it at: http://renamer.bounga.org

[ Parent | Reply to this comment ]

Posted by Adem (85.97.xx.xx) on Mon 11 Jul 2005 at 00:59
I have about a few millions of mail files in my Debian box.

I want to copy them to windows box using ssh.

But, the file names are 'modified-Base64' which means that they contain characters that are not acceptable on windows.

Using ls in a script did not help. There are too many files in folders.

Plus, all I want to do is to traverse the folders and simply substitute a different char for the offending chars (if the file name contains them).

Can someone help me out, please

[ Parent | Reply to this comment ]

Posted by muondude (206.117.xx.xx) on Tue 20 Sep 2005 at 06:41
[ View muondude's Scratchpad | View Weblogs ]

I often use Perl for these sort of tasks. Here are a couple of examples you can modify to solve your problem (substitute or remove the offending characters).

Command Line file renaming: rename a group of files with extension .html to .shtml extensions:

perl -e 'for (@ARGV) { ($new=$_) =~ s/(.+)(.)\.html$/$1$2.shtml/; rename $_, $new unless -e $new }' *.html
In your case you could use a regular expression for the offending characters and substitute them with whatever you like.


Convert a bunch of file names from UPPER case to lower case


perl -e 'for (@ARGV) { ($new=$_) =~ tr/[A-Z]/[a-z]/; rename $_, $new unless -e $new }' *.html


I strongly suggest you make a copy of some of your files first and test this out before letting it go on all your files.
You could also modify these perl scripts to be part of a shell script that loops over your 10^6 files, or write a full perl script to do that.

-- Sam

[ Parent | Reply to this comment ]

Posted by Anonymous (82.88.xx.xx) on Wed 21 Jun 2006 at 08:31
Hi Debian Admins :)

FYI on this topic: http://www.michael-forman.com/perl/ren-regexp.html


Fabio
http://www.rigadicomando.org
(what you see is what you Grep ;)



[ Parent | Reply to this comment ]

Posted by Anonymous (59.97.xx.xx) on Sat 27 Dec 2008 at 05:47
If you take a little time and go through your My Pictures folder and apply this technique(i.e<a href="http://dadecoders.blogspot.com/2007/11/how-to-rename-100-files-at -time.html">; rename all files at a time</a> to all your photos, the next time you try to find a specific photo you'll be glad that you did. And if you get into the practice of applying this technique right after you download your images to your computer, your digital photos will be well-organized and up-to-date as you accumulate digital memorie

open folder which contain images or any other files.select images(files) which you want to rename sequentially.
now whatever files you are selected are appear in blue coloured selected files.
just right click on the first image(file) and select<a href="http://dadecoders.blogspot.com/2007/11/how-to-rename-100-files-at -time.html">; rename</a>here type "marriage"then press enterNow you can see that all the files are name as
marriage
marriage (1)
marriage (2)....
,

[ Parent | Reply to this comment ]

Posted by a_ashabi (208.65.xx.xx) on Mon 5 Jan 2009 at 22:39
[ View Weblogs ]
Hi.I have a folder with 1000 file names like this:
14280_aljuc_1.jpg
14280_aljuc_3.jpg
14280_LBL.jpg
14280_SUP.jpg
100_Fkidbld_3.jpg
110_Fkidbld_1.jpg
1001_Soothing_1.jpg
10100_Half_1.jpg
...

if there is a middle part in the name of the files I want to rename them and delete the middle text.which means I need to have these results:
14280_1.jpg
14280_3.jpg
14280_LBL.jpg
14280_SUP.jpg
100_3.jpg
110_1.jpg
1001_1.jpg
10100_1.jpg

plz help.thanks

[ Parent | Reply to this comment ]

Posted by Anonymous (189.79.xx.xx) on Tue 29 Dec 2009 at 01:01
I think
rename -v 's/_[^_]*//'
will do the trick.

[ Parent | Reply to this comment ]

Posted by linulin (91.202.xx.xx) on Sat 1 May 2010 at 20:14

'vidir' should be mentioned. It allows editing directory contents in a (powerful) text editor.

Use "aptitude install moreutils" to get the utility.

--
...Bye..Dmitry.

[ Parent | Reply to this comment ]

Posted by Anonymous (214.3.xx.xx) on Fri 23 Jul 2010 at 20:54
Here is another solution to replace it anywhere in the file recursively:

for i in `find . |grep -i ORIGINALTEXT`; do mv $i ${i/ORIGINALTEXT/NEWTEXT}; done

I hope this will be useful for somebody without the use of the rename command. Should work almost anywhere....

[ Parent | Reply to this comment ]

Posted by Anonymous (99.110.xx.xx) on Sat 9 Oct 2010 at 10:27
This does not work in a situation I need it. I used a tool to extract some files to a directory. Instead of putting it in the "foo" directory it put foo\ in front of the file name. example: foo\foo.txt I moved the foo\foo.txt to the foo folder and now I'm trying to strip the "foo\" from the file name but the \ in the file name is causing a > prompt to come up on a couple of the posted commands here I've tried.

[ Parent | Reply to this comment ]

Posted by Anonymous (62.58.xx.xx) on Thu 20 Jan 2011 at 15:46
Tried this but it does not work with find :( It only appends the new basename to the existing basename. e.g
files:
./DE/NAV_SV_Sweden_INV_20091234_1.txt
./DE/NAV_SV_Sweden_INV_20091235_1.txt

$ find -name "*.txt" -exec mv \{\} "`basename \{\} .txt`.copying" \

Afterwards:
./DE/NAV_SV_Sweden_INV_20091234_1.txt.copying
./DE/NAV_SV_Sweden_INV_20091235_1.txt.copying

Damn.

[ Parent | Reply to this comment ]

Posted by Anonymous (50.98.xx.xx) on Mon 13 Jun 2011 at 07:30
using only the first two examples, and having no real skills with bash, i managed to encode 98 VOB files from a VIDEO_TS folder that were in the format of VTS_67_1.VOB using mencoder in one command (while ignoring all other filetypes and companion vob files of the format VTS_67_0.VOB) by doing this:
for i in VTS_??_1.VOB; do j=${i/_1.VOB}; j=${j/VTS_}; mencoder $i -oac mp3lame -ovc lavc -o $i.avi; done

[ Parent | Reply to this comment ]

Posted by Anonymous (131.146.xx.xx) on Thu 8 Dec 2011 at 17:53
awesome. thanks for the post. I've been doing this the "hard way" for years. just another bad habit of mine. haha

[ Parent | Reply to this comment ]

Posted by Anonymous (188.109.xx.xx) on Thu 29 Dec 2011 at 14:54
Hello,
I have files like this
1x01 - Serien Name - Serien title.avi

I want to change them to
Serien Name - 1x01 - Serien title.avi

I am not sure how I get parts of the name and replace them in another location

It would be nice if you could also comment what you have done.

Thanks,

Bene

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 525 votes ~ 7 comments )