Working with xenserver, and I want to perform a command on each file that is in a directory, grepping some stuff out of the output of the command and appending it in a file.
I'm clear on the command I want to use and how to grep out string(s) as needed.
But what I'm not clear on is how do I have it perform this command on each file, going to the next, until no more files are found.
grep $PATTERN *
would be sufficient. By default, grep would skip all subdirectories. However, if you want to grep through them, grep -r $PATTERN *
is the case.
In Linux, I normally use this command to recursively grep
for a particular text within a directory:
grep -rni "string" *
where
r = recursive i.e, search subdirectories within the current directory
n = to print the line numbers to stdout
i = case insensitive search
Use find. Seriously, it is the best way because then you can really see what files it's operating on:
find . -name "*.sql" -exec grep -H "slow" {} \;
Note, the -H is mac-specific, it shows the filename in the results.
xargs
rather than using -exec
this will be much faster, because -exec
spawns a new process for each grep, and the overhead becomes significant with a large number of files. Standard warnings about spaces in file names apply to xargs
.
xargs
syntax.
find . -iname "*.sql" -print0 | xargs -0 grep "slow"
grep -r
offers much the same functionality. The OP didn't seem to actually want to traverse subdirectories.
-exec
on modern find
supports shortcutting just like xargs
, and will actually be faster (because you cut out xargs
). Use -exec ... {} +
instead of -exec ... {} \;
To search in all sub-directories, but only in specific file types, use grep with --include
.
For example, searching recursively in current directory, for text in *.yml and *.yaml :
grep "text to search" -r . --include=*.{yml,yaml}
If you want to do multiple commands, you could use:
for I in `ls *.sql`
do
grep "foo" $I >> foo.log
grep "bar" $I >> bar.log
done
ls
to expand a wildcard (the shell already does it for you!) and quote your shell variables.
awk '/foo/ { print >"foo.log" } /bar/ { print >"bar.log" }' "$I"
searches for both patterns in a single pass over the input file. (Try grep -E 'foo|bar' "$I"
if you want to search for multiple patterns, but don't require multiple output files.)
Success story sharing
grep $PATTERN *.cpp *.h
. If you need more specific rules for what files should be grepped, usefind
command (check Rob's answer).*.scss
files in current directory but somewhere deeper in subdirs so grep does not look in all the files you wanted. You should use--include
option to tell grep to look recursively for files that matches specific patterns:grep -r x --include '*.scss' .
(note the quotes, they prevent the pattern from being expanded by the shell). Or just usefind
(see Rob's answer).grep -s
so you don't get a warning for each subdirectory thatgrep
skips. You should probably double-quote"$PATTERN"
here.