ChatGPT解决这个技术问题 Extra ChatGPT

How to perform grep operation on all files in a directory?

Working with xenserver, and I want to perform a command on each file that is in a directory, grepping some stuff out of the output of the command and appending it in a file.

I'm clear on the command I want to use and how to grep out string(s) as needed.

But what I'm not clear on is how do I have it perform this command on each file, going to the next, until no more files are found.


u
umi

grep $PATTERN * would be sufficient. By default, grep would skip all subdirectories. However, if you want to grep through them, grep -r $PATTERN * is the case.


"grep -r $PATTERN ." is sufficient. For recursive grep you do not need to specify all files
How do I include multiple extensions? Eg. "*.cpp", "*.h"?
@Tomáš Zato, just supply all your file patterns instead of *: grep $PATTERN *.cpp *.h. If you need more specific rules for what files should be grepped, use find command (check Rob's answer).
@Chris it's possible you don't have *.scss files in current directory but somewhere deeper in subdirs so grep does not look in all the files you wanted. You should use --include option to tell grep to look recursively for files that matches specific patterns: grep -r x --include '*.scss' . (note the quotes, they prevent the pattern from being expanded by the shell). Or just use find (see Rob's answer).
You want grep -s so you don't get a warning for each subdirectory that grep skips. You should probably double-quote "$PATTERN" here.
P
Philippe Fanaro

In Linux, I normally use this command to recursively grep for a particular text within a directory:

grep -rni "string" *

where

r = recursive i.e, search subdirectories within the current directory

n = to print the line numbers to stdout

i = case insensitive search


R
Rob

Use find. Seriously, it is the best way because then you can really see what files it's operating on:

find . -name "*.sql" -exec grep -H "slow" {} \;

Note, the -H is mac-specific, it shows the filename in the results.


If you decide to use find, pipe the output through xargs rather than using -exec this will be much faster, because -exec spawns a new process for each grep, and the overhead becomes significant with a large number of files. Standard warnings about spaces in file names apply to xargs.
What would this look like exactly? I always forget xargs syntax.
@BrianPeterson it would be something like that: find . -iname "*.sql" -print0 | xargs -0 grep "slow"
grep -r offers much the same functionality. The OP didn't seem to actually want to traverse subdirectories.
-exec on modern find supports shortcutting just like xargs, and will actually be faster (because you cut out xargs). Use -exec ... {} + instead of -exec ... {} \;
N
Noam Manos

To search in all sub-directories, but only in specific file types, use grep with --include.

For example, searching recursively in current directory, for text in *.yml and *.yaml :

grep "text to search" -r . --include=*.{yml,yaml}

b
bryan

If you want to do multiple commands, you could use:

for I in `ls *.sql`
do
    grep "foo" $I >> foo.log
    grep "bar" $I >> bar.log
done

Generally, probably try shellcheck.net on the code you post. Don't use ls to expand a wildcard (the shell already does it for you!) and quote your shell variables.
You probably want to use Awk instead for this, especially if the input file is big. awk '/foo/ { print >"foo.log" } /bar/ { print >"bar.log" }' "$I" searches for both patterns in a single pass over the input file. (Try grep -E 'foo|bar' "$I"if you want to search for multiple patterns, but don't require multiple output files.)