ChatGPT解决这个技术问题 Extra ChatGPT

How to search and replace using grep

I need to recursively search for a specified string within all files and subdirectories within a directory and replace this string with another string.

I know that the command to find it might look like this:

grep 'string_to_find' -r ./*

But how can I replace every instance of string_to_find with another string?

I don't believe grep can do this (I could be wrong). Easier ways would be to use sed or perl to do the replacing
Try to use sed -i 's/.*substring.*/replace/'
@Eddy_Em That will replace the entire line with replace. You need to use grouping to capture the part of the line before and after the substring and then put that in the replacement line. sed -i 's/\(.*\)substring\(.*\)/\1replace\2/'

r
rezizter

Another option is to use find and then pass it through sed.

find /path/to/files -type f -exec sed -i 's/oldstring/new string/g' {} \;

On OS X 10.10 Terminal, a proper extension string to parameter -i is required. For example, find /path/to/files -type f -exec sed -i "" "s/oldstring/new string/g" {} \; Anyway, providing empty string still creates a backup file unlike described in manual...
Why do I get "sed: RE error: illegal byte sequence". And yes, I added the -i "" for OS X. It works otherwise.
I had the illegal byte sequence issue on macOS 10.12, and this question/answer solved my issue: stackoverflow.com/questions/19242275/….
This touches every file so file times are modified; and converts line endings from CRLF to LF on Windows.
b
billtian

I got the answer.

grep -rl matchstring somedir/ | xargs sed -i 's/string1/string2/g'

This would scan through the matching files twice... once with grep and then again with sed. Using find method is more efficient but this method you mention does work.
On OS X you will need to change sed -i 's/str1/str2/g' to sed -i "" 's/str1/str2/g' for this to work.
@cmevoli with this method, grep goes through all the files and sed only scans the files matched by grep. With the find method in the other answer, find first lists all files, and then sed will scan through all the files in that directory. So this method is not necessarily slower, it depends on how many matches there are and the differences in search speeds between sed, grep and find.
OTOH this way lets you PREVIEW what grep finds BEFORE actually replacing, reducing the risk of failure greatly, especially for regex n00bs like myself
This is also useful when your grep replacement is more clever than sed. For example ripgrep obeys .gitignore while sed doesn't.
M
Matthias Braun

You could even do it like this:

Example

grep -rl 'windows' ./ | xargs sed -i 's/windows/linux/g'

This will search for the string 'windows' in all files relative to the current directory and replace 'windows' with 'linux' for each occurrence of the string in each file.


The grep is only useful if there are files which should not be modified. Running sed on all files will update the file's modification date but leave the contents unchanged if there are no matches.
@tripleee: Be careful with ... but [sed] leave the contents unchanged if there are no matches". When using -i, I believe sed changes the file time of every file it touches, even though the contents are unchanged. sed also converts line endings. I don't use sed on Windows in a Git repo because all CRLF are changed to LF.
This command needs a "" after the -i to denote that no backup files will be made after the in-place substitution takes place, at least in macosx. Check the man page for details. If you want a backup, this is where you put the extension of the file to be created.
For spaces is file names you need to do NULL termination on grep and xargs. stackoverflow.com/questions/17296525/…
I love the funny subtle subliminal command to replace Windows with Linux
M
Marc Juchli

This works best for me on OS X:

grep -r -l 'searchtext' . | sort | uniq | xargs perl -e "s/matchtext/replacetext/" -pi

Source: http://www.praj.com.au/post/23691181208/grep-replace-text-string-in-files


this is perfect! also works with ag: ag "search" -l -r . | sort | uniq | xargs perl -e 's/search/replace' -pi
@sebastiankeller Your Perl command lacks the final slash, which is a syntax error.
Why is the sort -u even part of this? In what circumstances would you expect grep -rl to produce the same file name twice?
m
minopret

Usually not with grep, but rather with sed -i 's/string_to_find/another_string/g' or perl -i.bak -pe 's/string_to_find/another_string/g'.


I think this is probably the easiest method on here to get the job done. Forcing grep when it is not needed is unnecessary.
W
Walf

Other solutions mix regex syntaxes. To use perl/PCRE patterns for both search and replace, and process only matching files, this works quite well:

grep -rlIZPi 'match1' | xargs -0r perl -pi -e 's/match2/replace/gi;'

match1 and match2 are usually identical but match2 can contain more advanced features that are only relevant to the substitution, e.g. capturing groups.

Translation: grep recursively and list matching filenames, each separated by null to protect any special characters; pipe any filenames to xargs which is expecting a null-separated list; if any filenames are received, pass them to perl to perform the actual substitutions.

For case-sensitive matching, drop the i flag from grep and the i pattern modifier from the s/// expression, but not the i flag from perl itself. To include binary files, remove the I flag from grep.


Perl itself is quite capable of recursing a file structure. In fact, there is a tool find2perl which ships with Perl which does this sort of thing without any xargs trickery.
@tripleee find doesn't search file contents, and the point is to only process matching files without writing a Perl program.
This is a nice solution for Windows, as it avoids the sed-based solutions' issue of converting the line endings. Thanks!
t
tsveti_iko

Be very careful when using find and sed in a git repo! If you don't exclude the binary files you can end up with this error:

error: bad index file sha1 signature 
fatal: index file corrupt

To solve this error you need to revert the sed by replacing your new_string with your old_string. This will revert your replaced strings, so you will be back to the beginning of the problem.

The correct way to search for a string and replace it is to skip find and use grep instead in order to ignore the binary files:

sed -ri -e "s/old_string/new_string/g" $(grep -Elr --binary-files=without-match "old_string" "/files_dir")

Credits for @hobs


t
tinnick

Here is what I would do:

find /path/to/dir -type f -iname "*filename*" -print0 | xargs -0 sed -i '/searchstring/s/old/new/g'

this will look for all files containing filename in the file's name under the /path/to/dir, than for every file found, search for the line with searchstring and replace old with new.

Though if you want to omit looking for a specific file with a filename string in the file's name, than simply do:

find /path/to/dir -type f -print0 | xargs -0 sed -i '/searchstring/s/old/new/g'

This will do the same thing above, but to all files found under /path/to/dir.


G
GuiltyDolphin

Another option would be to just use perl with globstar.

Enabling shopt -s globstar in your .bashrc (or wherever) allows the ** glob pattern to match all sub-directories and files recursively.

Thus using perl -pXe 's/SEARCH/REPLACE/g' -i ** will recursively replace SEARCH with REPLACE.

The -X flag tells perl to "disable all warnings" - which means that it won't complain about directories.

The globstar also allows you to do things like sed -i 's/SEARCH/REPLACE/g' **/*.ext if you wanted to replace SEARCH with REPLACE in all child files with the extension .ext.


"Another option would be to just use perl with globstar..." - Not on Posixy machines, like Solaris. That's why I am specifically looking for grep and sed.
K
Kpym

Modern rust tools can be used to do this job. For example to replace in all (non ignored) files "oldstring" and "oldString" with "newstring" and "newString" respectively you can :

Use fd and sd

fd -tf -x sd 'old([Ss]tring)' 'new$1' {}

Use ned

ned -R -p 'old([Ss]tring)' -r 'new$1' .

Use ruplacer

ruplacer --go 'old([Ss]tring)' 'new$1' .

Ignored files

To include ignored (by .gitignore) and hidden files you have to specify it :

use -IH for fd,

use --ignored --hiddenfor ruplacer.