I am trying to find out if it is possible to edit a file in a single sed command without manually streaming the edited content into a new file and then renaming the new file to the original file name. I tried the -i
option but my Solaris system said that -i
is an illegal option. Is there a different way?
-i
is an option in gnu sed, but is not in standard sed. However, it streams the content to a new file and then renames the file so it is not what you want.
The -i
option streams the edited content into a new file and then renames it behind the scenes, anyway.
Example:
sed -i 's/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g' filename
while on macOS you need:
sed -i '' 's/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g' filename
On a system where sed
does not have the ability to edit files in place, I think the better solution would be to use perl
:
perl -pi -e 's/foo/bar/g' file.txt
Although this does create a temporary file, it replaces the original because an empty in place suffix/extension has been supplied.
sudo
. In my mind, the question is about how to avoid manually creating and renaming a temporary file without having to install the latest GNU or BSD sed
. Just use Perl.
sed 's/foo/bar/g' file.txt > file.tmp && mv file.tmp file.txt
. Just because in-place editing does the rename using a temporary file, doesn't mean that he/she must perform a manual rename when the option is not available. There are other tools out there that can do what he/she wants, and Perl is the obvious choice. How is this not an answer to the original question?
sed
. The question was actually a very good one, but it's value has been diminished by people who either can't read man pages or can't be bothered to actually read questions here on SO.
Note that on OS X you might get strange errors like "invalid command code" or other strange errors when running this command. To fix this issue try
sed -i '' -e "s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g" <file>
This is because on the OSX version of sed
, the -i
option expects an extension
argument so your command is actually parsed as the extension
argument and the file path is interpreted as the command code. Source: https://stackoverflow.com/a/19457213
brew install gnu-sed
along with a PATH
will allow you to use the GNU version of sed. Thanks for mentioning; a definite head scratcher.
The following works fine on my mac
sed -i.bak 's/foo/bar/g' sample
We are replacing foo with bar in sample file. Backup of original file will be saved in sample.bak
For editing inline without backup, use the following command
sed -i'' 's/foo/bar/g' sample
One thing to note, sed
cannot write files on its own as the sole purpose of sed is to act as an editor on the "stream" (ie pipelines of stdin, stdout, stderr, and other >&n
buffers, sockets and the like). With this in mind you can use another command tee
to write the output back to the file. Another option is to create a patch from piping the content into diff
.
Tee method
sed '/regex/' <file> | tee <file>
Patch method
sed '/regex/' <file> | diff -p <file> /dev/stdin | patch
UPDATE:
Also, note that patch will get the file to change from line 1 of the diff output:
Patch does not need to know which file to access as this is found in the first line of the output from diff:
$ echo foobar | tee fubar
$ sed 's/oo/u/' fubar | diff -p fubar /dev/stdin
*** fubar 2014-03-15 18:06:09.000000000 -0500
--- /dev/stdin 2014-03-15 18:06:41.000000000 -0500
***************
*** 1 ****
! foobar
--- 1 ----
! fubar
$ sed 's/oo/u/' fubar | diff -p fubar /dev/stdin | patch
patching file fubar
/dev/stdin 2014-03-15
, answered Jan 7 - are you a time traveller?
/dev/stdin
doesn't exist, so you have to replace /dev/stdin
with '-', a single hyphen without the quotes, so the following commands should work: $ sed 's/oo/u/' fubar | diff -p fubar -
and $ sed 's/oo/u/' fubar | diff -p fubar - | patch
patch
will not truncate as well. I think saving output to another file and then cat
-ing into original would be safer.
Versions of sed
that support the -i
option for editing a file in place write to a temporary file and then rename the file.
Alternatively, you can just use ed
. For example, to change all occurrences of foo
to bar
in the file file.txt
, you can do:
echo ',s/foo/bar/g; w' | tr \; '\012' | ed -s file.txt
Syntax is similar to sed
, but certainly not exactly the same.
Even if you don't have a -i
supporting sed
, you can easily write a script to do the work for you. Instead of sed -i 's/foo/bar/g' file
, you could do inline file sed 's/foo/bar/g'
. Such a script is trivial to write. For example:
#!/bin/sh
IN=$1
shift
trap 'rm -f "$tmp"' 0
tmp=$( mktemp )
<"$IN" "$@" >"$tmp" && cat "$tmp" > "$IN" # preserve hard links
should be adequate for most uses.
inline
and invoke it as describe above: inline inputfile sed 's/foo/bar/g'
ed
only answer that really does in-place. First time I ever need ed
. Thank you. A little optimization is to use echo -e ",s/foo/bar/g\012 w"
or echo $',s/foo/bar/g\012 w'
where shell allows it to avoid extra tr
call.
ed
doesn't really do the modifications in-place. From strace
output, GNU ed
creates a temp file, writes the changes to the temp file, then rewrites the entire original file, preserving hard links. From truss
output, Solaris 11 ed
also uses a temp file, but renames the temp file to the original file name upon saving with the w
command, destroying any hard links.
ed
that ships with current macos (Mojave, whatever that marketing jargon means) still preserves hardlinks. YMMV
You could use vi
vi -c '%s/foo/bar/g' my.txt -c 'wq'
sed supports in-place editing. From man sed
:
-i[SUFFIX], --in-place[=SUFFIX]
edit files in place (makes backup if extension supplied)
Example:
Let's say you have a file hello.txt
with the text:
hello world!
If you want to keep a backup of the old file, use:
sed -i.bak 's/hello/bonjour' hello.txt
You will end up with two files: hello.txt
with the content:
bonjour world!
and hello.txt.bak
with the old content.
If you don't want to keep a copy, just don't pass the extension parameter.
If you are replacing the same amount of characters and after carefully reading “In-place” editing of files...
You can also use the redirection operator <>
to open the file to read and write:
sed 's/foo/bar/g' file 1<> file
See it live:
$ cat file
hello
i am here # see "here"
$ sed 's/here/away/' file 1<> file # Run the `sed` command
$ cat file
hello
i am away # this line is changed now
From Bash Reference Manual → 3.6.10 Opening File Descriptors for Reading and Writing:
The redirection operator [n]<>word causes the file whose name is the expansion of word to be opened for both reading and writing on file descriptor n, or on file descriptor 0 if n is not specified. If the file does not exist, it is created.
Like Moneypenny said in Skyfall: "Sometimes the old ways are best." Kincade said something similar later on.
$ printf ',s/false/true/g\nw\n' | ed {YourFileHere}
Happy editing in place. Added '\nw\n' to write the file. Apologies for delay answering request.
|
. @MattMontag
\n
. [range]s/<old>/<new>/<flag> is the form of the substitute command - a paradigm still seen in many other text editors (okay, vim, a direct descendent of ed) Anyway, the g
flag stands for "global", and means "Every instance on each line that you look at". The range 3,5
looks at lines 3-5. ,5
looks at StartOfFile-5, 3,
looks at lines 3-EndOfFile and ,
looks at StartOfFile-EndOfFile. A global substitue command on every line that replaces "false" with "true". Then, the write command, w
, is entered.
You didn't specify what shell you are using, but with zsh you could use the =( )
construct to achieve this. Something along the lines of:
cp =(sed ... file; sync) file
=( )
is similar to >( )
but creates a temporary file which is automatically deleted when cp
terminates.
mv file.txt file.tmp && sed 's/foo/bar/g' < file.tmp > file.txt
Should preserve all hardlinks, since output is directed back to overwrite the contents of the original file, and avoids any need for a special version of sed.
Very good examples. I had the challenge to edit in place many files and the -i option seems to be the only reasonable solution using it within the find command. Here the script to add "version:" in front of the first line of each file:
find . -name pkg.json -print -exec sed -i '.bak' '1 s/^/version /' {} \;
To resolve this issue on Mac I had to add some unix functions to core-utils following this.
brew install grep
==> Caveats
All commands have been installed with the prefix "g".
If you need to use these commands with their normal names, you
can add a "gnubin" directory to your PATH from your bashrc like:
PATH="/usr/local/opt/grep/libexec/gnubin:$PATH"
Call with gsed
instead of sed
. The mac default doesn't like how grep -rl
displays file names with the ./
preprended.
~/my-dir/configs$ grep -rl Promise . | xargs sed -i 's/Promise/Bluebirg/g'
sed: 1: "./test_config.js": invalid command code .
I also had to use xargs -I{} sed -i 's/Promise/Bluebirg/g' {}
for files with a space in the name.
In case you want to replace stings contain '/',you can use '?'. i.e. replace '/usr/local/bin/python' with '/usr/bin/python3' for all *.py files.
find . -name \*.py -exec sed -i 's?/usr/local/bin/python?/usr/bin/python3?g' {} \;
Success story sharing
sed -i "s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g" <file>