I am trying to delete empty lines using sed:
sed '/^$/d'
but I have no luck with it.
For example, I have these lines:
xxxxxx
yyyyyy
zzzzzz
and I want it to be like:
xxxxxx
yyyyyy
zzzzzz
What should be the code for this?
You may have spaces or tabs in your "empty" line. Use POSIX classes with sed
to remove all lines containing only whitespace:
sed '/^[[:space:]]*$/d'
A shorter version that uses ERE, for example with gnu sed:
sed -r '/^\s*$/d'
(Note that sed does NOT support PCRE.)
I am missing the awk
solution:
awk 'NF' file
Which would return:
xxxxxx
yyyyyy
zzzzzz
How does this work? Since NF
stands for "number of fields", those lines being empty have 0 fields, so that awk evaluates 0 to False and no line is printed; however, if there is at least one field, the evaluation is True and makes awk
perform its default action: print the current line.
$ time (topic companies <data.tpx | awk 'NF' - | awk -f dialog_menu.awk -)
real 0m0.006s
user 0m0.000s
sys 0m0.008s
$ time (topic companies <data.tpx | gsed '/^\s*$/d' | awk -f dialog_menu.awk -)
real 0m0.014s
user 0m0.002s
sys 0m0.006s
Would you know of a nifty way to include this into an awk-script like, e.g., a pattern? awk '/mypattern/ {do stuff...}'
sed '/^$/d'
should be fine, are you expecting to modify the file in place? If so you should use the -i
flag.
Maybe those lines are not empty, so if that's the case, look at this question Remove empty lines from txtfiles, remove spaces from start and end of line I believe that's what you're trying to achieve.
sed -i '/^$/d'
is one way of doing it.
sed
'/^[[:space:]]*$/d'
'/^\s*$/d'
'/^$/d'
-n '/^\s*$/!p'
grep
.
-v '^$'
-v '^\s*$'
-v '^[[:space:]]*$'
awk
/./
'NF'
'length'
'/^[ \t]*$/ {next;} {print}'
'!/^[ \t]*$/'
[]
should not be escaped in a bracket expression, so the code here isn't correct for \[\[:space:\]\]
or \[ \t\]
- should be [[:space:]]
and [ \t]
.
I believe this is the easiest and fastest one:
cat file.txt | grep .
If you need to ignore all white-space lines as well then try this:
cat file.txt | grep '\S'
Example:
s="\
\
a\
b\
\
Below is TAB:\
\
Below is space:\
\
c\
\
"; echo "$s" | grep . | wc -l; echo "$s" | grep '\S' | wc -l
outputs
7
5
cat
, grep
takes files as well: grep . file.txt
grep '\S'
is definitely not portable. If you have grep -P
then you can use grep -P '\S'
but it's not supported on all platforms, either.
grep .
compared to the other solutions is that it will highlight all the text in red. The other solutions can preserve the original colors. Compare unbuffer apt search foo | grep .
to unbuffer apt search foo | grep -v ^$
With help from the accepted answer here and the accepted answer above, I have used:
$ sed 's/^ *//; s/ *$//; /^$/d; /^\s*$/d' file.txt > output.txt
`s/^ *//` => left trim
`s/ *$//` => right trim
`/^$/d` => remove empty line
`/^\s*$/d` => delete lines which may contain white space
This covers all the bases and works perfectly for my needs. Kudos to the original posters @Kent and @kev
Another option without sed
, awk
, perl
, etc
strings $file > $output
strings - print the strings of printable characters in files.
strings
instead of string
?
--bytes=min-len
option to allow shorter lines.
You can say:
sed -n '/ / p' filename #there is a space between '//'
print all lines except the empty one(s)
and be quiet
The command you are trying is correct, just use -E flag with it.
sed -E '/^$/d'
-E flag makes sed catch extended regular expressions. More info here
-E
flag.
You are most likely seeing the unexpected behavior because your text file was created on Windows, so the end of line sequence is \r\n
. You can use dos2unix to convert it to a UNIX style text file before running sed or use
sed -r "/^\r?$/d"
to remove blank lines whether or not the carriage return is there.
-r
flag doing and is it possible to combine it with -i
to modify the file directly and avoid printing to screen. In addition, I think that this command would also work as sed -r "/^\r$/d"
This works in awk as well.
awk '!/^$/' file
xxxxxx
yyyyyy
zzzzzz
You can do something like that using "grep", too:
egrep -v "^$" file.txt
My bash
-specific answer is to recommend using perl
substitution operator with the global pattern g
flag for this, as follows:
$ perl -pe s'/^\n|^[\ ]*\n//g' $file
xxxxxx
yyyyyy
zzzzzz
This answer illustrates accounting for whether or not the empty lines have spaces in them ([\ ]*
), as well as using |
to separate multiple search terms/fields. Tested on macOS High Sierra and CentOS 6/7.
FYI, the OP's original code sed '/^$/d' $file
works just fine in bash
Terminal on macOS High Sierra and CentOS 6/7 Linux at a high-performance supercomputing cluster.
If you want to use modern Rust tools, you can consider:
ripgrep: cat datafile | rg '.' line with spaces is considered non empty cat datafile | rg '\S' line with spaces is considered empty rg '\S' datafile line with spaces is considered empty (-N can be added to remove line numbers for on screen display)
cat datafile | rg '.' line with spaces is considered non empty
cat datafile | rg '\S' line with spaces is considered empty
rg '\S' datafile line with spaces is considered empty (-N can be added to remove line numbers for on screen display)
sd cat datafile | sd '^\n' '' line with spaces is considered non empty cat datafile | sd '^\s*\n' '' line with spaces is considered empty sd '^\s*\n' '' datafile inplace edit
cat datafile | sd '^\n' '' line with spaces is considered non empty
cat datafile | sd '^\s*\n' '' line with spaces is considered empty
sd '^\s*\n' '' datafile inplace edit
NF is the command of awk you can use to delete empty lines in a file awk NF filename
and by using sed sed -r "/^\r?$/d"
For me with FreeBSD 10.1 with sed worked only this solution:
sed -e '/^[ ]*$/d' "testfile"
inside []
there are space and tab symbols.
test file contains:
fffffff next 1 tabline ffffffffffff
ffffffff next 1 Space line ffffffffffff
ffffffff empty 1 lines ffffffffffff
============ EOF =============
Using vim editor to remove empty lines
:%s/^$\n//g
Success story sharing
-r
sed -i "" '/^[[:space:]]*$/d'
<filename>
,^\s*$
will match all "empty" lines, empty here means, the line contains no chars, or the line contains only empty strings (E.g. spaces). All matched lines will be removed by sed, with thed
command.sed '/\S/!d' file