ChatGPT解决这个技术问题 Extra ChatGPT

What is the difference between sed and awk? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers. Want to improve this question? Update the question so it focuses on one problem only by editing this post. Closed 8 years ago. Improve this question

What is the difference between awk and sed ?

What kind of application are best use cases for sed and awk tools ?

A related QA on unix.stackexchange: Is there a basic tutorial for grep, awk and sed?
Check out this question and its answers, stackoverflow.com/questions/46734615/…. I believe you would get a feeling from it.

D
Dennis Williamson

sed is a stream editor. It works with streams of characters on a per-line basis. It has a primitive programming language that includes goto-style loops and simple conditionals (in addition to pattern matching and address matching). There are essentially only two "variables": pattern space and hold space. Readability of scripts can be difficult. Mathematical operations are extraordinarily awkward at best.

There are various versions of sed with different levels of support for command line options and language features.

awk is oriented toward delimited fields on a per-line basis. It has much more robust programming constructs including if/else, while, do/while and for (C-style and array iteration). There is complete support for variables and single-dimension associative arrays plus (IMO) kludgey multi-dimension arrays. Mathematical operations resemble those in C. It has printf and functions. The "K" in "AWK" stands for "Kernighan" as in "Kernighan and Ritchie" of the book "C Programming Language" fame (not to forget Aho and Weinberger). One could conceivably write a detector of academic plagiarism using awk.

GNU awk (gawk) has numerous extensions, including true multidimensional arrays in the latest version. There are other variations of awk including mawk and nawk.

Both programs use regular expressions for selecting and processing text.

I would tend to use sed where there are patterns in the text. For example, you could replace all the negative numbers in some text that are in the form "minus-sign followed by a sequence of digits" (e.g. "-231.45") with the "accountant's brackets" form (e.g. "(231.45)") using this (which has room for improvement):

sed 's/-\([0-9.]\+\)/(\1)/g' inputfile

I would use awk when the text looks more like rows and columns or, as awk refers to them "records" and "fields". If I was going to do a similar operation as above, but only on the third field in a simple comma delimited file I might do something like:

awk -F, 'BEGIN {OFS = ","} {gsub("-([0-9.]+)", "(" substr($3, 2) ")", $3); print}' inputfile

Of course those are just very simple examples that don't illustrate the full range of capabilities that each has to offer.


To see some examples of pushing the boundaries of sed: sed.sourceforge.net/#scripts
@DennisWilliamson - Am I at a disadvantage if I only learn awk ? Is awk far more commonly used than sed ?
@blasto: My recommendation is to learn both but with more emphasis on awk. Lots of the regular expression stuff applies to both (and other tools and languages). Use sed for simpler stuff and try to avoid the complex things. It's really cool that you can do loops and branches in sed, but the resulting command lines are complex and hard to read. The answer to your question really depends on what you're doing.
@DennisWilliamson - What would be most useful to an ETL developer ? ETL or Extract Transform and Load is a data-warehousing term. Put crudely, the job involves EXTRACTION of data from different disparate sources (such as DB's, excel files, csv files etc), TRANSFORMATION of the same and then LOADING into a datawarehouse (DW) for analysis, finding patterns in data, or just historical records. eg. End use of a DW - Algorithms applied to a DW of a grocery store which has data from the past 10 years might reveal that people who tend to buy apples also buy oranges or something similar.
Just an FYI for anyone stumbling across this on a mac, try "sed -E 's/-([0-9]+.[0-9]*)/(\1)/g'" for the first sed example
g
ghostdog74

1) What is the difference between awk and sed ?

Both are tools that transform text. BUT awk can do more things besides just manipulating text. Its a programming language by itself with most of the things you learn in programming, like arrays, loops, if/else flow control etc You can "program" in sed as well, but you won't want to maintain the code written in it.

2) What kind of application are best use cases for sed and awk tools ?

Conclusion: Use sed for very simple text parsing. Anything beyond that, awk is better. In fact, you can ditch sed altogether and just use awk. Since their functions overlap and awk can do more, just use awk. You will reduce your learning curve as well.


I find sed is much easier to learn though, so you need to account for that. As you learn to master awk, it might be useful to quickly learn sed to be able to use it faster for things you might not know how to do in awk yet.
Don't ditch sed, 's/search/replace' is way easier to type than awk's syntax and is what you need most of the time.
Better to use a mixture. For instance, if you are looking for a pattern in a huge file, use grep to find the line number, and then sed to edit the line. Otherwise, sed is going to be much slower to process the entire file.
If you're a craftsman, you learn how to use all your tools and you'll know which to use when. If you're just fixing stuff, a hammer and a screwdriver might just be all you need.
h
hek2mgl

Both tools are meant to work with text and there are tasks both tools can be used for.

For me the rule to separate them is: Use sed to automate tasks you would do otherwise in a text editor manually. That's why it is called stream editor. (You can use the same commands to edit text in vim). Use awk if you want to analyze text, meaning counting fields, calculate totals, extract and reorganize structures etc.

Also you should not forget about grep. Use grep if you only want to search/extract something in a text (file)