ChatGPT解决这个技术问题 Extra ChatGPT

Executing multi-line statements in the one-line command-line

I'm using Python with -c to execute a one-liner loop, i.e.:

python -c "for r in range(10): print 'rob'"

This works fine. However, if I import a module before the for loop, I get a syntax error:

python -c "import sys; for r in range(10): print 'rob'"

  File "<string>", line 1
    import sys; for r in range(10): print 'rob'
              ^
SyntaxError: invalid syntax

How can this be fixed?

It's important to me to have this as a one-liner so that I can include it in a Makefile.

Does this answer your question? Using a dictionary to select function to execute

P
Peter Mortensen

You could do

echo -e "import sys\nfor r in range(10): print 'rob'" | python

Or without pipes:

python -c "exec(\"import sys\nfor r in range(10): print 'rob'\")"

Or

(echo "import sys" ; echo "for r in range(10): print 'rob'") | python

Or SilentGhost's answer or Crast's answer.


The last option works great with python3 in Windows
@RudolfOlah hope you know it by now but just for reference, you need to wrap the print statement for python3+ versions like: python -c "exec(\"import sys\nfor r in range(10): print('rob')\")"
Second option also works great on Windows using Python 3.
P
Peter Mortensen

This style can be used in makefiles too (and in fact it is used quite often).

python - <<EOF
import random
for r in range(3): print(random.randint(1, 42))
EOF

Or with hard tabs:

python - <<-EOF
    import random
    for r in range(3): print(random.randint(1, 42))
EOF
# Important: Replace the indentation above w/ hard tabs.

In above case, leading TAB characters are removed too (and some structured outlook can be achieved).

Instead of EOF can stand any marker word not appearing in the here document at a beginning of a line (see also here documents in the bash man page or here).


Nice. To also pass arguments, simply place them after <<EOF. Note, however, that it's better to quote the delimiter - e.g., <<'EOF' - so as to protect the contents of the here-document from up-front shell expansions.
Great! This is a more streamlined solution to what I wanted to ask.
P
Peter Mortensen

The issue is not actually with the import statement. It's with anything being before the for loop. Or more specifically, anything appearing before an inlined block.

For example, these all work:

python -c "import sys; print 'rob'"
python -c "import sys; sys.stdout.write('rob\n')"

If import being a statement were an issue, this would work, but it doesn't:

python -c "__import__('sys'); for r in range(10): print 'rob'"

For your very basic example, you could rewrite it as this:

python -c "import sys; map(lambda x: sys.stdout.write('rob%d\n' % x), range(10))"

However, lambdas can only execute expressions, not statements or multiple statements, so you may still be unable to do the thing you want to do. However, between generator expressions, list comprehension, lambdas, sys.stdout.write, the "map" builtin, and some creative string interpolation, you can do some powerful one-liners.

The question is, how far do you want to go, and at what point is it not better to write a small .py file which your makefile executes instead?


C
Community


- To make this answer work with Python 3.x as well, print is called as a function: in 3.x, only print('foo') works, whereas 2.x also accepts print 'foo'.
- For a cross-platform perspective that includes Windows, see kxr's helpful answer.

In bash, ksh, or zsh:

Use an ANSI C-quoted string ($'...'), which allows using \n to represent newlines that are expanded to actual newlines before the string is passed to python:

python -c $'import sys\nfor r in range(10): print("rob")'

Note the \n between the import and for statements to effect a line break.

To pass shell-variable values to such a command, it is safest to use arguments and access them via sys.argv inside the Python script:

name='rob' # value to pass to the Python script
python -c $'import sys\nfor r in range(10): print(sys.argv[1])' "$name"

See below for a discussion of the pros and cons of using an (escape sequence-preprocessed) double-quoted command string with embedded shell-variable references.

To work safely with $'...' strings:

Double \ instances in your original source code. \ sequences - such as \n in this case, but also the usual suspects such as \t, \r, \b - are expanded by $'...' (see man printf for the supported escapes)

\ sequences - such as \n in this case, but also the usual suspects such as \t, \r, \b - are expanded by $'...' (see man printf for the supported escapes)

Escape ' instances as \'.

If you must remain POSIX-compliant:

Use printf with a command substitution:

python -c "$(printf %b 'import sys\nfor r in range(10): print("rob")')"

To work safely with this type of string:

Double \ instances in your original source code. \ sequences - such as \n in this case, but also the usual suspects such as \t, \r, \b - are expanded by printf (see man printf for the supported escape sequences).

\ sequences - such as \n in this case, but also the usual suspects such as \t, \r, \b - are expanded by printf (see man printf for the supported escape sequences).

Pass a single-quoted string to printf %b and escape embedded single quotes as '\'' (sic). Using single quotes protects the string's contents from interpretation by the shell. That said, for short Python scripts (as in this case) you can use a double-quoted string to incorporate shell variable values into your scripts - as long as you're aware of the associated pitfalls (see next point); e.g., the shell expands $HOME to the current user's home dir. in the following command: python -c "$(printf %b "import sys\nfor r in range(10): print('rob is $HOME')")" However, the generally preferred approach is to pass values from the shell via arguments, and access them via sys.argv in Python; the equivalent of the above command is: python -c "$(printf %b 'import sys\nfor r in range(10): print("rob is " + sys.argv[1])')" "$HOME" While using a double-quoted string is more convenient - it allows you to use embedded single quotes unescaped and embedded double quotes as \" - it also makes the string subject to interpretation by the shell, which may or may not be the intent; $ and ` characters in your source code that are not meant for the shell may cause a syntax error or alter the string unexpectedly. Additionally, the shell's own \ processing in double-quoted strings can get in the way; for instance, to get Python to produce literal output ro\b, you must pass ro\\b to it; with a '...' shell string and doubled \ instances, we get: python -c "$(printf %b 'import sys\nprint("ro\\\\bs")')" # ok: 'ro\bs' By contrast, this does not work as intended with a "..." shell string: python -c "$(printf %b "import sys\nprint('ro\\\\bs')")" # !! INCORRECT: 'rs' The shell interprets both "\b" and "\\b" as literal \b, requiring a dizzying number of additional \ instances to achieve the desired effect: python -c "$(printf %b "import sys\nprint('ro\\\\\\\\bs')")"

Using single quotes protects the string's contents from interpretation by the shell. That said, for short Python scripts (as in this case) you can use a double-quoted string to incorporate shell variable values into your scripts - as long as you're aware of the associated pitfalls (see next point); e.g., the shell expands $HOME to the current user's home dir. in the following command: python -c "$(printf %b "import sys\nfor r in range(10): print('rob is $HOME')")" However, the generally preferred approach is to pass values from the shell via arguments, and access them via sys.argv in Python; the equivalent of the above command is: python -c "$(printf %b 'import sys\nfor r in range(10): print("rob is " + sys.argv[1])')" "$HOME"

That said, for short Python scripts (as in this case) you can use a double-quoted string to incorporate shell variable values into your scripts - as long as you're aware of the associated pitfalls (see next point); e.g., the shell expands $HOME to the current user's home dir. in the following command: python -c "$(printf %b "import sys\nfor r in range(10): print('rob is $HOME')")"

python -c "$(printf %b "import sys\nfor r in range(10): print('rob is $HOME')")"

However, the generally preferred approach is to pass values from the shell via arguments, and access them via sys.argv in Python; the equivalent of the above command is: python -c "$(printf %b 'import sys\nfor r in range(10): print("rob is " + sys.argv[1])')" "$HOME"

python -c "$(printf %b 'import sys\nfor r in range(10): print("rob is " + sys.argv[1])')" "$HOME"

While using a double-quoted string is more convenient - it allows you to use embedded single quotes unescaped and embedded double quotes as \" - it also makes the string subject to interpretation by the shell, which may or may not be the intent; $ and ` characters in your source code that are not meant for the shell may cause a syntax error or alter the string unexpectedly. Additionally, the shell's own \ processing in double-quoted strings can get in the way; for instance, to get Python to produce literal output ro\b, you must pass ro\\b to it; with a '...' shell string and doubled \ instances, we get: python -c "$(printf %b 'import sys\nprint("ro\\\\bs")')" # ok: 'ro\bs' By contrast, this does not work as intended with a "..." shell string: python -c "$(printf %b "import sys\nprint('ro\\\\bs')")" # !! INCORRECT: 'rs' The shell interprets both "\b" and "\\b" as literal \b, requiring a dizzying number of additional \ instances to achieve the desired effect: python -c "$(printf %b "import sys\nprint('ro\\\\\\\\bs')")"

Additionally, the shell's own \ processing in double-quoted strings can get in the way; for instance, to get Python to produce literal output ro\b, you must pass ro\\b to it; with a '...' shell string and doubled \ instances, we get: python -c "$(printf %b 'import sys\nprint("ro\\\\bs")')" # ok: 'ro\bs' By contrast, this does not work as intended with a "..." shell string: python -c "$(printf %b "import sys\nprint('ro\\\\bs')")" # !! INCORRECT: 'rs' The shell interprets both "\b" and "\\b" as literal \b, requiring a dizzying number of additional \ instances to achieve the desired effect: python -c "$(printf %b "import sys\nprint('ro\\\\\\\\bs')")"

To pass the code via stdin rather than -c:

Note: I'm focusing on single-line solutions here; xorho's answer shows how to use a multi-line here-document - be sure to quote the delimiter, however; e.g., <<'EOF', unless you explicitly want the shell to expand the string up front (which comes with the caveats noted above).

In bash, ksh, or zsh:

Combine an ANSI C-quoted string ($'...') with a here-string (<<<...):

python - <<<$'import sys\nfor r in range(10): print("rob")'

- tells python explicitly to read from stdin (which it does by default). - is optional in this case, but if you also want to pass arguments to the scripts, you do need it to disambiguate the argument from a script filename:

python - 'rob' <<<$'import sys\nfor r in range(10): print(sys.argv[1])'

If you must remain POSIX-compliant:

Use printf as above, but with a pipeline so as to pass its output via stdin:

printf %b 'import sys\nfor r in range(10): print("rob")' | python

With an argument:

printf %b 'import sys\nfor r in range(10): print(sys.argv[1])' | python - 'rob'

R
Russia Must Remove Putin

Any idea how this can be fixed?

Your problem is created by the fact that Python statements, separated by ;, are only allowed to be "small statements", which are all one-liners. From the grammar file in the Python docs:

stmt: simple_stmt | compound_stmt simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE small_stmt: (expr_stmt | del_stmt | pass_stmt | flow_stmt | import_stmt | global_stmt | nonlocal_stmt | assert_stmt)

Compound statements can't be included on the same line with other statements via semicolons - so doing this with the -c flag becomes very inconvenient.

When demonstrating Python while in a bash shell environment, I find it very useful to include compound statements. The only simple way of doing this reliably is with heredocs (a posix shell thing).

Heredocs

Use a heredoc (created with <<) and Python's command line interface option, -:

$ python - <<-"EOF"
        import sys                    # 1 tab indent
        for r in range(10):           # 1 tab indent
            print('rob')              # 1 tab indent and 4 spaces
EOF

Adding the - after << (the <<-) allows you to use tabs to indent (Stackoverflow converts tabs to spaces, so I've indented 8 spaces to emphasize this). The leading tabs will be stripped.

You can do it without the tabs with just <<:

$ python - << "EOF"
import sys
for r in range(10):
    print('rob')
EOF

Putting quotes around EOF prevents parameter and arithmetic expansion. This makes the heredoc more robust.

Bash multiline strings

If you use double-quotes, you'll get shell-expansion:

$ python -c "
> import sys
> for p in '$PATH'.split(':'):
>     print(p)
> "
/usr/sbin
/usr/bin
/sbin
/bin
...

To avoid shell expansion use single-quotes:

$ python -c '
> import sys
> for p in "$PATH".split(":"):
>     print(p)
> '
$PATH

Note that we need to swap the quote characters on the literals in Python - we basically can't use quote character being interpreted by BASH. We can alternate them though, like we can in Python - but this already looks quite confusing, which is why I don't recommend this:

$ python -c '
import sys
for p in "'"$PATH"'".split(":"):
    print(p)
'
/usr/sbin
/usr/bin
/sbin
/bin
...

Critique of the accepted answer (and others)

This is not very readable:

echo -e "import sys\nfor r in range(10): print 'rob'" | python

Not very readable, and additionally difficult to debug in the case of an error:

python -c "exec(\"import sys\\nfor r in range(10): print 'rob'\")"

Perhaps a bit more readable, but still quite ugly:

(echo "import sys" ; echo "for r in range(10): print 'rob'") | python

You'll have a bad time if you have "'s in your python:

$ python -c "import sys > for r in range(10): print 'rob'"

Don't abuse map or list comprehensions to get for-loops:

python -c "import sys; map(lambda x: sys.stdout.write('rob%d\n' % x), range(10))"

These are all sad and bad. Don't do them.


++ for the grammar info; the (non-expanding) here-doc is handy and the most robust solution, but obviously not a one-liner. If a single-line solution is a must, using an ANSI C-quoted string (bash, ksh, or zsh) is a reasonable solution: python -c $'import sys\nfor r in range(10): print(\'rob\')' (you'll only have to worry about escaping single quotes (which you can avoid by using double quotes) and backslashes).
This is an amazing answer - both on the python and bash fronts
P
Peter Mortensen

Just use Return and type it on the next line:

python -c "import sys
for r in range(10): print 'rob'"

rob
rob
...

Seriously, you're going to sprain something if you keep doing this. python $(srcdir)/myscript.py for great justice.
P
Peter Mortensen
python2.6 -c "import sys; [sys.stdout.write('rob\n') for r in range(10)]"

works fine.

Use "[ ]" to inline your for loop.


This answer is probably way out of date. Can you update it?
P
Peter Mortensen

The problem is not with the import statement. The problem is that the control flow statements don't work inlined in a Python interpreter command. Replace that import statement with any other statement, and you'll see the same problem.

Think about it: Python can't possibly inline everything. It uses indentation to group control-flow.


P
Peter Mortensen

If your system is POSIX.2-compliant it should supply the printf utility:

printf "print 'zap'\nfor r in range(3): print 'rob'" | python

zap
rob
rob
rob

Good solution, but I suggest using printf %b '...' | python for added robustness, because it prevents printf from interpreting sequences such as %d (format specifiers) in the input string. Also, unless you explicitly want to apply shell expansions to your Python command string up front (which can get confusing), it's better to use ' (single quotes) for the outer quoting, to avoid both the expansions and the backlash processing that the shell applies to double-quoted strings.
P
Peter Mortensen

This variant is most portable for putting multi-line scripts on the command-line on Windows and Unix-like systems, Python 2 and Python 3, without pipes:

python -c "exec(\"import sys \nfor r in range(10): print('rob') \")"

(None of the other examples seen here so far did so.)

Neat on Windows is:

python -c exec"""import sys \nfor r in range(10): print 'rob' """
python -c exec("""import sys \nfor r in range(10): print('rob') """)

Neat on Bash on Unix-like systems is:

python -c $'import sys \nfor r in range(10): print("rob")'

This function turns any multiline-script into a portable command-one-liner:

def py2cmdline(script):
    exs = 'exec(%r)' % re.sub('\r\n|\r', '\n', script.rstrip())
    print('python -c "%s"' % exs.replace('"', r'\"'))

Usage:

>>> py2cmdline(getcliptext())
python -c "exec('print \'AA\tA\'\ntry:\n for i in 1, 2, 3:\n  print i / 0\nexcept:\n print \"\"\"longer\nmessage\"\"\"')"

The input was:

print 'AA    A'
try:
 for i in 1, 2, 3:
  print i / 0
except:
 print """longer
message"""

++ for the cross-platform angle and the converter. Your first command is as good as it gets in terms of portability (leaving PowerShell aside), but ultimately there is no single, fully robust cross-platform syntax, because the need to use double quotes then bears the risk of unwanted shell expansions, with Windows requiring escaping of different characters than POSIX-like shells. In PowerShell v3 or higher you can make your command lines work by inserting the "stop-parsing" option --% before the command-string argument.
@mklement0 "unwanted shell expansions": well, the insertion of shell expansions into something like print "path is %%PATH%%" resp. print "path is $PATH" is usually the option which one wants in a script or command-line - unless one escapes things as usual for the platform. Same with other languages. (The Python syntax itself does not regularly suggest the use of % and $'s in a competing way.)
If you insert shell-variable references directly into the Python source code, it will by definition not be portable. My point is that even if you instead constructed platform-specific references in separate variables that you pass as arguments to a single, "shell-free" Python command, that may not always work, because you cannot protect the double-quoted string portably: e.g., what if you need literal $foo in your Python code? If you escape it as \$foo for the benefit of POSIX-like shells, cmd.exe will still see the extra \ . It may be rare but it's worth knowing about.
Trying to do this in Windows PowerShell, but the problem is that python -c exec("""...""") doesn't produce any output whatsoever, no matter if the code in ... is able to be executed or not; I could put any gibberish there, and the result would be the same. I feel like the shell is "eating" both the stdout and stderr streams - how can I make it spit them out?
@Yury: PowerShell may also be eating double quotes. A workaround is to use CMD and then pipe the output to PowerShell...
P
Peter Mortensen

I'm not really a big Pythoner - but I found this syntax once, forgot where from, so I thought I'd document it:

If you use sys.stdout.write instead of print (the difference being, sys.stdout.write takes arguments as a function, in parentheses - whereas print doesn't), then for a one-liner, you can get away with inverting the order of the command and the for, removing the semicolon, and enclosing the command in square brackets, i.e.:

python -c "import sys; [sys.stdout.write('rob\n') for r in range(10)]"

I have no idea how this syntax would be called in Python :)

These square brackets in one-liners are "list comprehensions"; note this in Python 2.7:

STR=abc
echo $STR | python -c "import sys,re; a=(sys.stdout.write(line) for line in sys.stdin); print a"

Output:

<generator object <genexpr> at 0xb771461c>

So the command in round brackets/parenthesis is seen as a "generator object"; if we "iterate" through it by calling next() - then the command inside the parenthesis will be executed (note the "abc" in the output):

echo $STR | python -c "import sys,re; a=(sys.stdout.write(line) for line in sys.stdin); a.next() ; print a"

Output:

abc
<generator object <genexpr> at 0xb777b734>

If we now use square brackets - note that we don't need to call next() to have the command execute, it executes immediately upon assignment; however, later inspection reveals that a is None:

echo $STR | python -c "import sys,re; a=[sys.stdout.write(line) for line in sys.stdin]; print a"

Output:

abc
[None]

This doesn't leave much information to look for - for the square brackets case - but I stumbled upon this page which I think explains:

Python Tips And Tricks – First Edition - Python Tutorials | Dream.In.Code:

If you recall, the standard format of a single line generator is a kind of one line 'for' loop inside brackets. This will produce a 'one-shot' iterable object which is an object you can iterate over in only one direction and which you can't re-use once you reach the end. A 'list comprehension' looks almost the same as a regular one-line generator, except that the regular brackets - ( ) - are replaced by square brackets - [ ]. The major advantage of alist comprehension is that produces a 'list', rather than a 'one-shot' iterable object, so that you can go back and forth through it, add elements, sort, etc.

And indeed it is a list - it's just its first element becomes none as soon as it is executed:

echo $STR | python -c "import sys,re; print [sys.stdout.write(line) for line in sys.stdin].__class__"

Output:

abc
<type 'list'>

echo $STR | python -c "import sys,re; print [sys.stdout.write(line) for line in sys.stdin][0]"

Output:

abc
None

List comprehensions are otherwise documented in 5. Data Structures: 5.1.4. List Comprehensions — Python v2.7.4 documentation as "List comprehensions provide a concise way to create lists"; presumably, that's where the limited "executability" of lists comes into play in one-liners.

Well, hope I'm not terribly too off the mark here ...

And here is a one-liner command line with two non-nested for-loops; both enclosed within "list comprehension" square brackets:

echo $STR | python -c "import sys,re; a=[sys.stdout.write(line) for line in sys.stdin]; b=[sys.stdout.write(str(x)) for x in range(2)] ; print a ; print b"

Output:

abc
01[None]
[None, None]

Notice that the second "list" b now has two elements, since its for loop explicitly ran twice; however, the result of sys.stdout.write() in both cases was (apparently) None.


P
Peter Mortensen

single/double quotes and backslash everywhere:

python -c 'exec("import sys\nfor i in range(10): print \"bob\"")'

Much better:

python -c '
import sys
for i in range(10):
  print("bob")
'

Note: On some systems executable python is for Python 2 and may not be available. And executable python3 is for Python 3.


T
The Unfun Cat

This script provides a Perl-like command line interface:

Pyliner - Script to run arbitrary Python code on the command line (Python recipe)


I think they wanted something to fix that, not to use another tool. Anyway nice hint
P
Peter Mortensen

I wanted a solution with the following properties:

Readable Read standard input for processing output of other tools

Both requirements were not provided in the other answers, so here's how to read standard input while doing everything on the command line:

grep special_string -r | sort | python3 <(cat <<EOF
import sys
for line in sys.stdin:
    tokens = line.split()
    if len(tokens) == 4:
        print("%-45s %7.3f    %s    %s" % (tokens[0], float(tokens[1]), tokens[2], tokens[3]))
EOF
)

P
Peter Mortensen

When I needed to do this, I used:

python -c "$(echo -e "import sys\nsys.stdout.write('Hello, World!\\\n')")"

Note the triple backslash for the newline in the sys.stdout.write statement.


This works, but since you're using echo -e, which is nonstandard and requires bash, ksh, or zsh, you may as well use a $'...' string directly, which also simplifies the escaping: python -c $'import sys\nsys.stdout.write("Hello World!\\n")'
This answer works, the other answers do not work for Python3
But this is not for a compound statement (a for loop in the question)—that is the gist of the question. You haven't demonstrated that it works for that case. This is elaborated on in Russia Must Remove Putin's answer.
P
Peter Mortensen

If you don't want to touch standard input and simulate as if you had passed "python cmdfile.py", you can do the following from a Bash shell:

python <(printf "word=raw_input('Enter word: ')\nimport sys\nfor i in range(5):\n    print(word)")

As you can see, it allows you to use standard input for reading input data. Internally the shell creates the temporary file for the input command contents.


++ for not "using up" stdin with the script itself (though -c "$(...)" does the same, and is POSIX-compliant); to give the <(...) construct a name: process substitution; it also works in ksh and zsh.
P
Peter Mortensen

Use python -c with triple quotes:

python -c """
import os
os.system('pwd')
os.system('ls -l')
print('Hello, World!')
for _ in range(5):
    print(_)
"""

What is this underscore thingy? Like in Perl?
P
Peter Mortensen

There is one more option. sys.stdout.write returns None, which keeps the list empty:

cat somefile.log|python -c "import sys;[line for line in sys.stdin if sys.stdout.write(line*2)]"

The attempt at manual syntax highlighting (use view "Side-by-side Markdown") does not work at all... It is: <pre style='color:#000000;background:#ffffff;'><html><body style='color:#000000; background:#ffffff; '><pre>
cont' - cat somefile<span style='color:#800000; font-weight:bold; '>.</span>log<span style='color:#e34adc; '>|<span style='color:#7f0055; font-weight:bold; '>python -c "import</span> sys;[line <span style='color:#7f0055; font-weight:bold; '>for</span> line <span style='color:#7f0055; font-weight:bold; '>in</span> sys.stdin <span style='color:#7f0055; font-weight:bold; '>if</span> sys.stdout.write(line*2)]" </pre> (!!!!)
The <pre> tags are not balanced (two start tags and one end tag). Doesn't pre require an end tag (in some form)? (Even then it may not work.)
It may or may not have been broken by a later change (2020-2022).