ChatGPT解决这个技术问题 Extra ChatGPT

How can I open multiple files using "with open" in Python?

I want to change a couple of files at one time, iff I can write to all of them. I'm wondering if I somehow can combine the multiple open calls with the with statement:

try:
  with open('a', 'w') as a and open('b', 'w') as b:
    do_something()
except IOError as e:
  print 'Operation failed: %s' % e.strerror

If that's not possible, what would an elegant solution to this problem look like?


S
Sven Marnach

As of Python 2.7 (or 3.1 respectively) you can write

with open('a', 'w') as a, open('b', 'w') as b:
    do_something()

In earlier versions of Python, you can sometimes use contextlib.nested() to nest context managers. This won't work as expected for opening multiples files, though -- see the linked documentation for details.

In the rare case that you want to open a variable number of files all at the same time, you can use contextlib.ExitStack, starting from Python version 3.3:

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # Do something with "files"

Most of the time you have a variable set of files, you likely want to open them one after the other, though.


Unfortunately, according to the contextlib.nested docs, you shouldn't use it for file opening: "using nested() to open two files is a programming error as the first file will not be closed promptly if an exception is thrown when opening the second file."
is there a way to use with to open a variable list of files?
@monkut: Very good question (you could actually ask this as a separate question). Short answer: Yes, there is ExitStack as of Python 3.3. There is no easy way of doing this in any earlier version of Python.
Is it possible to have this syntax span multiple lines?
@tommy.carstensen: You can use the usual line continuation mechanisms. You should probably use backslash line continuation to break at the comma, as recommended by PEP 9.
M
Michael

Just replace and with , and you're done:

try:
    with open('a', 'w') as a, open('b', 'w') as b:
        do_something()
except IOError as e:
    print 'Operation failed: %s' % e.strerror

You should specify which versions of Python support this syntax.
M
Michael Ohlrogge

For opening many files at once or for long file paths, it may be useful to break things up over multiple lines. From the Python Style Guide as suggested by @Sven Marnach in comments to another answer:

with open('/path/to/InFile.ext', 'r') as file_1, \
     open('/path/to/OutFile.ext', 'w') as file_2:
    file_2.write(file_1.read())

With this indentation I get: "flake8: continuation line over-indented for visual indent"
@LouisM That sounds like something coming from your editor or environment, rather than base python. If it continues to be a problem for you, I'd recommend creating a new question relating to it and giving more detail on your editor and environment.
Yes it is definitely my editor, and it is only a warning. What I wanted to emphasize is that your indentation does not comply with PEP8. You should indent the second open() with 8 spaces instead of aligning it with the first one.
@LouisM PEP8 is a guideline, not rules, and in this case I would most certainly ignore it
Yes no problem with that, it might be useful for other people with automatic linters though :)
C
Chris_Rands

From Python 3.10 there is a new feature of Parenthesized context managers, which permits syntax like:

with (
    open("a", "w") as a,
    open("b", "w") as b
):
    do_something()

Interesting. Adds an extra pair of parentheses. Can it do anything that with open("a", "w") as a, open ("b", "w") as b: couldn't do?
@PatrickT It permits splitting the syntax over multiple lines without line breaks, which might be more readable for long examples
Got it! Thanks Chris!
C
Community

Since Python 3.3, you can use the class ExitStack from the contextlib module to safely
open an arbitrary number of files.

It can manage a dynamic number of context-aware objects, which means that it will prove especially useful if you don't know how many files you are going to handle.

In fact, the canonical use-case that is mentioned in the documentation is managing a dynamic number of files.

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # All opened files will automatically be closed at the end of
    # the with statement, even if attempts to open files later
    # in the list raise an exception

If you are interested in the details, here is a generic example in order to explain how ExitStack operates:

from contextlib import ExitStack

class X:
    num = 1

    def __init__(self):
        self.num = X.num
        X.num += 1

    def __repr__(self):
        cls = type(self)
        return '{cls.__name__}{self.num}'.format(cls=cls, self=self)

    def __enter__(self):
        print('enter {!r}'.format(self))
        return self.num

    def __exit__(self, exc_type, exc_value, traceback):
        print('exit {!r}'.format(self))
        return True

xs = [X() for _ in range(3)]

with ExitStack() as stack:
    print(len(stack._exit_callbacks)) # number of callbacks called on exit
    nums = [stack.enter_context(x) for x in xs]
    print(len(stack._exit_callbacks))

print(len(stack._exit_callbacks))
print(nums)

Output:

0
enter X1
enter X2
enter X3
3
exit X3
exit X2
exit X1
0
[1, 2, 3]

F
FatihAkici

Nested with statements will do the same job, and in my opinion, are more straightforward to deal with.

Let's say you have inFile.txt, and want to write it into two outFile's simultaneously.

with open("inFile.txt", 'r') as fr:
    with open("outFile1.txt", 'w') as fw1:
        with open("outFile2.txt", 'w') as fw2:
            for line in fr.readlines():
                fw1.writelines(line)
                fw2.writelines(line)

EDIT:

I don't understand the reason of the downvote. I tested my code before publishing my answer, and it works as desired: It writes to all of outFile's, just as the question asks. No duplicate writing or failing to write. So I am really curious to know why my answer is considered to be wrong, suboptimal or anything like that.


i don't know what someone else downvoted you, but I UPVOTED you because this is the only example that had three files (one input, two output) which happened to be just what I needed.
A friendly reminder to those poo-poohing python 2.6: CentOS 6 (which doesn't EOL until Nov 2020), still uses py2.6 by default. So this answer is (as of right now) still the best overall one IMO.
@FatihAkici The Zen of Python says: "Flat is better than nested". Unnecessarily nested code lowers readability and is considered a bad practice.
@ElRuso why is it more pythonic? Less indentation?
@stackexchange_account1111 yep, more detailed answer right above your question
A
Aashutosh jha

With python 2.6 It will not work, we have to use below way to open multiple files:

with open('a', 'w') as a:
    with open('b', 'w') as b:

P
Pedro Lobito

Late answer (8 yrs), but for someone looking to join multiple files into one, the following function may be of help:

def multi_open(_list):
    out=""
    for x in _list:
        try:
            with open(x) as f:
                out+=f.read()
        except:
            pass
            # print(f"Cannot open file {x}")
    return(out)

fl = ["C:/bdlog.txt", "C:/Jts/tws.vmoptions", "C:/not.exist"]
print(multi_open(fl))

2018-10-23 19:18:11.361 PROFILE  [Stop Drivers] [1ms]
2018-10-23 19:18:11.361 PROFILE  [Parental uninit] [0ms]
...
# This file contains VM parameters for Trader Workstation.
# Each parameter should be defined in a separate line and the
...