ChatGPT解决这个技术问题 Extra ChatGPT

PATH issue with pytest 'ImportError: No module named YadaYadaYada'

I used easy_install to install pytest on a mac and started writing tests for a project with a file structure likes so:

repo/
   |--app.py
   |--settings.py
   |--models.py
   |--tests/
          |--test_app.py

run py.test while in the repo directory, everything behaves as you would expect

but when I try that same thing on either linux or windows (both have pytest 2.2.3 on them) it barks whenever it hits its first import of something from my application path. Say for instance from app import some_def_in_app

Do I need to be editing my PATH to run py.test on these systems? Has Anyone experienced this?

Here is the way to fix it with setuptools.
Please check @hoefling answer and consider changing your accepted one, if SO allows after this long: much better!

A
Apteryx

I'm not sure why py.test does not add the current directory in the PYTHONPATH itself, but here's a workaround (to be executed from the root of your repository):

python -m pytest tests/

It works because Python adds the current directory in the PYTHONPATH for you.


It requires to rewrite relative imports to absolute ones, if you have the code for running the application not on the level, where you execute the command from. For example: project/test/all-my-tests and project/src/app.py and because of that change, one needs to call the app.py indirectly using a __main__.py file in project/src, so that one can use the call python -m src. Pretty messy stuff as far as I can tell.
@Zelphir: Using absolute imports is a recommended practice. Habnabit's has a good article about packaging best practices: blog.habnab.it/blog/2013/07/21/python-packages-and-you, and PEP8 says that "implicit relative imports should never be used and have been removed in Python 3." See: python.org/dev/peps/pep-0008.
@Apteryx You mean "project-absolute" right? Because things like /home/user/dev/projectxyz/src ... would be really bad and not run on other machines in most cases. I think what I meant is, that I have to always write the whole project root to module path even if a module is in the same folder as the file run. I didn't know that this is considered best practice, so that's a useful bit of information, thanks. I agree with most of pep8, although it is still not perfect.
@Zelphir, yes, that's what I meant. I believe the term absolute imports in Python always refer to "project-absolute". See: python.org/dev/peps/pep-0328/#rationale-for-absolute-imports. In fact, I'm pretty sure you can't import from random, absolute paths locations, at least using the default "import" mechanism.
I have added __init__.py in tests, that solved the problem. Now I can use pytest
h
hoefling

Recommended approach for pytest>=7: use the pythonpath setting

Recently, pytest has added a new core plugin that supports sys.path modifications via the pythonpath configuration value. The solution is thus much simpler now and doesn't require any workarounds anymore:

pyproject.toml example:

[tool.pytest.ini_options]
pythonpath = [
  "."
]

pytest.ini example:

[pytest]
pythonpath = .

The path entries are calculated relative to the rootdir, thus . adds repo directory to sys.path in this case.

Multiple path entries are also allowed: for a layout

repo/
├── src/
|   └── lib.py
├── app.py
└── tests
     ├── test_app.py
     └── test_lib.py

the configuration

[tool.pytest.ini_options]
pythonpath = [
  ".", "src",
]

or

[pytest]
pythonpath = . src

will add both app and lib modules to sys.path, so

import app
import lib

will both work.

Original answer (not recommended for recent pytest versions; use for pytest<7 only): conftest solution

The least invasive solution is adding an empty file named conftest.py in the repo/ directory:

$ touch repo/conftest.py

That's it. No need to write custom code for mangling the sys.path or remember to drag PYTHONPATH along, or placing __init__.py into dirs where it doesn't belong (using python -m pytest as suggested in Apteryx's answer is a good solution though!).

The project directory afterwards:

repo
├── conftest.py
├── app.py
├── settings.py
├── models.py
└── tests
     └── test_app.py

Explanation

pytest looks for the conftest modules on test collection to gather custom hooks and fixtures, and in order to import the custom objects from them, pytest adds the parent directory of the conftest.py to the sys.path (in this case the repo directory).

Other project structures

If you have other project structure, place the conftest.py in the package root dir (the one that contains packages but is not a package itself, so does not contain an __init__.py), for example:

repo
├── conftest.py
├── spam
│   ├── __init__.py
│   ├── bacon.py
│   └── egg.py
├── eggs
│   ├── __init__.py
│   └── sausage.py
└── tests
     ├── test_bacon.py
     └── test_egg.py

src layout

Although this approach can be used with the src layout (place conftest.py in the src dir):

repo
├── src
│   ├── conftest.py
│   ├── spam
│   │   ├── __init__.py
│   │   ├── bacon.py
│   │   └── egg.py
│   └── eggs 
│       ├── __init__.py
│       └── sausage.py
└── tests
     ├── test_bacon.py
     └── test_egg.py

beware that adding src to PYTHONPATH mitigates the meaning and benefits of the src layout! You will end up with testing the code from repository and not the installed package. If you need to do it, maybe you don't need the src dir at all.

Where to go from here

Of course, conftest modules are not just some files to help the source code discovery; it's where all the project-specific enhancements of the pytest framework and the customization of your test suite happen. pytest has a lot of information on conftest modules scattered throughout their docs; start with conftest.py: local per-directory plugins

Also, SO has an excellent question on conftest modules: In py.test, what is the use of conftest.py files?


This does not work, simple as that. Try putting a file at the root directory, and then importing that from a test. Setting PYTHONPATH to root works fine, this hack does not help at all.
@aaa90210 although I can't reproduce your issue (importing from a conftest in a root dir works on any level), you should never import from conftest files as it's a reserved name for pytest and it's strongly advised not to do so. By doing that, you plant seeds for future errors. Create another module named utils.py and place the code for reusing in tests there.
For some reason that lead to ModuleNotFoundError on travisCI when also installing the package. So watch out!
@Jarno can you give a link to the affected repository? If you get a ModuleNotFound by changing the sys.path, it is an indicator of errors in project/imports layout (most probably name shadowing).
Logically, conftest.py does not belong to the application code and, imo, placing it under src/ is not correct.
A
Aron Curzon

I had the same problem. I fixed it by adding an empty __init__.py file to my tests directory.


Note that this is NOT recommended by py.test: avoid “__init__.py” files in your test directories. This way your tests can run easily against an installed version of mypkg, independently from the installed package if it contains the tests or not. SRC: pytest.org/latest/goodpractises.html
I came here with the same question and found removing __init__.py from my tests directory solved it for me.
@mafro i don't see the problem?The tests don't have to be importable code, they are found by your test runner. Only the code TO BE TESTED should be an installed package/module, not the tests.
Adding an __init__.py in sub-directories of test/ makes absolute import work for running specific tests in that sub-directory against to-be installed modules. Thanks.
there you go: doc.pytest.org/en/latest/goodpractices.html really easy to find with google.
L
Lii

Yes, the source folder is not in Python's path if you cd to the tests directory.

You have 2 choices:

Add the path manually to the test files, something like this: import sys, os myPath = os.path.dirname(os.path.abspath(__file__)) sys.path.insert(0, myPath + '/../') Run the tests with the env var PYTHONPATH=../.


when am i cding to a directory? i am running py.test from my root. unless I am mistaken and you mean as pytest walks through my folders
if it was a cd issue, wouldn't i hit it on mac as well?
Oh, I misread and thought it doesn't work from the tests directory. still the trick in suggestion 1 would work. I only use Linux so I can't explain the behavior on other OSes.
do you have an import like that on all your test.py files?
yes, but my directory structure is usually slightly different - I usually keep /src and /test under the root directory.
S
Stefano Messina

Run pytest itself as a module with: python -m pytest tests


This seems to be a working solution, but can anyone explain WHY? I'd rather fix the underlying cause than just use python -m pytest without any explanation other than "because it works"
This happens when project hierarchy is for example: package/src package/tests and in tests you import from src. Executing as a module will consider imports as absolute than relative to the execution location.
This solution helped me, thanks! The cause of this was because of the conflict in Python version. pytest test works for earlier python version. In my situation, my python version is 3.7.1, python -m pytest tests works but not pytest tests.
From Pytest "Running pytest with python -m pytest [...] instead of pytest [...] yields nearly equivalent behaviour, except that the former call will add the current directory to sys.path."
This is the cleanest solution so far
F
Ford Guo

You can run with PYTHONPATH in project root

PYTHONPATH=. py.test

Or use pip install as editable import

pip install -e .   # install package using setup.py in editable mode

That didn't work for me with a test directory not in src directory structure and calling from the directory containing both test and src directory.
u
user13037517

I had the same problem in Flask.

When I added:

__init__.py

to tests folder, problem disappeared :)

Probably application couldn't recognize folder tests as module


Thanks for your answer. It is the most simple way to solve this question
J
Jeff MacDonald

I created this as an answer to your question and my own confusion. I hope it helps. Pay attention to PYTHONPATH in both the py.test command line and in the tox.ini.

https://github.com/jeffmacdonald/pytest_test

Specifically: You have to tell py.test and tox where to find the modules you are including.

With py.test you can do this:

PYTHONPATH=. py.test

And with tox, add this to your tox.ini:

[testenv]
deps= -r{toxinidir}/requirements.txt
commands=py.test
setenv =
    PYTHONPATH = {toxinidir}

Could you give a short explanation for the project you linked?
Maybe its just me, but the README on the project is pretty detailed, and my comment on stackoverflow says why I created the repo.
While it is not strictly necessary, it is a usual policy to have the main content of an answer in the answer itself because it ensures that the answer is understandable in x years from now when the linked resource may be long gone.
:) Oh well. Thats the internet for ya.
@JFabianMeier, good call. It's now 5 years later and the linked resource is, in fact, gone.
G
Gonzalo

I fixed it by removing the top-level __init__.py in the parent folder of my sources.


Fixed it for me. Can someone explain this?
this right here fixed it for me as well. definitely would love to see an explanation for this if anyone has it
i added init.py but was still facing same issues but this solution worked for me as well.. reason please?
S
Sociopath

I started getting weird ConftestImportFailure: ImportError('No module named ... errors when I had accidentally added __init__.py file to my src directory (which was not supposed to be a Python package, just a container of all source).


L
Luis Lezcano Airaldi

I was having the same problem when following the Flask tutorial and I found the answer on the official Pytest docs It's a little shift from the way I (and I think many others) are used to do things.

You have to create a setup.py file in your project's root directory with at least the following two lines:

from setuptools import setup, find_packages
setup(name="PACKAGENAME", packages=find_packages())

where PACKAGENAME is your app's name. Then you have to install it with pip:

pip install -e .

The -e flag tells pip to intall the package in editable or "develop" mode. So the next time you run pytest it should find your app in the standard PYTHONPATH.


t
tbrodbeck

It is a bit of a shame that this is an issue in python... But just add this environment variable is the most comfortable way IMO:

export PYTHONPATH=$PYTHONPATH:.

Just put this in you .zshrc/.bashrc.


This is indeed a shame, not a bit of a shame.
python -m <module> will add the current working directory (.) to the PYTHONPATH automatically.
n
nocibambi

I had a similar issue. pytest did not recognize a module installed in the environment I was working in.

I resolved it by also installing pytest into the same environment.


Although I was using pytest from inside a venv, I also had it installed globally which gave me this error. After uninstalling the global version and installing inside the venv it worked.
a
alecxe

Also if you run pytest within your virtual environment make sure pytest module is installed within your virtual environment. Activate your virtual env and run pip install pytest.


Exactly my problem. As I only use pytest as a test runner and since I also allow my virtualenv to "see" global packages, I thought I'd get cute and remove the pytest from the virtualenv and rely on the global one instead. The messages and errors are all over the map but I noticed my PYTHONPATH did not look at all like the virtualenvs. The other thing is that I could not pip install pytest in that virtualenv, it kept on telling me the requirement was already met (by the global pytest). Had to deactivate, pip uninstall pytest (global), activate, pip install pytest. Pytest: v6.25
P
Paweł Mucha

For me the problem was tests.py generated by Django along with tests directory. Removing tests.py solved the problem.


O
Oppy

I got this error as I used relative imports incorrectly. In the OP example, test_app.py should import functions using e.g.

from repo.app import *

However liberally __init__.py files are scattered around the file structure, this does not work and creates the kind of ImportError seen unless the files and test files are in the same directory.

from app import *

Here's an example of what I had to do with one of my projects:

Here’s my project structure:

microbit/
microbit/activity_indicator/activity_indicator.py
microbit/tests/test_activity_indicator.py

To be able to access activity_indicator.py from test_activity_indicator.py I needed to:

start test_activity_indicatory.py with the correct relative import:

    from microbit.activity_indicator.activity_indicator import *

put __init__.py files throughout the project structure:

    microbit/
    microbit/__init__.py
    microbit/activity_indicator/__init__.py
    microbit/activity_indicator/activity_indicator.py
    microbit/tests/__init__.py
    microbit/tests/test_activity_indicator.py

p
pbskumar

I was getting this error due to something even simpler (you could even say trivial). I hadn't installed the pytest module. So a simple apt install python-pytest fixed it for me.

'pytest' would have been listed in setup.py as a test dependency. Make sure you install the test requirements as well.


A
Arash Hatami

Since no one has suggested it, you could also pass the path to the tests in your pytest.ini file:

[pytest]
...
testpaths = repo/tests

see documentation: https://docs.pytest.org/en/6.2.x/customize.html#pytest-ini

Side effect for VScode: it should pick up the unit test in the UI.


E
Einhaender

According to a post on Medium by Dirk Avery (and supported by my personal experience) if you're using a virtual environment for your project then you can't use a system-wide install of pytest; you have to install it in the virtual environment and use that install.

In particular, if you have it installed in both places then simply running the pytest command won't work because it will be using the system install. As the other answers have described, one simple solution is to run python -m pytest instead of pytest; this works because it uses the environment's version of pytest. Alternatively, you can just uninstall the system's version of pytest; after reactivating the virtual environment the pytest command should work.


The only thing that worked for me so far was python -m pytest tests/.
B
Bruno Cavalleri

As pointed out by Luiz Lezcano Arialdi, the correct solution is to install your package as an editable package.

Since I am using pipenv, I thought about adding to his answer a step-by-step how to install the current path as an edible with pipenv, allowing to run pytest without the need of any mangling code or loose files.

You will need to have the following minimal folder structure (documentation):

package/
    package/
        __init__.py
        module.py
    tests/
        module_test.py
    setup.py

setup.py most have the following minium code (documentation):

import setuptools


setuptools.setup(name='package', # Change to your package name
                 packages=setuptools.find_packages())

Then you just need to run pipenv install --dev -e . and pipenv will install the current path as an editable package (the --dev flag is optional) (documentation).

Now you shoul be able to run pytest without problems.


S
SANN3

We have fixed the issue by adding the following environment variable.

PYTHONPATH=${PYTHONPATH}:${PWD}/src:${PWD}/test

B
Betterthan Kwora

If this pytest error appears not for your own package, but for a git-installed package in your package's requirements.txt, the solution is to switch to editable installation mode.

For example, suppose your package's requirements.txt had the following line:

git+https://github.com/foo/bar.git

You would instead replace it with the following:

-e git+https://github.com/foo/bar.git#egg=bar

B
Baydaa

Very often the tests were interrupted due to module being unable to be imported,After research, I found out that the system is looking at the file in the wrong place and we can easily overcome the problem by copying the file, containing the module, in the same folder as stated, in order to be properly imported. Another solution proposal would be to change the declaration for the import and show MutPy the correct path of the unit. However, due to the fact that multiple units can have this dependency, meaning we need to commit changes also in their declarations, we prefer to simply move the unit to the folder.


There are other answers that provide the OP's question, and they were posted some time ago. When posting an answer, please make sure you add either a new solution, or a substantially better explanation, especially when answering older questions. Sometimes it's better to post a comment on a particular answer.
As addition to the comment from @help-info.de: Here is the link to the guide for answering questions: stackoverflow.com/help/how-to-answer
d
davide burba

My solution:

create the conftest.py file in the test directory containing:

import os
import sys
sys.path.insert(0,os.path.dirname(os.path.realpath(__file__)) + "/relative/path/to/code/")

This will add the folder of interest to the python path without modifying every test file, setting env variable or messing with absolute/relative paths.