ChatGPT解决这个技术问题 Extra ChatGPT

Unable to get local issuer certificate when using requests in python

here is my code

import requests;
url='that website';
headers={
  'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
  'Accept-Language':'zh-CN,zh;q=0.9,en;q=0.8,ja;q=0.7',
  'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36'
};
r = requests.get(url,headers=headers);
print(r);
print(r.status_code);

then it ran into this:

requests.exceptions.SSLError: HTTPSConnectionPool(host='www.xxxxxx.com', port=44 3): Max retries exceeded with url: xxxxxxxx (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)')))

what should i do?

If you do have certificate, try r = requests.get(url,headers=headers, cert=("/path/to/file.crt", "/path/to/file.key"))
Normally the python installation has access to root certificate authorities. However on some OSes such as OSX, the root CA are empty. Check this answer, maybe this helps: stackoverflow.com/a/56031239/1617295
I found this awesome article explaining the cause of it: medium.com/@superseb/…
Are/Were you on a Mac by any chance? Check out this answer on how to install certificates: stackoverflow.com/a/53310545/778533

I
Indranil

It's not recommended to use verify = False in your organization's environments. This is essentially disabling SSL verification.

Sometimes, when you are behind a company proxy, it replaces the certificate chain with the ones of Proxy. Adding the certificates in cacert.pem used by certifi should solve the issue. I had similar issue. Here is what I did, to resolve the issue -

Find the path where cacert.pem is located -

Install certifi, if you don't have. Command: pip install certifi

import certifi
certifi.where()
C:\\Users\\[UserID]\\AppData\\Local\\Programs\\Python\\Python37-32\\lib\\site-packages\\certifi\\cacert.pem

Open the URL on a browser. Download the chain of certificates from the URL and save as Base64 encoded .cer files. Now open the cacert.pem in a notepad and just add every downloaded certificate contents (---Begin Certificate--- *** ---End Certificate---) at the end.


Hello, it looks like Python uses certifi module for SSL communications. This certifi module uses cacert.pem file to validate against the SSL certificate. Only the certificates chains that are stored in cacert.pem are considered valid. When any SSL certificate is not found in this file, causes "CERTIFICATE_VERIFY_FAILED" error.
If you get the error using urllib's urlopen, you can install certifi and then do urlopen(url, cafile="/path/to/file.pem", capath="/path/to/certifi/").read()
So that other don't have to dig to figure out how to do Step 2: How to download certificates from URL
Step 2 with Chrome on a Mac stackoverflow.com/questions/25940396/…
This worked for me too. Why must everything be a struggle to get the environment ready and working in python!! :-)
J
Jing He

If you have already tried to update the CA(root) Certificate using pip:

pip install --upgrade certifi

or have already downloaded the newest version of cacert.pem from https://curl.haxx.se/docs/caextract.html and replaced the old one in {Python_Installation_Location}\\lib\\site-packages\\certifi\\cacert.pem but it still does not work, then your client is probably missing the Intermediate Certificate in the trust chain.

Most browsers can automatically download the Intermediate Certificate using the URL in "Authority Info Access" section in the Certificate, but Python, Java, and openssl s_client cannot. They rely on the server proactively sending them the intermediate certificate.

https://i.stack.imgur.com/duYhd.png

If you speak Chinese you can read this awesome blog: https://www.cnblogs.com/sslwork/p/5986985.html and use this tool to check if the intermediate certificate is sent by / installed on the server or not: https://www.myssl.cn/tools/check-server-cert.html

If you do not, you can check this article: https://www.ssl.com/how-to/install-intermediate-certificates-avoid-ssl-tls-not-trusted/

We can also use openssl in Linux to cross-check this issue:

openssl s_client -connect yourwebsite:443

https://i.stack.imgur.com/Nc8KJ.png

My current solution for this problem is like @Indranil's suggestion (https://stackoverflow.com/a/57466119/4522434): Export the Intermediate Certificate in browser using base64 X.509 CER format; then use Notepad++ to open it and copy the content into the end of cacert.pem in {Python_Installation_Location}\\lib\\site-packages\\certifi\\cacert.pem


In the result of openssl command, CN = Common name, O = Organization, OU = Organization Unit, L = Locality, C = Country, S = State, ref link docs.oracle.com/cd/E24191_01/common/tutorials/…
Useful to know about "Authority Info Access", thanks!
S
Samuli P

Answers pointing to certifi are a good start and in this case there could be an additional step needed if on Windows.

pip install python-certifi-win32

The above package would patch the installation to include certificates from the local store without needing to manage store files manually. The patch was suggested to certifi but declined as "the purpose of certifi is not to be a cross-platform module to access the system certificate store." [https://github.com/certifi/python-certifi/pull/54#issuecomment-288085993]

The issue with local certificates traces to Python TLS/SSL and Windows Schannel. There is an open issue at Python [https://bugs.python.org/issue36011] and PEP that did not lead to a solution [https://www.python.org/dev/peps/pep-0543/#resolution]


Q
Quentin

If you're using macOS, search for "Install Certificates.command" file (it is usually in Macintosh HD > Applications > your_python_dir).

You can also find it with "command" + "break space" and paste "Install Certificates.command" in the field.

If you used brew to install python, your solution is there: brew installation of Python 3.6.1: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed


j
jstaab

I had the same problem. I was able to make requests against my server via the browser, but using python requests, I was getting the error mentioned above. Requests and certifi were both fully up to date; the problem ended up being my server's configuration.

The problem was that I had only installed the intermediate cert instead of the full cert chain.

In my case, following this article, I simply ran cat my-domain.crt my-domain.ca-bundle > my-domain.crt-combined and installed the crt-combined file on my server (via heroku's app settings interface) instead of the crt file.


t
thegreyd

You can also set REQUESTS_CA_BUNDLE env variable to force requests library to use your cert, that solved my issue.


N
Narayan Bhat

This should solve your problem

This is because the url is a https site instead of http. So it requires ssl verification using certificates. If you are working in your firms workstation, internal use sites will be accessible through the browser managed by your organization. The organization will have setup the certificates.

Atleast these certificates are needed

ROOT CA certificate

Intermediate CA certificate

Website ( domain ) certificate

The browsers will have these certificates configured, but python will not. So you need to do some manual work to get it working.

As Indranil suggests, using verify=False is not recommended. So download all the certificates as mentioned in the above link and follow the steps.


H
Husain Suksar

In macOS just open Macintosh HD

Now Select Application Then Select Python folder ( Python3.6, Python3.7 Whatever You are using just select this folder )

Then, double click on Install Certificates.command. Now your error should be solved.


关注公众号,不定期副业成功案例分享
Follow WeChat

Success story sharing

Want to stay one step ahead of the latest teleworks?

Subscribe Now