我在我的 Python 脚本中使用了很棒的 Requests 库:
import requests
r = requests.get("some-site.example")
print r.text
我想使用袜子代理。但是 Requests 现在只支持 HTTP 代理。
我怎样才能做到这一点?
现代方式:
pip install -U requests[socks]
然后
import requests
resp = requests.get('http://go.to',
proxies=dict(http='socks5://user:pass@host:port',
https='socks5://user:pass@host:port'))
从 2016 年 4 月 29 日发布的 requests
版本 2.10.0 开始,requests
支持 SOCKS。
它需要 PySocks,它可以与 pip install pysocks
一起安装。
示例用法:
import requests
proxies = {'http': "socks5://myproxy:9191"}
requests.get('http://example.org', proxies=proxies)
pip install -U requests[socks]
够了
requests
版本手动升级到支持 SOCKS (> 2.10.0) 的版本,请运行 pip: pip install requests==2.18.4
(2.18. 4),但检查:pypi.python.org/pypi/requests 以获得最新版本(此页面应在顶部标题中向您显示最新的稳定版本是什么)。
socks
模块名称与 qBittorrent
冲突,我需要删除/移动 ~/.local/share/data/qBittorrent/nova3/socks.py
并删除该 socks.pyc
,以分别解决错误消息 module 'socks' has no attribute 'create_connection'
和 bad magic number in 'socks':
。
如果有人尝试了所有这些较旧的答案,但仍然遇到以下问题:
requests.exceptions.ConnectionError:
SOCKSHTTPConnectionPool(host='myhost', port=80):
Max retries exceeded with url: /my/path
(Caused by NewConnectionError('<requests.packages.urllib3.contrib.socks.SOCKSConnection object at 0x106812bd0>:
Failed to establish a new connection:
[Errno 8] nodename nor servname provided, or not known',))
这可能是因为默认情况下,requests
配置为解析连接的 本地 端的 DNS 查询。
尝试将您的代理 URL 从 socks5://proxyhost:1234
更改为 socks5h://proxyhost:1234
。注意额外的 h
(它代表主机名解析)。
The PySocks package module default is to do remote resolution,我不确定为什么请求会导致 their integration 出现如此明显的分歧,但我们到了。
socks5h
方法如此比我之前担心必须做的猴子修补解决方法干净得多。
socks5h://
。一定是找错地方了。一定很喜欢。
您需要安装 pysocks ,我的版本是 1.0 并且代码适用于我:
import socket
import socks
import requests
ip='localhost' # change your proxy's ip
port = 0000 # change your proxy's port
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, ip, port)
socket.socket = socks.socksocket
url = u'http://ajax.googleapis.com/ajax/services/search/images?v=1.0&q=inurl%E8%A2%8B'
print(requests.get(url).text)
一旦 python requests
将与 SOCKS5
拉取请求合并,它将像使用 proxies
字典一样简单:
#proxy
# SOCKS5 proxy for HTTP/HTTPS
proxies = {
'http' : "socks5://myproxy:9191",
'https' : "socks5://myproxy:9191"
}
#headers
headers = {
}
url='http://example.com/'
res = requests.get(url, headers=headers, proxies=proxies)
另一种选择是,如果您不能等待 request
准备好,当您无法使用 requesocks
(例如由于缺少 pwd
内置模块而在 GoogleAppEngine 上)时,请使用上面提到的 PySocks :
从 repo 中获取 socks.py 文件并将副本放在您的根文件夹中;添加导入袜子和导入套接字
此时在使用 urllib2
之前配置和绑定套接字 - 在以下示例中:
import urllib2
import socket
import socks
socks.set_default_proxy(socks.SOCKS5, "myprivateproxy.example",port=9050)
socket.socket = socks.socksocket
res=urllib2.urlopen(url).read()
您可以使用 https_proxy
环境变量运行您的脚本。
如有必要,安装袜子支架。
pip install PySocks
pip install pysocks5
设置环境变量
export https_proxy=socks5://<hostname or ip>:<port>
运行你的脚本。此示例使用代理发出请求并显示 IP 地址:
echo Your real IP
python -c 'import requests;print(requests.get("http://ipinfo.io/ip").text)'
echo IP with socks-proxy
python -c 'import requests;print(requests.get("https://ipinfo.io/ip").text)'
# SOCKS5 proxy for HTTP/HTTPS
proxiesDict = {
'http' : "socks5://1.2.3.4:1080",
'https' : "socks5://1.2.3.4:1080"
}
# SOCKS4 proxy for HTTP/HTTPS
proxiesDict = {
'http' : "socks4://1.2.3.4:1080",
'https' : "socks4://1.2.3.4:1080"
}
# HTTP proxy for HTTP/HTTPS
proxiesDict = {
'http' : "1.2.3.4:1080",
'https' : "1.2.3.4:1080"
}
requesocks
?
我在 urllib3 中安装了 pysocks 和猴子补丁 create_connection,如下所示:
import socks
import socket
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS4, "127.0.0.1", 1080)
def create_connection(address, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None, socket_options=None):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith('['):
host = host.strip('[]')
err = None
for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socks.socksocket(af, socktype, proto)
# If provided, set socket level options before connecting.
# This is the only addition urllib3 makes to this function.
urllib3.util.connection._set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
sock.connect(sa)
return sock
except socket.error as e:
err = e
if sock is not None:
sock.close()
sock = None
if err is not None:
raise err
raise socket.error("getaddrinfo returns an empty list")
# monkeypatch
urllib3.util.connection.create_connection = create_connection
我可以在 Linux 上做到这一点。
$ pip3 install --user 'requests[socks]'
$ https_proxy=socks5://<hostname or ip>:<port> python3 -c \
> 'import requests;print(requests.get("https://httpbin.org/ip").text)'
不定期副业成功案例分享
bash -c "pip install -U requests[socks]"
,否则 zsh 会抱怨zsh: no matches found: requests[socks]
。pip install 'requests[socks]'
就足够了socks5h://localhost:8080
,因为我无法解析主机名。从 github.com/urllib3/urllib3/issues/1035 看来,“h”似乎告诉库服务器将是解析主机名的服务器。