Python-Selenium使用Chrome调用Http代理IP的代码样例
站大爷
官方
2024-01-20
1995 浏览
温馨提示:
1. 此代码样例分别为基于"终端IP"授权和"用户名+密码"授权方式使用Selenium+Chrome认证代理IP
2. 运行环境为python2/3 + selenium + Chrome + Chromedriver + Windows/Linux/macOS
3. 注意chromedriver版本要和Chrome版本对应
4. 样例中代理IP需更换为自己通过站大爷API提取链接获取到的代理IP
5. ${chromedriver_path}:您本机chromedriver驱动存放路径,如:"C:\chromedriver.exe"
6. selenium不是python原生库,需要安装才能使用:pip install selenium;4.6版本开始,无需手动下载driver
7. 样例中的"实例ID"和"密码"显示在站大爷控制台的"实例管理"里。
"终端IP"授权
from selenium import webdriver
import time
options = webdriver.ChromeOptions()
options.add_argument('--proxy-server=http://${168.168.168.168:88888}')
# ${chromedriver_path}: chromedriver驱动存放路径
driver=webdriver.Chrome(executable_path="${chromedriver_path}", options=options)
driver.get("https://example.com")
# 获取页面内容
print(driver.page_source)
# 延迟3秒后关闭当前窗口如果是最后一个窗口则退出
time.sleep(3)
driver.close()
"用户名+密码"授权
from selenium import webdriver
import string
import zipfile
import time
def create_proxyauth_extension(proxy_host, proxy_port, proxy_username, proxy_password, scheme='http', plugin_path=None):
if plugin_path is None:
plugin_path='vimm_chrome_proxyauth_plugin.zip'
manifest_json="""
{
"version":"1.0.0",
"manifest_version": 2,
"name":"Chrome Proxy",
"permissions": [
"proxy",
"tabs",
"unlimitedStorage",
"storage",
"<all_urls>",
"webRequest",
"webRequestBlocking"
],
"background":{
"scripts":["background.js"]
},
"minimum_chrome_version":"22.0.0"
}
"""
background_js=string.Template(
"""
var config={
mode:"fixed_servers",
rules:{
singleProxy:{
scheme:"${scheme}",
host:"${host}",
port:parseInt(${port})
},
bypassList:["foobar.com"]
}
};
chrome.proxy.settings.set({value:config,scope:"regular"},function(){});
function callbackFn(details) {
return {
authCredentials: {
username:"${username}",
password:"${password}"
}
};
}
chrome.webRequest.onAuthRequired.addListener(
callbackFn,
{urls: ["<all_urls>"]},
['blocking']
);
"""
).substitute(
host=proxy_host,
port=proxy_port,
username=proxy_username,
password=proxy_password,
scheme=scheme,
)
with zipfile.ZipFile(plugin_path,'w') as zp:
zp.writestr("manifest.json",manifest_json)
zp.writestr("background.js",background_js)
return plugin_path
proxyauth_plugin_path=create_proxyauth_extension(
proxy_host="168.168.168.168", # 代理IP
proxy_port="25852", # 端口号
# 用户名密码授权
proxy_username="实例id",
proxy_password="密码"
)
options=webdriver.ChromeOptions()
options.add_extension(proxyauth_plugin_path)
# ${chromedriver_path}: chromedriver驱动存放路径
driver=webdriver.Chrome(executable_path="${chromedriver_path}",options=options)
driver.get("https://example.com")
# 获取页面内容
print(driver.page_source)
# 延迟3秒后关闭当前窗口如果是最后一个窗口则退出
time.sleep(3)
driver.close()