当前位置: 代码迷 >> 综合 >> 用python selenium etree 获取Indiagogo众筹项目产品信息,把爬出到的数据存为csv格式和json格式,且实现网页长截图
  详细解决方案

用python selenium etree 获取Indiagogo众筹项目产品信息,把爬出到的数据存为csv格式和json格式,且实现网页长截图

热度:29   发布时间:2023-12-06 02:34:12.0

用python selenium etree 获取Indiagogo众筹项目产品信息,把爬出到的数据存为csv格式和json格式,且实现网页长截图,源代码如下:

import csv
import time
import requests
from random import randrange
from fake_useragent import UserAgent
from lxml import html
from selenium import webdriver
import jsonimport os.path
import multiprocessing as mp
from selenium.webdriver.chrome.options import Optionsclass getProduct():etree = html.etreeheaders = { "authority": "www.indiegogo.com",
"method": "POST",
"path": "/private_api/discover",
"scheme": "https",
"accept": "application/json, text/plain, */*",
"accept-encoding": "gzip, deflate, br",
"accept-language": "zh-CN,zh;q=0.9",
"content-length": '174',
"content-type": "application/json;charset=UTF-8",
"cookie": 'xxxx',
"origin": "https://www.indiegogo
  相关解决方案