多应用+插件架构,代码干净,二开方便,首家独创一键云编译技术,文档视频完善,免费商用码云13.8K 广告
### 1. 分析请求 分析网址:[https://m.weibo.cn/u/2145291155](https://m.weibo.cn/u/2145291155) ### ![](/assets/6.3-1.png) ### 2. 分析响应 链接:[https://m.weibo.cn/api/container/getIndex?type=uid&value=2145291155&containerid=1076032145291155](https://m.weibo.cn/api/container/getIndex?type=uid&value=2145291155&containerid=1076032145291155) ![](/assets/6.3-2.png) 下一条链接:[https://m.weibo.cn/api/container/getIndex?type=uid&value=2145291155&containerid=1076032145291155&page=2](https://m.weibo.cn/api/container/getIndex?type=uid&value=2145291155&containerid=1076032145291155&page=2) 可以后面多了一个page=2的参数 根据后面的链接可以发现page就是页数,value参数值是用户id,containerid参数值是1076032+用户id ### 3. 实战演练 {#3-实战演练} ``` from urllib.parse import urlencode import requests from pyquery import PyQuery as pq from pymongo import MongoClient headers = { 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8', 'Referer': 'https://germey.gitbooks.io/python3webspider/content/6.3-Ajax%E7%BB%93%E6%9E%9C%E6%8F%90%E5%8F%96.html', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.84 Safari/537.36', 'X-Requested-With': 'XMLHttpRequest', } client = MongoClient() db = client.weibo collection = db.weio max_page = 10 base_url = "https://m.weibo.cn/api/container/getIndex?" def get_page(page=0): params = { 'type':'uid', 'value':'2145291155', 'containerid':'1076032145291155', 'page':page, } url = base_url + urlencode(params) # print(url) try: response = requests.get(url,headers=headers) if response.status_code == 200: return response.json() except requests.ConnectionError as e: print(e.args) def parse_page(response): print(response) if response: items = response.get("data").get("cards") print(items) if items: for item in items: item = item.get("mblog") weibo = {} weibo.setdefault("id",item.get("id")) weibo.setdefault("text",item.get("text")) weibo.setdefault("attitudes",item.get("attitudes_count")) weibo.setdefault("comments",item.get("comments_count")) weibo.setdefault("reposts",item.get("reposts_count")) yield weibo def save_to_mongo(result): if collection.insert(result): print("Save to Mongo") if __name__ == "__main__": for page in range(1,max_page+1): response = get_page(page) results = parse_page(response) for result in results: print(result) save_to_mongo(result) ```