ThinkChat2.0新版上线,更智能更精彩,支持会话、画图、阅读、搜索等,送10W Token,即刻开启你的AI之旅 广告
[TOC] <br> ### requests 库介绍 requests库宣言:HTTP for Humans (给人用的 HTTP 库) 它强大,但是却非常非常的容易使用。也是我们最常用的一个库。 <center>***`掌握requests,不是应该,是必须...`***</center> 使用requests之前,我们必须要先安装 ```cmd pip install requests ``` --- ### 使用requests发送HTTP请求 ```python #!/usr/bin/env python # -*- coding: utf-8 -*- import requests # 设置请求参数,以字典的key-val形式存储 payload = {'key1': 'value1', 'key2': 'value2'} # 设置请求头信息,以字典的key-val形式存储 header = { 'User-Agent': 'Mozilla/5.0 (Linux; Android 4.1.1; MI 2 Build/JRO03L) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/37.0.0.0 Mobile MQQBrowser/6.8 TBS/036887 Safari/537.36 MicroMessenger/6.3.27.880 NetType/WIFI Language/zh_CN', 'Cookie': 'TestCookie=PTQA', 'Accept-Encoding': 'gzip', } # 使用requests发送一个GET请求 r1 = requests.get("http://httpbin.org/get", params=payload,headers=header) print("GET请求的响应结果:",r1.text) # 使用requests发送一个POST请求 r2 = requests.post("http://httpbin.org/post", data=payload,headers=header) print("POST请求的响应结果:",r2.text) ``` 运行结果如: ```cmd GET请求的响应结果: {"args":{"key1":"value1","key2":"value2"},"headers":{"Accept":"*/*","Accept-Encoding":"gzip","Connection":"close","Cookie":"TestCookie=PTQA","Host":"httpbin.org","User-Agent":"Mozilla/5.0 (Linux; Android 4.1.1; MI 2 Build/JRO03L) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/37.0.0.0 Mobile MQQBrowser/6.8 TBS/036887 Safari/537.36 MicroMessenger/6.3.27.880 NetType/WIFI Language/zh_CN"},"origin":"157.61.158.164","url":"http://httpbin.org/get?key1=value1&key2=value2"} POST请求的响应结果: {"args":{},"data":"","files":{},"form":{"key1":"value1","key2":"value2"},"headers":{"Accept":"*/*","Accept-Encoding":"gzip","Connection":"close","Content-Length":"23","Content-Type":"application/x-www-form-urlencoded","Cookie":"TestCookie=PTQA","Host":"httpbin.org","User-Agent":"Mozilla/5.0 (Linux; Android 4.1.1; MI 2 Build/JRO03L) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/37.0.0.0 Mobile MQQBrowser/6.8 TBS/036887 Safari/537.36 MicroMessenger/6.3.27.880 NetType/WIFI Language/zh_CN"},"json":null,"origin":"157.61.158.164","url":"http://httpbin.org/post"} ``` 请求参数与请求头,是非必须的,根据具体请求要求确定是否需要。 在python中,通过requests库发送一个HTTP请求,简单到不可思议,例如这样一行代码`requests.get(url=""http://httpbin.org/get")`就发起了一个GET请求。 --- ### 解析Response对象中的属性 ```python resp = requests.request("GET","http://httpbin.org/get") ``` **主要的属性** - ***resp.url***:最终响应的请求地址(当有302重定向时,可不是最初的请求地址哦) - ***resp.status_code***: HTTP响应码,如‘404’,‘200’等 - ***resp.headers***:HTTP响应头信息 - ***resp.history***: 请求历史记录列表 - ***resp.encoding***:在访问r.text 时进行解码的编码。 **属性方法@property** - ***resp.content***:返回响应内容,字节类型bytes - ***resp.text***:返回响应内容,unicode类型。编码格式由r.encoding决定的,在访问text属性前,允许我们设定合适的编码格式。 **常用方法** - ***resp.json()***:如果响应内容为json字符串,则会转换为json对象 《实例演示》: ```python #!/usr/bin/env python # -*- coding: utf-8 -*- import requests # 设置请求参数,以字典的key-val形式存储 payload = {'key1': 'value1', 'key2': 'value2'} # 设置请求头信息,以字典的key-val形式存储 header = { 'User-Agent': 'Mozilla/5.0 (Linux; Android 4.1.1; MI 2 Build/JRO03L) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/37.0.0.0 Mobile MQQBrowser/6.8 TBS/036887 Safari/537.36 MicroMessenger/6.3.27.880 NetType/WIFI Language/zh_CN', 'Cookie': 'TestCookie=PTQA', 'Accept-Encoding': 'gzip', } # 使用requests发送一个POST请求 resp = requests.post("http://postman-echo.com/post", data=payload,headers=header) print("请求的URL:",resp.url) print("响应码:",resp.status_code) print("响应头:",resp.headers) print("响应Cookie:",resp.cookies) print("请求历史:",resp.history) print("响应内容的编码格式:",resp.encoding) print("响应内容,字节类型bytes:",resp.content) print("响应内容,unicode类型:",resp.text) if resp.headers.get("content-type") == "application/json": print("响应内容,字典对象:",resp.json()) ``` 运行结果如下: ```cmd 请求的URL: http://postman-echo.com/post 响应码: 200 响应头: {'Content-Encoding': 'gzip', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Mon, 23 Jul 2018 13:53:47 GMT', 'ETag': 'W/"269-dp22/aBwvg6BLnokGzBSy+I+NSA"', 'Server': 'nginx', 'set-cookie': 'sails.sid=s%3AYcCxFgX3kgNdGk90nI37fFo4CV782M8U.rlE4FEN%2B9SKrbKIN9LMmouHjmT2TipR8mmdRUwjK%2FDs; Path=/; HttpOnly', 'Vary': 'Accept-Encoding', 'Content-Length': '416', 'Connection': 'keep-alive'} 响应Cookie: <RequestsCookieJar[<Cookie sails.sid=s%3AYcCxFgX3kgNdGk90nI37fFo4CV782M8U.rlE4FEN%2B9SKrbKIN9LMmouHjmT2TipR8mmdRUwjK%2FDs for postman-echo.com/>]> 请求历史: [] 响应内容的编码格式: utf-8 响应内容,字节类型bytes: b'{"args":{},"data":"","files":{},"form":{"key1":"value1","key2":"value2"},"headers":{"host":"postman-echo.com","content-length":"23","accept":"*/*","accept-encoding":"gzip","content-type":"application/x-www-form-urlencoded","cookie":"TestCookie=PTQA","user-agent":"Mozilla/5.0 (Linux; Android 4.1.1; MI 2 Build/JRO03L) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/37.0.0.0 Mobile MQQBrowser/6.8 TBS/036887 Safari/537.36 MicroMessenger/6.3.27.880 NetType/WIFI Language/zh_CN","x-forwarded-port":"80","x-forwarded-proto":"http"},"json":{"key1":"value1","key2":"value2"},"url":"http://postman-echo.com/post"}' 响应内容,unicode类型: {"args":{},"data":"","files":{},"form":{"key1":"value1","key2":"value2"},"headers":{"host":"postman-echo.com","content-length":"23","accept":"*/*","accept-encoding":"gzip","content-type":"application/x-www-form-urlencoded","cookie":"TestCookie=PTQA","user-agent":"Mozilla/5.0 (Linux; Android 4.1.1; MI 2 Build/JRO03L) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/37.0.0.0 Mobile MQQBrowser/6.8 TBS/036887 Safari/537.36 MicroMessenger/6.3.27.880 NetType/WIFI Language/zh_CN","x-forwarded-port":"80","x-forwarded-proto":"http"},"json":{"key1":"value1","key2":"value2"},"url":"http://postman-echo.com/post"} ``` --- ### 保持会话状态 由于HTTP的无状态特性,若不借助其他手段,远程的服务器就无法知道以前和客户端做了哪些通信。Cookie就是「其他手段」之一。 Cookie 一个典型的应用场景,就是用于记录用户在网站上的登录状态。 1.用户登录成功后,服务器下发一个(通常是加密了的)Cookie 文件。 2.客户端(通常是网页浏览器)将收到的 Cookie 文件保存起来。 3.下次客户端与服务器连接时,将 Cookie 文件发送给服务器,由服务器校验其含义,恢复登录状态(从而避免再次登录) #### 方式一:在请求中手工添加Cookie参数 ```python import requests # cookie通过requests的cookies参数传递 r1 = requests.get("http://httpbin.org/cookies", cookies={"is_login": "True"}) # cookie通过headers参数传递 r2 = requests.get("http://httpbin.org/cookies", headers={'cookie': 'is_login=True'}) print(r1.text) print(r2.text) ``` 运行结果如: ```cmd {"cookies":{"is_login":"True"}} {"cookies":{"is_login":"True"}} ``` 同时存在headers和cookies参数时,headers中的cookie会覆盖cookies参数中的cookie #### 方式二:使用Session自动携带Cookie session会自动管理cookie,一个session对象会保持同一个会话中的所有请求之间的cookie信息。 ```python import requests r1 = requests.post("https://postman-echo.com/response-headers?foo1=bar1&foo2=bar2") print("request第一次响应cookie:",r1.cookies) r2 = requests.post("https://postman-echo.com/response-headers?foo1=bar1&foo2=bar2") print("request第二次请求cookie:",r2.request._cookies) # print() print("") #创建Session实例 s = requests.Session() r3 = s.post("https://postman-echo.com/response-headers?foo1=bar1&foo2=bar2") print("session第一次响应cookie:",r3.cookies) r4 = s.post("https://postman-echo.com/response-headers?foo1=bar1&foo2=bar2") print("session第二次请求cookie:",r4.request._cookies) ``` 运行结果如: ```cmd request第一次响应cookie: <RequestsCookieJar[<Cookie sails.sid=s%3A6Ea1SO0RWIZKed43fQhhRVSvpBURT0h0.7xmVIYhg7xaqMpCpcJk7SlTFTg5bm4MY4eRjIuAm5L8 for postman-echo.com/>]> request第二次请求cookie: <RequestsCookieJar[]> session第一次响应cookie: <RequestsCookieJar[<Cookie sails.sid=s%3A60M5buOIVPZIAaBfuNaR3a0s3Q9ExQpK.uYhz9%2B1ZE5O4Hb8qUTCquf4w7af3CynDWNXJjQ9rGko for postman-echo.com/>]> session第二次请求cookie: <RequestsCookieJar[<Cookie sails.sid=s%3A60M5buOIVPZIAaBfuNaR3a0s3Q9ExQpK.uYhz9%2B1ZE5O4Hb8qUTCquf4w7af3CynDWNXJjQ9rGko for postman-echo.com/>]> ``` 通过以上结果对比可知,一个session对象会保持同一个会话中的所有请求之间的cookie信息。 --- ### requests.request 中那些重要的参数 如果你认真观看本书,当你读到这里的时候,应该体会到`源码才是最好的文档`。 通过阅读源码,我们可以看到request的参数有很多,这里不一一讲解,只强调其中两个比较重要的三个参数:timeout,allow_redirects,verify #### 设置超时时间 timeout 为防止服务器不能及时响应,大部分发至外部服务器的请求都应该带着 timeout 参数。如果没有 timeout,你的代码可能会挂起若干分钟甚至更长时间。 ```python r = requests.get('http://httpbin.org/get', timeout=5) ``` #### 进行SSL证书验证 verify 默认情况下, verify 是设置为 True 的,如果SSL认证失败,可以尝试将verify验证关闭,verify=False ```python r = requests.get('https://github.com', verify=True) ``` #### 重定向设置 allow_redirects 默认情况下,allow_redirects 是设置为True的,如果遇到301,302重定向,会自动跳转。如果我们不想让其自动跳转,可以设置 allow_redirects=False ```python import requests r1 = requests.get("http://cnblogs.com/") print("allow_redirects=True时的响应history:",r1.history) print("allow_redirects=True时的响应url:",r1.url) print("") r2 = requests.get("http://cnblogs.com/",allow_redirects=False) print("allow_redirects=False时的响应history:",r2.history) print("allow_redirects=False时的响应url:",r2.url) ``` 运行结果如下: ```cmd allow_redirects=True时的响应history: [<Response [301]>, <Response [301]>] allow_redirects=True时的响应url: https://www.cnblogs.com/ ![](https://box.kancloud.cn/2ff0bc02ec938fef8b6dd7b7f16ee11d_258x258.jpg) allow_redirects=False时的响应history: [] allow_redirects=False时的响应url: http://cnblogs.com/ ``` <hr style="margin-top:100px"> :-: ![](https://box.kancloud.cn/2ff0bc02ec938fef8b6dd7b7f16ee11d_258x258.jpg) ***微信扫一扫,关注“python测试开发圈”,了解更多测试教程!***