Of course! The requests library is one of the most popular and user-friendly libraries in Python for making HTTP requests. It's the de facto standard for interacting with web APIs, scraping websites, and communicating with web services.

Here’s a comprehensive guide covering everything from the basics to more advanced topics.

What is requests?

requests is an elegant and simple HTTP library for Python, built on top of the standard library's urllib3. It simplifies the process of sending HTTP/1.1 requests and makes it much more intuitive than the built-in modules.

Key Advantages:

• Simple and intuitive API: The code reads almost like plain English.
• Automatic Content Decoding: It automatically decodes response content (like gzip) into text.
• Session Objects: Allow you to persist parameters across requests.
• Built-in JSON Decoding: Easily handle JSON responses.
• Simple File Uploads: Easily upload files with multipart encoding.
• SSL/TLS Verification: Simple configuration for secure connections.

Installation

First, you need to install the library. It's not part of the Python standard library, so you'll need to use pip.

pip install requests

The Basic Workflow: GET Requests

The most common type of request is a GET request, used to retrieve data from a server. The basic workflow is always the same:

1. Import the requests library.
2. Make a request using requests.get().
3. Check the response status to see if the request was successful.
4. Access the response content (text, JSON, etc.).

Example: Making a GET Request

Let's fetch data from the JSONPlaceholder test API.

import requests
# The URL we want to send the request to
url = "https://jsonplaceholder.typicode.com/posts/1"
# 1. Make the GET request
response = requests.get(url)
# 2. Check the response status code
# The 'status_code' attribute tells us the result of the request.
# 200 OK means success.
if response.status_code == 200:
    print("Request successful!")
    print(f"Status Code: {response.status_code}")
    # 3. Access the response content
    # The .text attribute gives the raw response content as a string.
    print("\n--- Raw Response Text (JSON string) ---")
    print(response.text)
    # 4. Parse the JSON content
    # The .json() method parses the response text into a Python dictionary.
    # It will raise an error if the response is not valid JSON.
    data = response.json()
    print("\n--- Parsed JSON as Python Dictionary ---")
    print(data)
    print(f"\nTitle: {data['title']}")
    print(f"Body: {data['body']}")
else:
    print(f"Error: Request failed with status code {response.status_code}")

Other Common HTTP Methods

requests also makes it easy to use other HTTP methods like POST, PUT, DELETE, etc.

Example: POST Request (Creating Data)

POST is often used to send data to a server to create a new resource.

import requests
import json # For pretty printing the dictionary
url = "https://jsonplaceholder.typicode.com/posts"
# The data we want to send (this will be the request body)
payload = {: 'foo',
    'body': 'bar',
    'userId': 1
}
# Make the POST request
# The `json` argument automatically sets the Content-Type header to 'application/json'
# and serializes the Python dictionary to a JSON string.
response = requests.post(url, json=payload)
if response.status_code == 201: # 201 Created is the standard success code for POST
    print("Post successful!")
    print(f"Status Code: {response.status_code}")
    # The server usually returns the newly created object
    created_data = response.json()
    print("\n--- Server Response (Created Data) ---")
    print(json.dumps(created_data, indent=4))
else:
    print(f"Error: Request failed with status code {response.status_code}")

Example: DELETE Request

import requests
url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.delete(url)
if response.status_code == 200: # 200 OK is common for DELETE
    print("Delete successful!")
    print(f"Status Code: {response.status_code}")
    # The response body for a successful DELETE is often empty
    print(f"Response Text: '{response.text}'")
else:
    print(f"Error: Request failed with status code {response.status_code}")

Making Custom Requests (Headers, Parameters, etc.)

Real-world applications often require more than simple requests.

a) Adding URL Parameters

Instead of manually building URLs like .../search?query=python&page=1, you can use the params argument.

import requests
base_url = "https://jsonplaceholder.typicode.com/posts"
# Parameters are passed as a dictionary
params = {
    'userId': 1
}
response = requests.get(base_url, params=params)
if response.status_code == 200:
    posts = response.json()
    print(f"Found {len(posts)} posts for user 1.")
    for post in posts:
        print(f"- {post['id']}: {post['title']}")

b) Adding Custom Headers

Some APIs require you to send custom headers, like an API key for authentication.

import requests
url = "https://api.github.com"
# Custom headers
headers = {
    'Accept': 'application/vnd.github.v3+json',
    'User-Agent': 'My-Python-App/1.0' # It's good practice to identify your client
}
response = requests.get(url, headers=headers)
if response.status_code == 200:
    print("Successfully accessed GitHub API.")
    # The response contains info about the authenticated user (if any)
    print(response.json())

c) Sending Data in the Body (Form Data)

For non-JSON data, like form submissions, use the data argument.

import requests
url = "https://httpbin.org/post" # A service that echoes back what you send
# Form data is sent as key-value pairs
form_data = {
    'username': 'john_doe',
    'password': 'secret123'
}
response = requests.post(url, data=form_data)
if response.status_code == 200:
    print("Form data sent successfully.")
    # httpbin.org shows us the form data it received
    print("Server received the following form data:")
    print(response.json()['form'])

Handling Errors

Robust code must handle potential errors. requests provides a convenient way to do this.

• Connection Errors: Network issues (e.g., no internet).
• Timeouts: The server takes too long to respond.
• HTTP Error Status Codes: 4xx (client error) or 5xx (server error).

import requests
from requests.exceptions import RequestException, Timeout, HTTPError
url = "https://jsonplaceholder.typicode.com/non-existent-path"
try:
    # Set a timeout for the request (in seconds)
    response = requests.get(url, timeout=5)
    # If the response was an error (4xx or 5xx), raise an HTTPError
    response.raise_for_status() 
    # This line will only run if the request was successful (status code < 400)
    print("Success!")
    print(response.json())
except HTTPError as http_err:
    print(f"HTTP error occurred: {http_err}") # e.g., 404, 500
except Timeout as timeout_err:
    print(f"The request timed out: {timeout_err}")
except RequestException as req_err:
    # This is a base class for all requests-related exceptions
    print(f"An error occurred during the request: {req_err}")

Advanced: Using Session Objects

A Session object persists certain parameters across requests. This is extremely useful for:

• Keeping a cookie: Login to a website and stay logged in for subsequent requests.
• Setting headers once: If you need to send the same headers with every request.
• Connection pooling: Slightly improves performance for multiple requests to the same host.

import requests
# 1. Create a session object
with requests.Session() as session:
    # 2. Set parameters that will persist for all requests made with this session
    session.headers.update({'User-Agent': 'My-Cool-App/1.0'})
    # 3. First request (e.g., a login)
    # login_payload = {'username': 'user', 'password': 'pass'}
    # session.post('https://example.com/login', data=login_payload)
    # The session now holds the authentication cookie.
    # 4. Second request (e.g., accessing a protected page)
    # This request will automatically include the cookie from the login.
    response = session.get('https://jsonplaceholder.typicode.com/posts/1')
    if response.status_code == 200:
        print("Session request successful!")
        print(response.json())

Summary Table of Common Methods and Attributes