Of course. Let's break down how to use Scrapy with Python 3.5.

The Short Answer (TL;DR)

Official support for Python 3.5 was dropped in Scrapy version 2.0.

• Scrapy 1.8 was the last version to officially support Python 3.5.
• Scrapy 2.0+ requires Python 3.6 or newer.

This means if you are using Python 3.5, you must install an older version of Scrapy.

Detailed Explanation and Steps

Here’s a complete guide covering why this happened, how to install the correct version, and the implications.

Why the Version Mismatch?

Scrapy is a large and actively maintained project. To keep improving its features, performance, and security, the development team needs to use modern features of the Python language itself.

• Python 3.5 reached its End-of-Life (EOL) on September 13, 2025. This means it no longer receives security updates or bug fixes from the Python core team.
• Scrapy 2.0 (released in January 2025) introduced features that rely on Python 3.6+, such as f-strings for better readability and performance, and other syntax improvements.
• Dropping support for older Python versions allows the Scrapy team to reduce maintenance overhead, simplify their codebase, and focus on modern development practices.

How to Install Scrapy for Python 3.5

You must explicitly tell pip to install the last compatible version.

Step 1: Ensure Python 3.5 is active

First, make sure your terminal or command prompt is using the correct Python 3.5 environment.

# Check your Python version
python --version
# Should output: Python 3.5.x
# Or if you have multiple Python versions:
python3.5 --version

Step 2: Install the last Scrapy version for Python 3.5

Use the pip command with the operator to specify the exact version.

pip install scrapy==1.8.0

Recommendation: While 8.0 is the last version, you might try installing the very last version of the 8 series, which might have some final bug fixes. You can find the latest 8.x version on the PyPI Scrapy page. For example:

# As of late 2025, the latest 1.8.x version was 1.8.22
pip install scrapy==1.8.22

Important Considerations and Limitations of Using Scrapy 1.8

While you can get Scrapy running on Python 3.5, you are using an old and unsupported version. Be aware of the following:

• Security Vulnerabilities: Scrapy 1.8 does not have patches for security vulnerabilities discovered after its release. This is a significant risk if you are crawling public websites or handling sensitive data.
• No New Features: You will miss out on years of improvements, including major performance enhancements, new middleware components, better documentation, and bug fixes.
• Deprecation Warnings: You may see deprecation warnings from both Scrapy and its dependencies, as they are designed for newer Python versions.
• Dependency Issues: Other modern libraries you might want to use (e.g., for browser automation like selenium or for data processing like pandas) may no longer support Python 3.5, creating conflicts.

A Strong Recommendation: Upgrade Your Python

The best long-term solution is to upgrade your Python environment.

Why you should upgrade:

1. Security: You get critical security updates.
2. Performance: Newer Python versions (3.9, 3.10, 3.11) are significantly faster.
3. Access to Modern Features: You can use the latest versions of Scrapy and all other Python packages.
4. Community Support: It's much easier to get help when you are using supported, up-to-date software.

How to upgrade:

The easiest way to manage multiple Python versions is using a tool like pyenv.

1. Install pyenv: Follow the instructions for your operating system at pyenv.run.

2. Install a modern Python version:

   # Example: Install Python 3.11
   pyenv install 3.11.5

3. Set the local version for your project:

   # Navigate to your project directory
   cd /path/to/your/scrapy-project
   # Set the Python version for this directory
   pyenv local 3.11.5

4. Verify and Install Scrapy:

   # Check the version (it should now be 3.11.x)
   python --version
   # Now you can install the latest Scrapy
   pip install scrapy

Summary Table

Example: A Simple Scrapy Spider (for Scrapy 1.8)

Here is a basic "quotes" spider that will work with Scrapy 1.8 and Python 3.5. The code itself hasn't changed drastically, but the environment is what matters.

Create a project:

scrapy startproject quotesproject
cd quotesproject

Generate a spider:

scrapy genspider quotes quotes.toscrape.com

Edit the spider (quotesproject/spiders/quotes.py):

import scrapy
class QuotesSpider(scrapy.Spider):
    name = 'quotes'
    allowed_domains = ['quotes.toscrape.com']
    start_urls = ['http://quotes.toscrape.com']
    def parse(self, response):
        # The response.css() method is the same in old and new Scrapy
        for quote in response.css('div.quote'):
            yield {
                'text': quote.css('span.text::text').get(),
                'author': quote.css('small.author::text').get(),
                'tags': quote.css('div.tags a.tag::text').getall(),
            }
        # Follow pagination link
        next_page = response.css('li.next a::attr(href)').get()
        if next_page is not None:
            yield response.follow(next_page, self.parse)

Run the spider:

scrapy crawl quotes -o quotes.json

This will create a quotes.json file with the scraped data.