Scrapy爬虫报错HTTP status code is not handled or not allowed

爬取网页显示403 HTTP status code is not handled or not allowed

解决方法：

在settings.py里把代理关掉：

DOWNLOADER_MIDDLEWARES = {

'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None,

}