Questions tagged as 'scrapy'

Tagged questions
2
answers

click button with selenium

Hi, I'm new to Selenium. This is the code of the button where I want to click <a href="javascript:bp(26834)">Clic para ver</a> I navigate to the indicated page but it does not reflect the click action that should show a hidde...
asked by 14.09.2018 / 23:46
1
answer

Problem with relative links in Scrapy [closed]

I want to raise the complete news of the links that appear on the cover of an informative site. But the links are relative The site is link And the links look like this <div class="article-title"> <a href="/v2/arti...
asked by 10.07.2016 / 01:26
1
answer

python scrapy remove the blanks from the answer

I am trying to make a scrapy script and I am not capable of what the strip method removes the blanks ... It should be a string but it is an object. Any way to do it? import scrapy class GamesSpider(scrapy.Spider): name = "games" start...
asked by 20.12.2018 / 09:15
2
answers

Install Scrapy on Mac OS X error SSL pip

Good, I am currently trying to install Scrapy on my MacOS but they are all problems, the first thing I enter in terminal is: pip install scrapy And he returns me: You are using pip version 7.0.1, however version 9.0.1 is available. You sho...
asked by 20.11.2016 / 14:37
1
answer

Scrapy error is not recognized

I am trying to run a spider according to the " scrapy " library and when executing the command. scrapy runspider my_spider.py -o data.csv -t csv --set CLOSESPIDER_ITEMCOUNT=10 It signals me the following error    scrapy is not re...
asked by 14.12.2017 / 15:35
0
answers

scrapy splash login formdata input array name

I can not do the login of a form because the name of the inputs is as array, example: <input name="login[username]" value="" id="email" class="input-text required-entry validate-email" title="Dirección de email" type="text"> and <...
asked by 15.09.2017 / 04:20
1
answer

Format the scrapy spider response.css result

When I execute my spider the value I want to get I save them in a dictionary, but I also want to create a folder with the name of one of the results def parse(self, response): ml_item = ScrapyItem() mt_item = ScrapyItem()...
asked by 23.06.2017 / 08:37
1
answer

Iterate over the current selector with XPath and Scrapy

I am scribbling information but I am stuck in the for loop. First, when I receive the answer, I create an xpath "root" to call it. This xpath root has one and every one of the items in a tag ul Then I pull out each of the data I need, but...
asked by 28.09.2018 / 21:51
1
answer

scraping horizontal and vertical web with scrapy

I'm new to scraping and I'm doing horizontal and vertical scraping with scrapy. at the moment of executing my code, it generates the .csv file but empty, without the scraped information. This is my code. Can someone tell me what's wrong? _auth...
asked by 26.05.2018 / 01:11
0
answers

Scrapy extracts the url partially

My Scrapy spider works on everything, but extracting the url through xpath @href does not extract the complete url, always missing the final part of the url from the "? ......." sign which marks the variable GET of the url. Does anyone know if S...
asked by 05.01.2019 / 20:14