I want to extract the text content from the below HTML tag, but the <sup> tag is preventing me from getting the desired text. The text I want to extract is simply (4:6, 6:7). how can I extract this text at the same time escaping the <sup> tag. I tried this “//p/text()”, but I am only g…
Tag: scrapy
Parse property page URLs using xpath
I am trying to parse the main property page https://www.realtyatlas.co.za/search?areas%5B0%5D%5Btown%5D=Bellville&status=For%20Sale, more precisely I would like to extract the href from attribute class that is here, and make a follow link: However all the combinations I have tried result in None. I am als…
How to parse JavaScript Json into Python dict type, effeciently
I am looking for way to read javascript json data loaded into one of a script tag of this page. I have tried various re patterns posted on google and stackoveflow but got nothing. The Json Formatter shows an Invalid (RFC 8259). Here is a code The problem seems an invalid json format. The type of profile_json …
Scrapy + splash: can’t select element
I’m learning to use scrapy with splash. As an exercise, I’m trying to visit https://www.ubereats.com/stores/, click on the address text box, enter a location and then press the Enter button to move to next page containing the restaurants available for that location. I have the following lua code: …
Splash API/lua error: attempt to index local element (a nil value)
I’m writing a lua script that I want to use with scrapy + splash for a website. I want to write a script that enters a text and then clicks on a button. I have the following code: Right now I’m using the splash API to test if my code runs properly. When I click “Render!” I get the foll…