Parsing values from Google news

Tags: , , ,



From Google news I’m attempting to parse the results. For example, parse the title and text from the search “latest movie releases”, here is the URL:

https://www.google.com/search?client=firefox-b-d&tbm=nws&sxsrf=ALeKk01qAUzdE7UzK9aWPL9MYALHEk6aiQ%3A1599313588168&ei=tJZTX6vwCdWr1fAP6eGiyAk&q=latest+movie+releases&oq=latest+movie+releases&gs_l=psy-ab.3…299098.305542.0.305681.31.25.3.2.2.0.161.1719.22j3.25.0….0…1c.1.64.psy-ab..1.13.704…0j33i10k1.0.9TgaNbbee40

The results appear to use #rso in the id:

enter image description here

But the iterator over $('#rso').each is empty. What id or css element should I select in order to iterate over the divs of search results ?

Iterator code:

$('#rso').each(function (i, element) {
    console('div level 1')
    var title = $(this).find('.r').text();
    var link = $(this).find('.r').find('a').attr('href').replace('/url?q=', '').split('&')[0];
    var text = $(this).find('.st').text();
    var img = $(this).find('img.th').attr('src');
    savedData.push({
      title: title,
      link: link,
      text: text,
      img: img
    });
  });

Answer

You should use $$ instead

$$('#rso > div')

enter image description here

Reference

Console Utilities API Reference

$(selector) is alias to document.querySelector()

$$(selector) is alias to document.querySelectorAll()



Source: stackoverflow