Skip to content

Parsing values from Google news

From Google news I’m attempting to parse the results. For example, parse the title and text from the search “latest movie releases”, here is the URL:

https://www.google.com/search?client=firefox-b-d&tbm=nws&sxsrf=ALeKk01qAUzdE7UzK9aWPL9MYALHEk6aiQ%3A1599313588168&ei=tJZTX6vwCdWr1fAP6eGiyAk&q=latest+movie+releases&oq=latest+movie+releases&gs_l=psy-ab.3…299098.305542.0.305681.31.25.3.2.2.0.161.1719.22j3.25.0….0…1c.1.64.psy-ab..1.13.704…0j33i10k1.0.9TgaNbbee40

The results appear to use #rso in the id:

enter image description here

But the iterator over $('#rso').each is empty. What id or css element should I select in order to iterate over the divs of search results ?

Iterator code:

$('#rso').each(function (i, element) {
    console('div level 1')
    var title = $(this).find('.r').text();
    var link = $(this).find('.r').find('a').attr('href').replace('/url?q=', '').split('&')[0];
    var text = $(this).find('.st').text();
    var img = $(this).find('img.th').attr('src');
    savedData.push({
      title: title,
      link: link,
      text: text,
      img: img
    });
  });

Answer

You should use $$ instead

$$('#rso > div')

enter image description here

Reference

Console Utilities API Reference

$(selector) is alias to document.querySelector()

$$(selector) is alias to document.querySelectorAll()