Skip to content

Instantly share code, notes, and snippets.

@groundrace
Created January 15, 2020 15:33
Show Gist options
  • Save groundrace/b3925cd83eac8f3f791b0e840751e4f1 to your computer and use it in GitHub Desktop.
Save groundrace/b3925cd83eac8f3f791b0e840751e4f1 to your computer and use it in GitHub Desktop.
const puppeteer = require('puppeteer');
(async () => {
const search_string = 'google scraper';
const resultsObj = {search_term: search_string, results:[]};
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://google.com');
await page.type('input.gLFyf.gsfi', search_string);
page.keyboard.press('Enter');
await page.waitForSelector('div#resultStats');
await page.waitForSelector('div#center_col');
const elementHandles = await page.$$('.r > a');
const propertyJsHandles = await Promise.all(
elementHandles.map(handle => handle.getProperty('href'))
);
const hrefs2 = await Promise.all(
propertyJsHandles.map(handle => handle.jsonValue())
);
resultsObj.results = hrefs2;
console.log(resultsObj);
await page.screenshot({ path: 'screenshot.png' });
await browser.close();
})();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment