🐾 lycos.js

All the goodies you'll ever need to scrape the web

Documentation

• Get Started
• Examples
• API Reference

In-browser Playground

You can try the library on codesandbox, it uses a cors proxy fetcher to let you grab contents from any website inside your browser.

• CodeSandbox: https://codesandbox.io/s/njm2p72m

Installation

yarn add lycos
# or
npm i lycos

Features

• ⚡️️ All in one package to fetch and scrape data from the web
• ⭐ Node & Browser Support
• 💡 Powerful declarative API
• 🚀 Blazingly fast (supports concurrency)
• 🔧 Extensible

Quick Example

const lycos = require('lycos');

(async () => {
// Fetch the given url and return a page scraper
const page = await lycos.get('http://quotes.toscrape.com');

// Scrape all the quotes elements
const quoteElements = page.scrapeAll('.quote');

// For each quote element, scrape the text and the author 
const quotes = quoteElements.map(element => ({
    text: element.scrape('.text').text(),
    author: element.scrape('.author').text()
}));

// Shortcut to scrape the collection of quotes
const quotes = page.scrapeAll('.quote', {
  author: '.author@text',
  text: '.text@text'
});

// Shortcut to fetch and scrape
const quotes = await lycos
  .get('http://quotes.toscrape.com')
  .scrapeAll('.quote', {
    author: '.author@text',
    text: '.text@text'
  });

})();

Credits

• FB55: his work reprensents the core of this library.

• Matt Mueller and cheerio contributors : A good portion of the code and concepts are copied/derived from the cheerio and x-ray libraries.

License

MIT © 2019 Jimmy Laurent