Skip to content
Postman API Testing Tips

How To Use Cheerio In Postman

API Testing Blog

Scraping HTML with Cheerio in Postman

Postman is a powerful tool for API testing, but its capabilities extend beyond just interacting with APIs. With the help of the cheerio library, you can also scrape HTML content from websites directly within Postman, allowing you to analyze and extract data from web pages as part of your API testing workflow.

Installing Cheerio in Postman

To use Cheerio in Postman, you’ll need to install it as a global dependency. Here’s how:

  1. Open the Postman console: Go to the “Console” tab in Postman.
  2. Install the cheerio library: Use the following command in the console:
    Terminal window
    npm install cheerio --global
  3. Verify installation: Once installed, you can check if the package is available by using the require function: 
    console.log(require('cheerio'));
    This should display the Cheerio library object.

Using Cheerio in Postman Requests

Now that Cheerio is installed, you can use it within your Postman requests. Here’s a step-by-step guide:

  1. Define a GET request: Create a new request in Postman and set the method to “GET”. Enter the target URL you want to scrape in the request URL field.
  2. Add a “Pre-request Script” tab: Click on the “Pre-request Script” tab. This is where you’ll write the code to extract data using Cheerio.
  3. Import cheerio: Start by importing the Cheerio library in your script: 
    const cheerio = require('cheerio');
  4. Send the request and retrieve the response: You’ll need to send the GET request and get the response using the pm.sendRequest function: 
    pm.sendRequest(pm.request.url, function (err, response) {
        if (err) {
            console.error(err);
        } else {
            console.log('Response status:', response.statusCode);
            // Use the response body to parse with Cheerio
        }
    });
  5. Parse the HTML content with Cheerio: Once you have the response, you can use Cheerio to parse the HTML content: 
    const $ = cheerio.load(response.text);
  6. Extract the data you need: Use Cheerio’s jQuery-like syntax to select the elements you want and extract their data: 
    const titles = $('h1').text();
    const links = $('a').attr('href');
    // ... and so on
  7. Use the extracted data: You can now use the extracted data within your test assertions or further actions.

Example: Extracting product information

Let’s say you want to test an API that fetches product details from an e-commerce website. Here’s how you can use Cheerio to extract the product title, price, and image URL from the website’s product page:

pm.sendRequest(pm.request.url, function (err, response) {
    if (err) {
        console.error(err);
    } else {
        const $ = cheerio.load(response.text);


        const title = $('h1.product-title').text();
        const price = $('.product-price').text();
        const imageUrl = $('.product-image').attr('src');


        console.log('Title:', title);
        console.log('Price:', price);
        console.log('Image URL:', imageUrl);
    }
});

This script sends a GET request to the product page, parses the HTML using Cheerio, finds the title, price, and image URL elements, and outputs them to the console.

Considerations:

  • Website structure: The cheerio selectors depend on the HTML structure of the website you are scraping. Ensure you inspect the website’s HTML source code to find the correct selectors for the elements you need.
  • Performance: Scraping large amounts of data can affect the performance of your API tests. You might need to adjust the scripting logic or use a separate service for large-scale scraping tasks.
  • Data validity: As websites might change their HTML structure, ensure your selectors are updated to handle potential changes and avoid breaking your tests.

Conclusion:

Cheerio in Postman allows you to unlock powerful functionality for testing APIs and interacting with websites. By combining the power of API testing with web scraping, you can create a comprehensive testing framework that covers both API functionalities and the frontend presentation of data.

API Testing Blog