Home SEO Bulk Loading Efficiency Exams With PageSpeed Insights API & Python

Bulk Loading Efficiency Exams With PageSpeed Insights API & Python


Google provides PageSpeed Insights API to assist search engine marketing professionals and builders by mixing real-world knowledge with simulation knowledge,  offering load efficiency timing knowledge associated to internet pages.

The distinction between the Google PageSpeed Insights (PSI) and Lighthouse is that PSI entails each real-world and lab knowledge, whereas Lighthouse performs a web page loading simulation by modifying the connection and user-agent of the gadget.

One other level of distinction is that PSI doesn’t provide any data associated to internet accessibility, search engine marketing, or progressive internet apps (PWAs), whereas Lighthouse gives the entire above.

Thus, once we use PageSpeed Insights API for the majority URL loading efficiency take a look at, we received’t have any knowledge for accessibility.

Nonetheless, PSI gives extra data associated to the web page velocity efficiency, corresponding to “DOM Dimension,” “Deepest DOM Youngster Factor,” “Complete Process Rely,” and “DOM Content material Loaded” timing.

Yet another benefit of the PageSpeed Insights API is that it provides the “noticed metrics” and “precise metrics” completely different names.

On this information, you’ll be taught:

  • Find out how to create a production-level Python Script.
  • Find out how to use APIs with Python.
  • Find out how to assemble knowledge frames from API responses.
  • Find out how to analyze the API responses.
  • Find out how to parse URLs and course of URL requests’ responses.
  • Find out how to retailer the API responses with correct construction.

An instance output of the Web page Pace Insights API name with Python is under.

example output of the Page Speed InsightsScreenshot from creator, June 2022

Libraries For Utilizing PageSpeed Insights API With Python

The mandatory libraries to make use of PSI API with Python are under.

  • Advertools retrieves testing URLs from the sitemap of an internet site.
  • Pandas is to assemble the info body and flatten the JSON output of the API.
  • Requests are to make a request to the particular API endpoint.
  • JSON is to take the API response and put it into the particularly associated dictionary level.
  • Datetime is to switch the particular output file’s identify with the date of the second.
  • URLlib is to parse the take a look at topic web site URL.

How To Use PSI API With Python?

To make use of the PSI API with Python, comply with the steps under.

  • Get a PageSpeed Insights API key.
  • Import the mandatory libraries.
  • Parse the URL for the take a look at topic web site.
  • Take the Date of Second for file identify.
  • Take URLs into an inventory from a sitemap.
  • Select the metrics that you really want from PSI API.
  • Create a For Loop for taking the API Response for all URLs.
  • Assemble the info body with chosen PSI API metrics.
  • Output the leads to the type of XLSX.

1. Get PageSpeed Insights API Key

Use the PageSpeed Insights API Documentation to get the API Key.

Click on the “Get a Key” button under.

psi api key Picture from builders.google.com, June 2022

Select a mission that you’ve got created in Google Developer Console.

google developer console api projectPicture from builders.google.com, June 2022

Allow the PageSpeed Insights API on that particular mission.

page speed insights api enablePicture from builders.google.com, June 2022

You will want to make use of the particular API Key in your API Requests.

2. Import The Crucial Libraries

Use the traces under to import the elemental libraries.

    import advertools as adv
    import pandas as pd
    import requests
    import json
    from datetime import datetime
    from urllib.parse import urlparse

3. Parse The URL For The Check Topic Web site

To parse the URL of the topic web site, use the code construction under.

  area = urlparse(sitemap_url)
  area = area.netloc.cut up(".")[1]

The “area” variable is the parsed model of the sitemap URL.

The “netloc” represents the particular URL’s area part. After we cut up it with the “.” it takes the “center part” which represents the area identify.

Right here, “0” is for “www,” “1” for “area identify,” and “2” is for “area extension,” if we cut up it with “.”

4. Take The Date Of Second For File Identify

To take the date of the particular perform name second, use the “datetime.now” technique.

Datetime.now gives the particular time of the particular second. Use the “strftime” with the “%Y”, “”%m”, and “%d” values. “%Y” is for the 12 months. The “%m” and “%d” are numeric values for the particular month and the day.

 date = datetime.now().strftime("%Y_percentm_percentd")

5. Take URLs Into A Record From A Sitemap

To take the URLs into an inventory type from a sitemap file, use the code block under.

   sitemap = adv.sitemap_to_df(sitemap_url)
   sitemap_urls = sitemap["loc"].to_list()

For those who learn the Python Sitemap Well being Audit, you possibly can be taught additional details about the sitemaps.

6. Select The Metrics That You Need From PSI API

To decide on the PSI API response JSON properties, it’s best to see the JSON file itself.

It’s extremely related to the studying, parsing, and flattening of JSON objects.

It’s even associated to Semantic search engine marketing, because of the idea of “directed graph,” and “JSON-LD” structured knowledge.

On this article, we received’t give attention to inspecting the particular PSI API Response’s JSON hierarchies.

You may see the metrics that I’ve chosen to assemble from PSI API. It’s richer than the fundamental default output of PSI API, which solely provides the Core Internet Vitals Metrics, or Pace Index-Interplay to Subsequent Paint, Time to First Byte, and First Contentful Paint.

In fact, it additionally provides “solutions” by saying “Keep away from Chaining Vital Requests,” however there isn’t a must put a sentence into an information body.

Sooner or later, these solutions, and even each particular person chain occasion, their KB and MS values may be taken right into a single column with the identify “psi_suggestions.”

For a begin, you possibly can examine the metrics that I’ve chosen, and an necessary quantity of them shall be first for you.

PSI API Metrics, the primary part is under.

    fid = []
    lcp = []
    cls_ = []
    url = []
    fcp = []
    performance_score = []
    total_tasks = []
    total_tasks_time = []
    long_tasks = []
    dom_size = []
    maximum_dom_depth = []
    maximum_child_element = []
    observed_fcp  = []
    observed_fid = []
    observed_lcp = []
    observed_cls = []
    observed_fp = []
    observed_fmp = []
    observed_dom_content_loaded = []
    observed_speed_index = []
    observed_total_blocking_time = []
    observed_first_visual_change = []
    observed_last_visual_change = []
    observed_tti = []
    observed_max_potential_fid = []

This part consists of all of the noticed and simulated basic web page velocity metrics, together with some non-fundamental ones, like “DOM Content material Loaded,” or “First Significant Paint.”

The second part of PSI Metrics focuses on attainable byte and time financial savings from the unused code quantity.

    render_blocking_resources_ms_save = []
    unused_javascript_ms_save = []
    unused_javascript_byte_save = []
    unused_css_rules_ms_save = []
    unused_css_rules_bytes_save = []

A 3rd part of the PSI metrics focuses on server response time, responsive picture utilization advantages, or not, utilizing harms.

    possible_server_response_time_saving = []
    possible_responsive_image_ms_save = []

Notice: Total Efficiency Rating comes from “performance_score.”

7. Create A For Loop For Taking The API Response For All URLs

The for loop is to take the entire URLs from the sitemap file and use the PSI API for all of them one after the other. The for loop for PSI API automation has a number of sections.

The primary part of the PSI API for loop begins with duplicate URL prevention.

Within the sitemaps, you possibly can see a URL that seems a number of occasions. This part prevents it.

for i in sitemap_urls[:9]:
         # Stop the duplicate "/" trailing slash URL requests to override the knowledge.
         if i.endswith("/"):
               r = requests.get(f"https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url={i}&technique=cellular&locale=en&key={api_key}")
               r = requests.get(f"https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url={i}/&technique=cellular&locale=en&key={api_key}")

Keep in mind to examine the “api_key” on the finish of the endpoint for PageSpeed Insights API.

Test the standing code. Within the sitemaps, there is likely to be non-200 standing code URLs; these needs to be cleaned.

         if r.status_code == 200:
               data_ = json.masses(r.textual content)

The following part appends the particular metrics to the particular dictionary that now we have created earlier than “_data.”

               performance_score.append(data_["lighthouseResult"]["categories"]["performance"]["score"] * 100)

Subsequent part focuses on “complete job” depend, and DOM Dimension.


The following part takes the “DOM Depth” and “Deepest DOM Factor.”


The following part takes the particular noticed take a look at outcomes throughout our Web page Pace Insights API.


The following part takes the Unused Code quantity and the wasted bytes, in milliseconds together with the render-blocking sources.


The following part is to supply responsive picture advantages and server response timing.


The following part is to make the perform proceed to work in case there’s an error.


Instance Utilization Of Web page Pace Insights API With Python For Bulk Testing

To make use of the particular code blocks, put them right into a Python perform.

Run the script, and you’re going to get 29 web page speed-related metrics within the columns under.

pagespeed insights apiScreenshot from creator, June 2022


PageSpeed Insights API gives various kinds of web page loading efficiency metrics.

It demonstrates how Google engineers understand the idea of web page loading efficiency, and presumably use these metrics as a rating, UX, and quality-understanding perspective.

Utilizing Python for bulk web page velocity exams provides you a snapshot of the complete web site to assist analyze the attainable person expertise, crawl effectivity, conversion price, and rating enhancements.

Extra sources:

Featured Picture: Dundanim/Shutterstock



Please enter your comment!
Please enter your name here