How to bulk check redirects?

This article explores both methods, including how to automate your workflow using tools like Make and the dedicated HttpStatus app.

Using the web interface to bulk check redirects

The web interface is a straightforward way to check redirects and track HTTP status codes for up to 100 URLs per batch. While it doesn't support file uploads, it's a user-friendly solution for smaller datasets but requires some manual work.

Workflow for the web interface

1. Prepare a list of URLs: Copy a list of up to 100 URLs from a plain text file or spreadsheet.
2. Paste URLs into the web interface: Visit the web interface and paste your URLs directly into the textarea and click the 'Check status' button.
3. Analyze redirect chains: After submitting your URLs, the web interface will display (to open a detailed view, click the blue arrow to the left of the request URL.):
   • HTTP status codes for each request URL (e.g., 301, 404, 200)
   • Intermediate redirects for each URL
   • The final destination URL, providing full visibility of the redirect chain.
4. Export to Google Sheets: Use the "Export to Sheets" button to export (filtered) request URLs, status codes, locations and error messages to a Google Sheets document.
   

   NOTE  To activate the "Export to Sheets" button you have open the Settings screen and click on the 'Data export' tab. Add the URL of your Google Sheets document and grant HttpStatus access to your Google Sheets document as an 'Editor'. This 'Editor' role is only used to add data, no data is retrieved or read from the document.

5. Repeat: To perform additional bulk checks and add their results to the same Google Sheets document, repeat the previous steps.

Why you need to throttle requests?

When you check multiple URLs from the same host or domain, HttpStatus sends multiple requests to the same host at once. This can cause the server to become overloaded if it does not have enough resources to respond to all the requests.

This can have a number of consequences:

• It often triggers a rate limiter to send a '429 Too Many Requests' response to prevent congestion.
• If the server has insufficient resources and no rate limiter is configured, it may not be able to handle the number of requests and will send no response at all, causing a timeout for each request.
• As it takes more resources and time for the server to process the requests, for servers with limited resources, you will notice that for redirects within the same host, the redirect latency for those redirects will increase.

When checking URLs from different hosts, HttpStatus sends requests to several hosts at once. If the server(s) behind one or a few URLs do not respond in time, timeout errors will occur for those URLs, and sometimes for all subsequent URLs.

To prevent a host or domain's web server from throwing errors or responding with status code 429, you can limit the number of requests HttpStatus sends at one time using the 'Throttle requests' option in the settings.

By default, 100 requests are sent simultaneously. Limit the number of requests if you want to check many URLs for the same host or domain. The maximum number of simultaneous requests can be set to 100, 10, 5, 2 and 1 request(s) per second.

Automating redirect checks with the API

For larger datasets or continuous monitoring, the API offers greater flexibility. While the API processes one URL per request, it's highly adaptable for use in automated workflows. Users often combine the API with automation tools like Make to scale their operations effectively.

Streamline automation with Make and the HttpStatus application

What is Make?

Make (formerly Integromat) is a no-code platform that allows you to automate workflows by connecting different apps and services. HttpStatus has developed a dedicated app for Make, enabling users to integrate the API into their workflows without needing to write code.

How to use the HttpStatus app on Make

There is a detailed tutorial for getting started with Make and the HttpStatus app. Here's a high-level overview:

1. Design your workflow:
   • Use Make's drag-and-drop interface to create a workflow (scenario).
   • For example, set up a trigger to pull URLs from a Google Sheet, database, or other source.
2. Set up the HttpStatus app:
   • Install the app.
   • Access the HttpStatus app in Make.
   • Connect your HttpStatus API key to the app.
3. Automate redirect checks:
   • Use the HttpStatus app to process each URL sequentially.
   • Retrieve the redirect chain, HTTP status codes, and destination URL for every request.
4. Save and analyse results:
   • Output the API response to a Google Sheet, database, or other application for further analysis.
   • Schedule the workflow to run at regular intervals for continuous monitoring.

Web interface vs API: choosing the right approach

HttpStatus offers powerful tools for checking URL redirects, whether you're managing a small batch manually or processing thousands of URLs through automation.

• Use the web interface if you need a quick and simple solution for up to 100 URLs per batch.
• Leverage the API with the Make integration for automated and scalable workflows, especially for continuous or large-scale redirect checks.

Ready to get started? Explore the web interface, try the HttpStatus app for Make, and take your redirect management to the next level!