Static user agent for web import
Ada’s web import scraper now identifies itself with a static User-Agent: AdaWebCrawler header on every request. Websites that restrict crawling with an allowlist can add AdaWebCrawler to keep Ada’s scraper authorized — previously the user agent was dynamic and could not be reliably allowlisted.
Learn more in our Web import documentation.