Sure, you can tell us to run a crawler periodically by adding a "Schedule". You can schedule crawlers to run in Monthly, Weekly, Hourly or Minute intervals.
Some crawlers can collect multiple records from a single page, while others might need to go to 3 pages to get a single record. For example our Amazon Bestsellers crawler collects 50 records from one page, while our Indeed crawler needs to go through ...
The crawler works by sending requests simultaneously to get the data. The data needs to be processed and is not displayed instantly, it takes some time to display the data, so please be patient and check back periodically. This is especially the case ...
Our APIs or Crawlers only automate the process of extracting the data that's already on the website and is extractable. If the information isn't on the website or is in a format that is not extractable, we will not be able to get it for you at all. ...
Our Google maps crawler needs to go through a list of all businesses and then go into each business' detail page to get additional data about each business such as their telephone, address, open hours, website address, etc. Here is the business ...
When you subscribe to any paid plan here are the credit sharing options on based on whether the plan is for a Crawler or an API Crawlers You can pay for one plan and add as many crawlers as you like and share the payment and the plan across all ...