Throttling a Source

Sometimes, you may want to slow a data feed for testing purposes. In these cases, you can throttle your source – slow it to a specified number of rows per second; but you can adjust this default rate in project settings.

To throttle a source:

  1. Open the source from the Sources page.
  2. Click the throttle button in the top right of the window.
  3. The button changes to read “Throttled to 1 Row/Second”:
  4. To revert the source to normal (unthrottled) speed, click the button again.

Managing Throttling on the Sources Page

You can use the Throttle All and Unthrottle All buttons on the Sources page to change throttling for all sources at once.

To change throttling for individual sources once you have added them, click the rocket icon in the upper right hand corner of the source:

Export SQL

When you export the SQL, the throttling code - both for sources and for individual pipelines - can be removed (optionally). Follow these steps:

  1. Go to the SQL page using the scroll button
  2. Click on the “Export” button. Three options are available
    • Export with throttling removed (recommended)
    • Export with throttling included
    • As shown, including errors as comments

NOTE: You don’t get the same options when exporting or saving the project. If you plan to deploy your SQL by extracting it from the project or slab file json, you will have to make sure that throttling has already been removed and the project SQL has been re-created before it is saved.

Setting Project Standard Rate

Throttling of all sources is enabled or disabled using the buttons “Import SQLStream schema” , “Throttle all” and “Unthrottle all” available in the bottom left of the Sources page.

On the Project Settings page, you can also set the standard rate as required.

Note: StreamLab’s “Project standard throttle rate” refers to the rows/second rate, not the throttleScale parameter used by the underlying function).

Adding a Throttle to a Pipeline

For development and testing purpose, you can also use the “Throttle” guide operation to slow down the data flow. For more information, see the Throttle command.