The throttle UDX allows you to slow down data flows; this can be very helpful during development and testing to allow a developer to monitor data as it flows through s-Server. This UDX is also used by StreamLab – see Throttling a Source.
You can create the function that references the UDX using the following code:
CREATE OR REPLACE FUNCTION "throttle"
(inputRows CURSOR, throttleScale int)
RETURNS TABLE(inputRows.*)
LANGUAGE JAVA
PARAMETER STYLE SYSTEM DEFINED JAVA
NO SQL
EXTERNAL NAME 'class:com.sqlstream.plugin.timesync.ThrottleStream.throttle';
The UDX passes through all columns from the source cursor. Data columns are unchanged, but the ROWTIME is updated to System.currentTimeMillis(). So normally you would want to place this throttling on data that has an implicit ROWTIME – upstream from any step that promotes an data event time to ROWTIME.
In between each row, the UDX sleeps for throttleScale milliseconds. So if you set the throttleScale to 500, the next row will be emitted after half a second (if there is a next row available) or as soon as it arrives after that wait.
The larger you set throttleScale, the lower the record throughput will be.
If you want to restrict to a specific rows/second (rps) rate, you need to invert it:
throttleScale = 1000/rps
If you set throttleScale to 1, you will get the maximum of 1000 rows/second. More usefully:
throttleScale | rows/second |
---|---|
1 | 1000 |
2 | 500 |
3 | 333 |
4 | 250 |
5 | 200 |
6 | 166 |
7 | 143 |
8 | 125 |
9 | 111 |
10 | 100 |
20 | 50 |
40 | 25 |
50 | 20 |
100 | 10 |
200 | 5 |
250 | 4 |
333 | 3 |
500 | 2 |
1000 | 1 |
This UDX is also used for throttling in StreamLab. For more information, visit Throttling a source.