This topic contains information on tracing and within s-Server including the following:
SQLstream uses the standard Java logging interface (defined in the package java.util.logging) to trace execution for debugging purposes. Trace files can let you troubleshoot stream functioning during development. Errors logged to the trace log are also available in the global error stream. See the Streaming SQL Reference Guide for more details.
All errors that are logged to the trace log are also available in a system management stream called sys_boot.mgmt.ALL_TRACE. The latest (up to 1000) error messages are also available in a table called sys_boot.mgmt.SAVED_TRACE. You can use these to create visualizations of errors; they can be accessed from SQLline or anywhere that has a JDBC connection to s-Server. See the Global Error Stream for more details.
Trace files and ALL_TRACE / SAVED_TRACE also include Periodic Parser Statistics.
Trace files are useful in troubleshooting streams, because they allow you to see when a stream failed and why.
s-Server has two trace logs. The first is the most useful in troubleshooting streams.
To control the depth of the logging and the formatting of the output, Java logging uses external configuration files in the standard Java properties format. You determine the location of these files during installation. By default, SQLstream uses the following logging configuration files:
Component | Configuration File | Default Trace Log File(s) |
---|---|---|
SQLstream s-Server | /var/log/sqlstream/Trace.properties | /var/log/sqlstream//Trace.log/var/log/sqlstream/ApplicationTrace.log |
SQLstream s-Studio and other client tools | /var/log/sqlstream/ClientTrace.properties | /var/log/sqlstream/ClientTrace.log |
The SQLstream s-Server configuration controls tracing to three different log files:
You can modify the properties files to change the default logging. Before doing so, you should save a copy of the original, unmodified file.
Note: The installed files are made read-only by default so as to avoid accidental modification.
Because trace files are intended for debugging, by default, very little is logged to these files in normal conditions.
For debugging you can change the default tracing level by editing the Trace.properties or ClientTrace.properties file. To view more logging, open one of the properties files in a text editor and uncomment (remove the “#") lines such as the following (both from ClientTrace.properties):
# com.sqlstream.aspen.jdbc.level=FINE
# com.sqlstream.aspen.sdp.level=FINE
You can also change the tracing level for a particular package or class. For example, suppose you created a set of plugins that you want to debug at a high level of tracing. If the plugins are all part of a package called com.example.plugin, you can add the following to your Trace.properties:
com.example.plugin.level=FINEST
You may also change the logging level from the SQL command line (or as part of a SQL script) by using the SETTRACELEVEL procedure. This can be more convenient than updating the Trace.properties file and waiting for the new levels to take effect. To repeat the previous example:
CALL SETTRACELEVEL('com.example.plugin.level','FINEST');
See also the topic Error Logging in the Streaming SQL Reference Guide.
The default tracing configurations are intentionally set for production mode, with limits placed on the maximum size of a trace file, and the number of times the file will be rotated, controlled by these properties in Trace.properties::
java.util.logging.FileHandler.limit=10000000
java.util.logging.FileHandler.count=20
For testing, it is can be helpful to change these settings to allow more information to be retained in trace files. Just be careful that your tracefiles do not use up all available disk space.
To control the maximum file size and rollover count, change the following properties in the file /var/log/sqlstream/Trace.properties:
Trace.log | Property Name | Default |
---|---|---|
File size | java.util.logging.FileHandler.limit | 10000000 bytes |
Rollover count | java.util.logging.FileHandler.count | 20 |
ApplicationTrace.log | Property Name | Default |
---|---|---|
File size | com.sqlstream.aspen.trace.ApplicationLogConfigurator.limit | 10000000 bytes |
Rollover count | com.sqlstream.aspen.trace.ApplicationLogConfigurator.count | 20 |
For SQLstream s-Studio, use the following properties in /var/log/sqlstream/ClientTrace.properties:
ClientTrace.log | Property Name | Default |
---|---|---|
File size | java.util.logging.FileHandler.limit | 10000000 bytes |
Rollover count | java.util.logging.FileHandler.count | 20 |
If you set file size limit to zero, the file will never roll over, even if a rollover count is set.
You can override which logging configuration file SQLstream uses by overriding the value for the property java.util.logging.config.file. To do so, set the property in aspen.custom.properties.
$SQLSTREAM_HOME/aspen.custom.properties.
If this file is not present, you may need to create it or copy it from $SQLSTREAM_HOME/support.
To change the user interface logging configuration file, set the property in
$HOME/.SQLstreamrc.
For example, to use the file /usr/local/MyTrace.properties for user interface logging configuration, add the following line to the .SQLstreamrc file in your home directory:
java.util.logging.config.file=/usr/local/MyTrace.properties
The trace log contains information on how parsers are reading data from external sources such as log files, Kafka, Amazon Kinesis, and AMQP. This is known as Periodic Parser Statistics Logging. These sources are read through the Extensible Common Data Adapter. For more information on parsing data, see the topic Reading Data from Other Sources in the Integrating Guavus SQLstream with Other Systems guide.
You can control how often s-Server logs parser statistics with the following entry in Trace.properties:
com.sqlstream.aspen.namespace.common.input.logPeriod=10
This value determines how often in minutes that s-Server logs parser statistics. By default it is set to every 10 minutes.
For each Extensible Common Data reader open, the trace log lists information on the following:
Statistic | Description |
---|---|
Throughput | Bytes per second of input processed over the previous period. |
Runnable | Percent of time that the parser was running (or runnable but waiting for an available core to run in). |
Blocked waiting for input | Percentage of time that the parser was not runnable because it was waiting for more input to process. |
Blocked waiting for downstream writes | Percentage of time that the parser was not runnable because of backpressure from downstream processing. |
Statistics on runnable and blocked parsers are approximated based on sampling log parser statistics. You can control sampling and reporting by changing the following parameters in Trace.properties:
Parameter | Definition |
---|---|
com.sqlstream.aspen.namespace.common.input.logPeriod | How often in minutes to log parser statistics. Defaults to 10. Set to 0 to disable logging. |
com.sqlstream.aspen.namespace.common.input.probeFrequency | How may times per logPeriod should parser statistics be sampled. Defaults to 100. |
INFO: Parser for LOCALDB.TEST_READER.P_Pump: throughput=12598532bps, runnable 20%, blocked waiting for input 80%, blocked waiting to write downstream 0%