Logging
Posit Connect provides a structured logging implementation. Structured logging provides well-defined log formatting and flexible configuration, making it well-suited for log aggregators like Datadog or Splunk.
Logging configuration is managed by the Logging
configuration section and the examples below illustrate possible logging configurations.
In this example, service logs will be emitted as JSON
to STDOUT
:
; /etc/rstudio-connect/rstudio-connect.gcfg
[Logging]
ServiceLogFormat = JSON
ServiceLog = STDOUT
In this example, service logs will be emitted as TEXT
to the specified logfile, and access logs will be emitted as JSON
to STDERR
.
; /etc/rstudio-connect/rstudio-connect.gcfg
[Logging]
ServiceLogFormat = TEXT
ServiceLog = /var/log/logfile.log
AccessLogFormat = JSON
AccessLog = STDERR
A full list of structured logging configuration options is provided in the Appendix.
Using environment variables to configure logging
Similar to how it works for other configuration sections, logging can be configured using special environment variables.
For logging, however, Posit Connect follows a different variable precedence such that the environment variables specified at runtime will take precedence over the variables defined in a configuration file that gets passed to Posit Connect at runtime. This is done in order to properly configure logging in Posit Connect very early during startup.
The following example sets the respective environment variables needed to change output, format and level of the Service Log:
CONNECT_LOGGING_SERVICELOG=service.log
CONNECT_LOGGING_SERVICELOGFORMAT=JSON
CONNECT_LOGGING_SERVICELOGLEVEL=WARN
A more detailed description on how to use environment variables for configuring Posit Connect is provided in the Appendix.
Logging to STDOUT
/STDERR
Logging to STDOUT
and/or STDERR
may be useful when using Kubernetes to forward logs to a centralized location, or when using systemd
to aggregate logs in a single-host environment.
An example configuration which outputs logs to STDOUT
and/or STDERR
is provided below. In this example, access logs are emitted to STDERR
:
; /etc/rstudio-connect/rstudio-connect.gcfg
[Logging]
AccessLog = STDERR
Logging to a file
Migrating to structured logging using a logfile requires either:
Updating aggregation tooling to ingest logfiles from the new default locations listed in the Appendix.
Configuring Posit Connect to write logfiles to the basic logging locations, allowing aggregation tooling to continue to ingest logs without modification.
An example configuration which specifies a non-default logfile is provided below. In this example, access logs are emitted to a logfile at /var/logs/non-default.log
:
; /etc/rstudio-connect/rstudio-connect.gcfg
[Logging]
AccessLog = /var/logs/non-default.log
Logrotate
If logrotate
is available when Posit Connect is installed, a logrotate configuration will also be installed.
Each log type has a different default rotation frequency and retention period:
Log Type | Default Rotation Frequency | Default Retention Period |
---|---|---|
Service Logs | Daily | 30 Days |
Access Logs | Daily | 30 Days |
Audit Logs | Monthly | 12 Months |
When a logfile is rotated, the old logfile will be compressed and stored alongside the original log file with a .1.gz extension (then .2.gz, etc.). Compressed logs are then maintained for the retention period.
The default logrotate
configuration is provided below:
/var/log/rstudio/rstudio-connect/rstudio-connect.log {
rotate 30
daily
copytruncate
compress
delaycompress
notifempty
missingok
su root root
}
/var/log/rstudio/rstudio-connect/rstudio-connect.access.log {
rotate 30
daily
copytruncate
compress
delaycompress
notifempty
missingok
su root root
}
/var/log/rstudio/rstudio-connect/rstudio-connect.audit.log {
rotate 12
monthly
copytruncate
compress
delaycompress
missingok
su root root }
If you configure Posit Connect to write to a different file, you will need to update the logrotate
configuration accordingly.
Permissions
For some log aggregators (such as Datadog or CloudWatch), file permissions may need to be adjusted. Logfile(s) must be at least 0644
(group/world readable) and their enclosing directory must be at least 0755
(group/world searchable).
Alternatively, the aggregation agent could be added to a specific group or ACL, but this approach will be particular to the aggregation agent and distro.
Structured Logs
Structured logging emits three types of log events: Service Logs, Access Logs, and Audit Logs. Debug Logs are troubleshooting-oriented service logs that can be enabled for specific services. Application Logs are not affected when structured logging is enabled.
When modifying the format of structured logs, be sure to clear the output target location to avoid mixing log formats. For example, you may want to clear the service logfile before switching from TEXT
to JSON
or vice versa.
Service Logs
If you are using the official Posit Connect docker image, the server logs are written to stdout
by default and are not available at the default log path /var/log/rstudio/rstudio-connect/rstudio-connect.log
. If you prefer to log to a file, you will need to modify the value of the Logging.ServiceLog
setting.
Service logs capture the operating behavior of Posit Connect.
By default, service logs are written to /var/log/rstudio/rstudio-connect/rstudio-connect.log
in TEXT
format. The default logfile is owned by root
with permissions 0600
.
In Posit Connect editions prior to 2022.06
, basic logging emitted service logs to /var/log/rstudio-connect.log
by default. This differs from the current default for both basic and structured logging.
The following service log configuration options are available. See the Appendix for further details.
Configuration Parameter | Description |
---|---|
Logging.ServiceLog |
Output destination |
Logging.ServiceLogLevel |
Minimum severity level for logging messages |
Logging.ServiceLogFormat |
Output format |
Service Log Format
Regardless of format, service logs contain the following mandatory fields:
Field | Description |
---|---|
level |
Log level (info , warn , etc) |
msg |
Context-specific content describing the state of the service |
time |
Timestamp indicating when the log was generated |
Additional fields can be attached to service logs to provide service-specific contextual information.
An example service log in JSON
format is provided below:
{
"level":"info",
"msg":"Startup complete (~4.548667s)",
"time":"2022-04-07T15:38:39.472Z"
}
Here is an equivalent service log in TEXT
format, where the log entry’s fields are space-delimited <key>=<value>
pairs:
time="2022-04-11T17:01:34.465Z" level=info msg="Startup complete (~4.8366369s)"
Service Log Level
The service log level determines the minimum severity of service log entries. It is configured with the Logging.ServiceLogLevel
property. The default log level is INFO
, and available log level settings (in order of severity) are INFO
, WARN
and ERROR
.
Debug Logs
Debug logs provide troubleshooting information about specific services. They are enabled by setting the Logging
property to true
within the service’s configuration section. Available debug log properties are provided below:
TableauIntegration.Logging
LDAP.Logging
PAM.Logging
ProxyAuth.Logging
OAuth2.Logging
SAML.Logging
For example, the following configuration would enable debug logging for an LDAP auth workflow:
; /etc/rstudio-connect/rstudio-connect.gcfg
[LDAP]
Logging = true
Debug logs are a specific subset of service logs with the field level
set to debug
. Therefore, their formatting and output location are configured by Logging.ServiceLogFormat
and Logging.ServiceLog
, respectively.
Setting a debug log overrides any explicitly set Logging.ServiceLogLevel
configuration and forces the log level to DEBUG
. The DEBUG
log level is highly permissive and will record INFO
, WARN
, and ERROR
logs.
Access Logs
Access logs capture information about client requests received by Posit Connect.
Access log timestamps are generated when a request is received by Posit Connect, but access events are not written to the log stream until the response completes. Cross-comparing an access log stream’s sequential ordering with the timestamp of each logged event may appear non-chronological. This is an expected side effect of unique requests requiring different amounts of processing time.
By default, access logs are written to /var/log/rstudio/rstudio-connect/rstudio-connect.access.log
in COMMON
format. The default logfile is owned by root
with permissions 0600
.
The following access log configuration options are available. See the Appendix for further details.
Configuration Parameter | Description |
---|---|
Logging.AccessLog |
Output destination |
Logging.AccessLogFormat |
Output format |
Access Log Format
Posit Connect can format access logs according to three standards:
Structured JSON Format
Access logs which are formatted as structured JSON
contain the following fields:
Field | Description |
---|---|
type |
Log type (access) |
level |
Log level (info , warn , etc) |
time |
Timestamp indicating when request was received by Posit Connect |
endpoint |
Requested API endpoint |
method |
HTTP method attached to request |
size |
Response size |
status |
HTTP response status code |
An example access log in Posit Connect’s structured JSON
format is provided below:
{
"type":"access",
"level":"info",
"time":"2022-04-07T18:29:07.8109901Z",
"endpoint":"/connect/",
"method":"GET",
"size":2306,
"status":200
}
Common Format
See Apache’s Common Log Format documentation for a full description of the COMMON
log formatting standard.
An example COMMON
access log is provided below:
172.17.0.1 - - [11/Apr/2022:17:08:38 +0000] "GET /connect/ HTTP/1.1" 200 2306
Combined Format
See Apache’s Combined Log Format documentation for a full description of the COMBINED
log formatting standard.
An example COMBINED
access log is provided below:
172.17.0.1 - - [11/Apr/2022:17:10:33 +0000] "GET /connect HTTP/1.1" 307 45 "" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36"
Audit Logs
Audit logs capture information about the set of events defined by Posit Connect’s auditing system.
Audit logfiles contain actions performed through a specific Posit Connect server. In a clustered installation, the auditing files from each server need to be combined.
The Connect dashboard, Server API, and usermanager
tool present all auditing information, not per-server records.
The auditing system cannot be disabled and always records changes to the Posit Connect database (even when an optional log file is configured). Database audit entries are maintained indefinitely.
By default, audit logs are only recorded in the database. Admin users are able to view audit logs from the database in the dashboard.
The following audit log configuration options are available. See the Appendix for further details.
Configuration Parameter | Description |
---|---|
Logging.AuditLog |
Output destination |
Logging.AuditLogFormat |
Output format |
Audit Log Format
When logged to a file, audit logs can be in either TEXT
or JSON
formats. See the Appendix for more details about audit logs format types.
Note that different formats will always show the same fields. See the Audited Events page for more details about all available fields.
An example audit log in JSON
format is provided below:
{
"type":"audit",
"level":"info",
"action":"user_login",
"entry_id": 1,
"msg":"New session by user: 123-456-789 (user@posit.com)",
"time":"2022-04-07T18:47:21.769Z",
"user_description":"user_description",
"user_id":1
}
Here is an equivalent audit log in TEXT
format, where fields are space-delimited <key>=<value>
pairs:
time="2022-04-11T17:05:33.887Z" level=info msg="New session by user: 123-456-789 (user@posit.com)" action=user_login entry_id=1 type=audit user_description="user_description" user_id=1