The default logging configuration will store the logs locally, on the
amphora filesystem with file rotation.

Octavia Amphorae can offload their log files via the syslog protocol
to syslog receivers via the load balancer management network
(lb-mgmt-net). This allows log aggregation of both administrative logs
and also tenant traffic flow logs. The syslog receivers can either be
local to the load balancer management network or routable via the load
balancer management network. By default any syslog receiver that
supports UDP or TCP syslog protocol can be used, however the operator
also has the option to create an override rsyslog configuration template
to enable other features or protocols their Amphora image may
support.

This guide will discuss the features of Amphora log offloading and
how to configure them.

Administrative Logs

The administrative log offloading feature of the Amphora covers all of the
system logging inside the Amphora except for the tenant flow logs. Tenant flow
logs can be sent to and processed by the same syslog receiver used by
the administrative logs, but they are configured separately.

All administrative log messages will be sent using the native log
format for the application sending the message.

Enabling Administrative
Log Offloading

One or more syslog receiver endpoints must be configured in the
Octavia configuration file to enable administrative log offloading. The
first endpoint will be the primary endpoint to receive the syslog
packets. Read the rsyslog failover considerations section for
information about how to use multiple target servers.

To configure administrative log offloading, set the following setting
in your Octavia configuration file for all of the controllers and
restart them:

[amphora_agent]
admin_log_targets = 192.0.2.1:10514

In this example, the syslog receiver will be 192.0.2.1 on port 10514.
If log_protocol is not specified UDP will be used.

The load balancer related administrative logs will be sent using a
LOG_LOCAL[0-7] facility. The facility number defaults to 1, but is
configurable using the administrative_log_facility setting in the
Octavia configuration file.

To configure administrative log facility, set the following setting
in your Octavia configuration file for all of the controllers and
restart them:

[amphora_agent]
administrative_log_facility = 1

Forwarding All
Administrative Logs

By default, the Amphorae will only forward load balancer related
administrative logs, such as the haproxy admin logs, keepalived, and
Amphora agent logs.
You can optionally configure the Amphorae to send all of the
administrative logs from the Amphora, such as the kernel, system, and security
logs. Even with this setting the tenant flow logs will not be included.
You can configure tenant flow log forwarding in the Tenant Flow Logs section.

The load balancer related administrative logs will be sent using the
LOG_LOCAL[0-7] configured using the administrative_log_facility setting.
All other administrative log messages will use their native syslog
facilities.

To configure the Amphorae to forward all administrative logs, set the
following setting in your Octavia configuration file for all of the
controllers and restart them:

[amphora_agent]
forward_all_logs = True

Tenant Flow Logs

Enabling Tenant Flow Log
Offloading

One or more syslog receiver endpoints must be configured in the
Octavia configuration file to enable tenant flow log offloading. The
first endpoint will be the primary endpoint to receive the syslog
packets. The endpoints configured for tenant flow log offloading may be
the same endpoints as the administrative log offloading configuration.
Read the rsyslog failover considerations section for
information about how to use multiple target servers.

To configure tenant flow log offloading, set the following setting in
your Octavia configuration file for all of the controllers and restart
them:

[amphora_agent]
tenant_log_targets = 192.0.2.1:10514

In this example, the syslog receiver will be 192.0.2.1 on port 10514.
If log_protocol is not specified UDP will be used.

The load balancer related tenant flow logs will be sent using a
LOG_LOCAL[0-7] facility. The facility number defaults to 0, but is
configurable using the user_log_facility setting in the Octavia
configuration file.

To configure the tenant flow log facility, set the following setting
in your Octavia configuration file for all of the controllers and
restart them:

[amphora_agent]
user_log_facility = 0

Tenant Flow Log Format

The default tenant flow log format is:

project_id loadbalancer_id listener_id client_ip client_port data_time
request_string http_status bytes_read bytes_uploaded
client_certificate_verify(0 or 1) client_certificate_distinguised_name
pool_id member_id processing_time(ms) termination_state

Any field that is unknown or not applicable to the connection will
have a ‘-‘ character in its place.

An example log entry when using rsyslog as the syslog receiver
is:

Jun 12 00:44:13 amphora-3e0239c3-5496-4215-b76c-6abbe18de573 haproxy[1644]: 5408b89aa45b48c69a53dca1aaec58db fd8f23df-960b-4b12-ba62-2b1dff661ee7 261ecfc2-9e8e-4bba-9ec2-3c903459a895 172.24.4.1 41152 12/Jun/2019:00:44:13.030 "GET / HTTP/1.1" 200 76 73 - "" e37e0e04-68a3-435b-876c-cffe4f2138a4 6f2720b3-27dc-4496-9039-1aafe2fee105 4 --

Custom Tenant Flow Log
Format

You can optionally specify a custom log format for the tenant flow
logs. This string follows the HAProxy log format variables with the
exception of the “{{ project_id }}” and “{{ lb_id }}” variables that
will be replaced by the Octavia Amphora driver. These custom variables are
optional.

See the HAProxy documentation for Custom
log format variable definitions.

To configure a custom log format, set the following setting in your
Octavia configuration file for all of the controllers and restart
them:

[haproxy_amphora]
user_log_format = '{{ project_id }} {{ lb_id }} %f %ci %cp %t %{+Q}r %ST %B %U %[ssl_c_verify] %{+Q}[ssl_c_s_dn] %b %s %Tt %tsc'

Failover Considerations

In order to provide protection against potential data loss because of
downtime of a single syslog server, it may be a advisable to use
multiple log targets. In such configuration log_protocol needs
to be set to TCP. With the UDP syslog protocol, RSyslog is
unable to detect if the primary endpoint has failed.

Also pay attention to the log_retry_count and
log_retry_interval settings when using multiple log targets.
You might want to set log_retry_count to 0 and use a higher
value for log_retry_interval. Values up to 1800 (30 minutes)
are possible. That way the failover will happen immediately after the
client detects that the server became unavailable. In such case, that
server won’t be used again for at least log_retry_interval
seconds after that event. In the following example the primary syslog
receiver will be 192.0.2.1 on port 10514. The backup syslog receiver
will be 2001:db8:1::10 on port 10514.

[amphora_agent]
admin_log_targets = 192.0.2.1:10514, 2001:db8:1::10:10514
tenant_log_targets = 192.0.2.1:10514, 2001:db8:1::10:10514
log_protocol = TCP
log_retry_count = 0
log_retry_interval = 1800

Disabling Logging

There may be cases where you need to disable logging inside the Amphora, such as complying
with regulatory standards. Octavia provides multiple options for
disabling Amphora
logging.

Disable Local Log Storage

This setting stops log entries from being written to the disk inside
the Amphora. Logs can
still be sent via Amphora log offloading if log offloading is
configured for the Amphorae. Enabling this setting may provide a
performance benefit to the load balancer.

To disable local log storage in the Amphora, set the following setting in your Octavia
configuration file for all of the controllers and restart them:

[amphora_agent]
disable_local_log_storage = True

Disable Tenant Flow Logging

This setting allows you to disable tenant flow logging irrespective
of the other logging configuration settings. It will take precedent over
the other settings. When this setting is enabled, no tenant flow
(connection) logs will be written to the disk inside the Amphora or be sent via the
Amphora log
offloading.

To disable tenant flow logging, set the following setting in your
Octavia configuration file for all of the controllers and restart
them:

[haproxy_amphora]
connection_logging = False