Observability

One of Approzium's primary goals is a high level of security-oriented observability.

Logging

Every database authentication request is assigned a request_id, which is included in all logs generated by the request. We recommend you set "APPROZIUM_LOG_FORMAT=json" and "APPROZIUM_LOG_LEVEL=info" when running Approzium. This will result in the following:

• Logs will be outputted in a JSON format that's easily consumable by log viewing tools such as the ELK stack and DataDog.
• The request_id field may easily be searched for a request's full lifecycle.
• If the caller's identity for the request_id was confirmed, it will be logged.
• Any malicious-looking activity that occurred in the request will be logged at the WARN level.

Metrics

Metrics are exposed at the /v1/metrics/prometheus endpoint.

Performance-Centric Metrics

These metrics help understand performance. The number of requests, responses, and error responses help describe the load Approzium is under. Total request milliseconds is an end-to-end metric regarding how long Approzium handles a request. The other request milliseconds are for helping to understand where Approzium's request time is being spent. We do execute some requests in parallel to minimize request time, so the sum of sub-request time may exceed total request time.

The AWS request being executed is the get_caller_identity call. The password request is where we call your configured back-end (HashiCorp Vault, for example) to retrieve passwords that will be used to build database connections.

Total requests and total error responses can be considered significant to security because if an attacker is attempting to DOS or fuzz Approzium, these will rise at an unexpectedly high level.

METRIC	IS SECURITY RELATED	TYPE	DESCRIPTION
approzium_total_requests	yes	counter	Total number of requests
approzium_total_responses	no	counter	Total number of responses
approzium_total_error_responses	yes	counter	Total number of error responses
approzium_total_request_milliseconds	no	gauge	Total request milliseconds
approzium_total_aws_request_milliseconds	no	gauge	Total AWS request milliseconds
approzium_total_password_request_milliseconds	no	gauge	Total password retrieval milliseconds

Security-Centric Metrics

These metrics are focused on understanding whether your system may be under attack.

When the Approzium SDK attempts to gain a database connection, it must send the Approzium server a few things:

1. The identity it claims to be. This is sent in advance so Approzium can minimize request time by getting database creds while simultaneously confirming the caller's identity. It will only return the connection if they match.
2. Proof of its actual identity. For example, for AWS, a signed get caller identity request is sent, which includes a cryptographically signed string that AWS can use to confirm the caller's identity.
3. The database to which it wants access. The database requested is fully at the caller's discretion, but they must have been granted access in advance to receive it.

So, when an attacker begins trying to get something they shouldn't, the following metrics will be incremented.

METRIC	IS SECURITY RELATED	TYPE	DESCRIPTION
approzium_total_identity_matching_attempts	maybe	counter	Total checks of whether the identity a caller claims matches their actual identity
approzium_total_identity_matching_failures	yes	counter	Total calls where caller claimed an identity that was not their actual identity
approzium_total_identity_verification_attempts	maybe	counter	Total attempts to verify caller identity
approzium_total_identity_verification_failures	yes	counter	Total failures to verify caller identity
approzium_total_password_retrieval_attempts	maybe	counter	The number of times a caller has requested a password from the database to authenticate
approzium_total_password_retrieval_failures	yes	counter	The number of times a caller has failed to retrieve a password for any reason
approzium_total_password_retrieval_unauthorized	yes	counter	The number of times a caller has been unauthorized to retrieve a password

If the caller is attempting to get around 1, claiming to be someone they're not, approzium_total_identity_matching_failures will rise. We also provide approzium_total_identity_matching_attempts in case you prefer to observe this as a ratio.

If the caller is attempting to get around 2, providing invalid proof of their identity, approzium_total_identity_verification_failures will rise. We also provide approzium_total_identity_verification_attempts in case you prefer to observe this as a ratio.

If the caller is attempting to get around 3, requesting access to a database they shouldn't have, approzium_total_password_retrieval_unauthorized will rise. However, there's a potential second vector here - they could be simply combing to see whether values exist for other databases. That's why we also provide approzium_total_password_retrieval_failures, which will rise in such a case. These both also offer approzium_total_password_retrieval_attempts if a ratio is preferred.

Tracing

Coming soon!