-
Notifications
You must be signed in to change notification settings - Fork 887
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Logs API to be user-facing #4225
Changes from 11 commits
45f6847
ffe4a1b
4dadc75
90610d2
452f61b
4ab9a0a
76c4644
daaaa1f
7c609c0
847b691
3ea0a68
1409d01
bbc8ab1
3b32a2d
71c0986
22d628d
de72b99
beff5c5
6e2511d
ba78baa
d3aefef
cc0f262
47468d4
d286fc2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -203,26 +203,6 @@ Disclaimer: this list of features is still a work in progress, please refer to t | |
| Can plug custom LogRecordExporter | | | + | | + | | | + | | + | | | | ||
| Trace Context Injection | | | + | | + | | | + | | + | + | | | ||
|
||
## Events | ||
|
||
Features for the [Events API](specification/logs/event-api.md) and the [Events SDK](specification/logs/event-sdk.md). | ||
Disclaimer: Events are currently in Development status - work in progress. | ||
|
||
| Feature | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This table is (was) incomplete, several languages already have this capability and users of this API. JS, Java (Android) and recently Python |
||
|----------------------------------------------------------------------------|----------|----|------|----|--------|------|--------|-----|------|-----|------|-------| | ||
| [EventLoggerProvider](specification/logs/event-api.md#eventloggerprovider) | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | | ||
| Get EventLogger | | | | | | | | | | | | | | ||
| Get EventLogger accepts version | X | | | | | | | | | | | | | ||
| Get EventLogger accepts schema_url | X | | | | | | | | | | | | | ||
| Get EventLogger accepts attributes | X | | | | | | | | | | | | | ||
| [EventLogger](specification/logs/event-api.md#eventlogger) | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | | ||
| Emit event accepts name | | | | | | | | | | | | | | ||
| Emit event accepts AnyValue body | X | | | | | | | | | | | | | ||
| Emit event accepts severity | X | | | | | | | | | | | | | ||
| Emit event accepts timestamp | X | | | | | | | | | | | | | ||
| Emit event accepts attributes | X | | | | | | | | | | | | | ||
| Emit event accepts context | X | | | | | | | | | | | | | ||
|
||
## Resource | ||
|
||
| Feature | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -140,16 +140,18 @@ Given the above state of the logging space we took the following approach: | |
OpenTelemetry log data model. OpenTelemetry Collector can read such logs and | ||
translate them to OpenTelemetry log data model. | ||
|
||
- OpenTelemetry defines a Logs Bridge API | ||
for [emitting LogRecords](./bridge-api.md#emit-a-logrecord). Application developers are | ||
NOT encouraged to call this API directly. It is provided for library authors | ||
to build [log appender](../glossary.md#log-appender--bridge), which use | ||
the API to bridge between existing logging libraries and the OpenTelemetry log | ||
data model. Existing logging libraries generally provide a much richer set of | ||
features than what is defined in OpenTelemetry. It is NOT a goal of | ||
OpenTelemetry to ship a feature-rich logging library. | ||
|
||
- OpenTelemetry defines an [SDK](./sdk.md) implementation of the [Bridge API](./bridge-api.md), | ||
- OpenTelemetry defines a Logs API for [emitting LogRecords](./api.md#emit-a-logrecord). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This NEEDS to include comments about also emitting events as that is the intention There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is described in API and Data Model. Do we need to repeat it? Can you suggest a change? |
||
It is provided for library authors to build [log appenders](../glossary.md#log-appender--bridge), | ||
which use the API to bridge between existing logging libraries and the | ||
OpenTelemetry log data model. | ||
The [API](./api.md) is also provided for instrumentation libraries to emit log | ||
records following semantic conventions (also known as OpenTelemetry Events). | ||
pellared marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Existing logging libraries generally provide a much richer set of features | ||
than what is defined in OpenTelemetry. It is NOT a goal of OpenTelemetry to | ||
ship a feature-rich logging library. However, nothing prevents application | ||
cijothomas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
developers to use it directly. | ||
|
||
- OpenTelemetry defines an [SDK](./sdk.md) implementation of the [API](./api.md), | ||
which enables configuration of [processing](./sdk.md#logrecordprocessor) | ||
and [exporting](./sdk.md#logrecordexporter) LogRecords. | ||
|
||
|
@@ -230,10 +232,9 @@ processor. | |
### Infrastructure Logs | ||
|
||
These are logs generated by various infrastructure components, such as | ||
Kubernetes events (if you are wondering why events are discussed in the context | ||
of logs see [Event API Data model](./event-api.md#event-data-model)). Like system logs, the | ||
infrastructure logs lack a trace context and can be enriched by the resource | ||
context - information about the node, pod, container, etc. | ||
Kubernetes events. Like system logs, the infrastructure logs lack a trace | ||
context and can be enriched by the resource context - information about | ||
the node, pod, container, etc. | ||
|
||
OpenTelemetry Collector or other agents can be used to query logs from most | ||
common infrastructure controllers. | ||
|
@@ -359,7 +360,7 @@ as parsers, log tailing and rotation. It also enables the possibility to send | |
logs directly to the logging backend without using a log collection agent. | ||
|
||
To facilitate both approaches described above OpenTelemetry provides | ||
a [Bridge API](./bridge-api.md) and [SDK](./sdk.md), which can be used together with existing | ||
a [API](./api.md) and [SDK](./sdk.md), which can be used together with existing | ||
logging libraries to automatically inject the trace context in the emitted logs, | ||
and provide an easy way to send the logs via OTLP. Instead of | ||
modifying each logging statement, [log appenders](../glossary.md#log-appender--bridge) | ||
|
@@ -444,10 +445,9 @@ standard output. | |
|
||
## Specifications | ||
|
||
* [Logs Bridge API](./bridge-api.md) | ||
* [Logs API](./api.md) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about having links to the old API's stating that the this new API will encompass the existing functionality. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I do not want to create yet another API. |
||
* [Logs SDK](./sdk.md) | ||
* [Logs Data Model](./data-model.md) | ||
* [Event API](./event-api.md) | ||
* [Trace Context in non-OTLP Log Formats](../compatibility/logging_trace_context.md) | ||
|
||
## References | ||
|
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
@@ -1,4 +1,4 @@ | ||||||||
# Logs Bridge API | ||||||||
# Logs API | ||||||||
|
||||||||
**Status**: [Stable](../document-status.md), except where otherwise specified | ||||||||
|
||||||||
|
@@ -9,6 +9,7 @@ | |||||||
|
||||||||
<!-- toc --> | ||||||||
|
||||||||
- [Overview](#overview) | ||||||||
- [LoggerProvider](#loggerprovider) | ||||||||
* [LoggerProvider operations](#loggerprovider-operations) | ||||||||
+ [Get a Logger](#get-a-logger) | ||||||||
|
@@ -18,21 +19,33 @@ | |||||||
+ [Enabled](#enabled) | ||||||||
- [Optional and required parameters](#optional-and-required-parameters) | ||||||||
- [Concurrency requirements](#concurrency-requirements) | ||||||||
- [Artifact Naming](#artifact-naming) | ||||||||
- [References](#references) | ||||||||
|
||||||||
<!-- tocstop --> | ||||||||
|
||||||||
</details> | ||||||||
|
||||||||
<b>Note: this document defines a log *backend* API. The API is not intended | ||||||||
to be called by application developers directly. It is provided for logging | ||||||||
library authors to build | ||||||||
[log appenders](./supplementary-guidelines.md#how-to-create-a-log4j-log-appender), | ||||||||
## Overview | ||||||||
|
||||||||
This document defines a log API. | ||||||||
|
||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As part of OTep #265 it was determined that there SHOULD be a public facing "Logging" API to enable the emitting of "Events" (as a type of OpenTelemetry Log) as well as other general "Logs" <- I'm implicitly inferring based on discussions that occurred as part of #265 (ie. while all One of the key words here is These can include "Event", "Audit", "Security", "Transaction", "Message", "Server", "File" and many, many more, and just as "Events are a As part of any Logging API we SHOULD provide a front-end API and guidance for all of the CORE fundamental concepts to "try" to make it clear about what will and won't occur when you do or don't use the specific API. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I changed the PR so that Logs API can be used to emit events but I have not removed the Events API. |
||||||||
The API serves following use cases. | ||||||||
|
||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There are many additional features / requirements that exist and are yet to be defined as we move this API forward (some of which was starting to occur in the Event API) as it was moving (or trying to move) towards stability. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this should rather require refining the SDK and not the API. |
||||||||
It is provided for instrumentation libraries to emit events (log records following | ||||||||
OpenTelemetry Semantic Conventions). | ||||||||
|
||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It should include an As the And it should also
And (if) future versions of OpenTelemetry evolve to change the internal representation of the |
||||||||
It is provided for logging library authors to build | ||||||||
[log appenders/bridges](./supplementary-guidelines.md#how-to-create-a-log4j-log-appender), | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @MSNev can you highlight any specific blocking concerns you have about this PR beyond #4225 (comment), and open tracking issues for the rest? Once we get this initial restructuring PR landed then we'll be able to start sending PRs to try to address those. Thanks. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is up to the bridge implementation how the logging library would be mapped to OTel Logs Data Model. I think we should rather create some "guidance" / "recommendations" for log bridge authors. We cannot control how the bridges are going to be implemented. For sure this should be out of scope of this PR. |
||||||||
which use this API to bridge between existing logging libraries and the | ||||||||
OpenTelemetry log data model.</b> | ||||||||
OpenTelemetry log data model. | ||||||||
|
||||||||
The Logs Bridge API consist of these main components: | ||||||||
It is provided for application developers to emit structured log records | ||||||||
(including OpenTelemetry Events). | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should call out the "difference" between the processing / handling / semantics of an
Problem Statement and confusion
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I find it as out of scope. We need such mechanism for all log records. From last SIG meeting agenda:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Am I stating anywhere that we don't -- NO. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. based on the current version of spec, it's totally valid to emit events using log bridge. We don't have a requirement to support separate pipelines today and this PR does not change status quo opentelemetry-specification/specification/logs/event-api.md Lines 79 to 81 in fbaf846
|
||||||||
|
||||||||
Languages MAY provide a separate Logs Bridge API if they need different | ||||||||
ergonomics for consumers that are building log appenders/bridges. | ||||||||
|
||||||||
The Logs API consists of these main components: | ||||||||
|
||||||||
* [LoggerProvider](#loggerprovider) is the entry point of the API. It provides access to `Logger`s. | ||||||||
* [Logger](#logger) is responsible for emitting logs as | ||||||||
|
@@ -167,22 +180,13 @@ provide it. | |||||||
|
||||||||
## Concurrency requirements | ||||||||
|
||||||||
For languages which support concurrent execution the Logs Bridge APIs provide | ||||||||
For languages which support concurrent execution the Logs APIs provide | ||||||||
specific guarantees and safeties. | ||||||||
|
||||||||
**LoggerProvider** - all methods are safe to be called concurrently. | ||||||||
|
||||||||
**Logger** - all methods are safe to be called concurrently. | ||||||||
|
||||||||
## Artifact Naming | ||||||||
|
||||||||
The Logs Bridge API is not intended to be called by application developers | ||||||||
directly, and SHOULD include documentation that discourages direct use. However, | ||||||||
in the event OpenTelemetry were to add a user facing API, the Logs Bridge API would | ||||||||
be a natural starting point. Therefore, Log Bridge API artifact, package, and class | ||||||||
names MUST NOT include the terms "bridge", "appender", or any other qualifier | ||||||||
that would prevent evolution into a user facing API. | ||||||||
|
||||||||
## References | ||||||||
|
||||||||
- [OTEP0150 Logging Library SDK Prototype Specification](https://github.com/open-telemetry/oteps/blob/main/text/logs/0150-logging-library-sdk.md) |
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Merge Events API and SDK
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I decided to simply bring back the Events API and SDK. We can address it later as a separate PR.