Wide Events and Cannonical Log Lines for Go

Cannonical Log Lines for Go.md

https://thefridaydeploy.substack.com/p/less-is-more-with-opentelemetry-spans?r=36rml&triedRedirect=true

https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/blob/main/observability/distributed-tracing/main.go

https://github.com/GoogleCloudPlatform/opentelemetry-demo/blob/main/kubernetes/opentelemetry-demo.yaml

https://github.com/mterhar/otel-go-batch ( https://www.honeycomb.io/blog/exotic-trace-shapes )

https://blog.bitdrift.io/post/observability-3-0

https://jeremymorrell.dev/blog/a-practitioners-guide-to-wide-events/

All events can be represented as structured logs, but not all structured logs are events. An event contains information about what it took for a service to perform a "unit of work". Treating an event as a "unit of work" lets you adjust what it means depending on the goals of the observer. An event is a conceptual abstraction and a structured log is one possible representation of that abstraction. I like the idea of a "Arbitrarily wide structured events" / "canonical log line" collecting all possibly relevant key-values like a lint roller collects dust. Putting all the important telemetry data in a single log line makes it easy for everyone to query for answers, without the need for complex joins or manual piecing of things together with request IDs, especially during production outages. Arbitrarily wide structured events that describe the request and its context, one event per request per service

• https://www.honeycomb.io/blog/structured-events-basis-observability

• An event is a conceptual abstraction and a structured log is one possible representation of that abstraction. The interesting part of the conversation is where to draw the lines around that abstraction; the technical implementation part is how to represent that event.

• Explicit propagation requires explicitly passing the active context from function to function as an argument (Go).

For distributed context propagation, OpenTelemetry supports several protocols that define how to serialize and pass context data:

• W3C trace context in traceparent header, for example, traceparent=00-84b54e9330faae5350f0dd8673c98146-279fa73bc935cc05-01.

• https://www.w3.org/TR/trace-context/#version-format

• https://github.com/kozlov-ma/sesc-backend/blob/b675d5c37224df4c450ac392806628eb9d629e0e/internal/slogsink/slogsink.go#L8

• https://github.com/AndrosHQ/lib-go canonical log lines with log/slog

• https://github.com/uptrace/uptrace-go/blob/master/example/metrics/main.go

• https://uptrace.dev/guides/opentelemetry-zap

• https://github.com/StevenACoffman/sesc-backend/tree/main/pkg/event

• https://github.com/dmathieu/owe

• https://github.com/maragudk/snorkel

• https://github.com/BTBurke/cannon

• https://github.com/rushsteve1/fp/blob/2284460bcfe9729b4a3052eaf6e234310c6dd131/wide/wide.go#L24

• https://baselime.io/blog/canonical-log-lines

• https://github.com/jonjohnsonjr/logspan

• https://github.com/jonjohnsonjr/til/blob/main/post/otel.md

• https://github.com/tl-tek/slog-openobserve

• https://github.com/ucarion/log

• https://brandur.org/nanoglyphs/025-logs

• https://github.com/robbyt/go-loglater

• https://github.com/openobserve/correlation

• https://www.w3.org/TR/trace-context/#version-format

• https://github.com/honeycombio/libhoney-go/blob/862c32cde92946c3ee787f5ee5e91d8a67c04d22/examples/wiki-manual-tracing/wiki.go

• https://github.com/instana/go-sensor/blob/main/w3ctrace/version.go#L156

• https://github.com/metalmatze/signal/

• https://github.com/kubeshop/tracetest

• https://web.archive.org/web/20190213162559/https://lightstep.com/blog/three-pillars-zero-answers-towards-new-scorecard-observability/ They’re a dead simple concept. In addition to normal logging made during a request, emit one, big, unified log line at the end that glues together all the key vitals together into one place:

canonical_log_line http_method=GET http_path=/v1/clusters duration=0.050 db_duration=0.045 db_num_queries=3 user=usr_123 status=200

• https://github.com/honeycombio/libhoney-go/blob/862c32cde92946c3ee787f5ee5e91d8a67c04d22/examples/wiki-manual-tracing/wiki.go

• https://github.com/smallstep/logging/blob/master/tracing/traceparent.go

• https://cloud.google.com/trace/docs/setup/go-ot#config-otel

• https://stackoverflow.com/questions/78828729/new-trace-and-span-generated-even-after-using-traceid-and-spanid-from-remote-con

• https://cloud.google.com/trace/docs/setup/go-ot

• https://codelabs.developers.google.com/codelabs/otel-cloudtrace#0

• https://cloud.google.com/pubsub/docs/open-telemetry-tracing

• https://pkg.go.dev/go.opentelemetry.io/contrib/bridges/otelzap

• https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry

https://signoz.io/guides/zap-logger/

https://github.com/GoogleCloudPlatform/microservices-demo

https://github.com/uptrace/opentelemetry-go-extra/tree/main/otelzap

Logging systems can’t afford to store data about every transaction anymore because the cost of those transactional logs is proportional to the number of microservices touched by an average transaction. Not to mention that the logs themselves are less useful (independent of cost) due to the analytical need to understand concurrency and causality in microservice transactions. So conventional logging isn’t sufficient in our brave new architectural world.

https://web.archive.org/web/20190213162559/https://lightstep.com/blog/three-pillars-zero-answers-towards-new-scorecard-observability/

https://github.com/mreider/sushi0/blob/ec2bdd964ef7aba133bf6a39c5ae05f28cf2a3f7/.github/workflows/deploy-things.yml#L23

    - name: End Docker Buildx
      run: |
        time_end=$(date +%Y-%m-%dT%H:%M:%S.%NZ)
        if [ "$GITHUB_EVENT_NAME" = "repository_dispatch" ]; then
          otel-cli span create \
          --endpoint=${{ secrets.DYNATRACE_ENDPOINT }}/api/v2/otlp/v1/traces \
          --otlp-headers="Authorization=Api-Token ${{ secrets.DYNATRACE_TOKEN}}" \
          --service=workflow \
          --name="Docker Build" \
          --force-parent-span-id=${{ env.parent_span_id }} \
          --start=${{ env.docker_time_start }}  \
          --end=$time_end \
          --kind=internal \
          --force-trace-id=${{ env.trace_id }} \
          --status-code="ok" \
          --status-description="Docker Built Successfully" \
          --verbose
        fi
        echo "deploy_time_start=$(date +%Y-%m-%dT%H:%M:%S.%NZ)" >> $GITHUB_ENV