Are you prepared to begin your journey on the highway to amassing telemetry knowledge out of your functions?
On this sequence, you may discover learn how to undertake OpenTelemetry (OTel) and learn how to instrument an utility to gather tracing telemetry. You will learn to leverage out-of-the-box computerized instrumentation instruments and perceive when it is necessary to discover extra superior handbook instrumentation on your functions. By the tip of this sequence, you may have an understanding of how telemetry travels out of your functions to the OpenTelemetry Collector, and be able to deliver OpenTelemetry to your future tasks. Every part mentioned right here is supported by a hands-on, self-paced workshop authored by Paige Cruz.
The earlier article zoomed into the utilization of Jaeger to assist builders when visually exploring their telemetry knowledge. On this article, we’ll assist builders with manually instrumenting so as to add metadata particular to their utility or enterprise permitting them to derive higher insights quicker.
It’s assumed that you simply adopted the earlier articles in establishing each OpenTelemetry and the instance Python utility challenge, but when not, return and see the earlier articles as it is not lined right here.
To this point on this sequence, we have used computerized and programmatic instrumentation to gather tracing knowledge for our system interactions. It might be way more attention-grabbing to manually instrument our utility, including metadata particular to our enterprise that helps derive higher insights quicker.
Including Span Attributes
Including span attributes means we’re monitoring key, worth pairs containing metadata to annotate a span with details about the operation it’s monitoring. For instance, to set this up for our Python utility, we use the syntax:
span.set_attribute("KEY","VALUE")
To use this to our utility (that you simply put in and used within the earlier articles from this sequence), we open the file handbook/app.py, import the get_current_span from the API, and add an attribute monitoring the depend of homepage hundreds to the index() technique as follows with the brand new code proven in daring:
...
from opentelemetry import hint
from opentelemetry.hint import set_tracer_provider, get_current_span
from opentelemetry.sdk.hint import TracerProvider
from opentelemetry.sdk.hint.export import SimpleSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
...
@app.route("https://dzone.com/")
def index():
span = hint.get_current_span()
international HITS
HITS = HITS + 1
span.set_attribute("hits", HITS)
msg = f'This webpage has been considered {HITS} occasions'
return msg
...
We now can construct a brand new container picture and run the pod to confirm as follows:
$ podman construct -t hello-otel:handbook -f handbook/Buildfile-manual $ podman play kube handbook/app_pod.yaml
With the pod working open a browser to http://localhost:8080 and open the Jaeger UI at http://localhost:16686, trying to find traces of the operation / which ought to look one thing like this:
Click on on a hint and make sure the hits attribute which is in span particulars:
This verifies that the handbook instrumentation with its attributes is working for tracing our utility homepage. What about tracing nested spans?
Let’s attempt manually instrumenting for nested spans. However first, we have to cease the working pod with the next:
$ podman play kube handbook/app_pod.yaml --down
Now let’s improve our monitoring of nested spans.
Enhancing Programmatic Instrumentation
The instrumented requests library supplies knowledge about requests to the Canine API in our utility, however not customized strategies get_breed() or validate_breed(). To seize that work in /doggo traces we will manually instrument nested spans to replicate their hierarchical relationship. The next creates a brand new span throughout the present hint context:
tracer.start_as_current_span()
So as to add this, open handbook/app.py and get entry to the international tracer. That is what really creates and manages spans, proven beneath in daring for the brand new code:
...
app = Flask("hello-otel")
FlaskInstrumentor().instrument_app(app)
Jinja2Instrumentor().instrument()
RequestsInstrumentor().instrument()
tracer = supplier.get_tracer(app.identify)
...
Additional down the file, we discover the get_breed() technique and create a nested span as follows:
...
def get_breed(url):
with tracer.start_as_current_span("get_breed"):
path = urllib3.util.parse_url(url).path
match = re.search(r"/breeds/([^/]+)/", path)
if match:
outcome = match.group(1)
return outcome
...
We now have to rebuild the container picture and run the pod to confirm as follows:
$ podman construct -t hello-otel:handbook -f handbook/Buildfile-manual $ podman play kube handbook/app_pod.yaml
With the pod working open a browser to http://localhost:8080/doggo and make a couple of requests by refreshing the web page. Open the Jaeger UI at http://localhost:16686, trying to find traces for the operation /doggo. It’s best to see that the span depend has elevated however to substantiate, click on on a hint:
Now let’s confirm we’re receiving tracing knowledge from the nested spans. The hint waterfall ought to present one span for the general request to /doggo, the GET request to the Canine API, and the operation get_breed() to provide a fuller image of the place time is spent fulfilling requests to /doggo:
To date we have used the hint waterfall to visualise our traces. Let’s attempt an alternative choice by clicking on the menu within the higher proper nook and deciding on Hint Graph:
By clicking on the T over on the right-hand vertical menu bar, we will colour the crucial path or the spans that instantly contribute to the slowest path of processing this request. That is how we will view a hint graph by time:
On this instance, we’re solely working with small traces. You’ll be able to think about with bigger hint samples, akin to this instance with 160 spans, a birds-eye view generally is a very highly effective device when troubleshooting extra advanced traces:
Now let’s attempt including extra span occasions to our utility, however first, we have to cease the working pod so we will add extra code to the appliance:
$ podman play kube handbook/app_pod.yaml --down
Now let’s add span occasions.
Including Span Occasions
An occasion incorporates a structured log with a identify, a number of attributes, and a timestamp. These are used so as to add structured annotations to a present span:
span.add_event(identify, attributes)
So as to add one to our utility, open the file handbook/app.py and add a span occasion representing the results of the cube roll to the roll_dice() technique as follows. Notice the brand new code is in daring kind:
...
@app.route("/rolldice")
def roll_dice():
sp = hint.get_current_span()
outcome = do_roll()
sp.add_event("rolled dice",attributes={"result":outcome})
return outcome
...
We now have to rebuild the container picture and run the pod to confirm as follows:
$ podman construct -t hello-otel:handbook -f handbook/Buildfile-manual $ podman play kube handbook/app_pod.yaml
With the pod working, open a browser to http://localhost:8080/rolldice and make a couple of requests by refreshing the web page. Open the Jaeger UI at http://localhost:16686, seek for traces for the operation /rolldice, and click on on a hint:
It’s of word that the timestamp related to this span occasion is relative to the beginning time of the hint itself:
Now let’s attempt exploring span standing, however first, we have to cease the working pod so we will add extra code to the appliance:
$ podman play kube handbook/app_pod.yaml --down
Now let’s examine what defines a span standing.
Including Span Standing
A span standing is usually used to establish spans that haven’t been accomplished efficiently and could be set any time earlier than the span is completed. Spans can have a standing of Unset, OK, or Error.
span.set_status(standing, description)
The /doggo web page permits customers to seek for pictures of a selected canine breed. If the search time period is invalid, an error message is returned to the consumer and the request efficiently returns 200 OK. If we take into account invalid searches to be errors, we have to instrument validate_breed() to hint this exercise. This is not going to make your complete hint failed, however noting the error on a span might be useful to our utility visibility.
So as to add a span standing, open the file handbook/app.py and find validate_breed(). Create a nested span and add an attribute for the breed to assist us perceive what the consumer was trying to find as follows with daring kind indicating our new code:
...
def validate_breed(breed):
with tracer.start_as_current_span("validate_breed") as span:
span.set_attribute("breed", breed)
if breed not in breeds:
elevate ValueError("No breed found.")
return
...
We now have to rebuild the container picture and run the pod to confirm as follows:
$ podman construct -t hello-otel:handbook -f handbook/Buildfile-manual $ podman play kube handbook/app_pod.yaml
With the pod working, open a browser to http://localhost:8080/doggo and make a couple of requests by refreshing the web page. Seek for legitimate canine breeds like Doberman or Akita and nonsense phrases like woof:
Now open the Jaeger UI at http://localhost:16686 and seek for traces for the operation /doggo. We’re verifying that there’s not less than one hint with a profitable request marked with an error. Click on that hint to see the detailed view:
Confirm that the error message is recorded on the search_breed span and that the breed is recorded as a span attribute:
As we now have seen, combining programmatic and handbook instrumentation to boost our utility visibility supplied by tracing knowledge is one other highly effective addition to our toolbox. We have explored and added span attributes, enhanced our programmatic instrumentation, added span occasions, and explored tracing span standing.
These examples use code from a Python utility that you may discover within the supplied hands-on workshop.
What’s Subsequent?
This text detailed manually instrumenting our utility so as to add metadata particular to our enterprise wants, permitting us to derive higher insights quicker.
In our subsequent article, we might be turning to manually instrumenting metrics for our utility.