Back to the release learning center

Introduction

Application telemetry is crucial for system tracking and observability as it provides real-time data and insights into the performance, health, and usage patterns of the system. By leveraging telemetry, businesses can proactively identify and resolve issues, optimize operational efficiency, and enhance the overall user experience.

Using the warehousing telemetry feature in Dynamics 365 F&O and building meaningful metrics with it can help system administrators and warehouse managers track warehouse infrastructure reliability and business process efficiency. This feature allows integration between Dynamics 365 F&O environment and the designated Application Insights resource in Azure.

Through examples, this blog post aims to show how the telemetry transactions are organized and how they can be used to create different metrics.

Setup

Since all the telemetry data needs to be exported to Azure, you must have an Application Insights resource available in your Azure tenant. The full configuration process of connecting your D365 F&O environment to Application Insights is straightforward and well-described in Microsoft Docs.

Keep in mind that turning on User sessions (custom events) and Warehouse events options in D365 F&O Monitoring and telemetry parameters form is mandatory for warehousing mobile app's and warehousing processes' telemetry data, but if you would like to get the exceptions happening due to user activities in the Warehousing mobile app, you will need to turn on the X++ exceptions (Failures) option as well. Note that this option will export any X++ exception across the system, not just those related to warehousing.

Overview of telemetry data in Application Insights

There are different ways to view the telemetry data stored in Application Insights. The quickest one is the Transaction search in the Investigate section. This will provide you with all the logged transactions in the past period (24 hours by default but can be changed). You can also set additional filters, like types or specific transaction property values.

The warehouse events are logged as custom events in Application Insights, while the errors that users might receive during their work in the system or the warehousing app are logged as exceptions.

Most of the activities in D365 F&O or the mobile app being tracked have two transactions logged in Application insights – a start transaction and a stop transaction. This makes it convenient to see the timestamps of when a particular process step has started and when it was completed. The "stop" transaction also provides the property of elapsed time. Additionally, each transaction has an eventId property, which helps differentiate what type of activity it is, and an activityId property, which is the same for every start and stop transaction pair.

Depending on where the event originates from, the transaction log will have different properties that can be used to create monitoring metrics. We can separate these transactions into those coming from warehousing processes in D365 F&O, like load validation and release, wave processing, etc., and those coming from warehouse mobile app usage.

Warehouse processes telemetry transactions

Warehouse process telemetry transactions result from executing warehouse activities related to loads, shipments, waves, and work creation or processing. One activity can generate multiple telemetry transactions by sending events for each step that is part of the activity.

Additionally, some steps in the activity can have sub-steps which are also sent to the Application Insights. In this case, the start and stop events of the sub-step will be nested between the start and stop events of the "parent" step.

Let's take the load posting activity, for example. It consists of multiple steps, one of which is creating the shipment(s). This step triggers the shipment(s) creation activity, which will be represented as a nested sub-activity in the telemetry data. This way, we can track the time needed to create the shipments for the load and see how much time it took to create the individual shipment.

Each execution of a warehouse process gets one activity id assigned to it in the telemetry transactions. This allows us to track all the steps for a particular process execution by using the activityId. A combination of activityId, eventId and eventStatus properties on the transactions can be used to match the start and stop transaction pairs. However, in case a warehouse process has multiple executions of the same event type (i.e., finding location during wave processing), a chronological ordering of the transactions is important before pairing them.

The data that the telemetry transactions contain depends on which process they are generated from. Still, they have the ids of the entities (loads, waves, etc.) being processed, which allows us to track them in D365 F&O and investigate further.

Warehouse mobile app telemetry transactions

These telemetry transactions are generated every time a user navigates between two screens in the warehousing mobile app or when the mobile app is unable to reach or execute the API call to the server. Each transaction has the mobile device id property which helps track the device that is generating the telemetry transaction.

The most common is the Next page request, which occurs when a user navigates to the next screen in their mobile app. This request has its start and stop telemetry transactions, which share the same activity Id and mobile device request activity Id. The stop transaction also has the work execute mode and step, which helps establish the point in the work execution process the user is in at the given moment.

If a user action in the mobile app generates an execution error in D365 F&O, a system exception transaction will be logged in Application Insights as well (if it was configured as described at the beginning of this article). The exception is related to the Next page request transactions via the same activity Id.

Another useful transaction type is the Client round trip transaction which is logged whenever the mobile app makes a call to the server following the Next page request transactions. It provides information about the mobile app device, such as model, battery state, last known location, and the request processing times. However, the activity Id of these types of transactions is always blank, so they cannot be paired with any other transaction.

Creating reports on warehouse telemetry

Now that we understand the types of warehousing telemetry transactions that get logged in Application Insights, let’s look at how we can use them to create actionable and meaningful reports. For this, we will combine the KQL to query the transactions and Power BI to display the reports.

Querying the telemetry transactions

For querying the telemetry transactions, we will use the KQL query editor that is built-in Application Insights. You can find it in the Monitoring > Logs section of Application Insights.

KQL (Kusto Query Language) is a query language that retrieves and manipulates data stored in Application Insights. Its primary purpose is to enable users to perform complex queries quickly and efficiently. KQL is well-suited for analyzing large volumes of structured, semi-structured, and unstructured data. Using a clear and concise syntax, it allows for filtering, sorting, aggregating, and transforming data, making it an invaluable tool for creating detailed and insightful reports from telemetry transactions and other datasets.

KQL queries created in Application insights can be used in Excel or Power BI for additional data analysis of the query results.

Microsoft provides the KQL queries that filter each of the warehousing management and mobile app telemetry transactions on their GitHub. These are good starting points for making advanced queries for our reporting.

Power BI reporting

Once the KQL query is completed, we can export it and use it in Power BI for additional analysis and the creation of report dashboards. The KQL query is translated into M language, allowing it to be used in Excel or Power BI.

Example 1 - Warehouse mobile device exceptions report

For our example of warehousing telemetry reporting, let's examine how to extract the exceptions coming from the warehouse mobile device and create a chart of exceptions per mobile device in Power BI.

Note that, if the Exceptions parameter is turned on, the exceptions generated across the whole system will be logged in application insights, so the warehouse app ones will need to be filtered for our reports.

Step 1 - Create a KQL query to extract the warehouse mobile device exception logs.

As shown in a previous section of this post, the exceptions coming from mobile devices are related to the next page start and stop requests. Getting these transactions is achieved by filtering all the transactions of type System.Exception that are connected, via activity Id, to transactions whose event Id is next page request (WHS000003).

The following KQL query can be used to list these transactions:

Step 2 - Export the query to Power BI format.

In the Logs section, above the KQL query editor, there is an Export button. One of the export options is the Power BI (as an M query).

This option will create a text file with the translation of the KQL query into the Power Query Formula Language (M language) and clear instructions on how to use it in the Power BI desktop application.

Step 3 - Follow the instructions in the file to get the telemetry data into PowerBI.

In the Power BI desktop app, in the Home tab select the Get Data > Blank query option.

In the query editor window select the Advanced query option and paste the M language query from the file exported in the previous step.

Press Done and then Close & Apply.

Step 4 - Create a Power BI report based on the retrieved data.

Now that the query is ready in Power BI, a report can be created. In this case, we want to make a chart showing the number of errors reported by each warehousing mobile device. Additionally, we can add a drill-down option to show which errors were reported for the selected mobile device, so we know if the errors are caused by human error or a system issue.

A clustered column chart can be used for this purpose. Add the mobileDeviceId and outerMessage fields to the X-axis and Count of outerMessage to the Y-axis.

Notice that, in our example, the mobile device starting with “362" has a higher number of errors compared to the rest. If we click on that red bar, we can drill down and see specific errors and numbers of their occurrence.

By utilizing Power BI to visualize the telemetry data, we can effectively identify and address errors reported by warehousing mobile devices. This process not only enhances our ability to pinpoint the root causes of issues—whether they are due to human error or system malfunctions—but also empowers us to take proactive measures to improve overall operational efficiency. The detailed insights gained from such reports are invaluable for making informed decisions and ensuring the smooth functioning of our warehousing processes.

Example 2 - Keeping track of WiFi strength around warehouse locations.

For this report, we will use a different type of telemetry transaction, which was mentioned in the previous section – Client Round Trip. This transaction shows, among other data, the last warehouse location of the mobile app and the Wi-Fi strength during the round-trip call to the server. By combining this data in our Power BI report, we can keep track of the Wi-Fi strength at our locations. The steps are the same as in the previous example so this one will not be in so much detail.

In this case we can use the KQL query for client round trip transactions provided by Microsoft on their Github.

In Power BI, we will create a clustered bar chart to present this information. Configuration of the chart is the following:

X-axis – Count of wifiSignalStrength

Y-axis – lastKnownWMSLocation field

Legend – wifiSignalStrength

The chart shows which locations the Wi-Fi signal is degrading around, so we can preemptively perform maintenance on our network to keep the warehousing activities running smoothly at  these locations.

These are just some of the ways we can benefit from warehousing telemetry data. By exploring other types of transactions available, we can construct additional reports on the efficiency of our warehouse processes.

In conclusion, by utilizing warehouse telemetry data, we can visualize critical metrics and trends. This enables us to identify potential issues early and take proactive measures, ultimately ensuring smoother operations, enhancing efficiency, and optimizing overall process performance.