Logo

dev-resources.site

for different kinds of informations.

Azure Logs Analytics for CosmosDB

Published at
12/30/2024
Categories
azure
monitoring
architecture
nosql
Author
martin_pi
Author
9 person written this
martin_pi
open
Azure Logs Analytics for CosmosDB

Azure CosmosDB allows you to enable and store logs in Log Analytics, which is helpful for troubleshooting and performance analysis.


Enabling Log Analytics on CosmosDB

  1. Access the CosmosDB account and, in the left panel, go to Monitoring > Diagnostic Settings.

Image description

  1. A message will prompt you about enabling "Full Text Query." Click Not Now.

Image description

  1. If no Diagnostic Settings are enabled, click Add Diagnostic Settings.

Image description

  1. Select the logs you want to store in Log Analytics.

Image description

Key Categories of Logs

Category Description
DataPlaneRequest Captures every data plane operation (e.g., create, update, delete, retrieve) for the CosmosDB account.
QueryRuntimeStatistics Details query operations executed against a SQL API account, with optional full-text query logging.
PartitionKeyStatistics Provides insights into logical partition keys with disproportionate storage utilization.
PartitionKeyRUConsumption Tracks RU consumption for logical partition keys in each region and identifies hot partitions.
ControlPlaneRequest Logs all control plane operations, including indexing policy updates, backups, and account modifications.

Important: While enabling logs is useful, it can incur additional costs in the Log Analytics workspace depending on database usage. Regularly monitor log consumption.

Checking Log Volume

Use the following KQL query to verify log volume:

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.DOCUMENTDB"
| summarize count() by Category
| order by count_ desc
Enter fullscreen mode Exit fullscreen mode

Image description

For example, in a 24-hour period, over 112 million logs were stored in the DataPlaneRequest category.


Accessing Logs in CosmosDB

To access logs, navigate to Monitoring > Logs in the CosmosDB account.

Image description

Log Analytics provides pre-defined queries separated into Alerts and Diagnostics:

Alerts Queries

Image description

Diagnostics Queries

Image description

Sample Query

The following query identifies top operations by consumed RUs in the last 24 hours:

// Top operations by consumed Request Units (RUs) in last 24 hours
AzureDiagnostics
| where TimeGenerated >= ago(24h)
| where Category == "DataPlaneRequests"
| summarize numberOfOperations = count(), totalConsumedRU = sum(todouble(requestCharge_s)) by databaseName_s, collectionName_s, OperationName, requestResourceType_s, requestResourceId_s, _ResourceId
| extend averageRUPerOperation = totalConsumedRU / numberOfOperations
| order by numberOfOperations
Enter fullscreen mode Exit fullscreen mode

Image description

Available Logs in CosmosDB

On the left panel, you can view logs available for CosmosDB:

Image description


Enabling Full Text Query

By default, CosmosDB does not enable full-text query logging. Enabling this feature improves query visibility.

Comparison

Default Logs (without full-text query):

SELECT c.p1, c.p2, c.p3 FROM c WHERE c.p1="123"
Enter fullscreen mode Exit fullscreen mode

Logs with Full Text Query Enabled:

SELECT c.id, c.firstname, c.lastname FROM c WHERE c.id="123"
Enter fullscreen mode Exit fullscreen mode

How to Enable Full Text Query

  1. Navigate to Diagnostic Settings.
  2. Click Go to Feature when prompted to enable full-text query logging.

Image description

  1. Click Diagnostics full-text query and enable it.

Image description


Conclusion

Log Analytics is a powerful tool for monitoring and troubleshooting CosmosDB. Enabling appropriate logs can help identify issues like hot partitions, query bottlenecks, and resource utilization patterns. However, it is crucial to balance the logs collected with the cost implications in Log Analytics. Using targeted queries and insights will maximize efficiency while keeping costs under control.

Future posts will delve into advanced troubleshooting techniques for CosmosDB.


References

nosql Article's
30 articles in total
Favicon
O que é o Apache Cassandra e quando usar?
Favicon
Efficient Batch Writing to DynamoDB with Python: A Step-by-Step Guide
Favicon
SQL VS NoSQL
Favicon
MongoDB: How to setup replica sets
Favicon
Do you think schema flexibility justifies using NoSQL? Think twice.
Favicon
Series de tiempo en MongoDB
Favicon
What I Learned from the 'Amazon DynamoDB for Serverless Architectures' Course on AWS Skill Builder
Favicon
MongoDB Command Shortcuts: The Ultimate Guide
Favicon
MongoDB: Startup replica sets with a config file
Favicon
Azure Logs Analytics for CosmosDB
Favicon
Choosing the Right Database: A Simplified Guide
Favicon
Understanding the Differences Between NoSQL and SQL Databases
Favicon
Part 2 - CosmosDB Logical Partition and the Impact on Partition Key Choice
Favicon
Partitions in Azure Cosmos DB: A Common Discussion with Customers
Favicon
Database Sharding: Simplifying Data Scalability
Favicon
HTTP and GraphQL
Favicon
New possibilities with GraphQL
Favicon
NoSQL delivers quick value
Favicon
Navigating Databases: From SQL to NoSQL
Favicon
Selecting the Right Database for the Job
Favicon
NewSQL: Bridging the Gap Between SQL and NoSQL
Favicon
Weekly Updates - October 18, 2024
Favicon
Overcoming MongoDB Limitations with Fauna
Favicon
MongoDB Developer Day Manila 2024: A Recap - A Deep Dive into the Future of Data
Favicon
How to choose the right database?
Favicon
SQL vs. NoSQL: Key Differences, Use Cases, and Choosing the Right Database for Your Project
Favicon
Top 5 SQL questions asked in interviews
Favicon
Weekly Updates - Nov 8, 2024
Favicon
Plain Javascript Refresher for those feeling left behind or not knowing where to start w/ Functions, Arrays, Loops, JSON & NoSQL
Favicon
Mastering DynamoDB: Batch Operations Explained

Featured ones: