dev-resources.site
for different kinds of informations.
WarpStream Newsletter #5: Dealing with Rejection, Schema Validation, and Time Lag
Welcome to the fifth issue of the WarpStream newsletter where we share our latest blogs on:
- Backpressure in distributed systems.
- WarpStream schema validation.
- Measuring consumer group lag in time instead of offsets.
As well as our most recent product updates. Connect with us on social media and other platforms to stay up to date via the links in the social footer at the bottom of this email.
New Blog Posts
Dealing with rejection (in distributed systems)
At its core, backpressure is a really simple concept. When the system is nearing overload, it should start “saying no” by slowing down or rejecting requests. Of course, the big question is: How do we know when we should reject a request? The latest version of the WarpStream BYOC Agent includes backpressure support for Product and Fetch requests.
Announcing WarpStream schema validation
We are excited to announce that users can now connect WarpStream Agents to any Kafka-compatible Schema Registry and validate that records conform to the provided schema. WarpStream validates not only that the schema ID encoded in a given record matches the schema ID in the Schema Registry, but also that the record actually conforms with the provided schema (field level validation).
The Kafka metric you’re not using: stop counting messages, start measuring time
Monitoring consumer groups can be a challenge. We explain why the usual way of measuring consumer group lag (using offsets) isn’t always the best and show you an alternative approach (time lag) that makes it much easier to monitor and troubleshoot them. The best part? Time lag measurement is built directly into WarpStream — no third-party tooling is needed.
Recent Product Updates
In addition to our blog about schema validation, which we shared above, you can also check out our related doc pages on Agent schema validation and connecting to an external schema registry.
You can upgrade to the latest version of the WarpStream BYOC Agents to benefit from our new-and-improved backpressuring system. Check out our official changelog to see the full list of Agent updates.
Try WarpStream With $400 in Free Credits
WarpStream is free to try. After you create your account, it will be loaded with $400 in free credits so you can test how easy it is to set up and use WarpStream.
Connect With Us
Featured ones: