Logo

dev-resources.site

for different kinds of informations.

Emergent Software Principles I've Learned

Published at
1/11/2025
Categories
softwaredevelopment
softwareengineering
Author
Hayden Rear
Categories
2 categories in total
softwaredevelopment
open
softwareengineering
open
Emergent Software Principles I've Learned

Code as Data and Data Structure as a Program

The best code is provably correct, and the most provably correct code is a data structure. Trying to write your program as a data structure makes it more testable and more provably correct. Moreover, using the best data structure for the job makes the code much more resilient and a source of great abstraction. Learn your data structures and then learn them again, always be learning your data structures! You can always start with an off-the-shelf version and then create a customized version as a performance optimization. Then you can compare the naive version to the custom version.

Example: using a trie for file paths, using a graph for program analysis.

MicroBench

Typically test based development (basically all development) leads to code that just barely passes. So then when you increase the load a little bit (or add just a little bit more data), it could break. This is the importance of micro-benchmarking.

Scalability as Smallest Units of Work Under Infinite Load - Theory of Constraints

Anything can break under sufficient load. Make sure that the bottleneck scales for the load you need.

Pascal's Triangle as Emergent Refactors

When I first learned recursion I spent hours banging my head against the wall implementing pascal's triangle. Later on I learned that much of emergent refactors resemble some n-dimensional version of pascal's triangle. As we refactor to add a needed emergent layer of abstraction, that layer of abstraction becomes the next layer in the base of this n-dimensional "pyramidal" structure. Make sure the layers are firm foundations for the next layer.

Result Type as Point of Coupling - Decoupling Away "Distributed" as Metadata

Typically developers depend on some stream or IO provided by some other developer, making it hard to decouple the implementation. As microservices are refactored away, result types will become more important.

The Weakest Chain in the System Should be You - Observability Matters

If you're system doesn't provide a mechanism to observe the issue before it becomes a problem, then there's something wrong with the way you deploy code.

Example: before you release your code, you look through all of the errors you had - from the perspective of structured logging. So then you see the error - you ignore it. But it's not an ignorable error! In this case you made a mistake. But the system for letting you know that there is a bug is good!

Defining things only in terms of the most complex case, the programmer's dilemma

The programmers dilemma is I want to do the least amount possible but I need an emergent design out of the most complex set of cases for the minimum viable product.

This is coming from a greater truth related to the resilience of a system being based on the maximum amount of testing done, and the way the testing relates to the provable resilience of the system... resulting in failures when you push just a little bit harder, just a little bit beyond what you've tested before, and specifically how that test relates to the provable resilience of the system.

The Single Unifying Thread

The most generic implementation, typically ByteBuf or something similar in IO, for instance. It should be the emergent data structure from the emergent design of your system.

Levels of Abstraction ~ Levels of Testing

Being able to replicate bugs quickly in some ways depends on testing capability. If I find a bug in the system and I have a testing architecture set up, then I can recreate the bug quickly and with minimal setup by running the test with a different input, or running the test a bit differently, or cloning a test setup and running it a different way.

Example: you find a bug in prod and it's one of these "difficult to replicate" issues. At the very least you should have a test harness that works a starting point for replication. So many developers everywhere just run the thing from beginning to end, resulting in these ridiculous turnaround times. Testing is all about creating tests at the layers of complexity to replicate bugs, hopefully replicating them at the time you first write the code, but if necessary being able to use the test setup to do so.

Featured ones: