In several projects I’m working on, directly and indirectly, a recurring need has emerged — something I call “immutable metadata-less storage.” This post lays out the problem, why it matters, and my thoughts on a potential specification.
The Core Problem: Data vs. Metadata
Storing data is easy. Storing data about data — metadata — is hard.
Most systems today mix these two responsibilities. Whether it’s a simple key-value store, an object store, or a highly consistent RDBMS, the rules for storing raw data are very different from the rules for managing how that data evolves over time.
For example:
A video can be stored once and never touched again — trivial.
But what happens when that video needs to be replaced, blurred, or hidden because someone posted personally identifiable information?
Suddenly, we need to track edits, references, permissions, and consistency.
That’s the metadata problem. Managing lifecycle, references, and invariants is always more complex than just dropping bytes onto disk.
In high-throughput, low-latency systems, the metadata layer — not the raw storage layer — is where most of the engineering pain lives.
The Key Insight: Separate Them
Let’s assume a clean separation:
Data storage → Holds immutable objects. Videos, images, documents, binary blobs — anything large and permanent.
Metadata storage → Holds references to those immutable objects and manages who can see, edit, or delete them.
Once you decouple the two, things get much simpler:
Metadata stays small. If something is more than a tweet’s worth of text, you store a reference to the data, not the data itself.
Data can eventually be deleted safely — but only when nothing in metadata references it.
Deletion doesn’t need to be instant. Objects can be removed in batches every few hours. As long as access control is enforced at the metadata layer, there’s no risk in delayed deletion.
Even backups follow this principle. Cloud providers don’t edit tapes; they just drop encryption keys, effectively “deleting” data without touching it.
S3 is Too Powerful, How to Simplify it
Amazon S3 is the closest thing we have to a de facto object storage “standard,” and they’ve polished it over the years. But S3 is not metadata-free. Far from it.
Modern S3 actually provides strong consistency guarantees and even supports atomic compare-and-swap operations. For example, you can say:
“Write this as version 11 only if the current version is still 10.”
If many servers send competing writes for version 11, exactly one succeeds.
And here’s the real kicker: S3 is designed to manage metadata, too.
Achieving consistency when it comes to metadata, while still delivering S3’s famous durability, is incredibly difficult. This is great, of course, but not everyone needs it.
This overpower of S3 is also exactly what I want to simplify with this RFC.
The Goal: Immutable, Metadata-Free Storage
We want a system that focuses purely on passive storage:
Eleven nines durability as a baseline.
No mutations — once written, bytes never change.
Idempotent writes — writing the same value to the same key is always safe.
Safe deletions — supported, but delayed and explicitly mediated by metadata.
Here’s the critical rule:
The client, not the storage, is responsible for ensuring same keys always map to same values.
Now, try pitching this to engineers who also want versioning, updates, and soft deletes at the same time:
This is exactly why we need a clean separation between data and metadata.
Handling Conflicts
Of course, mistakes happen. Collisions are rare, but not impossible — even SHA256 isn’t guaranteed to be collision-free.
If two different values ever appear under the same key, the storage system should:
Detect it.
Temporarily halt writes (~10 seconds of downtime is fine).
Resolve the conflict cleanly before resuming.
This keeps the system lean and avoids expensive metadata machinery.
Exactly. The system promises durability, not full transactional metadata management.
A Web3-Inspired Design
Here’s where my Web3 experience comes in handy. We can enforce correctness using signatures and deposits:
Every write request is signed by a registered private key.
Each writer stakes a deposit (e.g. $1,000) to register its public key as legal to sign write requests.
If a writer misbehaves — writing different values under the same key — part of the deposit is slashed as a penalty.
Because when things go wrong, here’s what we really need:
This [economic] model discourages misuse and keeps the system self-policing.
Enter the Control Plane
To manage conflicts and finality, we introduce a lightweight Control Plane.
This can be:
A small blockchain (in Web3 settings), or
A centralized cluster (etcd, Consul, or Kafka) in traditional deployments.
The Control Plane handles:
Conflict arbitration → Decides winners in write collisions.
Epoch tracking → Publishes random nonces every ~500ms.
Finality guarantees → Ensures writes are “locked” after a couple of epochs.
Permissions & throttling → Controls who can write, how much, and how often.
Deposit slashing → Enforces accountability for misbehaving clients.
The beauty is that this Control Plane stays tiny: conflict arbitration costs strictly under O(log(N)) where N is number of objects stored. As in: negligible, as long as the system is being put to good use.
Readers vs. Writers
Readers don’t need to interact with the Control Plane directly. They can:
Use caching aggressively.
Fetch immutable data without worrying about versioning.
Rely on metadata services to enforce access control.
Because the data never changes, read paths become trivial.
Why This Matters
This isn’t just a thought experiment. A clean separation between immutable data and metadata makes systems:
Faster → Storage becomes dumb and optimized for throughput.
Safer → Conflicts are rare and well-defined.
Cheaper → Control logic moves out of the hot path, reducing infrastructure cost.
Scalable → Web3-inspired governance adds natural incentives for correctness.
So when someone asks you if your “immutable” system actually allows edits, the answer is clear.
The API Contract
This post is about the problem and a rough sketch of a spec.
The write and read APIs.
The guarantees for durability and finality.
The exact role of the Control Plane.
Pricing, billing, and incentives for participants.
This post would be incomplete without a rough spec. So here it comes.
Mutation requests should be signed with the private key.
The public counterpart of which is registered as a valid writer.
Mutation request bodies should contain the latest or second-latest nonce.
Handled by the client library, or the gateway, of course.
The body of the mutation request contains:
Required: The key under which the value should be written.
Required: The SHA256 of the value to write.
Optional: The very data blob value, the body.
Optional: The timeout for this request, default to T = ~3 seconds.
Optional: The desired durability, default to D = “eleven nines”.
In addition to write requests we also have:
Delete requests, and Merge requests.
The Merge requests are mutation requests that concatenate values.
So that to write a large blob, many small blobs are persisted first, then merged.
The original parts can also be retrieved, the overhead is negligible.
The response always contains the degree to which the persisted data is durably stored.
The value — the data blob — can be absent for the “check persistence status” requests.
The semantics of the call is the following:
The request is allowed to take up to T seconds.
As the timeout is close, the gateway / client library must return.
In fact, the client library is guaranteed to return within T.
The return value may well be “uncertain”.
If the desired degree of durability is not there yet.
The durability degree, D, is generally free-form.
Some defaults, such as “eleven nines”, are present.
The user can have more fine-grained control.
“At least five nodes in at least three AZs on at least two continents” et. al.
These custom modes would need to be registered with the Control Plane.
If the desired durability is already in place, the request returns sooner.
As in, T is “max. time to wait until D is in place, as part of this request”.
This spec is missing error handling, but the gist should be clear.
What’s Next
I believe this design can work — both inside Web3 ecosystems and in traditional infrastructure. It could become the foundation for a metadata-free, immutable storage standard.
And yes — before shipping, we’ll have to smash it with load tests until it breaks.
Thank you for reading!
Full disclosure: I wrote a longer post and had GPT-5 simplify it for me. Let me know in the comments if you prefer this format, or the longer one — I’m still calibrating on this front!