Generating IDs in Distributed Systems

Generating unique identifiers in a single-server application is a trivial problem. The database has a sequence, every insert gets the next value, uniqueness is guaranteed by construction. Generating unique identifiers across a distributed system — where multiple independent processes need to produce IDs without coordinating with each other — is genuinely hard, and the techniques developed to solve it underpin most of the modern ID formats in common use.

This article works through the problem space: what makes distributed ID generation difficult, what trade-offs different solutions make, and which existing formats to reach for in which circumstances.

The fundamental problem

A distributed ID generator has to satisfy several constraints that pull in different directions:

Uniqueness. No two generators should ever produce the same ID, even when they're running concurrently with no knowledge of each other.
Compactness. IDs shouldn't be so large that storage, indexing, and transmission costs become significant.
Generation cost. IDs should be cheap to produce — ideally a few microseconds at most, with no network round-trips.
Ordering. For many use cases, especially database primary keys, IDs that sort in approximately the order they were created have substantial practical benefits.
Information hiding. For other use cases, IDs that leak information (creation time, generator identity, sequence position) are a problem.
Coordination-free generation. The whole point is to avoid having a central authority issue IDs, since that central authority is a latency bottleneck and a single point of failure.

No single format optimises all of these. Every distributed ID format makes deliberate trade-offs across this space, and the right choice depends on which constraints matter most for the system being built.

The coordination spectrum

Distributed ID generation lives on a spectrum between "fully coordinated" (a single source of truth issues every ID) and "fully independent" (every generator works in isolation and uniqueness is statistical). Most practical formats sit somewhere in the middle.

Fully coordinated — a single centralised sequence, like an auto-incrementing primary key in a single-master database. Perfect uniqueness, perfect ordering, no information hiding, but a hard scaling ceiling and a latency floor set by the round-trip to the coordinator.

Coordinated allocation, independent generation — generators acquire blocks of IDs from a central authority in batches, then issue IDs from their local block independently. Reduces the coordinator's load, but still requires occasional coordination and adds complexity (what happens if a block holder crashes with unused IDs?).

Negotiated identity, independent generation — generators receive a unique identity (a "worker ID" or "node ID") once at startup, then use that identity as part of every generated ID. Subsequent generation requires no coordination. The classic example is Twitter's Snowflake.

Statistical uniqueness — generators use enough randomness that collisions are negligibly unlikely even without coordination. UUID v4 is the canonical example.

Statistical uniqueness with embedded time — like statistical uniqueness, but the high bits encode a timestamp so IDs sort chronologically. UUID v7, ULID, and KSUID all fit this pattern.

The choice of strategy is the most consequential decision in distributed ID design. Everything else — encoding, length, formatting — is secondary.

Approaches in use

Auto-incrementing integers with sharding

The simplest extension of single-server sequences to distributed systems: partition the keyspace by allocating different ranges or step sizes to different servers. Server 1 generates 1, 3, 5, 7…; server 2 generates 2, 4, 6, 8…; or server 1 gets 1–1,000,000, server 2 gets 1,000,001–2,000,000.

This works for small, stable clusters where the partitioning can be configured manually. It breaks down when new nodes need to join dynamically, when nodes can fail and lose their unused range, or when the total keyspace needs to be carefully managed to avoid exhaustion.

Real systems sometimes use this pattern under the hood for performance — Postgres's bigserial with multiple writers in a partitioned setup, for example — but rarely as the primary distributed ID strategy.

Twitter Snowflake

Snowflake, introduced by Twitter in 2010, is the canonical example of negotiated-identity ID generation. A Snowflake ID is 64 bits, structured as:

41 bits of timestamp (milliseconds since a custom epoch)
10 bits of machine ID (datacenter ID + worker ID)
12 bits of sequence number (incremented within the same millisecond)

The machine ID is the negotiated part — each generator instance gets a unique value at startup, often via Zookeeper or a similar coordination service. Once a generator has its machine ID, every subsequent ID is produced locally without network calls.

Snowflake IDs are 64-bit integers, fit naturally into database integer columns, and sort chronologically. They leak the machine ID and the creation time, but that's often acceptable in internal systems.

The pattern has been widely adopted with variations. Discord uses Snowflake-style IDs for messages and users. Instagram uses a similar approach. Sony developed Sonyflake, which extends the time range at the cost of fewer per-millisecond IDs. Twitter eventually open-sourced Snowflake and several reimplementations exist.

Snowflake is a strong choice when:

You have a moderate number of stable generator instances (tens to hundreds, not thousands)
You want 64-bit IDs that fit naturally in integer columns
You can run a small coordination service for machine ID assignment
Information leakage about generation time and machine identity is acceptable

It's less suitable when you need an arbitrary number of ephemeral generators (lambdas, browser clients, mobile devices), or when you can't run a coordination service.

UUID v1 and v6

Both encode a timestamp and a "node" identifier in a 128-bit value. v1 lays out the timestamp with least-significant bits first; v6 reorders the same fields to make the value sortable.

The node identifier was originally meant to be the generating machine's MAC address, providing spatial uniqueness without coordination — the MAC address itself was the unique identity. In practice, MAC address access is unreliable (browsers can't read it, virtualised systems may not have a real one) and using it leaks information. Most modern implementations use a random node value instead.

For greenfield distributed systems, v1 and v6 are rarely the right choice. v7 supersedes them for most purposes, with simpler structure and no legacy MAC concerns.

UUID v4

122 bits of randomness, no structure beyond version and variant markers. UUID v4 takes the statistical uniqueness path: collisions are mathematically possible but practically impossible. At a billion IDs per second sustained, you'd need to run for about 85 years to have a 50% chance of any collision.

v4 is the right choice when:

You need IDs from arbitrary generators (including browser clients) with no coordination
You don't need time-ordering
You specifically want IDs that leak no information about when or where they were generated
Database insert performance is not the critical bottleneck

It's less suitable for high-write database primary keys (where v7's ordering helps with index locality) and for systems where the larger 128-bit size is a real constraint.

UUID v7

The current default for new distributed ID generation in most contexts. v7 combines v4's coordination-free generation with time-ordering similar to Snowflake:

48 bits of Unix millisecond timestamp
74 bits of cryptographic randomness (after version and variant)

The randomness is sufficient to avoid collisions across any practical number of generators without any coordination. The timestamp provides sort order. There's no machine identifier, so no need to negotiate node identities, and no information leakage beyond the timestamp itself.

v7 is the right choice when:

You need distributed generation without coordination
Time-ordering matters (most database primary key scenarios)
The 128-bit size is acceptable
Creation-time leakage is acceptable

It supersedes Snowflake for use cases where the 64-bit constraint isn't critical, and supersedes v4 for use cases where ordering is valuable.

ULID and KSUID

ULID (Universally Unique Lexicographically Sortable Identifier) and KSUID (K-Sortable Unique IDentifier) are conceptually similar to UUID v7 but with different encodings and slightly different trade-offs.

ULID is 128 bits, encoded as 26 characters of Crockford base32. The first 48 bits are a Unix millisecond timestamp; the remaining 80 bits are random. Compared to v7, ULID has slightly more random bits, uses a case-insensitive encoding that's friendlier to URLs and transcription, and predates v7 by eight years (the v7 design was influenced by ULID).

KSUID is 160 bits, encoded as 27 characters of base62. The first 32 bits are a Unix timestamp in seconds (not milliseconds); the remaining 128 bits are random. KSUIDs are larger than UUIDs but have more random bits, and the second-precision timestamp avoids some of the clock-skew edge cases that affect millisecond-precision formats.

Both formats are well-supported in their respective ecosystems (ULID is popular in JavaScript and Go; KSUID was created by Segment and is widely used in their stack). For new projects without an existing preference, UUID v7 has the weight of RFC standardisation and growing native runtime support; ULID has older library support and a more URL-friendly encoding.

Coordinated allocation patterns

A pattern worth knowing about even though it doesn't have a standardised format: generators request blocks of IDs from a central authority, then issue from the local block without further coordination. The classic example is HiLo (high-low) allocation used by some ORM systems.

A generator requests "the next 1000 IDs starting at 47,000." It then issues 47,000, 47,001, 47,002 locally until the block is exhausted, at which point it requests another block. The coordination cost is amortised across the block size.

This works well for systems with stable generators that need integer IDs, can tolerate some wasted IDs when generators restart with unused allocations, and have access to a coordination service. It's less common in modern systems because v7 and Snowflake-style formats avoid the coordination requirement entirely.

The decision framework

For a new distributed system needing ID generation, work through these questions in order:

1. Do IDs need to fit in 64 bits?

If yes (legacy integer columns, very high-volume systems where every byte matters): Snowflake or a Snowflake variant. Pay the coordination cost for machine IDs.

If no: continue.

2. Do IDs need to be coordination-free?

If you can run a coordination service for one-time machine ID assignment and your generator population is stable: Snowflake is still in play.

If generators are ephemeral, can't coordinate, or must work in environments without network access (browsers, offline-first mobile clients): UUID v4, v7, or ULID.

3. Does ordering matter?

If IDs are database primary keys, are used for chronological access patterns, or benefit from natural sorting: UUID v7 or ULID.

If IDs should specifically not leak creation time, or ordering is irrelevant: UUID v4.

4. Are there ecosystem constraints?

If you're in a JavaScript ecosystem and tooling expects ULID or NanoID-style IDs: prefer those.

If you're in a stack that natively supports UUIDs (.NET, Java, Python, Go, most relational databases): prefer UUID.

If the system spans multiple languages and ecosystems: UUID's universal support is hard to beat.

5. Are there URL or display constraints?

If IDs appear prominently in URLs that users see and care about: consider ULID (shorter than UUID, case-insensitive) or NanoID (shortest, most URL-friendly, but no time-ordering).

If IDs are mostly internal: UUID or Snowflake.

The vast majority of new distributed systems land on UUID v7 after working through these questions. It's coordination-free, time-ordered, well-supported, and has no significant disadvantages for typical use cases. The exceptions — Snowflake for 64-bit requirements, v4 for privacy-sensitive IDs, ULID or NanoID for URL-friendly formats — are real but specific.

What about coordinating for stronger guarantees?

Some systems need guarantees that purely independent generation can't provide:

Strict monotonic ordering — every ID must be strictly greater than every previously-generated ID in the system, with no possibility of out-of-order arrival. Achievable only with a central sequencer or with carefully-managed hybrid logical clocks. Most systems don't actually need this and pay a substantial cost trying to achieve it.

Globally consistent timestamps — IDs whose embedded time is consistent across all generators within a tight bound. Spanner uses TrueTime (specialised hardware and protocols) to achieve this. Most systems use NTP and accept the few-millisecond clock skew across servers as a tolerable imprecision.

Application-level conflict resolution — using IDs as tiebreakers when multiple writers conflict. v7's embedded timestamp is sometimes used for this purpose informally; CRDTs and other formal conflict-resolution schemes use richer structures.

These are real requirements for some systems but exotic enough that most projects shouldn't try to solve them with custom ID generation. If you're considering implementing a custom distributed ID scheme to address one of these, the right next step is usually a literature review (the academic work on logical clocks and CRDTs is extensive and high-quality) rather than implementation.

Common mistakes

A few patterns to avoid:

Reinventing UUID v7 or Snowflake. If your custom design ends up looking like "timestamp plus randomness plus machine ID," you're reimplementing one of these. The standard versions have been tested across far more deployments than any custom variant will see.

Underestimating clock skew. Time-based formats (v7, ULID, KSUID, Snowflake) all assume reasonably synchronised clocks across generators. NTP gives most systems millisecond-level synchronisation; that's usually fine. Systems running on VMs that pause and resume, on containers with imprecise time sources, or on hardware with drifting clocks can produce IDs that appear out of order. Plan for this.

Treating IDs as secrets. Distributed IDs are identifiers, not authentication tokens. Even v4's randomness isn't a guarantee of unguessability across very large populations. Use purpose-built tokens for authentication, not raw UUIDs.

Optimising for the wrong axis. "Faster generation" rarely matters; "smaller IDs" rarely matters; "more random bits" rarely matters. What matters in most real systems is ecosystem fit, ordering properties, and operational simplicity. Optimise for those.

Summary

Distributed ID generation is a well-explored problem with mature solutions. UUID v7 is the right default for most new systems — it's coordination-free, time-ordered, universally supported, and has no significant downsides for typical use cases. Snowflake remains a strong choice when 64-bit IDs are required. UUID v4 is the right choice when ordering should not be inferable from the ID. ULID and KSUID are reasonable alternatives in their respective ecosystems.

The choice rarely turns out to be the limiting factor in system design. Pick the format that fits your ecosystem and constraints, and spend your engineering effort on the harder problems — consistency, partitioning, fault tolerance — that the ID format itself can't solve.

For deeper coverage of specific formats, see UUID versions explained, UUID vs auto-increment primary keys, NanoID vs UUID, and UUID v7 in PostgreSQL — a practical guide. Generate UUIDs, NanoIDs, ULIDs, and other identifiers in your browser with the UUID v7 generator, ULID generator, and NanoID generator.