unique id generation at high scale

Moving away from relational db allows us to work with huge data, which would otherwise be not feasible. But, it also mandates us to look into the issues that we didn't have to consider much about.

One of such thing was generating unique ids. With mysql you get auto-incremented ids which work absolutely fine for most of the use cases. But, for other systems, in context of nosql/distributed systems, that job is on system designer's shoulder. Inherently one-size fits all solution is difficult to come with.

There are many solutions proposed for this ubiquitous problem.

Twitter has developed snowflake which solves it.  They guarantee id generation with the following constraints :

• minimum 10k ids per second per process
• response rate 2ms (plus network latency

The other solution I found is  flake  by boundary. It achieves the goal of generating ids with good performance and adequate bit size 128 which snowflake lacked.

I will update more information on it later, as I confront with more of these.