The system should provide a functionality to insert, modify and read the data.
- One instance of the data store should handle 250 000 messages per second, where each message is no more than 200 bytes.
- A query for a record, with a unique index, should be returned in no more than 10 milliseconds with a 10 billion records database.
- The capacity of a given response stream should be 100 Mbps.
- The response streaming capacity of a datastore server should be 300 Mbps (divided across several clients), while receiving 100 000 messages per second.
- A transaction should be acknowledges on the message bus within 100 microseconds.
- Fail over within 30 seconds on the same site.
It is important to note that consistency and fault-tolerance is an important part when building distributed systems, however the leverage of the internal NASDAQ OMX messaging system INET provides a build-in support for this functionality. In the further posts (at least I really hope 🙂 lol) I will discover how the load will be handled and which techniques will be used to provide a high availability level and specific faults recovery.