For that purpose a distributed database is an ideal solution.Įverything above is worthless if performance of the in-memory data structure is lagging behind the business need. Simply put, there should be a way to distribute the queue’s load across a cluster. When queue is growing, it would be better to distribute the queue content across multiple servers. So there must be a mechanism to periodically write in-memory snapshots to some persistent storage. All of a sudden if something went wrong in the server, whole data set will be vanished in to thin air in a second. Holding a large data set inside the memory is inefficient and risky. I was assigned the task of designing a new data structure to address this problem. Since the queue is a in-memory data structure, its rapid growth affects the overall server performance. Problem arose with this approach is Queue is getting filled rapidly with frames and theres no control over its upper limit. Once a frame has been read from the socket, it will be placed in a In-memory queue so that it can be used for further processing. Initially server connects to stock exchanges and listens for incoming market data frames. The source code for this article and other from our data structure series are available on our GitHub repository.Recently I joined a company which provides real-time stock market information to its clients through a proprietary stock price dissemination server.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |