Recently I started to look at Handoop MapReduce framework (originally published by Google) . This along with GFS and Bigtable have empowered Google from its early days until today. Unfortunately, at my workplace, we are not actively using this. However, I tried to do a small project on it using…

I’ve been using SNS/SQS at work, and recently also tried EventBridge at my hobby project. I found them all been very useful in different scenarios, and would like to share my feedback on them. SQS — A message queue that decouples sender and receiver. Important configs: Visibility timeout — The length of time…

What is a message queue? A message queue is like a buffer that decouples sending and receiving of messages. It has: Async operation by nature: the producer no need to wait for consumer to retrieve and process the message Decouple: separating the post and receipt of messages to allow multiple producers/receivers post/receive through one/multiple queue. Workflow: …

Event driven architecture has been on the market for years. It is closely linked with cloud, microservices, asynchronous workflow from its birth. In a nutshell, the event-driven architecture is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events (Source) . An event is a…

I did more learning on distributed systems this year, so would like to share some of my learning notes and experiments with you :) . Category: a. Horizontal Partitioning: Different rows are split into different partitions. b. Vertical Partitioning: Different columns are split into different partitions (maybe some denormalisation are needed…

A series of programming design patterns illustration with examples with JavaScript/Python — Simply put, the Observer pattern offers a Publication/Subscription model where observers subscribe to an event and get notified when the event happens. It is one the most important models in event-driven architecture. Essentially it has 2 participants, the observer(s) and the observable. Observable: links to one event type

A series of programming design patterns illustration with examples with JavaScript/Python — Simply put, the Proxy pattern is a way to avoid direct access to an object through a substitute, aka. Proxy. The proxy object looks like the same to the client as the real subject, so they will need to have the same interface. Requests from the client are passed to…

A series of programming design patterns illustration with examples with JavaScript/Python — Simply put, the Flyweight pattern is a Structural Design Pattern that help reduce memory load by minimising the number of objects created of the same type. …

A series of programming design patterns illustration with examples with JavaScript/Python — Simply put, the facade pattern is used to serve as a high level interface for the client to interact with a set of more complex subsystems. It hides the implementation details of the underlying systems and exposes an unified, sometimes even single method to the client. The benefits of it…