There has been a spate of news flooding the media after the sudden demise of arguably the musical sensation of the century. There are people who still cannot get over his sudden demise, people who are sick of the continued media coverage and those who are indifferent. This BLOG is a few words from me to express my feelings on the passing of a man whose music I loved growing up and still do.
Going back in time, one day during my schooling years, a friend of mine had this audio cassette that he said I must hear as the songs from it were just super. I heard "Wanna be starting something" and I was of gyrating to the beats. Not the most famous song of M.J, and I cannot claim I understood the lyrics, but I liked it. Especially, I had my own version of "Mama-se, mama-sa, ma-ma-coo-sa" that I would be dancing to ;-)
Then I heard Beat It and Billie Jean. Hearing the vocals and guitaring on the former (Eddie Van Halen on the guitar), hearing the introduction on the latter had me hooked! I must admit that the first time I heard him, I was of the impression that the singer was a female.
Then started the research into who this artiste was. Steps back in time hearing "Off the wall" and others got me more interested in M.J. Following suite was trying to get videos of him performing, something not easily available in India at the time. I watched the moon walk that he performed for the Motown anniversary. Recollecting the time, I remember all my friends trying their very best to do this moon walk. What a dance step. This person was more than just a singer that I liked, he was a dancer that I could never hope to be. I had found an artist who was a complete entertainer. Before you knew it, my brother and I had most of the Jackson songs in our collection, even if it meant begging my parents to buy a tape for one song considering we already had the rest in our collection. Thanks dad and mom for humoring us.
At the time, in India one could either purchase L.P records or Tapes. Music quality from the tapes were not the best, however, even with the quality available I remember my brother and I enjoying the music and trying very hard to decipher the lyrics. No internet available at that time :-)
Time flew, the next album we eagerly awaited and got was "Bad". M.J had changed his face considerably by now, the songs were different compared to "Thriller". It took me time for the songs to grow on me, his looks although strange were not much of a deterrent for me, that was his personal life so to say. M.J still rocked!
Continued to watch sporadic news that trickled in about M.J, him sleeping in an Oxygen Chamber, "Bubbles" the chimp, more surgeries etc etc....To be honest, again these oddities as reported by the media did not affect me in a way to reduce my admiration of the man.
Time flew on, then came the "Pedophile" accusations and scandals. At this point, it was more than eccentric indulgences of a man who has more money than what he knew to do with. I really did not know what to believe. As a fan, I hoped that my childhood idol would be exonerated. As an evaluator, I was of the opinion, a man is innocent till proven guilty. Cut it anyway, the fact that he was a great artist could not be blemished by the results of the court cases. If you choose to, blame it on the legal system or the compromising accusers, there was no conviction. As a bystander, I can only accept the result. Guilty or otherwise, I primary held the parents of the children who claimed abuse responsible for allowing such a situation to present itself in the first place. Did they really need to let their children sleep over in a strangers home??? Come on!
After all these incidents, one would hear of a one off concert where M.J would be in news and then silence ensuing for months, sometimes years on end. Financial problems, and the "Martin Bashir" interview are what I followed. M.J's appeal and record selling ability had dwindled, I however still like the song "Rock my world" though.
Years pass, I hear that he is about to perform again in 2009; a curtains concert and that tickets are sold out like hot cakes. Happy for the man. Also hear that he is in the process of making a new album. Happy to hear the news and hoping he will be back with an album to match the genius of "Thriller".
In my office, I hear, M.J has been hospitalized. Do not give much interest to the news as I do not feel it was serious. This is M.J we are talking about, a great dancer, a physically fit person, at least that was the impression imprinted in my brain growing up. Soon after a colleague of mine says that some sites are reporting M.J is dead. I refuse to believe saying its not been announced on "CNN", its not been announced on "BBC". Denial at work here...
Soon after, my trusted news sites report that M.J has in fact passed away. Saddened now. This just felt chronologically wrong, out of order. How could M.J pass on when others that I had placed ahead of him in the passing order have not? Shocked and saddened at the loss...still not ready to believe the obvious. News that he died in a car accident, News that he was murdered were all acceptable outcomes for my sensibility, but sudden death by heart failure ???
Reach home and am hooked on CNN watching the events unfold. M.J's demise reminded me of my own mortality in a way...the random factor had wreaked havoc for me. Watching CNN get into news about his weirdness was not what I expected. I had hoped that a person such as M.J would be more remembered for his music and creative genius. Felt disappointed that the media were focusing more on the negativity involved in his life.
In the end, there are people who hate him, have convicted him in their minds as a pedophile and there are people who believe he was innocent and then there are majority who simply respected the man for his immense talent. I am part of the latter faction and I will miss the possible creative and artistic contributions he would have provided were he alive. The world has lost a great artiste.
Everyone will remember M.J in their own way. Like one person said, one will remember where one was on the day M.J passed away, I for one definitely will. The man has done considerable work for charity and causes. Maybe more than many of will ever be able to do. Remember the man for what he did good. I admit that he was a strange bird....however please try to remember him for his abilities and contributions as an entertainer/singer.
From my end, every time the song "Billie Jean" or "Beat It" starts on my Boxster, I will have a smile on my face and be in an upbeat mood...it takes me back to my childhood, it reminds me of the good times my brother and I had as kids...thanks M.J...I will miss your singing..Eddie Van Halen or Slash on the guitar..the pedal is hitting the floor....
Search This Blog
Friday, June 26, 2009
Tuesday, June 23, 2009
Scaling in the JMS World...in particular Topics
Scalability and Availability in JMS. For the former, as load on the messaging system increases, one would want to be able to service the increased messages without performance degradation. For the latter, one should ensure the reduction if not elimination of single point of failures, i.e., ensuring system is available for use.
Most JMS Providers have some sort of mechanism for addressing the above. This BLOG is however only going to concentrate on Oracle WebLogic.
WebLogic has an interesting way in which JMS Destinations (Queues and Topics) are scaled via Distributed or Virtual Destinations. A Virtual Destination can be considered a sort of router/load balancer that distributes messages it receives over to actual or physical destinations. Each physical destination would reside in separate WebLogic instances on potentially separate hardware. See figure below where a virtual destination "test_queue" has physical members test_queue_1, test_queue_2 and test_queue_3 on separate WebLogic instances:
As an example of Virtual Queues in action, in the conceptual figure shown below, Producers send messages to the Virtual Queue which then directs the message to one of the physical destinations
P : Producers
VQ: Virtual Queue
PQ-X: Physical Queue, where X is the number of the Queue
C: Consumer
M: Message
In the case of a Virtual Topic, messages sent to the Virtual Topic are sprayed on to each and every physical member as shown in the figure below where VT and PT-X represent Virutal and Physical topics respectively:
Scaling Queues:
From the figure shown regarding Queues, it is pretty simple to follow how the system scales and that it will continue to service new messages and consume them should one of the servers in the messaging cluster fails. A message will be consumed at most by one of many consumers successfully. Adding Consumers to the end of the physical members allow the architecture to process messages rapidly, i.e., Competing Consumer Design Pattern.
Now, what if one of the servers having physical members went down, for example VQ-1 went down? From the perspective of availability, messages are still flowing and getting consumed from the other available member, i.e., VQ-2. What about the messages that were in the physical member whose server went down? One has multiple options. The first and most straight forward is to restart the server and re-connect the consumers to drain the queue. WebLogic allows for auto-service migration in the event of failure. This is another feature that can assist in recovering from failure. Queues readily tend to present themselves for scaling. One note on the same, if message ordering is put into play, all bets are off :-)..more on the same on a subsequent blog.
Scaling Topics:
Topics unlike Queues present an interesting challenge. In the case of a topic, a message is delivered to all listeners of the topic and each listener is typically different than its peer in how it handles the message. For example a message posted to an order topic, might have a consumer that processes the order while another one looks at inventory calculations:
Transfer MDB:
It is pretty trivial to create the Transfer MDB. One would typically like to re-use the Transfer MDB Java Code with only configuration changes for different applications. The same can easily achieved via separate projects where target destination can be provided via @Resource. A simple transfer MDB is shown below:
The CODE for the above MDB and an example Maven Project that uses WebLogic is available for download HERE. Note that the code is not specific to WebLogic and can really be used in any container, for example JBoss.
Conclusion:
Scaling topics via an MDB and target queues just felt like an obvious pattern to me. WebLogic, like many JMS Providers has the concept of a Message Bridge. Initial investigation led me to believe that a message bridge more serves the purpose of transferring a message from one provider to to another rather than serving to scale a Topic and distributed messages to multiple destinations.
There may be cases as mentioned where the number of subscribers are not known. In such a case, a topic might work and it becomes the responsibility of the subscribers to scale on their end, for example multi-threading upon receiving the message.
What really would be nice if the JMS specification had the concept of "Durable Competing Consumers" where a Durable subscriber could register multiple instances of itself such that a message to a topic would be delivered to at most it or one of its clones :-). I believe there are some Messaging providers who permit such a pattern via customization that extend the JMS specifications.
Before bidding adieu, this has been one of my most figurative blogs, all Ye artists, look at the diagrams with a forgiving eye, "I am a programmer not an artist!" :-)
Most JMS Providers have some sort of mechanism for addressing the above. This BLOG is however only going to concentrate on Oracle WebLogic.
WebLogic has an interesting way in which JMS Destinations (Queues and Topics) are scaled via Distributed or Virtual Destinations. A Virtual Destination can be considered a sort of router/load balancer that distributes messages it receives over to actual or physical destinations. Each physical destination would reside in separate WebLogic instances on potentially separate hardware. See figure below where a virtual destination "test_queue" has physical members test_queue_1, test_queue_2 and test_queue_3 on separate WebLogic instances:
As an example of Virtual Queues in action, in the conceptual figure shown below, Producers send messages to the Virtual Queue which then directs the message to one of the physical destinations
P : Producers
VQ: Virtual Queue
PQ-X: Physical Queue, where X is the number of the Queue
C: Consumer
M: Message
In the case of a Virtual Topic, messages sent to the Virtual Topic are sprayed on to each and every physical member as shown in the figure below where VT and PT-X represent Virutal and Physical topics respectively:
Scaling Queues:
From the figure shown regarding Queues, it is pretty simple to follow how the system scales and that it will continue to service new messages and consume them should one of the servers in the messaging cluster fails. A message will be consumed at most by one of many consumers successfully. Adding Consumers to the end of the physical members allow the architecture to process messages rapidly, i.e., Competing Consumer Design Pattern.
Now, what if one of the servers having physical members went down, for example VQ-1 went down? From the perspective of availability, messages are still flowing and getting consumed from the other available member, i.e., VQ-2. What about the messages that were in the physical member whose server went down? One has multiple options. The first and most straight forward is to restart the server and re-connect the consumers to drain the queue. WebLogic allows for auto-service migration in the event of failure. This is another feature that can assist in recovering from failure. Queues readily tend to present themselves for scaling. One note on the same, if message ordering is put into play, all bets are off :-)..more on the same on a subsequent blog.
Scaling Topics:
Topics unlike Queues present an interesting challenge. In the case of a topic, a message is delivered to all listeners of the topic and each listener is typically different than its peer in how it handles the message. For example a message posted to an order topic, might have a consumer that processes the order while another one looks at inventory calculations:
Clearly we cannot have two consumers that process the same order and it is redundant to have duplicate consumers perform the same inventory calculations.
Messaging systems have the concept of a Durable Subscriber, i.e., a Message listener who registers with the topic such that they are delivered messages sent to the topic even when the listener is not connected. Message providers store the message on behalf of the durable subscriber and forward the same when the subscriber comes back on line.
In WebLogic one cannot connect to a durable subscriber using a distributed (virtual) destination JNDI. One has to explicitly register with a physical member. This is one gripe that I have with the WebLogic solution. Adding to the problem, should one connect to one physical member and if that member goes down, attempting to establish the durable subscription on other available members are treated like totally new subscriptions. In consolation, if the server goes down that contains durable subscriptions, then messages will be stored on other physical members of the virtual topic on behalf of the durable subscribers, and when the server is restored, messages get forwarded to the durable subscribers when they comes back on line as shown below:
Now, as each subscriber of the topic has a unique agenda from the other subscribers, the fact that it is the sole entity that services the message, the same represents a single point of failure and a proves a scalability inhibitor as it cannot concurrently process multiple messages.
So how can one scale a topic so that a message on a given type of subscriber can be rapidly processed? One direction is have the consumer of a Topic post the message onto a Queue which can then be processed by competing consumers. One way this can be achieved is to have a Message Driven Bean withing the container whose sole purpose is to transfer the message from the topic to a queue for those consumers that need to scale. An example of the same is shown below:
In the above shown example, as one consumer of the topic needs processing to rapidly occur of messages received, an MDB (registered as a durable subscriber of the topic) is used to transfer every message onto a Queue which is then consumed by competing consumers. As other consumers, DC-1 and DC-2 are not in need of scaling, they remain durable single subscribers.
The above solution works to scale the durable subscriber of a topic. However, in WebLogic when using distributed topics the durable MDB still represents a single point of failure.
A question to ask at this point is, "Do you really need a topic for the use case at hand?" Topics are great IMO when the subscribers tend to be dynamic in nature or do not need to be durable, an example of the former is a stock quote topic and an example of the latter is a event listener that requires notification of an event.
Messaging systems have the concept of a Durable Subscriber, i.e., a Message listener who registers with the topic such that they are delivered messages sent to the topic even when the listener is not connected. Message providers store the message on behalf of the durable subscriber and forward the same when the subscriber comes back on line.
In WebLogic one cannot connect to a durable subscriber using a distributed (virtual) destination JNDI. One has to explicitly register with a physical member. This is one gripe that I have with the WebLogic solution. Adding to the problem, should one connect to one physical member and if that member goes down, attempting to establish the durable subscription on other available members are treated like totally new subscriptions. In consolation, if the server goes down that contains durable subscriptions, then messages will be stored on other physical members of the virtual topic on behalf of the durable subscribers, and when the server is restored, messages get forwarded to the durable subscribers when they comes back on line as shown below:
Now, as each subscriber of the topic has a unique agenda from the other subscribers, the fact that it is the sole entity that services the message, the same represents a single point of failure and a proves a scalability inhibitor as it cannot concurrently process multiple messages.
So how can one scale a topic so that a message on a given type of subscriber can be rapidly processed? One direction is have the consumer of a Topic post the message onto a Queue which can then be processed by competing consumers. One way this can be achieved is to have a Message Driven Bean withing the container whose sole purpose is to transfer the message from the topic to a queue for those consumers that need to scale. An example of the same is shown below:
In the above shown example, as one consumer of the topic needs processing to rapidly occur of messages received, an MDB (registered as a durable subscriber of the topic) is used to transfer every message onto a Queue which is then consumed by competing consumers. As other consumers, DC-1 and DC-2 are not in need of scaling, they remain durable single subscribers.
The above solution works to scale the durable subscriber of a topic. However, in WebLogic when using distributed topics the durable MDB still represents a single point of failure.
A question to ask at this point is, "Do you really need a topic for the use case at hand?" Topics are great IMO when the subscribers tend to be dynamic in nature or do not need to be durable, an example of the former is a stock quote topic and an example of the latter is a event listener that requires notification of an event.
When the number of subscribers are a known or finite in amount, the use of a queue coupled with the MDB can easily achieve a scalable solution as shown below where the MDB transfers messages on the in-queue to subsequent out-queues where competing consumers can scale and consume the messages:
Transfer MDB:
It is pretty trivial to create the Transfer MDB. One would typically like to re-use the Transfer MDB Java Code with only configuration changes for different applications. The same can easily achieved via separate projects where target destination can be provided via @Resource. A simple transfer MDB is shown below:
@MessageDriven
@TransactionAttribute(TransactionAttributeType.REQUIRED)
public class MessageRouterMdb implements MessageListener {
/**
* Map this jms connection factory to the actual one in the weblogic-ejb-jar.xml deployment
* descriptor of the ejb project. The Connection factory to be used will be the one that is
* specific to this mdb.
*/
@Resource(name = "jms/connectionFactory")
private ConnectionFactory connectionFactory;
/**
* Injected in Message Driven Context
*/
@Resource
private MessageDrivenContext mdc;
/**
* Define a comma separated list of destinations JNDI names in the ejb-jar.xml via an environment
* entry.
*/
@Resource
private String targetDestinations;
// Initialized
private List<Destination> targetJmsDestinations;
private MessageProducer messageProducer;
private Connection connection;
private Session session;
@PostConstruct
public void init() {
try {
InitialContext ctx = new InitialContext();
connection = connectionFactory.createConnection();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
messageProducer = session.createProducer(null);
String[] targetDestArray = StringUtils.split(targetDestinations, ",");
targetJmsDestinations = new ArrayList<Destination>();
for (String targetDest : targetDestArray) {
targetJmsDestinations.add((Destination) ctx.lookup(targetDest));
}
connection.start();
}
catch (Exception e) {
throw new RuntimeException("Error Initializing Message Router Mdb", e);
}
}
public void onMessage(Message message) {
try {
// Loop through target destinations and send message to each destination
for (Destination dest : targetJmsDestinations) {
messageProducer.send(dest, message);
}
}
catch (Exception e) {
mdc.setRollbackOnly();
}
}
@PreDestroy
public void cleanup() {
// Close open jms resources here...
}
}
The CODE for the above MDB and an example Maven Project that uses WebLogic is available for download HERE. Note that the code is not specific to WebLogic and can really be used in any container, for example JBoss.
Conclusion:
Scaling topics via an MDB and target queues just felt like an obvious pattern to me. WebLogic, like many JMS Providers has the concept of a Message Bridge. Initial investigation led me to believe that a message bridge more serves the purpose of transferring a message from one provider to to another rather than serving to scale a Topic and distributed messages to multiple destinations.
There may be cases as mentioned where the number of subscribers are not known. In such a case, a topic might work and it becomes the responsibility of the subscribers to scale on their end, for example multi-threading upon receiving the message.
What really would be nice if the JMS specification had the concept of "Durable Competing Consumers" where a Durable subscriber could register multiple instances of itself such that a message to a topic would be delivered to at most it or one of its clones :-). I believe there are some Messaging providers who permit such a pattern via customization that extend the JMS specifications.
Before bidding adieu, this has been one of my most figurative blogs, all Ye artists, look at the diagrams with a forgiving eye, "I am a programmer not an artist!" :-)
Subscribe to:
Posts (Atom)
