The 25 Best Quotes from Tech History. Think of my colleagues at work will enjoy the one on JINI. My favourite quote of course lies with HAL.
http://www.computerworld.com/s/article/9141008/25_best_quotes_from_tech_history
Search This Blog
Tuesday, November 17, 2009
Saturday, October 31, 2009
Ubuntu Karmic and Windows 7
Did two upgrades this weekend. Upgraded a laptop from Vista to Windows 7 and a desktop from Ubuntu Jaunty to Karmic, both 64 bit. Pleased with both upgrades so far. Vista was one of MS biggest lows as far as their OS version goes, Windows 7 seems more peppier in general so far.
I have been very pleased with Ubuntu ever since I installed the same. Have previously been a user of Mandriva, Fedora and Suse. Found mandriva comprehensive in its software offering but not very stable. I feel that Suse may be great to compete with Redhat on the server market but is no where as user friendly as Ubuntu. What I like about Ubuntu is that its pretty solid, convenient and just so easy to use. It detects the need for propriety drivers and installs in a breeze. Have dual monitor support without any xorg.conf tweaking. Printer, web cam all worked out of the box. Have it now running Skype as well without a problem.
After installing Ubuntu, I followed the following BLOG on a TODO list that really helped.
Anyway, got rid of the standard 1948 created Gnome menu in favour of AWN and now my desktop rocks.
A note for those installing eclipse 64 bit. One will find that clicking on some buttons does not work and one has use the keyboard actions to activate the buttons. The way around this problem is to set the following "export GDK_NATIVE_WINDOWS=true" prior to launching eclipse.
On the Windows 7 front, I prefer using Chrome over firefox or IE as its so much more faster on many sites that I frequent. The upgrade from Dell from Vista to Win 7 was pretty simple. Cannot watch youtube on 64 bit IE yet due to 64 flash plugin not available (no so on my Ubuntu though :-)) Don't get me wrong, I don't have anything against Win 7 (except $$$) but when compared with what FREE brings, I find it difficult to justify the extra $$$. I however do need to a Win machine due to the fact that there is no software for the iphone of any value on Linux and that there are sites that I go to such as CWTV to watch videos requiring Win or Mac. Another seller for the Win 7 is my few PC games that I have. BTW, I did not buy Win 7, its a free upgrade option.
The Mrs also is happy with Ubuntu...and as long as she is happy, I am as well :-)
I have been very pleased with Ubuntu ever since I installed the same. Have previously been a user of Mandriva, Fedora and Suse. Found mandriva comprehensive in its software offering but not very stable. I feel that Suse may be great to compete with Redhat on the server market but is no where as user friendly as Ubuntu. What I like about Ubuntu is that its pretty solid, convenient and just so easy to use. It detects the need for propriety drivers and installs in a breeze. Have dual monitor support without any xorg.conf tweaking. Printer, web cam all worked out of the box. Have it now running Skype as well without a problem.
After installing Ubuntu, I followed the following BLOG on a TODO list that really helped.
Anyway, got rid of the standard 1948 created Gnome menu in favour of AWN and now my desktop rocks.
A note for those installing eclipse 64 bit. One will find that clicking on some buttons does not work and one has use the keyboard actions to activate the buttons. The way around this problem is to set the following "export GDK_NATIVE_WINDOWS=true" prior to launching eclipse.
On the Windows 7 front, I prefer using Chrome over firefox or IE as its so much more faster on many sites that I frequent. The upgrade from Dell from Vista to Win 7 was pretty simple. Cannot watch youtube on 64 bit IE yet due to 64 flash plugin not available (no so on my Ubuntu though :-)) Don't get me wrong, I don't have anything against Win 7 (except $$$) but when compared with what FREE brings, I find it difficult to justify the extra $$$. I however do need to a Win machine due to the fact that there is no software for the iphone of any value on Linux and that there are sites that I go to such as CWTV to watch videos requiring Win or Mac. Another seller for the Win 7 is my few PC games that I have. BTW, I did not buy Win 7, its a free upgrade option.
The Mrs also is happy with Ubuntu...and as long as she is happy, I am as well :-)
Friday, October 16, 2009
Exceptions in RESTful HTTP with Restlet and JAXB
REST, or rather, RESTful HTTP and how exceptions can be handled therein is what this posting is about.
For those of us who have worked with SOAP and faults, we are familiar with how exceptions are handled. SOAP Stacks upon encountering an exception will throw a SOAP fault which is then marshalled into an exception on the consumer end of the service. SOAP faults in a way provide a protocol over standard HTTP.
The scope of this example is as follows:
e. Simple mechanism to throw exceptions on the Client:
Clients are aware of the Exceptions that can be expected from a Service and can expect them to be thrown appropriately. For example:
Note that it would have been nice if we could specify the exceptions that a method could throw using generics. That feature however is restricted to a single exception type and thus accommodating checked exceptions are hard.
f. Resource Classes throw Exceptions on failed code states:
Resource classes will be able to throw any type of WebServiceException and have it marshalled to the client. The Resource classes must however ensure they populate the Exception with the appropriate Http Status Code. For example, an OrderResource on create might throw an OrderNotValidException or an OrderException when a POST operation is invoked on it. At the same time, a GET on the OrderResource might throw a OrderNotFoundException.
g. Marshalling of exceptions occurs via a Restlet Status Service:
A custom extension of the Restlet Status Service is responsible for marshalling all thrown Exceptions. The same is shown below:
I have included herewith a Maven example that will demonstrate the client, service and exception management in use. The maven module titled "exceptionnmodule" contains the exception handling code. The integration will demonstrate different exceptions that are thrown by the server and re-thrown by the client code. In particular, you can see the the following abilities of the framework:
a. Ability to throw a simple RuntimeException from the Service and see it marshalled/unmarshalled as WebServiceException
b. Ability to throw an instance of WebServiceException
c. Ability to throw an exception that is a subtype of WebServiceException but without any addition body. For example, the OrderException.
d. Ability to throw a sub type of the WebServiceException that has additional body elements. For example, OrderValidationException.
One can witness the server stack, cause, HTTP Status, specific exception types. Simply run a "mvn install" from the top level project.
Although the example uses Restlet 2.0, the framework has no dependencies on the same. There might be many areas for improvement in the concept and I would like to hear about them if a reader of this blog has ideas. Also, in this example, I have used code from a posting by Ian Robertson on Artima regarding Reflecting Generics.
Finally, I don't know how this BLOG will format. I am trying something new, I hope it woiks!
Download the Code HERE!
For those of us who have worked with SOAP and faults, we are familiar with how exceptions are handled. SOAP Stacks upon encountering an exception will throw a SOAP fault which is then marshalled into an exception on the consumer end of the service. SOAP faults in a way provide a protocol over standard HTTP.
When working with REST we have the concept of Resources and their Representations. An exception in a RESTful architecture is really a tuple of an HTTP Status Code and a Representation that describes the same.
The HTTP Status code in themselves are quite indicative, delving a bit into the same:
The HTTP Status code in themselves are quite indicative, delving a bit into the same:
- 1XX Series - Informational - Request received and continuing processing
- 2XX Success - Action received, understood and accepted
- 3XX Redirection - Further action required to complete the request
- 4XX Client Error - Request is badly formatter or the server cannot process the request
- 5XX Server Error - Server failure on a a potentially valid request
What I am trying to do in this BLOG is a demonstrate how exception handling can be accomplished with a framework like Restlet. I am not demonstrating a specification by any means or a wrapper protocol. For the sake of this demonstration, I will be using Restlet and JAXB. In particular, the example will rely on the XmlAdapters of JAXB extensively coupled with the Java Reflection API and Annotations. The XmlAdapters serve to translate to and from Exceptions to JAXB Representions
The scope of this example is as follows:
a. Exceptions supported are of a particular type and its subclasses only:
Exceptions are restricted to WebServiceException and its sub classes. In addition, the WebServiceException is a RuntimeException. No checked exceptions. Any exceptions thrown by a Restlet Resource that are not an instance of WebServiceException will be converted to the same prior to being marshalled. Any WebServiceException marshalled, must have a JAXB Object associated with it.
Exceptions are restricted to WebServiceException and its sub classes. In addition, the WebServiceException is a RuntimeException. No checked exceptions. Any exceptions thrown by a Restlet Resource that are not an instance of WebServiceException will be converted to the same prior to being marshalled. Any WebServiceException marshalled, must have a JAXB Object associated with it.
@XmlJavaTypeAdapter(WebServiceExceptionAdapter.class)
public class WebServiceException extends RuntimeException {
// Http Status code
private int status;
}
// This class extends WebServiceException but has the same body as
// WebServiceException. For this reason, there is no special XmlAdapter required.
public class OrderException extends WebServiceException {
..
}
b. XML as the media type for Transferring Exceptions:
XML is the sole media type supported for exceptions in this example. Note that it is quite trivial to change the same to accommodate additional media types like JSON etc if required. Every Exception's representation is wrapped around with a WebServiceExceptionWrapperDto. The same is to allow for sub typing. Note that the targetException will be marshalled by an appropriate XmlAdapter at Runtime.
XML is the sole media type supported for exceptions in this example. Note that it is quite trivial to change the same to accommodate additional media types like JSON etc if required. Every Exception's representation is wrapped around with a WebServiceExceptionWrapperDto. The same is to allow for sub typing. Note that the targetException will be marshalled by an appropriate XmlAdapter at Runtime.
@XmlType(name = "ExceptionWrapper", propOrder = { "targetException" })
@XmlRootElement
public class WebServiceExceptionWrapperDto<E extends WebServiceException> {
@XmlElement(name = "targetException", required = true)
public E targetException;
}
c. Rich Exceptions:
Exceptions that can be richer, i.e., more than just message, cause and stack. For example validation messages, codes, inner objects etc. A simple example of the same is shown below where the OrderValidationException contains a description and a custom validation error code. The same could easily have been a complex object if required:
Exceptions that can be richer, i.e., more than just message, cause and stack. For example validation messages, codes, inner objects etc. A simple example of the same is shown below where the OrderValidationException contains a description and a custom validation error code. The same could easily have been a complex object if required:
@XmlJavaTypeAdapter(OrderValidationExceptionAdapter.class)
public class OrderValidationException extends WebServiceException {
// A code representing the error
private int validationCode;
// A description indicating the cause of the validation error
private String validationMessage;
...
}
// Note that the OrderValidationException Adapter is behaving like a Template method pattern
// Base marshalling to an exception is handled by the parent class with the child only providing
// specific additions. The methods marshall() and unmarshall() are implemented in the parent class
// and will populate the base properties of the Exception and representation accordingly
public class OrderValidationExceptionAdapter extends
AbstractExceptionXmlAdapter<OrderValidationExceptionDto,OrderValidationException> {
@Override
public OrderValidationException toException(OrderValidationException created,
OrderValidationExceptionDto dto) throws Exception {
created.setValidationCode(dto.validationCode);
created.setValidationMessage(dto.validationMessage);
return created;
}
@Override
public OrderValidationExceptionDto toRepresentation(OrderValidationExceptionDto created,
OrderValidationException t) throws Exception {
created.validationCode = t.getValidationCode();
created.validationMessage = t.getValidationMessage();
return created;
}
d. Control over which parts of an Exception are marshalled:
Provide server stack, cause and message to client. At the same time, ability to prevent certain exceptions from transmitting stack. One might or might not want to marshall server stack in certain cases. The same will be accomplished via an annotation that resides on an Exception thrown. Note that the same can be enhanced for further control. In the example shown below, throwing of the BarException will not result in the marshalling of the stack or cause:
Provide server stack, cause and message to client. At the same time, ability to prevent certain exceptions from transmitting stack. One might or might not want to marshall server stack in certain cases. The same will be accomplished via an annotation that resides on an Exception thrown. Note that the same can be enhanced for further control. In the example shown below, throwing of the BarException will not result in the marshalling of the stack or cause:
/**
* Foo Exception when thrown should not include server stack and any cause
*/
@ExceptionMarshallingParams(includeStack=false, marshallCause=false)
public class BarException extends WebServiceException {
}
e. Simple mechanism to throw exceptions on the Client:
Clients are aware of the Exceptions that can be expected from a Service and can expect them to be thrown appropriately. For example:
public OrderDto createOrder(OrderDto order) throws OrderValidationException, OrderException {
JaxbRepresentation<OrderDto> orderRep = new JaxbRepresentation<OrderDto>(order);
Response response = client.post(baseUri + ORDERS_RESOURCE_URI, orderRep);
if (response.getStatus().isSuccess()) {
orderRep = new JaxbRepresentation<OrderDto>(response.getEntity(), OrderDto.class);
try {
return orderRep.getObject();
}
catch (IOException e) {
throw new OrderException(e.getMessage(), e, response.getStatus());
}
}
// Not a Response we expected..throw one of the following exceptions expected instead
throw ExceptionUtil
.getWebServiceException(response.getEntity(), OrderValidationException.class, OrderException.class);
}
public class ExceptionUtil {
...
public static WebServiceException getWebServiceException(Representation xmlRep,
Class... expectedExceptions) {
StringBuilder packages = new StringBuilder(20);
packages.append(WebServiceException.class.getPackage().getName());
for (Class clazz : expectedExceptions) {
String ctxPath = getContextPath(clazz);
packages.append(":").append(ctxPath);
}
JaxbRepresentation<WebServiceExceptionWrapperDto<WebServiceException>> wsRep
= new JaxbRepresentation<WebServiceExceptionWrapperDto<WebServiceException>>(
xmlRep, packages.toString());
try {
return wsRep.getObject().targetException;
}
catch (IOException e) {
throw new RuntimeException("Error unmarshalling exception", e);
}
}
...
}
Note that it would have been nice if we could specify the exceptions that a method could throw using generics. That feature however is restricted to a single exception type and thus accommodating checked exceptions are hard.
f. Resource Classes throw Exceptions on failed code states:
Resource classes will be able to throw any type of WebServiceException and have it marshalled to the client. The Resource classes must however ensure they populate the Exception with the appropriate Http Status Code. For example, an OrderResource on create might throw an OrderNotValidException or an OrderException when a POST operation is invoked on it. At the same time, a GET on the OrderResource might throw a OrderNotFoundException.
public class OrderResource extends ServerResource {
@Post("xml")
public OrderDto createOrder(OrderDto dto) {
try {
dto = persistOrder(dto);
setStatus(Status.SUCCESS_CREATED);
return dto;
} catch (OrderValidationException ove) {
ove.setStatus(Status.CLIENT_ERROR_BAD_REQUEST);
throw ove;
}
catch (OrderException e) {
getResponse().setStatus(Status.SERVER_ERROR_INTERNAL);
throw e;
}
}
@Get("xml")
public OrderDto getOrder() {
....
Order order = null;
try {
order = orderService.getOrder(orderId);
}
catch (OrderNotFoundException nfe) {
nfe.setStatus(Status.CLIENT_ERROR_NOT_FOUND);
throw nfe;
}
return (OrderDto) beanMapper.map(order, OrderDto.class);
}
....
}
public class HealthResource extends ServerResource {
private static int counter = 0;
@Get
public String isHealthy() {
// Simply to prove a point....
switch (counter) {
case 0:
counter++;
// Throw a WebServiceException
throw new WebServiceException("System is unhealthy", Status.CLIENT_ERROR_UNAUTHORIZED);
case 1:
counter++;
// throw a RuntimeExeception..this will be transparently marshalled
throw new RuntimeException("Some unexpected exception thrown as a Runtime");
}
getResponse().setStatus(Status.SUCCESS_OK);
return "OK YEAH!";
}
}
g. Marshalling of exceptions occurs via a Restlet Status Service:
A custom extension of the Restlet Status Service is responsible for marshalling all thrown Exceptions. The same is shown below:
public class JaxbExceptionStatusService extends StatusService {
....
public Status getStatus(Throwable throwable, Request request, Response response) {
Throwable cause = throwable.getCause();
WebServiceException exception;
String contextPath = ExceptionUtil.BASE_EXCEPTION_PACKAGE;
if (cause == null || !(cause instanceof WebServiceException)) {
// If not a WebService Exception then throw a WebService Exception
exception = new WebServiceException(..., Status.SERVER_ERROR_INTERNAL);
} else {
exception = (WebServiceException) cause;
String clazzContext = ExceptionUtil.getContextPath(cause.getClass());
if (!contextPath.equals(cause.getClass().getPackage().getName())
&& !clazzContext.equals(contextPath)) {
contextPath = contextPath + ":" + clazzContext;
}
if (exception.getStatus() == 0) {
exception.setStatus(Status.SERVER_ERROR_INTERNAL);
}
}
// Create the Exception Wrapper
WebServiceExceptionWrapperDto<WebServiceException> exceptionWrapper = new WebServiceExceptionWrapperDto<WebServiceException>();
exceptionWrapper.targetException = exception;
JaxbRepresentation<WebServiceExceptionWrapperDto<?>> rep = new JaxbRepresentation<WebServiceExceptionWrapperDto<?>>(
exceptionWrapper);
rep.setContextPath(contextPath);
response.setEntity(rep);
return Status.valueOf(exception.getStatus());
}
}
I have included herewith a Maven example that will demonstrate the client, service and exception management in use. The maven module titled "exceptionnmodule" contains the exception handling code. The integration will demonstrate different exceptions that are thrown by the server and re-thrown by the client code. In particular, you can see the the following abilities of the framework:
a. Ability to throw a simple RuntimeException from the Service and see it marshalled/unmarshalled as WebServiceException
b. Ability to throw an instance of WebServiceException
c. Ability to throw an exception that is a subtype of WebServiceException but without any addition body. For example, the OrderException.
d. Ability to throw a sub type of the WebServiceException that has additional body elements. For example, OrderValidationException.
One can witness the server stack, cause, HTTP Status, specific exception types. Simply run a "mvn install" from the top level project.
Although the example uses Restlet 2.0, the framework has no dependencies on the same. There might be many areas for improvement in the concept and I would like to hear about them if a reader of this blog has ideas. Also, in this example, I have used code from a posting by Ian Robertson on Artima regarding Reflecting Generics.
Finally, I don't know how this BLOG will format. I am trying something new, I hope it woiks!
Download the Code HERE!
Wednesday, August 26, 2009
Restlet 2.0 example using Spring, Maven, Annotations and a Custom Converter
Its been some time since I posted regarding REST. Restlet 2.0 is on its way, I figured its time I updated the example I had created some time ago to a Restlet 2.0 milestone release. If you are continuing to read this BLOG, I would recommend a read of the Restlet 1.1 BLOG I mentioned prior to reading this one.
So whats new with Restlet 2.0? A read of the Resource API refactoring page is recommended.
One of the major features that I liked about JAX-RS over the core Restlet API while working with the former is that it is annotation driven versus class driven. Automatic content negotiation and the calling of the appropriate method in your Server Resource class for different Content-Types is really best left to the container. A method intended to serve content should not have if/else blocks for different content, something one would need to do in Restlet 1.1. Restlet 2.0 seems to take the same into consideration by providing the "best of both worlds"
There were major changes that occurred on Restlet version 1.2 that the team at Restlet decided to change the release to a 2.0. Further details of the change can be obtained at http://blog.noelios.com/2009/05/27/restlet-2-0-m3-released/
Restlet 2.0 has a major change in the way Resources are viewed. In particular a split into a ClientResource and a ServerResource. Details of the same can be viewed on the Restlet Resource API refactoring WIKI page. In addition, there has been a refactoring of certain core classes to different packages. I also noticed that the Restlet API and Restlet engine are shipped as a single jar org.restlet.jar. I could not find the com.neolios packages any more. Finally, there is a jse and j2ee edition of Restlet available.
So, eager to try out the Restlet 2.0 annotations, I reworked my example to use Restlet 2.0 while simplifying the Spring integration present.
Compared to my previous example, I have reduced my Spring applicationContext.xml to be more thinner by using a more annotated approach.
<context:component-scan base-package="com.welflex"> </context:component-scan> <!-- Spring Application. Note there are no mapping of resources here --> <bean id="application" class="org.restlet.Application"> <property name="name" value="orderApplication"></property> <property name="root" ref="root" /> <!-- Added to handle Exceptions centrally --> <property name="statusService" ref="applicationStatusService"/> </bean> <bean id="root" name="router" class="com.welflex.order.rest.SpringBeanRouter"/> <bean id="beanMapper" class="net.sf.dozer.util.mapping.DozerBeanMapper"> <property name="mappingFiles"> <list> <value>dozerBeanMapping.xml</value> </list> </property> </bean>The OrdersResource has changed to:
@Component(value = "/orders") @Scope(value = BeanDefinition.SCOPE_PROTOTYPE) public class OrdersResource extends ServerResource { ...... @Post public JaxbRepresentation<OrderDTO> createOrder(Representation rep) { ... return new JaxbRepresentation<OrderDTO>(MediaType.APPLICATION_XML, dto); } }Note that the OrderResource class now extends ServerResource unlike in my previous example which extended the Resource class. Also note that the mapping of the resources to paths are defined in the OrderResource class itself unlike in the xml file in my previous example. The POST method, called createOrder() accepts and returns XML. The return type is a JaxbRepresentation. If we desired, we could have another method in the code that accepts JSON content and returns JSON content that is titled createJSONRepresentation(). Depending on the content type requested, the appropriate method would be called. This is much better than having an if/else block in an acceptRepresentation(Variant v) method. Take a look at the Products Resource, that returns a JSON representation when a GET method is invoked. Note that this method will only return JSON content.
@Component(value = "/products") @Scope(value = BeanDefinition.SCOPE_PROTOTYPE) public class ProductsResource extends ServerResource { ...... @Get public JsonRepresentation getProducts() { ..... return new JsonRepresentation(jsonString); } }Pretty nice, I would say. To spoil the party, one of the things I liked about the JAXRS implementations was that I could annotate a Resource with the media type it supported and if it were a JAXB object it would automatically marshall the XML, if it were JSON, it would automatically marshal the same as well. The same can be accomplished in Restlet by using a Converter Service that converts a java object to and from a media type. There are plans for adding JAXB converters amongst others in 2.0 M5. In short, by using a Converter Service, I would like the createOrder() method to have the following signature, i.e., method returns a JAXB Object, however the container knows how to marshal that into XML back to the caller:
@Post("xml":"xml") public OrderDTO createOrder(OrderDTO inOrder) { return outOrderDTO; }So what about the ClientResource class? How does that work? Well with Restlet 2.0, one can continue to use the Restlet Client class or use the ClientResource class. The ClientResource is not thread safe. An example of how the ClientResource can be used is shown below in the OrderClientImpl class:
public class OrderClientImpl implements OrderClient { private static final String ORDERS_RESOURCE_URI = "/orders"; private final String baseUri; public OrderClientImpl(String baseUri) { this.baseUri = baseUri; } public OrderDTO createOrder(OrderDTO order) throws IOException { ClientResource ordersResource = new ClientResource(baseUri + ORDERS_RESOURCE_URI); try { return ordersResource.post(order, OrderDTO.class); } catch (ResourceException e) { throw new IOException(e); } } public void updateOrder(OrderDTO order) throws OrderException { JaxbRepresentation<OrderDTO> orderCmd = new JaxbRepresentation<OrderDTO>( MediaType.APPLICATION_XML, order); ClientResource ordersResource = new ClientResource(baseUri + ORDERS_RESOURCE_URI + "/" + String.valueOf(order.getOrderId())); try { ordersResource.put(orderCmd); } catch (ResourceException e) { throw new OrderException("Order Update Failed", e); } } public OrderDTO getOrder(Long orderId) throws OrderNotFoundException, IOException { ....} } public void deleteOrder(Long orderId) throws OrderException {....} }I quite like the use of ClientResource versus the Restlet Client as I feel its more RESTful, i.e., talking to a Resource using HTTP verbs. However, again we witness the JAXB representation creeping in as we saw with the service due to the lack of a converter. I decided to create my own Converter for JAXB to see if I can get it to work and simplify the code base. One of the things I wanted to add just for my pleasure is a way to propagate server stack traces to the client. This is NOT what one would typically include in a representation of a resource as one would not want clients getting details of the working of a server, however, as this is my playground, anything goes ;-). After some tinkering, the following is my Jaxb Converter, it assumes that all JAXB root level objects have the annotation @XmlRootElement:
public class CustomXmlConverter extends XmlConverter { ...... @Override public List<Class<?>> getObjectClasses(Variant source) { List<Class<?>> result = super.getObjectClasses(source); result = addObjectClass(result, JaxbRepresentation.class); result = addObjectClass(result, Object.class); return result; } @Override public List<VariantInfo> getVariants(Class<?> source) { List<VariantInfo> result = super.getVariants(source); if (source.getAnnotation(XmlRootElement.class) != null || source.isAssignableFrom(JaxbRepresentation.class)) { result = addVariant(result, VARIANT_APPLICATION_ALL_XML); result = addVariant(result, VARIANT_APPLICATION_XML); result = addVariant(result, VARIANT_TEXT_XML); } return result; } @SuppressWarnings("unchecked") @Override public <T> T toObject(Representation source, Class<T> target, UniformResource resource) throws IOException { ..... if (target.getAnnotation(XmlRootElement.class) != null) { return new JaxbRepresentation<T>(source, target).getObject(); } throw new IllegalStateException("Should not have got here"); } @Override public Representation toRepresentation(Object source, Variant target, UniformResource resource) throws IOException { Representation result = null; if (source.getClass().getAnnotation(XmlRootElement.class) != null) { result = new JaxbRepresentation<Object>(source); } else { result = super.toRepresentation(source, target, resource); } return result; } ..... }Great, so we have a Converter, how do we make Restlet use this converter for XML. In my common maven module (shared between client and web), I define a file in META-INF/services/org.restlet.engine.converter.ConverterHelper, and in that file, I have a single entry defining my custom converter. Now when the client and service code use the common library, they will use the Custom Converter I have defined for JAXB. Thus, to illustrate the change, my new resources for Orders and Order are shown below:
@Component(value = "/orders") @Scope(value = BeanDefinition.SCOPE_PROTOTYPE) public class OrdersResource extends ServerResource { ..... @Post("xml") public OrderDTO createOrder(OrderDTO dto) { .... return createdOrderDto; } ... } @Component(value = "/orders/{id}") @Scope(value = BeanDefinition.SCOPE_PROTOTYPE) public class OrderResource extends ServerResource { @Put(value = "xml") public void updateOrder(OrderDTO orderDTO) { try { String id = (String) getRequest().getAttributes().get("id"); orderDTO.setOrderId(Long.parseLong(id)); persistOrder(orderDTO); } catch (RuntimeException e) { getResponse().setStatus(Status.CLIENT_ERROR_BAD_REQUEST); throw new OrderException("Failed to update an order", e); } } ..... }We are no longer returning Representations in the above code. Quite similar to JAXRS, wouldn't you say ;-)? On the Client end, the OrderClientImpl no longer deals with Representations but only OrderDTOs as shown below:
public class OrderClientImpl implements OrderClient { .... public OrderDTO createOrder(OrderDTO order) throws OrderException { ClientResource ordersResource = new ClientResource(baseUri + ORDERS_RESOURCE_URI); try { return ordersResource.post(order, OrderDTO.class); } catch (ResourceException e) { throw new OrderException("Error Creating an Order", e); } } public void updateOrder(OrderDTO order) throws OrderException { ClientResource ordersResource = new ClientResource(baseUri + ORDERS_RESOURCE_URI + "/" + String.valueOf(order.getOrderId())); try { ordersResource.put(order); } catch (ResourceException e) { throw new OrderException("Order Update Failed", e); } } }I like the simplicity of the above and cannot wait to see the release of Restlet 2.0 that has the additional converters built in. For those interested, I have a maven project example available for download. The project provided uses the custom JAXB converter. To execute the project, one would need to use JDK1.6.X and maven 2.0.9 or higher. To view an integration test in action execute, "mvn install" from the root level of the project. The result should be something like:
------------------------------------------------------- T E S T S ------------------------------------------------------- Running com.welflex.order.IntegrationTest log4j:WARN No appenders could be found for logger (com.welflex.client.ProductClientImpl). log4j:WARN Please initialize the log4j system properly. Aug 28, 2009 8:59:55 PM org.restlet.engine.http.StreamClientHelper start INFO: Starting the HTTP client 20:59:55 DEBUG - com.welflex.order.rest.ProductsResource.getProducts(44) | Getting Products in JSON Format Number of Available Products:3 Product Id:663123 Product Id:9912123 Storing the order... Aug 28, 2009 8:59:55 PM org.restlet.engine.http.StreamClientHelper start INFO: Starting the HTTP client 20:59:56 DEBUG - com.welflex.order.rest.OrdersResource.createOrder(45) | Call to post an order:.......... Updating the order... Order successfully Retrieving the order... Deleting the order.. 20:59:56 DEBUG - com.welflex.order.rest.OrderResource.getOrder(86) | Requested order with id:5499731937442305515 20:59:56 DEBUG - com.welflex.order.rest.ApplicationStatusService.getStatus(21) | In Order Status Service :class org.restlet.resource.ResourceException Expected Order to not be found. Look at the server stack we got back com.welflex.exception.OrderNotFoundException: Error obtaining Order at com.welflex.client.OrderClientImpl.getOrder(OrderClientImpl.java:60) at com.welflex.order.IntegrationTest.testLifeCycle(IntegrationTest.java:131) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:59)More on Restlet 2.0 once its released. For now, maybe I should probably look toward cleaning up the JaxbConverter and submitting it to Restlet or better still head in the search of kindred spirits, if you know what I mean ;-)
Tuesday, August 11, 2009
Documentating Software, my 2c
Documentation....Aaargh..why in heavens name would anyone waste time on it? The following few words are a window into my experiences and thoughts on documentation in IT organizations.
Sadly, most organizations cannot or will not afford, the luxury of an additional resource who is dedicated to just the task of documentation, i,e., a technical writer. So as developers and architects where does that leave us ? I have heard statements such as:
What I find myself questioning is:
I am a very visual person, many a time a figure to me means more than a 1000 words. If there is a diagram that describes the stack the company uses, the applications, the services, even if partially wrong, immediately gives me a description of the architecture and/or project. That is a big win as far as acclimatising me to the environment.
When different folk ask the same question to me many times over, to me, that is another clearly indication of a lack of documentation. If a topic is constantly questioned, it deserves a place IMHO for documentation such that the next time the question arises, I point the person posing the question to the documentation instead of the repeated sermon that only I hold the copyrights to. DRY (Don't repeat yourself) does not apply to code alone...
Documentation is often created with great detail and appears as a big win, however if it starts becoming stale, then it loses its value and significance and becomes more of a "White Elephant". A conversation I had with someone in the past:
Someone: "Can you help me with this?"
Me: "Sure, did you check the documentation on the same?"
Someone: "Yeah but that is so outdated, so I don't bother looking at it any more"
The above conversation totally supports the argument that bad or documentation that are stale are definite deterrents to readers.
The following represent a few directions regarding documentation that I will try to adhere to:
Sadly, most organizations cannot or will not afford, the luxury of an additional resource who is dedicated to just the task of documentation, i,e., a technical writer. So as developers and architects where does that leave us ? I have heard statements such as:
- "Your code should be clear enough such that documenting the same is redundant"
- "Documentation is not much value unless its kept up to date, bad or expired documentation is worse than no documentation"
- "It is more important that the code has enough test coverage. The tests if good, document the code use"
- "We do not have the time, it is lowest on our priority list. We got bigger fish to fry."
What I find myself questioning is:
- If the code is so self documenting or clear, and if you find people keep asking the same question over an over, is it really so self documenting ?
- If one created documentation, clearly it was considered important. If important, then it would need to updated with the same love when created.
- And please, there is always time...:-)
I am a very visual person, many a time a figure to me means more than a 1000 words. If there is a diagram that describes the stack the company uses, the applications, the services, even if partially wrong, immediately gives me a description of the architecture and/or project. That is a big win as far as acclimatising me to the environment.
When different folk ask the same question to me many times over, to me, that is another clearly indication of a lack of documentation. If a topic is constantly questioned, it deserves a place IMHO for documentation such that the next time the question arises, I point the person posing the question to the documentation instead of the repeated sermon that only I hold the copyrights to. DRY (Don't repeat yourself) does not apply to code alone...
Documentation is often created with great detail and appears as a big win, however if it starts becoming stale, then it loses its value and significance and becomes more of a "White Elephant". A conversation I had with someone in the past:
Someone: "Can you help me with this?"
Me: "Sure, did you check the documentation on the same?"
Someone: "Yeah but that is so outdated, so I don't bother looking at it any more"
The above conversation totally supports the argument that bad or documentation that are stale are definite deterrents to readers.
The following represent a few directions regarding documentation that I will try to adhere to:
- Enterprise or Project Documentation or "The BIG Picture documentation": From an Enterprise architect perspective, providing a high level description/diagram of the entire enterprise and how it all fits or works together can be valuable. The documentation does not have to drill into the microscopic details but instead provide a more a high level view that helps an entrant understand how all the parts of an enterprise fit together. From the perspective of a team lead/lead developer, it would be beneficial if documentation were provided regarding the boundaries and dependencies of a particular project. There is no need for details of a database schema, however there is need to indicate that the application talks to a particular schema on a particular database . These documents might even be able to help detect problems in the architecture, as now, you are looking at "The Big Picture" From previous experience, I also feel that documenting "Big" decisions that affect a project can have great value. With time, we tend to forget the reasons for choosing a direction. A place to consult back as to "Why?" can prove valuable.
- Stack Documentation: A diagram and/or description indicating the various technology stack in play. For example, "JBoss - App Server, Spring Framework, HP UX - 10.2 deployment OS, ..". This might be encompassed within the scope of point no.1. Again, stacks are rarely changed, and if they are, they probabaly require immediate attention.
- Repeated Query Documentation: If any question posed to a developer or architect is being asked more than once, regardless of its longevity deserves due diligence in the documentation space. In lawyer terms, it would classify as "Asked and answered" :-). If you enjoy the interaction and enjoy repeating the same answer, well then more power to you , please disregard the documentation angle.
- Redundant Documentation : Code that contains redundant or unnecessary documentation (i.e., code is self explanatory) serves no purpose. I violate this sometimes and need to stop. For example, getters and setter of a POJO (Plain Ole Java object) rarely requires documenting if the method name is clear. Institute naming standards that are enforced. Don't bother with class or schema diagrams, any developer worth his salt will be able to easily find free tools to reverse engineer classes or database schema's and be able to get the most current versions of the same.
- Documenting Complex Code: Code that is unavoidably complex, deserve either strong documentation or better still refactoring.
- Job security as it applies to documentation: Job security is not improved by keeping the details in your head. People are smart and folks that tend to do that will not last in a smart organization. Organizations lacking strong leaders, well get strong and smart leaders :-)
- When to document and how much ? : Things that are in constant flux are hard to document, wait for stabilization. However, do follow up after stabilization occurs. Make sure your code works first, then comes the documentation. If you have good documentation but code that is broken, thats not a win. In the same vein, I would go further to say, Unit Tests and Integration Tests are more valuable than documentation. Place yourself in the position of a person who will utilize the documentation, make the content detailed enough for your intended audience. For example, in an enterprise level document, do not bother listing individual Web Services or Database Schemas, it is enough to state that Web Services communicate with databases and or other Web Services. The detail in the documentation should also be only that much that you can afford to keep current moving forward. Finally piggy backing on this point, broken windows or no windows from the top can send a bad message. What I mean is, "If an architect or lead themselves do not document, then it becomes a hard sell to a developer that they need to document"
- Detail and Artistry of Documentation: Reiterating, "Make it right before making it pretty". A straw diagram caught on a white board via a cell phone camera during the design, if posted on a WIKI is often as effective as a colorful work of art. High ceremony documentation is almost too much and often unnecessary. Concise and to the point documentation works in most cases. People are smart, they often like a start. Its like being provided a map into Yellowstone regarding the sights to see. One does not require information on the concentration of sulfur at each site, just where the sites are located, we'll figure out the sulfur content at each site :-)
- Tools for documentation: Use a tool that works for collaborative authoring such as a WIKI. Almost any software development project IMO deserves a WIKI or equivalent presence. Media WIKI and such are free to use, so a smart person would avail such tools. Use technology where applicable.
- Collaborative Documentation: Documentation on IT projects should not belong to a single owner but be considered a collaborative effort. In other words, every member of the team should feel that they can add or correct toward the documentation. If a person comes asking a question and you give an answer, if you ask the person to document the same as you feels its valuable and is missing, you might be surprised as to how gladly they are willing to contribute. On the same note when you find documentation that is outdated and more importantly wrong, deprecate the documentation at the very top if you do not have the time to make it current.
- How to get started, documentation: As an entrant into a new IT environment, one often needs to get set up and be ready to get productive. A road map documentation that provides a user of "What, Why, Where, Who" can be quite useful.
- Selling documentation to the business: Getting business to account for time slots for code testing itself can be a daunting task in some organizations, getting a time slot in the project budget for documentation can be harder sale. However, as a lead or person making an estimate of a project, it is important to bake in the same. Documentation is money, however you need to sell the same and its value to the business. Lack of critical documentation is something that can come back to haunt you. "Protect your skin and those of your kinsmen" :-)
Tuesday, June 23, 2009
Scaling in the JMS World...in particular Topics
Scalability and Availability in JMS. For the former, as load on the messaging system increases, one would want to be able to service the increased messages without performance degradation. For the latter, one should ensure the reduction if not elimination of single point of failures, i.e., ensuring system is available for use.
Most JMS Providers have some sort of mechanism for addressing the above. This BLOG is however only going to concentrate on Oracle WebLogic.
WebLogic has an interesting way in which JMS Destinations (Queues and Topics) are scaled via Distributed or Virtual Destinations. A Virtual Destination can be considered a sort of router/load balancer that distributes messages it receives over to actual or physical destinations. Each physical destination would reside in separate WebLogic instances on potentially separate hardware. See figure below where a virtual destination "test_queue" has physical members test_queue_1, test_queue_2 and test_queue_3 on separate WebLogic instances:
As an example of Virtual Queues in action, in the conceptual figure shown below, Producers send messages to the Virtual Queue which then directs the message to one of the physical destinations
P : Producers
VQ: Virtual Queue
PQ-X: Physical Queue, where X is the number of the Queue
C: Consumer
M: Message
In the case of a Virtual Topic, messages sent to the Virtual Topic are sprayed on to each and every physical member as shown in the figure below where VT and PT-X represent Virutal and Physical topics respectively:
Scaling Queues:
From the figure shown regarding Queues, it is pretty simple to follow how the system scales and that it will continue to service new messages and consume them should one of the servers in the messaging cluster fails. A message will be consumed at most by one of many consumers successfully. Adding Consumers to the end of the physical members allow the architecture to process messages rapidly, i.e., Competing Consumer Design Pattern.
Now, what if one of the servers having physical members went down, for example VQ-1 went down? From the perspective of availability, messages are still flowing and getting consumed from the other available member, i.e., VQ-2. What about the messages that were in the physical member whose server went down? One has multiple options. The first and most straight forward is to restart the server and re-connect the consumers to drain the queue. WebLogic allows for auto-service migration in the event of failure. This is another feature that can assist in recovering from failure. Queues readily tend to present themselves for scaling. One note on the same, if message ordering is put into play, all bets are off :-)..more on the same on a subsequent blog.
Scaling Topics:
Topics unlike Queues present an interesting challenge. In the case of a topic, a message is delivered to all listeners of the topic and each listener is typically different than its peer in how it handles the message. For example a message posted to an order topic, might have a consumer that processes the order while another one looks at inventory calculations:
Transfer MDB:
It is pretty trivial to create the Transfer MDB. One would typically like to re-use the Transfer MDB Java Code with only configuration changes for different applications. The same can easily achieved via separate projects where target destination can be provided via @Resource. A simple transfer MDB is shown below:
The CODE for the above MDB and an example Maven Project that uses WebLogic is available for download HERE. Note that the code is not specific to WebLogic and can really be used in any container, for example JBoss.
Conclusion:
Scaling topics via an MDB and target queues just felt like an obvious pattern to me. WebLogic, like many JMS Providers has the concept of a Message Bridge. Initial investigation led me to believe that a message bridge more serves the purpose of transferring a message from one provider to to another rather than serving to scale a Topic and distributed messages to multiple destinations.
There may be cases as mentioned where the number of subscribers are not known. In such a case, a topic might work and it becomes the responsibility of the subscribers to scale on their end, for example multi-threading upon receiving the message.
What really would be nice if the JMS specification had the concept of "Durable Competing Consumers" where a Durable subscriber could register multiple instances of itself such that a message to a topic would be delivered to at most it or one of its clones :-). I believe there are some Messaging providers who permit such a pattern via customization that extend the JMS specifications.
Before bidding adieu, this has been one of my most figurative blogs, all Ye artists, look at the diagrams with a forgiving eye, "I am a programmer not an artist!" :-)
Most JMS Providers have some sort of mechanism for addressing the above. This BLOG is however only going to concentrate on Oracle WebLogic.
WebLogic has an interesting way in which JMS Destinations (Queues and Topics) are scaled via Distributed or Virtual Destinations. A Virtual Destination can be considered a sort of router/load balancer that distributes messages it receives over to actual or physical destinations. Each physical destination would reside in separate WebLogic instances on potentially separate hardware. See figure below where a virtual destination "test_queue" has physical members test_queue_1, test_queue_2 and test_queue_3 on separate WebLogic instances:
As an example of Virtual Queues in action, in the conceptual figure shown below, Producers send messages to the Virtual Queue which then directs the message to one of the physical destinations
P : Producers
VQ: Virtual Queue
PQ-X: Physical Queue, where X is the number of the Queue
C: Consumer
M: Message
In the case of a Virtual Topic, messages sent to the Virtual Topic are sprayed on to each and every physical member as shown in the figure below where VT and PT-X represent Virutal and Physical topics respectively:
Scaling Queues:
From the figure shown regarding Queues, it is pretty simple to follow how the system scales and that it will continue to service new messages and consume them should one of the servers in the messaging cluster fails. A message will be consumed at most by one of many consumers successfully. Adding Consumers to the end of the physical members allow the architecture to process messages rapidly, i.e., Competing Consumer Design Pattern.
Now, what if one of the servers having physical members went down, for example VQ-1 went down? From the perspective of availability, messages are still flowing and getting consumed from the other available member, i.e., VQ-2. What about the messages that were in the physical member whose server went down? One has multiple options. The first and most straight forward is to restart the server and re-connect the consumers to drain the queue. WebLogic allows for auto-service migration in the event of failure. This is another feature that can assist in recovering from failure. Queues readily tend to present themselves for scaling. One note on the same, if message ordering is put into play, all bets are off :-)..more on the same on a subsequent blog.
Scaling Topics:
Topics unlike Queues present an interesting challenge. In the case of a topic, a message is delivered to all listeners of the topic and each listener is typically different than its peer in how it handles the message. For example a message posted to an order topic, might have a consumer that processes the order while another one looks at inventory calculations:
Clearly we cannot have two consumers that process the same order and it is redundant to have duplicate consumers perform the same inventory calculations.
Messaging systems have the concept of a Durable Subscriber, i.e., a Message listener who registers with the topic such that they are delivered messages sent to the topic even when the listener is not connected. Message providers store the message on behalf of the durable subscriber and forward the same when the subscriber comes back on line.
In WebLogic one cannot connect to a durable subscriber using a distributed (virtual) destination JNDI. One has to explicitly register with a physical member. This is one gripe that I have with the WebLogic solution. Adding to the problem, should one connect to one physical member and if that member goes down, attempting to establish the durable subscription on other available members are treated like totally new subscriptions. In consolation, if the server goes down that contains durable subscriptions, then messages will be stored on other physical members of the virtual topic on behalf of the durable subscribers, and when the server is restored, messages get forwarded to the durable subscribers when they comes back on line as shown below:
Now, as each subscriber of the topic has a unique agenda from the other subscribers, the fact that it is the sole entity that services the message, the same represents a single point of failure and a proves a scalability inhibitor as it cannot concurrently process multiple messages.
So how can one scale a topic so that a message on a given type of subscriber can be rapidly processed? One direction is have the consumer of a Topic post the message onto a Queue which can then be processed by competing consumers. One way this can be achieved is to have a Message Driven Bean withing the container whose sole purpose is to transfer the message from the topic to a queue for those consumers that need to scale. An example of the same is shown below:
In the above shown example, as one consumer of the topic needs processing to rapidly occur of messages received, an MDB (registered as a durable subscriber of the topic) is used to transfer every message onto a Queue which is then consumed by competing consumers. As other consumers, DC-1 and DC-2 are not in need of scaling, they remain durable single subscribers.
The above solution works to scale the durable subscriber of a topic. However, in WebLogic when using distributed topics the durable MDB still represents a single point of failure.
A question to ask at this point is, "Do you really need a topic for the use case at hand?" Topics are great IMO when the subscribers tend to be dynamic in nature or do not need to be durable, an example of the former is a stock quote topic and an example of the latter is a event listener that requires notification of an event.
Messaging systems have the concept of a Durable Subscriber, i.e., a Message listener who registers with the topic such that they are delivered messages sent to the topic even when the listener is not connected. Message providers store the message on behalf of the durable subscriber and forward the same when the subscriber comes back on line.
In WebLogic one cannot connect to a durable subscriber using a distributed (virtual) destination JNDI. One has to explicitly register with a physical member. This is one gripe that I have with the WebLogic solution. Adding to the problem, should one connect to one physical member and if that member goes down, attempting to establish the durable subscription on other available members are treated like totally new subscriptions. In consolation, if the server goes down that contains durable subscriptions, then messages will be stored on other physical members of the virtual topic on behalf of the durable subscribers, and when the server is restored, messages get forwarded to the durable subscribers when they comes back on line as shown below:
Now, as each subscriber of the topic has a unique agenda from the other subscribers, the fact that it is the sole entity that services the message, the same represents a single point of failure and a proves a scalability inhibitor as it cannot concurrently process multiple messages.
So how can one scale a topic so that a message on a given type of subscriber can be rapidly processed? One direction is have the consumer of a Topic post the message onto a Queue which can then be processed by competing consumers. One way this can be achieved is to have a Message Driven Bean withing the container whose sole purpose is to transfer the message from the topic to a queue for those consumers that need to scale. An example of the same is shown below:
In the above shown example, as one consumer of the topic needs processing to rapidly occur of messages received, an MDB (registered as a durable subscriber of the topic) is used to transfer every message onto a Queue which is then consumed by competing consumers. As other consumers, DC-1 and DC-2 are not in need of scaling, they remain durable single subscribers.
The above solution works to scale the durable subscriber of a topic. However, in WebLogic when using distributed topics the durable MDB still represents a single point of failure.
A question to ask at this point is, "Do you really need a topic for the use case at hand?" Topics are great IMO when the subscribers tend to be dynamic in nature or do not need to be durable, an example of the former is a stock quote topic and an example of the latter is a event listener that requires notification of an event.
When the number of subscribers are a known or finite in amount, the use of a queue coupled with the MDB can easily achieve a scalable solution as shown below where the MDB transfers messages on the in-queue to subsequent out-queues where competing consumers can scale and consume the messages:
Transfer MDB:
It is pretty trivial to create the Transfer MDB. One would typically like to re-use the Transfer MDB Java Code with only configuration changes for different applications. The same can easily achieved via separate projects where target destination can be provided via @Resource. A simple transfer MDB is shown below:
@MessageDriven
@TransactionAttribute(TransactionAttributeType.REQUIRED)
public class MessageRouterMdb implements MessageListener {
/**
* Map this jms connection factory to the actual one in the weblogic-ejb-jar.xml deployment
* descriptor of the ejb project. The Connection factory to be used will be the one that is
* specific to this mdb.
*/
@Resource(name = "jms/connectionFactory")
private ConnectionFactory connectionFactory;
/**
* Injected in Message Driven Context
*/
@Resource
private MessageDrivenContext mdc;
/**
* Define a comma separated list of destinations JNDI names in the ejb-jar.xml via an environment
* entry.
*/
@Resource
private String targetDestinations;
// Initialized
private List<Destination> targetJmsDestinations;
private MessageProducer messageProducer;
private Connection connection;
private Session session;
@PostConstruct
public void init() {
try {
InitialContext ctx = new InitialContext();
connection = connectionFactory.createConnection();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
messageProducer = session.createProducer(null);
String[] targetDestArray = StringUtils.split(targetDestinations, ",");
targetJmsDestinations = new ArrayList<Destination>();
for (String targetDest : targetDestArray) {
targetJmsDestinations.add((Destination) ctx.lookup(targetDest));
}
connection.start();
}
catch (Exception e) {
throw new RuntimeException("Error Initializing Message Router Mdb", e);
}
}
public void onMessage(Message message) {
try {
// Loop through target destinations and send message to each destination
for (Destination dest : targetJmsDestinations) {
messageProducer.send(dest, message);
}
}
catch (Exception e) {
mdc.setRollbackOnly();
}
}
@PreDestroy
public void cleanup() {
// Close open jms resources here...
}
}
The CODE for the above MDB and an example Maven Project that uses WebLogic is available for download HERE. Note that the code is not specific to WebLogic and can really be used in any container, for example JBoss.
Conclusion:
Scaling topics via an MDB and target queues just felt like an obvious pattern to me. WebLogic, like many JMS Providers has the concept of a Message Bridge. Initial investigation led me to believe that a message bridge more serves the purpose of transferring a message from one provider to to another rather than serving to scale a Topic and distributed messages to multiple destinations.
There may be cases as mentioned where the number of subscribers are not known. In such a case, a topic might work and it becomes the responsibility of the subscribers to scale on their end, for example multi-threading upon receiving the message.
What really would be nice if the JMS specification had the concept of "Durable Competing Consumers" where a Durable subscriber could register multiple instances of itself such that a message to a topic would be delivered to at most it or one of its clones :-). I believe there are some Messaging providers who permit such a pattern via customization that extend the JMS specifications.
Before bidding adieu, this has been one of my most figurative blogs, all Ye artists, look at the diagrams with a forgiving eye, "I am a programmer not an artist!" :-)
Friday, May 22, 2009
Ending the drought...
Its been a month since I posted anything on my BLOG, cannot let May end with a drought. It has been rather a trying month for me, the reasons I am saving for a later melodramatic post ;-)...
Right now I am very happy. At the company I currently work at, we released out a JMS architecture with minimal hiccups. It was very gratifying to witness the entire team involved work and mesh so well in order to make the architecture work. Everyone from Architects, Developers, System Administrators, Testers worked so well together. I must state that my level of confidence in the team and players involved was so much that I did not even have to be there during the go live in person. Put it in a word, my current organization is so much about "WE" and not "ME". That is a motto that works IMHO. Guess my current organization is very talented in its selection process of the people they want on board.
I truly enjoyed working with all the folks on the various facets involving the deployment of the architecture, frameworks, deployment, projects etc. We suffered a few hiccups after the deploy as well but the messaging solution itself was 100% solid. In fact in my own retarded way, I appreciated the fact that the failures occurred as they served to demonstrate that the failure path in fact worked ;-)
Personally, I have experienced considerable growth in the areas of JMS Messaging. However, every once so often I again am humbled by what I don't know or did not anticipate.
The project as a whole was like our baby that finally saw the light..and danced on...I hope it keeps dancing :-))))))))))) Even if does not, I believe we have a team strong enough to put it on crutches to make it so...
I can't wait for further adoption of the architecture...so thrilled, had a blast...
More BLOG's coming up soon, the little GREY cells were on vacation (not dead, just vacationing), you should see something from them soon....
Right now I am very happy. At the company I currently work at, we released out a JMS architecture with minimal hiccups. It was very gratifying to witness the entire team involved work and mesh so well in order to make the architecture work. Everyone from Architects, Developers, System Administrators, Testers worked so well together. I must state that my level of confidence in the team and players involved was so much that I did not even have to be there during the go live in person. Put it in a word, my current organization is so much about "WE" and not "ME". That is a motto that works IMHO. Guess my current organization is very talented in its selection process of the people they want on board.
I truly enjoyed working with all the folks on the various facets involving the deployment of the architecture, frameworks, deployment, projects etc. We suffered a few hiccups after the deploy as well but the messaging solution itself was 100% solid. In fact in my own retarded way, I appreciated the fact that the failures occurred as they served to demonstrate that the failure path in fact worked ;-)
Personally, I have experienced considerable growth in the areas of JMS Messaging. However, every once so often I again am humbled by what I don't know or did not anticipate.
The project as a whole was like our baby that finally saw the light..and danced on...I hope it keeps dancing :-))))))))))) Even if does not, I believe we have a team strong enough to put it on crutches to make it so...
I can't wait for further adoption of the architecture...so thrilled, had a blast...
More BLOG's coming up soon, the little GREY cells were on vacation (not dead, just vacationing), you should see something from them soon....
Friday, April 24, 2009
Browsing Weblogic JMS Distrubuted Queues
On of the interesting features of Weblogic is that when javax.jms.QueueBrowsers are created, they are pinned to a particular physical member of a distributed destination. What that means is that a QueueBrowser created for a distributed destination will only be able to browse messages on the member it gets pinned to and not the whole queue.
One can try to create X number of queue browsers based of the number of servers in the provider URL, for example, "t3://localhost:8002, localhost:8003", and hope that no two queue browsers will pin to the same destination, i.e., at the mercy of Weblogic round robin if possible. Even if we do manage to do that, one still needs to iterate over X separate enumerations in order to manage the message. Consider an integration test where one sends a message to a distributed destination and would have to create X queue browsers to determine whether or not the message is present.
What would be really nice is if the Weblogic runtime gave us a QueueBrowser that provided an enumeration of enumerations of the physical members of a distributed queue. That however is not present. So, I decided to create a simple example that would help me with the same. At the same time, I am interested in understanding the JMX capabilities of Weblogic.
The example will create a QueueBrowser and obtain an enumeration from each of the physical members of the distributed destination and provide back an enumeration of the respective enumerations. In the case of uniformly created distributed destinations, the JNDI name of the physical members will be of the format JMSServerName@JNDIName. If one knows the JMSServerName thats great. However, one should not have to rely on the same as JMSServers might be added/removed. It would be nice if the distributed member JNDI names could be located on the fly by using Weblogic's JMX capabilities and thats what I intend to do. Note that the enumerations obtained in Weblogic are snapshots of the current state of the queue and are not dynamic, i.e., new messages added being dynamically reflected in the enumeration. Finally, I am providing a java.util.Enumeration for the JMS messages instead of an java.util.Iterator as Enumeration does not have a "remove" method and I hate putting an "UnsupportedOperationException" in the Iterator and also I would like to be true to the JMS API.
So first obtaining the JNDI name of physical members of a distributed queue. Weblogic JMX is a tree for objects. Once we obtain the ServerRuntimeMBean information, we can obtain information about the JMSRuntimeMBeans and then the different JMSServers and finally the JNDI names of the JMS Servers as shown below:
Now that we have a way of obtaining the distributed members, the following represents the QueueBrowser that enumerates over the different members and consolidating the same into a single enumeration. I have a second enumeration that provides information regarding the JMServer that hosts the given message:
Using this QueueBrowser is rather straight forward and can be done via:
The code for the same is available HERE. The zip file contains only the JAVA files. One would need to compile the same with weblogic's client jar's and then use them. Let me know if you have problems running the code or if you have any tips regarding the code.
Later...
One can try to create X number of queue browsers based of the number of servers in the provider URL, for example, "t3://localhost:8002, localhost:8003", and hope that no two queue browsers will pin to the same destination, i.e., at the mercy of Weblogic round robin if possible. Even if we do manage to do that, one still needs to iterate over X separate enumerations in order to manage the message. Consider an integration test where one sends a message to a distributed destination and would have to create X queue browsers to determine whether or not the message is present.
What would be really nice is if the Weblogic runtime gave us a QueueBrowser that provided an enumeration of enumerations of the physical members of a distributed queue. That however is not present. So, I decided to create a simple example that would help me with the same. At the same time, I am interested in understanding the JMX capabilities of Weblogic.
The example will create a QueueBrowser and obtain an enumeration from each of the physical members of the distributed destination and provide back an enumeration of the respective enumerations. In the case of uniformly created distributed destinations, the JNDI name of the physical members will be of the format JMSServerName@JNDIName. If one knows the JMSServerName thats great. However, one should not have to rely on the same as JMSServers might be added/removed. It would be nice if the distributed member JNDI names could be located on the fly by using Weblogic's JMX capabilities and thats what I intend to do. Note that the enumerations obtained in Weblogic are snapshots of the current state of the queue and are not dynamic, i.e., new messages added being dynamically reflected in the enumeration. Finally, I am providing a java.util.Enumeration for the JMS messages instead of an java.util.Iterator as Enumeration does not have a "remove" method and I hate putting an "UnsupportedOperationException" in the Iterator and also I would like to be true to the JMS API.
So first obtaining the JNDI name of physical members of a distributed queue. Weblogic JMX is a tree for objects. Once we obtain the ServerRuntimeMBean information, we can obtain information about the JMSRuntimeMBeans and then the different JMSServers and finally the JNDI names of the JMS Servers as shown below:
public class WeblogicMBeanHelper {
private final MBeanServerConnection connection;
private final JMXConnector connector;
private final ObjectName service;
public WeblogicMBeanHelper(String url, String userName, String password) {
try {
service = new ObjectName(
"com.bea:Name=DomainRuntimeService,Type=weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean");
}
catch (MalformedObjectNameException e) {
throw new AssertionError(e.getMessage());
}
......
......
JMXServiceURL serviceURL;
try {
serviceURL = new JMXServiceURL("t3", hostName, Integer.valueOf(port), jndiroot + mserver);
Hashtable<String, String> h = new Hashtable<String, String>();
h.put(Context.SECURITY_PRINCIPAL, userName);
h.put(Context.SECURITY_CREDENTIALS, password);
h.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES, "weblogic.management.remote");
connector = JMXConnectorFactory.connect(serviceURL, h);
connection = connector.getMBeanServerConnection();
}
catch (Exception e) {
throw new RuntimeException(e);
}
}
public Iterable<String> getDistributedMemberJndiNames(String distributedDestJndiName) {
Iterable<String> serverNames = getJmsServerNames();
Set<String> distributedDestNames = new TreeSet<String>();
for (String serverName : serverNames) {
distributedDestNames.add(serverName + "@" + distributedDestJndiName);
}
return distributedDestNames;
}
public Iterable<String> getJmsServerNames() {
.....
}
public Iterable<ObjectName> getJMSServers() {
....
}
public Iterable<ObjectName> getJMSRuntimes() {
....
}
public List<ObjectName> getServerRuntimeMBeans() {
try {
return Arrays.asList((ObjectName[]) connection.getAttribute(service, "ServerRuntimes"));
}
catch (Exception e) {
throw new RuntimeException("Error obtaining Server Runtime Information", e);
}
}
public void close() {
// Close the connector
....
}
}
public class WeblogicMBeanHelper {
private final MBeanServerConnection connection;
private final JMXConnector connector;
private final ObjectName service;
public WeblogicMBeanHelper(String url, String userName, String password) {
try {
service = new ObjectName(
"com.bea:Name=DomainRuntimeService,Type=weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean");
}
catch (MalformedObjectNameException e) {
throw new AssertionError(e.getMessage());
}
......
......
JMXServiceURL serviceURL;
try {
serviceURL = new JMXServiceURL("t3", hostName, Integer.valueOf(port), jndiroot + mserver);
Hashtable<String, String> h = new Hashtable<String, String>();
h.put(Context.SECURITY_PRINCIPAL, userName);
h.put(Context.SECURITY_CREDENTIALS, password);
h.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES, "weblogic.management.remote");
connector = JMXConnectorFactory.connect(serviceURL, h);
connection = connector.getMBeanServerConnection();
}
catch (Exception e) {
throw new RuntimeException(e);
}
}
public Iterable<String> getDistributedMemberJndiNames(String distributedDestJndiName) {
Iterable<String> serverNames = getJmsServerNames();
Set<String> distributedDestNames = new TreeSet<String>();
for (String serverName : serverNames) {
distributedDestNames.add(serverName + "@" + distributedDestJndiName);
}
return distributedDestNames;
}
public Iterable<String> getJmsServerNames() {
.....
}
public Iterable<ObjectName> getJMSServers() {
....
}
public Iterable<ObjectName> getJMSRuntimes() {
....
}
public List<ObjectName> getServerRuntimeMBeans() {
try {
return Arrays.asList((ObjectName[]) connection.getAttribute(service, "ServerRuntimes"));
}
catch (Exception e) {
throw new RuntimeException("Error obtaining Server Runtime Information", e);
}
}
public void close() {
// Close the connector
....
}
}
Now that we have a way of obtaining the distributed members, the following represents the QueueBrowser that enumerates over the different members and consolidating the same into a single enumeration. I have a second enumeration that provides information regarding the JMServer that hosts the given message:
public class DistribuedQueueBrowser {
...
private final Iterable<String> queueNames;
public DistribuedQueueBrowser(String adminUrl, String providerUrl,
String distributedDestinationName, String userName, String password) throws Exception {
ctx = getInitialContext(providerUrl, userName, password);
WeblogicMBeanHelper helper = null;
try {
helper = new WeblogicMBeanHelper(adminUrl, userName, password);
queueNames = helper.getDistributedMemberJndiNames(distributedDestinationName);
}
finally {
if (helper != null) {
helper.close();
}
}
// Set up connection/session..
....
}
private InitialContext getInitialContext(String providerUrl, String userName, String password) throws Exception {
....
}
// Retrun an Enumeration of Messages only
public Enumeration<Message> getEnumeration() throws JMSException, NamingException {
return new JmsMessageEnumeration(getMessageEnumeratorMap());
}
@SuppressWarnings("unchecked") private Map<String, Enumeration<Message>> getMessageEnumeratorMap() throws JMSException,
NamingException {
Map<String, Enumeration<Message>> serverMessageMap = new HashMap<String, Enumeration<Message>>();
for (String queueName : queueNames) {
String serverDq[] = StringUtils.split(queueName, "@");
Queue queue = (Queue) ctx.lookup(queueName);
javax.jms.QueueBrowser qb = session.createBrowser(queue);
serverMessageMap.put(serverDq[0], qb.getEnumeration());
}
return serverMessageMap;
}
// Return an enumeration of ServerLocatedMessage that contains the
// the jms server that houses the message
public Enumeration<ServerLocatedMessage> getServerLocatedEnumeration() throws JMSException,
NamingException {
return new ServerLocatedMessageEnumeration(getMessageEnumeratorMap());
}
public static class ServerLocatedMessage {
private final Message message;
private final String jmsServerName;
public ServerLocatedMessage(String jmsServerName, Message message) {
this.message = message;
this.jmsServerName = jmsServerName;
}
...
}
private static abstract class AbstractMessageEnumeration<T> implements Enumeration<T> {
Map.Entry<String, Enumeration<Message>> current;
private Enumeration<Message> currMessageEnumer;
private final Iterator<Map.Entry<String, Enumeration<Message>>> iterator;
public AbstractMessageEnumeration(Map<String, Enumeration<Message>> map) {
iterator = map.entrySet().iterator();
current = iterator.hasNext()
? iterator.next()
: null;
currMessageEnumer = current != null
? current.getValue()
: new Enumeration<Message>() {
public boolean hasMoreElements() {
return false;
}
public Message nextElement() {
throw new NoSuchElementException();
}
};
}
Enumeration<Message> getEnumeration() {
if (current == null || currMessageEnumer.hasMoreElements()) {
return currMessageEnumer;
}
while (iterator.hasNext()) {
current = iterator.next();
currMessageEnumer = current.getValue();
if (currMessageEnumer.hasMoreElements()) {
return currMessageEnumer;
}
}
return currMessageEnumer;
}
public boolean hasMoreElements() {
return getEnumeration().hasMoreElements();
}
}
// Wraps the JMS Message within a ServerLocatedMessage object
// which contains the server name along with the message
private static class ServerLocatedMessageEnumeration extends
AbstractMessageEnumeration<ServerLocatedMessage> {
public ServerLocatedMessageEnumeration(Map<String, Enumeration<Message>> map) {
super(map);
}
public ServerLocatedMessage nextElement() {
Message message = getEnumeration().nextElement();
return new ServerLocatedMessage(current.getKey(), message);
}
}
private static class JmsMessageEnumeration extends AbstractMessageEnumeration<Message> {
public JmsMessageEnumeration(Map<String, Enumeration<Message>> map) {
super(map);
}
public Message nextElement() {
return getEnumeration().nextElement();
}
}
public void close() {
......
}
}
Using this QueueBrowser is rather straight forward and can be done via:
// Note that the first argument is the admin url and the second is the
// managed server url.
DistribuedQueueBrowser qb = new DistribuedQueueBrowser("t3://localhost:7001",
"t3://localhost:8001",
"test_queue", "weblogic", "weblogic");
Enumeration<Message> i = qb.getEnumeration();
while (i.hasMoreElements()) {
Message m = i.nextElement();
System.out.println("Message:" + m);
}
Enumeration<ServerLocatedMessage> sli = qb.getServerLocatedEnumeration();
while (sli.hasMoreElements()) {
ServerLocatedMessage m = sli.nextElement();
System.out.println(m);
}
}
The code for the same is available HERE. The zip file contains only the JAVA files. One would need to compile the same with weblogic's client jar's and then use them. Let me know if you have problems running the code or if you have any tips regarding the code.
Later...
Subscribe to:
Posts (Atom)