Maven Integration Testing of Spring Boot Cloud Netflix Eureka Services with Docker

Introduction

My previous blog was about using Spring Cloud Netflix and my experiences with it.  In this BLOG, I will share how one could perform a maven integration test that involves multiple Spring Boot services that use Netflix Eureka as a service registry. I have utilized a similar example as the one I used in my BLOG about Reactive Programming with Jersey and RxJava where a product is assembled from different JAX-RS microservices. For this BLOG though, each of the microservices are written in Spring Boot using Spring Cloud Netflix Eureka for service discovery. A service which I am calling the Product Gateway, assembles data from the independent product based microservices to hydrate a Product.

Product Gateway

A mention of the Product Gateway is warranted for completeness. The following represents the ProductGateway classes where the ObservableProductResource uses a ProductService which in turn utilizes the REST clients of the different services and RxJava to hydrate a Product.

The ObservableProductResource Spring Controller class is shown below which uses a DeferredResult for Asynchronous processing:

@RestController
public class ObservableProductResource {  
  @Inject
  private ProductService productService;
  
  @RequestMapping("/products/{productId}")
  public DeferredResult<Product> get(@PathVariable Long productId) {
    DeferredResult<Product> deferredResult = new DeferredResult<Product>();
    Observable<Product> productObservable = productService.getProduct(productId);

    productObservable.observeOn(Schedulers.io())
    .subscribe(productToSet -> deferredResult.setResult(productToSet), t1 -> deferredResult.setErrorResult(t1));
    
    return deferredResult;
  }
}

Each Micro Service client is declaratively created using Netflix Feign. As an example of one such client, the BaseProductClient, is shown below:

@FeignClient("baseproduct")
public interface BaseProductClient {
  @RequestMapping(method = RequestMethod.GET, value = "/baseProduct/{id}", consumes = "application/json")
  BaseProduct getBaseProduct(@PathVariable("id") Long id);
}

What does the Integration Test do?

The primary purpose is to test the actual end to end integration of the Product Gateway Service. As a maven integration test, the expectancy is that it would entail:

• Starting an instance of Eureka
• Starting Product related services registering them with Eureka
• Starting Product Gateway Service and registering it with Eureka
• Issuing a call to Product Gateway Service to obtain said Product
• Product Gateway Service discovering instances of Product microservices like Inventory, Reviews and Price from Eureka
• Product Gateway issuing calls to each of the services using RxJava and hydrating a Product
• Asserting the retrieval of the Product and shutting down the different services

The test itself would be a maven JUnit integration test.

As the services are bundled as JARs with embedded containers, they present a challenge to start up and tear down during an integration test. 

One option is to create equivalent WAR based artifacts for testing purposes only and use the maven-cargo plugin to deploy each of them under a separate context of the container and test the gateway. That however does mean creating a WAR that might never really be used apart from testing purposes.

Another option to start the different services is using the exec maven plugin and/or some flavor(hack) to launch external JVMs.

Yet another option is write custom class loader logic [to prevent stomping of properties and classes of individual microservices] and launch the different services in the same integration test JVM.

All these are options but what appealed to me was to use Docker containers to start each of these microservice JVMs and run the integration test.  So why Docker? Docker seems a natural fit to compose an application and distribute it across a development environment as a consistent artifact. The benefits during micro service based integration testing where one can simply pull in different docker images such as services, data stores etc of specific versions without dealing with environment based conflicts is what I find appealing.

Creating Docker Images 

As part of building each of the web services, it would be ideal to create a Docker image. There are many maven plugins out there to create Docker images [actually too many]. In the example, we have used the one from Spotify.  The building of the Docker image using the Spotify plugin for Spring Boot applications is nicely explained in the BLOG from spring.io, Spring Boot with Docker.

What I would see happening is that as part of the build process, the Docker image would be published to a docker repository which is internal to an organization and then made available for other consumers.

Integration Test

As part of the pre-integration test phase of maven, we would like to start up the Docker containers representing the different services. In order for the gateway container to work with the other service containers, we need to be able to link the Docker containers. I was not able to find a way to do that using the Spotify plugin. What I instead found myself doing is utilizing another maven plugin for Docker by Roland HuB which has much better documentation and more features. Shown below is the plugin configuration for the integration test.

<plugin>
  <groupId>org.jolokia</groupId>
  <artifactId>docker-maven-plugin</artifactId>
  <version>0.13.6</version>
  <configuration>
    <logDate>default</logDate>
    <autoPull>true</autoPull>
    <images>
      <image>
        <!-- Eureka Server -->
        <alias>eureka</alias>
        <name>docker/eureka</name>
        <run>
          <wait>
            <http>
              <url>http://localhost:8761</url>
              <method>GET</method>
              <status>200</status>
            </http>
            <time>30000</time>
          </wait>
          <log>
            <prefix>EEEE</prefix>
            <color>green</color> <!-- Color the output green -->
          </log>
          <ports>
            <port>8761:8761</port> <!-- Local to container port mapping -->
          </ports>
          <env>
            <eureka.instance.hostname>eureka</eureka.instance.hostname> <!-- Override host name property -->
          </env>
        </run>
      </image>
      <image>
        <alias>baseproduct</alias>
        <name>docker/baseproduct</name>
        <run>
          <wait>
            <http>
              <url>http://localhost:9090</url>
              <method>GET</method>
              <status>200</status>
            </http>
            <time>30000</time>
          </wait>
          <log>
            <prefix>EEEE</prefix>
            <color>blue</color>
          </log>
          <ports>
            <port>9090:9090</port>
          </ports>
          <links>
            <link>eureka</link> <!-- Link to Eureka Docker image -->
          </links>
          <env>
            <!-- Notice the system property overriding of the eureka service Url -->
            <eureka.client.serviceUrl.defaultZone>http://eureka:8761/eureka/</eureka.client.serviceUrl.defaultZone>
          </env>
        </run>
      </image>
      <!--....Other service containers like price, review, inventory-->
      <image>
        <alias>product-gateway</alias>
        <name>docker/product-gateway</name>
        <run>
          <wait>
            <http>
              <url>http://localhost:9094</url>
              <method>GET</method>
              <status>200</status>
            </http>
            <time>30000</time>
          </wait>
          <log>
            <prefix>EEEE</prefix>
            <color>blue</color>
          </log>
          <ports>
            <port>9094:9094</port>
          </ports>
          <links>
            <!-- Links to all other containers -->
            <link>eureka</link>
            <link>baseproduct</link>
            <link>price</link>
            <link>inventory</link>
            <link>review</link>
          </links>
          <env>
            <eureka.client.serviceUrl.defaultZone>http://eureka:8761/eureka/</eureka.client.serviceUrl.defaultZone>
            <!-- Setting this property to prefer ip address, else Integration will fail as it does not know host name of product-gateway container-->
            <eureka.instance.prefer-ip-address>true</eureka.instance.prefer-ip-address>
          </env>
        </run>
      </image>
    </images>
  </configuration>
  <executions>
    <execution>
      <id>start</id>
      <phase>pre-integration-test</phase>
      <goals>
        <goal>start</goal>
      </goals>
    </execution>
    <execution>
      <id>stop</id>
      <phase>post-integration-test</phase>
      <goals>
        <goal>stop</goal>
      </goals>
    </execution>
  </executions>
</plugin>

One of the nice features is the syntax color prefix of each containers messages, this gives one a sense of visual separation among the multitude of containers that are started. The Integration Test itself is shown below:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = { ProductGatewayIntegrationTest.IntegrationTestConfig.class })
public class ProductGatewayIntegrationTest {
  private static final Logger LOGGER = Logger.getLogger(ProductGatewayIntegrationTest.class);
  
  /**
   * A Feign Client to obtain a Product
   */
  @FeignClient("product-gateway")
  public static interface ProductClient {
    @RequestMapping(method = RequestMethod.GET, value = "/products/{productId}", consumes = "application/json")
    Product getProduct(@PathVariable("productId") Long productId);
  }

  @EnableFeignClients
  @EnableDiscoveryClient
  @EnableAutoConfiguration
  @ComponentScan
  @Configuration
  public static class IntegrationTestConfig {}

  // Ribbon Load Balancer Client used for testing to ensure an instance is available before invoking call
  @Autowired
  LoadBalancerClient loadBalancerClient;

  @Inject
  private ProductClient productClient;

  static final Long PRODUCT_ID = 9310301L;

  @Test(timeout = 30000)
  public void getProduct() throws InterruptedException {
    waitForGatewayDiscovery();
    Product product = productClient.getProduct(PRODUCT_ID);
    assertNotNull(product);
  }

  /**
   * Waits for the product gateway service to register with Eureka
   * and be available on the client.
   */
  private void waitForGatewayDiscovery() {
    while (!Thread.currentThread().isInterrupted()) {
      LOGGER.debug("Checking to see if an instance of product-gateway is available..");
      ServiceInstance choose = loadBalancerClient.choose("product-gateway");
      if (choose != null) {
        LOGGER.debug("An instance of product-gateway was found. Test can proceed.");
        break;
      }
      try {
        LOGGER.debug("Sleeping for a second waiting for service discovery to catch up");
        Thread.sleep(1000);
      }
      catch (InterruptedException e) {
        Thread.currentThread().interrupt();
      }
    }
  }
}

The test uses the LoadBalancerClient client from Ribbon to ensure an instance of 'product-gateway' can be discovered from Eureka prior to using the Product client to invoke the gateway service to obtain back a product.

Running the Example

The first thing you need to do is make sure you have Docker installed on your machine. Once you have Docker installed, clone the example from github (https://github.com/sanjayvacharya/sleeplessinslc/tree/master/product-gateway-docker) and then execute a mvn install from the root level of the project. This will result in the creation of Docker images and the running of the Docker based integration tests of the product gateway. Cheers!