WSO2 API Manager with message routing to different endpoints types

Introduction

The scenario depicted here is for a simple route data to endpoint, based on HTTP method and API URL. I show two aproaches. One simple using only HTTP method – PUT. And second advanced with specify URL routing. Data will be routed to different endpoint. As an example, of endpoint we use RabbitMQ message queue. How to configure endpoint for RabbitMQ I refer to this post[1]. And for more complex scenarios of routing messages I recommended to use WSO2 Enterprise Integrator instead of WSO2 API Manager.

Steps to start

To create simple routing messages in WSO2 API manager we modify in Runtime Configuratons our Message Mediation component.

As schown on image, we go to APIs -> Runtime Configurations -> Message Mediation. Before we can upload our routing mediation, we must create artifact as XML file like below. We called it rabbitmq.router.xml

Simple solution example

<?xml version="1.0" encoding="UTF-8"?><sequence xmlns="http://ws.apache.org/ns/synapse" name="rabbitmq.router" onError="fault">
  <filter source="$axis2:HTTP_METHOD" regex="PUT">
    <then>
      <property name="exchangeName" value="wso2.api" scope="default" type="STRING"/>
      <property name="routingKey" value="data.put" scope="default" type="STRING"/>
      <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
      <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" type="STRING"/>
      <header xmlns:ns="http://org.apache.synapse/xsd" name="To" scope="default" expression="fn:concat('rabbitmq:/?rabbitmq.connection.factory=CachedRabbitMQConnectionFactory&rabbitmq.queue.delivery.mode=2&rabbitmq.exchange.name=',$ctx:exchangeName,'&rabbitmq.queue.routing.key=',$ctx:routingKey)"/>
      <send/>  
    </then>
    <else/>
  </filter>
</sequence>

In simple aproach, we use filter mediator based on $axis2:HTTP_METHOD. For PUT method we set exchangeName and routingKey. Based on this values we build header for sending to Rabbit MQ. All other http method will be passed to default endpoint, only PUT will be redirect to Rabbit MQ.

Advence solution example

In more advenced solution, we can additionaly route messages using pattern in our URI. For this example we have want to route to Rabbit MQ endpoints defined as:
/data/sample/{sampleId}/id/{id}/status
/data/complex/{complexId}/guid/{guid}

Using appropriate regex in switch mediator we filter our URI and set diffrent exchangeName and routingKey properties. Moreover we use dynamic URI elements to concatenate exchange name and routing key for more flexible usecases – filtering topics in Rabbit MQ. All other approach is the same as in simple example. This approach as XML is listed below.

<?xml version="1.0" encoding="UTF-8"?><sequence xmlns="http://ws.apache.org/ns/synapse" name="rabbitmq.router" onError="fault">
  <filter source="$axis2:HTTP_METHOD" regex="PUT">
    <then>
      <property name="exchangeName" value="null" scope="default" type="STRING"/>
      <property name="routingKey" value="null" scope="default" type="STRING"/>
      <switch source="get-property('To')">
        <case regex=".*\/data\/sample\/([^\/]*)\/id\/([^\/]*)\/status">
          <property name="routingKey" expression="concat('sample.',get-property('uri.var.sampleId'),'.id.',get-property('uri.var.id'))" scope="default" type="STRING"/>
          <property name="exchangeName" value="sample.api.data" scope="default" type="STRING"/>
        </case>
        <case regex=".*\/data\/complex\/([^\/]*)\/guid\/([^\/]*)">
          <property name="routingKey" expression="concat('complex.',get-property('uri.var.complexId'),'.id.',get-property('uri.var.guid'))" scope="default" type="STRING"/>
          <property name="exchangeName" value="complex.api.data" scope="default" type="STRING"/>
        </case>
      </switch>
      <filter xpath="$ctx:exchangeName != 'null' and $ctx:routingKey != 'null'">
        <then>
          <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
          <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" type="STRING"/>
          <header xmlns:ns="http://org.apache.synapse/xsd" name="To" scope="default" expression="fn:concat('rabbitmq:/?rabbitmq.connection.factory=CachedRabbitMQConnectionFactory&rabbitmq.queue.delivery.mode=2&rabbitmq.exchange.name=',$ctx:exchangeName,'&rabbitmq.queue.routing.key=',$ctx:routingKey)"/>
          <send/>
        </then>
        <else/>
     </filter>
    </then>
    <else/>
  </filter>
</sequence>

Summary

In this article, I have described a fairly simple way to separate API traffic to different endpoints. It can be use for one API on front, with all benefits of WSO2 API Manager, and have sepparate backend endpoints. For example for bigger traffic. Or can be use in CQRS-style separating queres to backend and commands to queue system like Rabbit MQ.

How to permanently change or add configuration in WSO2 API Manger 3.1.0

What is the problem?

In my previous post, i show how to modify configuration in axis2.xml to set API Manager to work with RabbitMQ as endpoint. Sadly it doesn’t work on WSO2 API Manager version 3.1.0. From this version, the deployment was changed. The standard configuration is overwritten every WSO2 API Manager is started. That is because, first the configuration files are automatic generated from templates, before WSO2 API Manager starts.

What we can do?

After a small investigation, I found the solution. Because, configuration files are generating, from Jinja2 templates – files with extension .j2, we need to edit it. The simplest way is to found them and add or change what we need to change, and then save them. In WSO2 API Manager 3.1.0 the files can be found in <API-M_HOME>\repository\resources\conf\templates\. You will see, the structure of directories, in this path is familiar. You can found the axis2 template here:
<API-M_HOME>\repository\resources\conf\templates\repository\conf\axis2\axis2.xml.j2 Now you can open this file with your text editor, and permanently change what you want. Be careful, because it is not XML but j2 template.

Can we do it in better way?

Yes we can. Considering my previous post, where i was configuring RabbitMQ introduced changes in accordance with the Jinja2 template documentation. So my changes was looking like following:

{% if transport.rabbitmq.connection is defined %}
	<transportSender name="rabbitmq" class="org.apache.axis2.transport.rabbitmq.RabbitMQSender">
		<parameter name="CachedRabbitMQConnectionFactory" locked="false">
			<parameter name="rabbitmq.server.host.name" locked="false">{{transport.rabbitmq.connection.parameters.host.name}}</parameter>
			<parameter name="rabbitmq.server.port" locked="false">{{transport.rabbitmq.connection.parameters.port}}</parameter>
			<parameter name="rabbitmq.server.user.name" locked="false">{{transport.rabbitmq.connection.parameters.user.name}}</parameter>
			<parameter name="rabbitmq.server.password" locked="false">{{transport.rabbitmq.connection.parameters.password}}</parameter>
	</transportSender>
{% endif %}

It simply add the configuration node TransportSender for RabbitMQ, but it is adding dynamically. You can See the notation with double bracket – that is part of the Jinja2 template. But wher comes from this dynamicly variables? All of those templates variables, are defined in: <API-M_HOME>\repository\resources\conf\default.json. For my configuration, I need to add in json configuration rows like this:

"transport.rabbitmq.connection.parameters.host.name": "localhost",
"transport.rabbitmq.connection.parameters.port": "5672",
"transport.rabbitmq.connection.parameters.user.name": "guest",
"transport.rabbitmq.connection.parameters.password": "guest",

After that, is done, when i start my WSO2 API Manager 3.1.0, a proper axis2.xml are generated including my RabbitMQ config and works exacly what i expected.

Additional information

In addition to this product, I see, that newly released WSO2 products are also based on configuration using j2 templates, including WSO2 Micro Integrator.

How to use WSO2 API Manager as RabbitMQ message producer

Introduction

In integration world, you often got some not usual scenarios. In one hand you would like have benefits witch WSO2 API Manager offers. On the other hand you look for benefits of asynchronous communication. In this scenario I show, how to configure WSO2 API Manager, to expose REST API to act as data feeder to RabittMQ.

Solution

  1. Firstly, we need to enable RabbitMQ sender in the API Manager configuration file.
    <API-M_HOME>\repository\conf\axis2\ axis2.xml
    We do this by adding the following configuration. In other words you must set your RabbitMQ IP address, user and password.
<transportSender name="rabbitmq" class="org.apache.axis2.transport.rabbitmq.RabbitMQSender"> 
    <parameter name="CachedRabbitMQConnectionFactory" locked="false">
    <parameter name="rabbitmq.server.host.name" locked="false">RabbitMQIp</parameter>
	<parameter name="rabbitmq.server.port" locked="false">5672</parameter>
	<parameter name="rabbitmq.server.user.name" locked="false">guest</parameter>
	<parameter name="rabbitmq.server.password" locked="false">guest</parameter>
    </parameter>
</transportSender>

2. Copy RabbitMQ client library files to location:
<API-M_HOME>\repository\components\lib
I was using: amqp-client-5.8.0.jar

3. After the above steps are done, start WSO2 API Manger and RabbitMQ.

4. For passing messages to RabbitMQ, we need to create mediation extension. In this extension we set asynchronous behavior. API Manager will response to client without waiting for response from backend and push message to RabbitMQ exchange. For this we need to set two properties, and use send to RabbitMQ endpoint. In the URI we set connectionstring parameters. We can also add rabbitmq.queue.delivery.mode=2, if you want to mark message as persistent in RabbitMQ. After that, the sequence should look like the following.

<sequence xmlns="http://ws.apache.org/ns/synapse" name="pass.to.rabbitmq">  
   <property name="OUT_ONLY" value="true"/>
   <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2"/>
   <send>
     <endpoint>
       <address uri="rabbitmq:/?rabbitmq.connection.factory=CachedRabbitMQConnectionFactory&rabbitmq.queue.route.key=route&rabbitmq.exchange.name=exchange"/>
     </endpoint>
   </send>
</sequence>

5. Now we need to create an API with API Manager. I suggest to create PUT or POST HTTP verb when defining resources.

6. Next in API Manager Publisher, in Runtime Configuration, we upload Message Mediation as Custom Policies that we created in step 4. and use our created pass.to.rabbitmq sequence.

After this is done, next steps are straight forward for API Manager APIs. Publish this API to the API Store. Now you can subscribe from API Store, and use it as regular one.

Summary

Inspiration for this post was this article, which describes passing data to JMS from WSO2 API Manager.