WSO2 Plugin for Treblle

treblle-wso2

The official Treblle plugin for WSO2.

The extension retrieves data from Global Synapse Handler in the API Gateway and creates a payload to send to Treblle. The data is added onto a queue once received and is processed by a worker thread. The worker thread sends the data asynchronously to Treblle, if the data is successfully sent, the data is removed from the queue. If the data is not successfully sent, the worker thread will attempt to send the data again, after 1 retry attempt, the event will be dropped.

Prerequisites

Before you begin, make sure you have:

Java: Version 8 or later (java -version)
Maven: Version 3.6.0 or later (mvn -version)

Step 1: Install WSO2 API Manager

Download the WSO2 API Manager 4.3.0 from the official WSO2 website.

Unzip the archive:

unzip wso2am-4.3.0.zip
cd /path/wso2am-4.3.0/bin/
export JAVA_HOME=$(/usr/libexec/java_home -v 11) ./api-manager.sh

Open the Admin Console in your browser:
- URL: https://localhost:9443/carbon
- Username: admin
- Password: admin

Step 2: Create an API in WSO2 API Manager

Log in to the Publisher Portal at https://localhost:9443/publisher.

Use the default credentials:
- Username: admin
- Password: admin
Create a New API:
- Choose the API type (e.g., REST API, SOAP API).
- Provide details like API Name, Context Path, Version, and Endpoint URL.
- Publish the API.
Test the API:
- Log in to the Developer Portal at https://localhost:9443/devportal/apis.
Subscribe to the API:
- Go to the Subscription section.
- Use the Subscription and Key Generation Wizard to generate an access token:

Provide details such as:
- Application Name.
- Description.
- Shared Quota.
Generate the access token and copy it. You will use this token to authenticate your requests.

Step 3: Define API Resources

In the WSO2 Publisher, go to the Resources section.
Define the endpoints for your API, such as /users.

Step 4: Test the API

Return to the Developer Portal and locate the API you subscribed to.
Click the Try Out option to test your API directly within the portal.
Select the key type for testing:
- Sandbox: For testing the API in a non-production environment.
- Production: For live use in production.
Choose the method you want to test (e.g., GET).
Execute the request by providing the required parameters.

Step 5: Set Up Treblle

Follow these steps to integrate Treblle with your WSO2 API Manager:

Visit the Treblle website and log in or sign up for an account if you don’t have one.
Once logged in, create a new project by providing the following details:
- API Name.
- Base URL.
- Environment.
- Platform.
After creating the project, Treblle will generate:
- API Key
- Project ID
These credentials are essential for integrating Treblle with your WSO2 API Manager.

Step 6: Install Treblle Data Publisher Extension

Clone the Treblle Data Publisher repository:

git clone https://github.com/Treblle/treblle-wso2.git
cd /path/treblle-wso2/wso2am-4.3.0

Build the project:
Terminal window
```
mvn clean install
```
This will generate the necessary .jar file for the Treblle Data Publisher extension.
Copy the generated JAR file to WSO2 API Manager:
Terminal window
```
cp target/treblle-data-publisher-*.jar /path/wso2am-4.3.0/repository/components/lib
```
If mvn clean install is giving ERRORS. Please download the aspectjweaver-1.8.5.jar file from the WSO2 Nexus repository manually.

Assuming your downloaded JAR is in your Downloads folder:
Terminal window
```
mv ~/Downloads/aspectjweaver-1.8.5.jar ~/.m2/repository/org/aspectj/aspectjweaver/1.8.5/
```

Step 7: Configure WSO2 API Manager for Treblle

Update the deployment.toml file (Add at the beginning of the file):

/wso2am-4.3.0/repository/conf/deployment.toml

[synapse_handlers.treblle_publisher]
enabled=true
class="com.treblle.wso2publisher.handlers.APILogHandler"

Update log4j2.properties:

To enable the logs to populate under the default wso2carbon.log, you can create a custom appender to log the entries to a separate log file.

 /wso2am-4.3.0/repository/conf/log4j2.properties

logger.treblle_publisher.name = com.treblle.wso2publisher
logger.treblle_publisher.level = INFO
logger.treblle_publisher.appenderRef.CARBON_LOGFILE.ref = CARBON_LOGFILE

Set environment variables:

Open a terminal and add the following environment variables. Alternatively, set these variables in your .bash_profile or .bashrc to make them persistent across sessions.

For MacOS/Linux:

# Required variables
export TREBLLE_API_KEY="your-api-key"           # Your Treblle API key
export TREBLLE_PROJECT_ID="your-project-id"      # Your Treblle project ID
export TREBLLE_GATEWAY_URL="your-gateway-url"    # Your WSO2 gateway URL

# Optional variables with defaults
export TREBLLE_QUEUE_SIZE=20000                  # Max size of the request queue
export TREBLLE_WORKER_THREADS=1                  # Number of worker threads
export ADDITIONAL_MASK_KEYWORDS="Authorization,token"  # Fields to mask
export TREBLLE_ENABLED_TENANT_DOMAINS="carbon.super"  # Enabled tenant domains

For Windows:

:: Required variables
set TREBLLE_API_KEY=your-api-key
set TREBLLE_PROJECT_ID=your-project-id
set TREBLLE_GATEWAY_URL=https://your-wso2-gateway-url

:: Optional variables
set TREBLLE_QUEUE_SIZE=20000                     :: Default queue size
set TREBLLE_WORKER_THREADS=1                     :: Default worker threads
set ADDITIONAL_MASK_KEYWORDS=Authorization,token  :: Default masked fields
set TREBLLE_ENABLED_TENANT_DOMAINS=carbon.super  :: Default tenant domain

Verifying Environment Variables:

To check that your environment variables are properly set up for Treblle:

# For Linux/macOS
echo $TREBLLE_API_KEY
echo $TREBLLE_PROJECT_ID
echo $TREBLLE_GATEWAY_URL

# For Windows
echo %TREBLLE_API_KEY%
echo %TREBLLE_PROJECT_ID%
echo %TREBLLE_GATEWAY_URL%

Step 8: Configure OAuth2 Authentication

To generate an OAuth2 Token, log in to the Developer Portal at https://localhost:9443/devportal.
Under the Subscription section, click on your created application.
Select either Production or Sandbox as the key type.
Click Generate Key for the chosen environment and copy the generated access token.
This token is essential for accessing protected resources in your API.

Step 9: Testing API Requests with OAuth2 Authentication

After setting up your WSO2 API and Treblle integration, you’ll need to test the API with proper authentication. Here are curl commands for different scenarios:

Generate an OAuth2 Access Token First, obtain an access token:

  curl -k -X POST \
    https://localhost:8243/token \
    -H "Authorization: Basic $(echo -n client_id:client_secret | base64)" \
    -H "Content-Type: application/x-www-form-urlencoded" \
    -d "grant_type=password&username=admin&password=admin&scope=default"

Replace client_id and client_secret with the credentials from your application.

Retrieve Users from Your API

Use the token to make a GET request:

  curl -k -X GET \
    "https://localhost:8243/user-management/v1/users" \
    -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

Get a Specific User

  curl -k -X GET \
    "https://localhost:8243/user-management/v1/users/1" \
    -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

Create a New User

 curl -k -X POST \
  "https://localhost:8243/user-management/v1/users" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "john.doe@example.com",
    "first_name": "John",
    "last_name": "Doe"
  }'

Update a User

curl -k -X PUT \
  "https://localhost:8243/user-management/v1/users/1" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "john.doe@example.com",
    "first_name": "John",
    "last_name": "Smith"
  }'

Delete a User

curl -k -X DELETE \
  "https://localhost:8243/user-management/v1/users/1" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

Testing the Treblle Integration

To verify that Treblle is capturing your API requests correctly, make a series of API calls:

curl -k -X GET "https://localhost:8243/user-management/v1/users" -H "Authorization: Bearer YOUR_ACCESS_TOKEN"
curl -k -X GET "https://localhost:8243/user-management/v1/users/1" -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

Then check the Treblle dashboard to confirm the requests are being logged.

Step 10: Restart WSO2 API Manager

Restart the server to apply the configurations:

cd /path/wso2am-4.3.0/bin
./api-manager.sh restart

Step 11: Test API Observability with Treblle

Use cURL to send a request:

Replace “access_token” with a valid OAuth2 token.

  curl -k -X GET "https://localhost:8243/<api-context>/<version>" -H "Authorization: Bearer <access_token>"

Troubleshooting the Integration

If you’re not seeing requests in Treblle, check the logs:

 # View WSO2 logs
  tail -f /path/wso2am-4.3.0/repository/logs/wso2carbon.log | grep "treblle"

Dashboard Overview

The Treblle dashboard provides real-time insights into API requests, compliance checks, and potential issues.

Customize the View: Use the customization options at the top right of the dashboard to tailor the information displayed according to your needs.

Detailed API Request Logs

Request List: View a detailed list of API requests, including key metrics and diagnostic information.

Inspect Individual Requests: