How to Streamline JWT-Based Traffic Routing with Istio, Keycloak, and AWS EKS
25.06.2024
3 minues
![](https://blog.dedicatted.com/wp-content/uploads/2024/06/47.png)
Here at Dedicatted, we always strive to find the best solutions to implement in our projects, ensuring we fully meet all customer expectations.
Today, we will present an interesting case involving using AWS EKS, Istio, Keycloak, and JWT-based traffic routing. The case is specific, but sharing this knowledge will benefit the community and engineers working on similar projects.
This article will use Istio to configure routing based on JWT claims. The client’s infrastructure runs on the AWS cloud service, with each user having access to their application in separate Kubernetes clusters. Keycloak is used for user authentication, and each user has their ID, which is included in the JWT token. The cluster runs a microservices application with an Application Load Balancer (ALB) configured in front of it. The application can also independently validate JWT tokens.
Problem
The client requires a single AWS CloudFront frontend to route requests to a specific backend located in different AWS EKS Clusters based on the user’s JWT claim.
Initially, we attempted to use Kong to solve this problem, but its open-source plugins were not sufficiently developed to handle this task effectively.
Challenges
One of the main challenges is mastering and configuring Istio. This tool offers robust functionality, but familiarizing yourself with it has a high learning curve, making it complex for beginners.
Another challenge is handling preflight requests, which occur during cross-origin HTTP requests. When a web application in one domain requests a resource in another domain, protocol, or port, it is called a cross-origin request. Preflight requests do not contain JWT tokens, which adds an extra complexity when setting up Istio.
Additionally, there are challenges related to using load balancers in AWS. While AWS provides various load balancers, working with Istio requires specific adjustments and configurations to integrate effectively with them. At first glance, using AWS load balancers may seem straightforward, but combining them with Istio necessitates careful configuration.
Solution
First, you need to install Istio. Visit the official Istio website and download the archive using the application. Follow the instructions in the official documentation to complete the installation. Don’t forget to open the necessary ports for Istio.
It’s also worth installing Kiali. The web interface provided by Kiali will help you understand the structure of Istio and can also display errors or issues in the configuration.
Depending on your configuration, Istio may install a Network Load Balancer (NLB) or a Classic Load Balancer (CLB). However, to meet our requirements, we need an Application Load Balancer (ALB), which operates at the OSI model’s application layer (layer 7) and allows us to perform actions on the request. To change the balancer type, modify the Ingress Gateway service type from Load Balancer to NodePort and configure the ALB.
Istio ingress gateway YAML configuration:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: alb-ingress-gateway
namespace: istio-system
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
alb.ingress.kubernetes.io/actions.ssl-redirect: '443'
alb.ingress.kubernetes.io/load-balancer-name: alb-ingress-gateway
alb.ingress.kubernetes.io/target-type: instance
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:
labels:
app: alb-ingress-gateway
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: istio-ingressgateway
port:
number: 80
Next, you need to create a Gateway and attach it to our Ingress Gateway:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: ex-gateway
namespace: ex-client
spec:
selector:
app: istio-ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
Gateway listens on port 80 because the ALB terminates SSL/TLS connections.
Let’s add our VirtualService, which describes the conditions for routing traffic. So, if client_id = app1, traffic should be sent to api.example1.com. If client_id = app2, traffic should be forwarded to api.example2.com. You also need to add CORS rules. For now, we will comment on the header’s block so you can test if our application is responding to requests.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: client-vs
namespace: ex-client
spec:
hosts:
- "istio.com"
- "api.example1.com"
- "api.example2.com"
gateways:
- ex-client/ex-gateway
- mesh
http:
- name: "ex-client"
match:
- gateways:
- ex-client/ex-gateway
# headers:
# "@request.auth.claims.client_id":
# exact: ex-client
route:
- destination:
host: api.example1.com
port:
number: 80
corsPolicy:
allowOrigins:
- exact: "*"
allowMethods:
- POST
- GET
- OPTIONS
allowHeaders:
- Authorization
- Access-Control-Allow-Origin
allowCredentials: true
- name: "ex-client2"
match:
- gateways:
- ex-client/ex-gateway
# headers:
# "@request.auth.claims.client_id":
# exact: ex-client2
route:
- destination:
host: api.example2.com
port:
number: 80
corsPolicy:
allowOrigins:
- exact: "*"
allowMethods:
- POST
- GET
- OPTIONS
allowHeaders:
- Authorization
- Access-Control-Allow-Origin
To send traffic from outside the cluster, you need to add a ServiceEntry. However, first, we need to explain something. This example uses a microservice application fronted by an Internet-facing ALB, which, depending on the path, sends traffic to a specific service.
Try launching your ALB, digging, and using the received IP addresses to knock on the application. As you can see, nothing worked.
Let’s install an Internet-facing NLB in front of the application and direct it to the ALB target group. When running NLB, you must assign an elastic IP to at least one subnet so that Istio can send traffic to apps in different clusters. Also, you need to delete the ‘host’ in ALB. You can make ALB internal. After all the manipulations, ServiceEntry looks like this:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: ex-client
namespace: istio-system
spec:
hosts:
- api.example1.com
ports:
- number: 80
name: http
protocol: HTTP
resolution: STATIC
endpoints:
- address: "34.230.122.162"
location: MESH_EXTERNAL
—--
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: ex-client2
namespace: istio-system
spec:
hosts:
- api.example2.com
ports:
- number: 80
name: http
protocol: HTTP
resolution: STATIC
endpoints:
- address: "174.129.253.100"
location: MESH_EXTERNAL
At the moment, the virtual services are almost identical, so when you comment on one of them, the other will begin to receive traffic. Also, you can find this out by checking the Kiali “Istio Config” dashboard.
Keycloak
We need to add authentication to validate JWT tokens. To do this, you need to create a RequestAuthentication resource:
apiVersion: security.istio.io/v1
kind: RequestAuthentication
metadata:
name: ingress-jwt
namespace: istio-system
spec:
selector:
matchLabels:
app: istio-ingressgateway
jwtRules:
- issuer: "https://auth.keycloak.com/realms/example"
jwksUri: "https://auth.keycloak.com/realms/example/protocol/openid-connect/certs"
forwardOriginalToken: true
forwardOriginalToken is responsible for keeping the original JWT token to the upstream request.
After this, you need to add an authorization policy allowing traffic to be sent further.
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: authpolicyjwt
namespace: istio-system
spec:
selector:
matchLabels:
app: istio-ingressgateway
action: ALLOW
rules:
- to:
- operation:
methods: ["OPTIONS"]
path: ["/*"]
- when:
- key: request.auth.claims[client_id]
values: ["ex-client"]
- when:
- key: request.auth.claims[client_id]
values: ["ex-client2"]
Since the JWT plugin tries to authenticate all requests, we encounter preflight requests without a JWT token.
To solve the problem, you need to apply a filter that launches the CORS plugin before the JWT plugin:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: cors-bypass-jwt
namespace: istio-system
spec:
workloadSelector:
labels:
app: istio-ingressgateway
configPatches:
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
subFilter:
name: "envoy.filters.http.jwt_authn"
patch:
operation: MERGE
value:
name: "envoy.filters.http.jwt_authn"
typed_config:
"@type": "type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication"
bypass_cors_preflight: true
Also, you need to apply a filter that sends a 200 response to preflight requests:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: allow-preflight-requests
namespace: istio-system # Adjust based on your namespace
spec:
workloadSelector:
labels:
app: istio-ingressgateway # Apply to Istio ingress gateway or another relevant workload
configPatches:
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
subFilter:
name: "envoy.filters.http.router"
patch:
operation: INSERT_BEFORE
value:
name: envoy.lua
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
inlineCode: |
function envoy_on_request(request_handle)
-- If the request method is OPTIONS, just return with no further processing
local method = request_handle:headers():get(":method")
if method == "OPTIONS" then
request_handle:respond(
{
[":status"] = "200",
["access-control-allow-origin"] = "*",
["access-control-allow-methods"] = "GET, POST, OPTIONS, PUT, DELETE",
["access-control-allow-headers"] = "Authorization, Content-Type, Access-Control-Allow-Origin"
}
)
end
end
Almost done.
To get routing using JWT claims, you must uncomment a “headers” block to VirtualService.
Now, let’s check if everything is working as expected.
Let’s log in to our app with a user that has client_id “ex-client.”
After that, we log in with a user with client_id “ex-client2.”
As you can see, everything is working as expected.
Conclusion
The Dedicatted team achieved the goal of creating a unified frontend configuration with traffic routing based on JWT claims using Istio and user authentication with Keycloak. This approach allowed us to satisfy client’s needs with the management of user authentication to platform systems with the Keycloak & Istio bundle in the scope of several AWS EKS Clusters and simplify the CI/CD process, as we now only need to build a single frontend application. We also improved page load times by configuring caching on the AWS CloudFront side.
Based on the implemented solution, the Dedicatted team successfully covered project needs in the desired scope and satisfied business requirements.
Authors:
- George Levytskyy
- Mykyta Ponomarenko