What You’ll Learn

• How to deploy Calisti to your Kubernetes cluster
• How to visualize your microservices applications using Service Mesh Manager (SMM)
• How to inject traffic faults and tap traffic between microservices components
• How to add external virtual machine (VM) resources to a Kubernetes-based application

What You’ll Need

• A free Calisti account
• Access to an Amazon Web Services (AWS) account with billing information on file (more information in the tutorial)

AWS Instances and Cost

Prior to starting the lab, you’ll need to create an AWS account. This tutorial will depend on using a large AWS Elastic Compute Cloud (EC2) instance, which does not qualify for the free tier within the AWS cloud. This instance, as configured and deployed in us-east-1, will bill at about $1.25 per hour, which includes the instance and the additional root-block storage. This instance may bill differently in other regions and will require a payment method on file for the instance to be able to be provisioned. We will be using Terraform to provision the instance, which will allow us to easily destroy the instance after we are done with the tutorial. Please remember that this lab will not be free using the AWS resources.

A Note About AWS Regions

AWS has many different regions in which its data centers and cloud offerings are available. Generally speaking, it is advantageous (from a latency perspective) to choose a region that is geographically close to you. However, it is important to note that different services within the AWS console may either be “global” in scope (such as Identity and Access Management [IAM], wherein changes in this service will affect all regions), or regional (like EC2, such that a key pair generated in one region cannot be used for another region). The scope of the service will be indicated in the upper-right corner of the service window.

Adding a Payment Method

Once you are signed up with an AWS account, you will need to access the billing platform by typing billing in the AWS Service search bar and selecting it. Once in the billing service, click Payment preferences and add a new payment method. By default, all major credit and debit cards are accepted, but please ensure that you delete the lab when complete using Terraform so that you will not be charged for usage beyond the exercises within this tutorial.

Creating Access Keys

Once billing information has been input, you will need to create access keys to allow your local machine to provision the EC2 instance using Terraform. This task is accomplished by searching for IAM in the AWS Service search bar and selecting it.

When the IAM service page appears, go to the left-hand side of the window and click Users, which will bring up a list of the users within the account. Select your username, which will bring up a new page with various information about your user. Select Security Credentials in the main window pane, and scroll down to Access keys and click Create access key.

In the resulting window, select Other, followed by Next. In the next window, you can set an optional description for the access key.

Once you click Next, you will be presented with the access key and secret key for your account, along with the option to download a CSV file of these credentials. Once you leave this screen, you will not be able to access the secret key. You may want to download the CSV file for safekeeping, but do not commit any of these credentials to a public repository; these credentials will allow anyone to access your account without your knowledge!

Creating an EC2 Key Pair

The final step of gathering the AWS prerequisites will be to create a new EC2 key pair. You can do this by searching for EC2 in the AWS service search bar and selecting it.

From there, select Key pairs in the left pane (under Network and Security), and select Create key pair.

Provide a name, ensure that both RSA and .pem are selected, and then click Create key pair.

This action will download the key pair to your local machine. Please do not delete this key pair because it will be required to perform SSH to the cloud instance that will be stood up in EC2, which we will accomplish next.

Installing Terraform and Gathering Files

The EC2 instance will be provisioned in the AWS cloud using Terraform. By using Terraform, we can package the required files for deployment and ensure that every instance is created in a similar way. Additionally, Terraform will allow us to destroy all configured resources after we are done to ensure that we are not charged for usage outside of the lab exploration. In order to use Terraform, you’ll need to install it using the instructions found on https://terraform.io/downloads for your operating system. If you choose to use the binary download, please download it and ensure that it resides in $PATH or the folder in which you plan on executing the Terraform files.

Next, you’ll need to download the code for this lab found here: https://github.com/CiscoLearning/calisti-tutorial-sample-code/. This repository will contain all the required Terraform for building the AWS EC2 instance as well as files that will be needed for the completion of the tutorial. Once the code has been cloned, you’ll need to navigate to the calisti-tutorial-sample-code/terraform/terraform.tfvars file and add in the required information that we acquired in the previous step (AWS IAM keys, AWS EC2 key pair name, and EC2 key pair location).

Once this file has been modified to reflect your information, the final piece will be ensuring the region that you have created the EC2 key pair in (and which region you decided to use). The region location is declared at the top of the panoptica-main.tf file.

Once these changes have been made, move your terminal into the calisti-tutorial-sample-code/terraform folder and perform a terraform init, followed by a terraform apply -auto-approve. (We’re skipping the plan step because we know what will be created.) This process should only take a couple of minutes, and at the end, you should have a successfully built AWS VM. The output at the end of the Terraform run should indicate the fully qualified domain name (FQDN) of your new AWS EC2 instance, which has been fully updated and had Docker, kubectl, kind, caddy, k9s, and terraform installed, as well as the copy of files in the calisti-tutorial-sample-code folder cloned directly from GitHub.

Once the EC2 instance build is completed, there will be output generated by the Terraform configuration. This will print the required SSH command, including the location of the SSH key from the included variables, as well as the FQDN, which is not known until the instance is created. You can copy and paste this command and hit <ENTER> to connect to the VM.

The goal of these first tasks is to deploy a Kubernetes demo cluster based on kind (Kubernetes in Docker) that will serve as the Kubernetes cluster for Calisti and the included demo application. The included shell script will build a cluster with one control plane node and four worker nodes, and it is located in the calisti-tutorial-sample-code/calisti/cluster folder within the EC2 instance.

Create Kind Cluster

To instantiate the kind cluster, we will reference the included shell script found at the folder location above. This file, when executed, will build a kind cluster using the 1.23.13 image, wait for the cluster to become fully active, and then install MetalLB within the cluster in Layer 2 mode. (MetalLB enables Kubernetes services of type LoadBalancer to be installed on bare-metal Kubernetes installations.) If you wish to analyze the full cluster creation script, you can view the file by executing cat calisti-tutorial-sample-code/calisti/cluster/cluster_setup.sh.

When this shell script is run, you will see output similar to the following:

ubuntu@ip-172-31-0-66:~/calisti$ bash cluster_setup.sh
Creating cluster "demo1" ...
 ✓ Ensuring node image (kindest/node:v1.23.13) 🖼
 ✓ Preparing nodes 📦 📦 📦 📦 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
 ✓ Joining worker nodes 🚜
Set kubectl context to "kind-demo1"
You can now use your cluster with:

kubectl cluster-info --context kind-demo1

Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
Waiting for matallb to be deployed...
Deploying metallb
namespace/metallb-system created
customresourcedefinition.apiextensions.k8s.io/addresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created
customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created
customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created
serviceaccount/controller created
serviceaccount/speaker created
role.rbac.authorization.k8s.io/controller created
role.rbac.authorization.k8s.io/pod-lister created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/controller created
rolebinding.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
secret/webhook-server-cert created
service/webhook-service created
deployment.apps/controller created
daemonset.apps/speaker created
validatingwebhookconfiguration.admissionregistration.k8s.io/metallb-webhook-configuration created
Waiting for metallb to be ready...
pod/controller-7967ffcf8-569k6 condition met
pod/speaker-7bhz7 condition met
pod/speaker-fg2ff condition met
pod/speaker-pjd2k condition met
pod/speaker-pmwh4 condition met
pod/speaker-vhrhj condition met
ipaddresspool.metallb.io/metallb-ippool created
l2advertisement.metallb.io/metallb-l2-mode created

To verify that the entire cluster has been bootstrapped and is ready for Calisti installation, you can use a kubectl command to verify the status of all pods within the cluster:

ubuntu@ip-172-31-0-66:~/calisti$ kubectl get pods -A
NAMESPACE            NAME                                          READY   STATUS    RESTARTS   AGE
kube-system          coredns-64897985d-c7mmd                       1/1     Running   0          13m
kube-system          coredns-64897985d-kq2kt                       1/1     Running   0          13m
kube-system          etcd-demo1-control-plane                      1/1     Running   0          13m
kube-system          kindnet-6z6cm                                 1/1     Running   0          13m
kube-system          kindnet-b88bq                                 1/1     Running   0          13m
kube-system          kindnet-cfj26                                 1/1     Running   0          13m
kube-system          kindnet-k64rq                                 1/1     Running   0          13m
kube-system          kindnet-kpnww                                 1/1     Running   0          13m
kube-system          kube-apiserver-demo1-control-plane            1/1     Running   0          13m
kube-system          kube-controller-manager-demo1-control-plane   1/1     Running   0          13m
kube-system          kube-proxy-5lk8z                              1/1     Running   0          13m
kube-system          kube-proxy-8t467                              1/1     Running   0          13m
kube-system          kube-proxy-hfwvw                              1/1     Running   0          13m
kube-system          kube-proxy-m8c2c                              1/1     Running   0          13m
kube-system          kube-proxy-zgkmj                              1/1     Running   0          13m
kube-system          kube-scheduler-demo1-control-plane            1/1     Running   0          13m
local-path-storage   local-path-provisioner-58dc9cd8d9-ss289       1/1     Running   0          13m
metallb-system       controller-7967ffcf8-569k6                    1/1     Running   0          12m
metallb-system       speaker-7bhz7                                 1/1     Running   0          12m
metallb-system       speaker-fg2ff                                 1/1     Running   0          12m
metallb-system       speaker-pjd2k                                 1/1     Running   0          12m
metallb-system       speaker-pmwh4                                 1/1     Running   0          12m
metallb-system       speaker-vhrhj                                 1/1     Running   0          12m

All pods should be in the running state. Notice that we have a separate Kubernetes namespace created for MetalLB, which was created as part of the cluster build script. We’re now ready to deploy Calisti to the cluster.

Installing Calisti and the Demo Application

In order to install Calisti, you’ll need to create a free Calisti account. Once logged in, click the button to Connect cluster, which will take you to a screen that documents the system requirements for the application.

We’re using kind on a large EC2 instance—c5a.8xlarge has 32 vCPU and 64 GB RAM, and we’ve given plenty of GP2 storage to the system as well—so we meet these specs.

Clicking continue will bring you to a screen wherein you’ll enter an installation name and, if required, a proxy server. While this installation method automates a large portion of the Calisti installation into the cluster, we’ll use another method that will allow us to install all features and ensure that we can access the dashboard on our EC2 instance.

Begin by clicking the Download link for the Calisti CLI file for Linux. This will allow us to install SMM and Streaming Data Manager (SDM) into the cluster, as well as the demo application.

In order to install the demo application and access the Calisti dashboard, we’ll need to copy over the Calisti binaries to our EC2 instance. Ensure that you download the Linux binaries, and then copy them to the EC2 instance using Secure Copy Protocol (SCP).

scp -i {KEY_LOCATION} Downloads/smm_1.12.0_linux_amd64.tar.gz ubuntu@{EC2_FQDN}:.

As an example of the above:

scp -i ~/.ssh/qms-keypair.pem Downloads/smm_1.12.0_linux_amd64.tar.gz ubuntu@ec2-18-209-69-87.compute-1.amazonaws.com:../

You’ll then need to decompress and untar the compressed file and ensure that the resulting files are executable:

cd ~
tar xvzf smm__1.12.0_linux_amd64.tar.gz
chmod +x smm supertubes

Next, you’ll need to gather the shell command from here, under 02. Get your Activation Credentials. Copy the command (an example is given below) and paste it to the terminal of your EC2 instance.

SMM_REGISTRY_PASSWORD={USER_PASSWORD} ./smm activate \
--host=registry.eticloud.io \
--prefix=smm \
--user={USER_STRING}

This should provide output similar to the following, and your cluster should now be able to access the SMM repositories.

ubuntu@ip-172-31-0-66:~$ SMM_REGISTRY_PASSWORD={USER_PASSWORD} ./smm activate \
> --host=registry.eticloud.io \
> --prefix=smm \
> --user={USER_STRING}
? Are you sure to use the following context? kind-demo1 (API Server: https://127.0.0.1:36263) Yes
✓ Configuring docker image access for Calisti.
✓ If you want to change these settings later, please use the 'registry' and 'activate' commands

Once this step is complete, you can install Calisti, which we will use for the rest of our lab.

./smm install --non-interactive -a --install-sdm -a --additional-cp-settings ~/calisti-tutorial-sample-code/calisti/smm/enable-dashboard-expose.yaml

The output will indicate that it is creating a set of custom resource definitions (CRDs) and then creating Kubernetes deployments in different namespaces. When it is complete, you can perform a kubectl get pods -A. When all pods are in the Running state, you can invoke the following command to check the status of the Istio deployment, upon which SMM is built:

./smm istio cluster status -c ~/.kube/demo1.kconf

If everything is deployed correctly, the following output will be shown:

logged in as kubernetes-admin
Clusters
---
Name        Type   Provider  Regions  Version   Distribution  Status  Message
kind-demo1  Local  kind      []       v1.23.13  KIND          Ready


ControlPlanes
---
Cluster     Name                        Version  Trust Domain     Pods                                                  Proxies
kind-demo1  sdm-icp-v115x.istio-system  1.15.3   [cluster.local]  [istiod-sdm-icp-v115x-7fd5ccc9d9-l2zfq.istio-system]  8/8
kind-demo1  cp-v115x.istio-system       1.15.3   [cluster.local]  [istiod-cp-v115x-6bdfb6b4bd-c7qpb.istio-system]       22/22

We’ll now deploy our demo app, which will be installed using the smm binary:

./smm demoapp install --non-interactive

This will take some time. When the command completes, we want to ensure that all pods in the smm-demo namespace are in the running state.

kubectl get pods -n smm-demo

The output should appear similar to the following:

ubuntu@ip-172-31-14-56:~$ kubectl get pods -n smm-demo
NAME                                  READY   STATUS    RESTARTS   AGE
analytics-v1-6c9fd4c7d9-lbq77         2/2     Running   0          43s
bombardier-5f59948978-cdvnw           2/2     Running   0          43s
bookings-v1-6b89b9d965-spgrm          2/2     Running   0          43s
catalog-v1-7dd5b79cf-s2z6s            2/2     Running   0          43s
database-v1-6896cd4b59-kr67p          2/2     Running   0          43s
frontpage-v1-5976b889-jkgqw           2/2     Running   0          43s
movies-v1-9594fff5f-2zm4h             2/2     Running   0          43s
movies-v2-5559c5567c-q8n2s            2/2     Running   0          43s
movies-v3-649b99d977-ttp4j            2/2     Running   0          43s
mysql-669466cc8d-dmg44                2/2     Running   0          42s
notifications-v1-79bc79c89b-zj9xz     2/2     Running   0          42s
payments-v1-547884bfdf-6wpcn          2/2     Running   0          42s
postgresql-768b8dbd7b-khxvs           2/2     Running   0          42s
recommendations-v1-5bdd4cdf5f-cxld2   2/2     Running   0          42s

Finally, we need to expose the Calisti dashboard outside of the cluster. We will accomplish this in two steps. The first step is to determine the IP address of the ingress load-balancer that is deployed as part of Calisti inside of our Kubernetes cluster. The second step is to use Caddy to forward an external port on our EC2 instance to this host port. This is accomplished with the following:

INGRESSIP=$(kubectl get svc smm-ingressgateway-external -n smm-system -o jsonpath="{.status.loadBalancer.ingress[0].ip}")
echo $INGRESSIP
caddy reverse-proxy --from :8080 --to ${INGRESSIP}:80 > /dev/null 2>&1 &

We can now access opening a web browser and the FQDN of our EC2 instance on port 8080, like http://{EC2_FQDN};8080. We can log in by generating a token from the smm CLI via our SSH session, and then pasting that into the web browser and logging in.

./smm login --non-interactive

Once logged in, you will be presented with a dashboard indicating the current state of the smm-demo namespace. We’re now ready to explore the capabilities of SMM.

SMM Overview

SMM is a multi and hybrid cloud-enabled service mesh platform for constructing modern applications. Built on Kubernetes and our Istio distribution, SMM enables flexibility, portability, and consistency across on-premises data centers and cloud environments.

SMM helps you to confidently scale your microservices over single- and multicluster environments and to make daily operational routines standardized and more efficient. The componentization and scaling of modern applications inevitably leads to a number of optimization and management issues:

• How do you spot bottlenecks?
• Are all components functioning correctly?
• How are connections between components secured?
• How does one reliably upgrade service components?

SMM helps you accomplish these tasks and many others in a simple and scalable way, by leveraging the Istio service mesh and building many automations around it.

Architecture

The open source Cisco Istio operator helps to install, upgrade, and manage Istio deployments. Its unique features include managing multiple ingress/egress gateways in a declarative fashion, and automated and fine-tuned multicluster management.

The core components of SMM are:

• SMM backend, exposing a GraphQL application programming interface (API)
• SMM user interface (a web interface)
• SMM CLI
• SMM operator

The soul of SMM is its backend, which exposes a GraphQL API. The SMM user interface (dashboard) and CLI interact with this API. The SMM operator is an optional component that helps with a declarative installation method to support GitOps workflows.

External out-of-the-box integrations include:

• Prometheus
• Grafana
• Jaeger
• Cert manager

Calisti SMM automatically installs and configures these components by default to be able to work with Istio, but it can also integrate with your own Prometheus, Grafana, Jaeger, or Cert manager.

Topology

The topology page of the SMM web interface displays the topology of services and workloads inside the mesh and annotates it with real-time information about latency, throughput, or HTTP request failures. You can also display historical data by adjusting the timeline.

The topology view is almost entirely based on metrics—metrics received from Prometheus and enhanced with information from Kubernetes.

The topology page serves as a starting point of diagnosing problems within the mesh. SMM is integrated with Grafana and Jaeger for easy access to in-depth monitoring and distributed traces of various services.

Select the smm-demo namespace and display its topology.

The nodes in the graph are services or workloads, while the arrows represent network connections between different services. This is based on Istio metrics retrieved from Prometheus. You can click and zoom into the services; note how the traffic protocols along with the requests per second (RPS) are also shown in the topology view.

We can easily observe the various microservices in the demo application:

• The frontpage microservice calls bookings, catalog, and PostgreSQL microservices to populate the page.
• The bookings microservice calls the analytics and payments microservices.
• The payments microservice calls the notifications microservice.
• The catalog microservice calls the movie microservices.
• There are three versions of the movies microservice, with version2 using MySQL and version3 using a different database.

Calisti is also able to show the details for services such as MySQL and PostgreSQL; these metrics are not available in Istio and is a value-add provided by Calisti. Click the postgresql service, and in the pop-up window, scroll down to note how it shows details such as SQL transactions per second.

Select one node in the graph (for example, the postgresql service) and display its details. By drilling down and selecting the pod, it is also possible to display its logs directly in the dashboard (click the icon).

Generate Traffic Load

Note: By default, when the demo app is deployed, traffic generation is already started. This topic serves as an explanation for the traffic generation functions, but it can be skipped if desired.

Most of the data displayed and the features provided in the Calisti interface is based on analyzing the traffic received by the different applications in the cluster. Calisti provides several mechanisms to generate traffic.

If there is no traffic generated, the topology cannot be displayed; an option to generate traffic is displayed instead.

If the topology is displayed, triggering the traffic generation can be done using the HTTP button on the left.

Let’s generate some traffic on the frontend service from the smm-demo namespace.

Going back to the overview page of the dashboard, we can now see the traffic increasing.

For the demoapp application, constant traffic can also be generated using the CLI.

Note: Traffic generation is started automatically when the demo app is installed; there is no need to start this traffic load again.

Start with

smm demoapp load start

and stop with

smm demoapp load stop

Calisti offers Mutual Transport Layer Security (mTLS) as a solution for service-to-service authentication. It accomplishes this using the sidecar pattern, meaning that each application container has a sidecar Envoy proxy container running beside it in the same pod.

When a service receives or sends network traffic, the traffic always goes through the Envoy proxies first. When mTLS is enabled between two services, the client side and server side Envoy proxies verify each other’s identities before sending requests.

If the verification is successful, then the client-side proxy encrypts the traffic, and sends it to the server-side proxy. The server-side proxy decrypts the traffic and forwards it locally to the actual destination service.

In Calisti, you can manage the mTLS settings using:

• The user interface at the service level

• The PeerAuthentication custom resource—within the mesh, within a namespace, and at the service level

The following policies are available:

• STRICT: Accept only mTLS traffic.

• PERMISSIVE: Accept both plaintext/unencrypted traffic and mTLS traffic at the same time.

• DISABLE: Accept plaintext/unencrypted traffic only.

• DEFAULT: Use the global mTLS policy.

In order to view the status of mTLS within the mesh, from within the topology view under Edge Labels, select Security. If a link has a green lock, it means that it uses mTLS.

Changing Service-Specific mTLS Settings with the User Interface

Select the analytics service in the topology and go to MTLS Policies. Select DISABLE for all ports and APPLY CHANGES.

After 10 to 20 seconds, you should see the lock now being open and red when going back to the topology, meaning traffic going to that service will not be encrypted.

Change mTLS Settings Using PeerAuthentication Custom Resource

We’ll now use a custom resource definition to re-enable mTLS for the analytics service. The CRD for this is defined below …

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: analytics
  namespace: smm-demo
spec:
  mtls:
    mode: STRICT

… which is provided in the ~/calisti-tutorial-sample-code/calisti/reenable-mtls.yaml file. Let’s apply this CRD using kubectl.

kubectl apply -f ~/calisti-tutorial-sample-code/calisti/reenable-mtls.yaml

Note: It will take 5 to 10 minutes to see the results in the topology view. There is no need to wait for it to show as enabled; you can check it later on in the tutorial if desired.

Switch back to display the request rate/throughput on the topology links.

The traffic tap feature of SMM enables you to monitor live access logs of the Istio sidecar proxies. Each sidecar proxy outputs access information for the individual HTTP requests or HTTP/gRPC streams.

The access logs contain information about the reporter proxy, source and destination workloads, request, response, and timings.

Tapping Traffic Using the CLI

To watch the access logs for an individual namespace, workload, or pod, use the smm tap command. Be sure to have some live traffic generated, using any of the previous methods for this to work. (This exists because we are using the demoapp application.)

For smm-demo namespace

cd ~
./smm tap ns/smm-demo

Note: This will produce a lot of traffic output in the terminal.

Press Ctrl+C in the terminal to stop. The output should be similar to the following:

It is also possible to filter on workload …

./smm tap --ns smm-demo workload/bookings-v1

… and also use filter tags to only display the relevant lines:

./smm tap ns/smm-demo --method GET --response-code 500,599

Note: No output will be displayed if there is no traffic matching those error codes.

Tapping Traffic with the User Interface

The traffic tapping functionality is also available in the user interface, including setting different filters. To see this within the user interface, navigate to the TRAFFIC TAP menu item.

You’ll then want to select the smm-demo namespace as a source, and click START STREAMING.

Once the traffic appears within the user interface, if you select one of the requests from within the list, it will pop out a details pane to the right of the screen, providing information to debug and analyze the traffic. If additional filters are needed, the FILTERS menu (when clicked) will drop down additional menu options to filter the requests tapped.

Calisti also provides distributed tracing—the process of tracking individual requests throughout their whole call stack in the system.

With distributed tracing in place, it is possible to visualize full call stacks, to see which service called which service, how long each call took, and how much were the network latencies between them. It is possible to tell where a request failed or which service took too much time to respond. To collect and visualize this information, Istio comes with tools like Jaeger, which is installed automatically by default when installing SMM.

The demo application uses Golang services, which are configured to propagate the necessary tracing headers.

Once load is sent to the application, traces can be perceived right away.

Jaeger is exposed through an ingress gateway, and the links are present on the user interface, both on the graph and list view.

In order to access the Jaeger user interface, return to the topology view for the application. For our investigation, we’ll look at the bookings microservice by clicking its icon within the topology view. On the resulting pane that appears from the right of the screen, scroll until you see the Traces link; hovering over it will indicate that it will open the Jaeger dashboard.

In the Jaeger user interface, you can see the whole call stack in the microservices architecture. You can see when exactly the root request was started and how much each request took.

Fault injection is a system testing method that involves the deliberate introduction of network faults and errors into a system. It can be used to identify design or configuration weaknesses and to ensure that the system can handle faults and recover from error conditions.

With SMM, you can inject failures at the application layer to test the resiliency of the services within your application or cluster. You can configure faults to be injected into requests that match specific conditions to simulate service failures and higher latency between services. There are two types of failures:

• Delay adds a time delay before forwarding the requests, emulating various failures such as network issues, an overloaded upstream service, and so on.

• Abort cancels the HTTP request attempts and returns error codes to a downstream service, giving the impression that the upstream service is faulty.

SMM uses Istio’s (Envoy) fault injection feature under the hood.

The Calisti dashboard offers a powerful YAML editor in order to make the management of different k8s resources easier.

We will use both the YAML editor and kubectl to apply the needed configurations.

Let’s begin by creating a DestinationRule for the payments service using some YAML and kubectl. The DestinationRule will be located under the sample code directory and called paymentsDR.yaml.

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: payments-destination
  namespace: smm-demo
  labels:
    istio.io/rev: cp-v115x.istio-system
spec:
  host: payments
  subsets:
    - name: v1
      labels:
        version: v1

We can apply this DestinationRule using the following command:

kubectl apply -f ~/calisti-tutorial-sample-code/calisti/smm/paymentsDR.yaml

After applying the rule, navigate to the topology pane within the Calisti user interface and select the payments microservice. Select DESTINATION RULES on the pane that appears on the right after selecting the service. You should be able to see the created DestinationRule in the user interface.

We’ll now create a VirtualService for Fault Injection Abort, this time using the Calisti YAML editor. Click VIRTUAL SERVICE at the top of the right-hand pane, and click CREATE NEW.

This will bring up a template editor window. Underneath the template text, there is a drop-down window that will allow you to select from a set of pre-created templates to apply. Select Fault Injection Abort Template. This will now change the YAML underneath to represent the desired template.

Next, ensure that your template mimics what is shown below by changing the outlined words. Prior to creating, ensure that the template is valid by clicking the Validate button. If the template validates, click the Create button. This will configure the service to respond with a 503 error on the 20 percent of the requests it receives.

Click the OVERVIEW tab of the right-hand pane and scroll down to where you can see the Error Rate value. You’ll see this value begin to increase to about 20 percent, as well as see the graphs begin to converge on the 20 percent error value.

Gateways are used to manage inbound and outbound traffic for your mesh, letting you specify which traffic you want to enter or leave the mesh. Gateway configurations are applied to standalone Envoy proxies that are running at the edge of the mesh, rather than sidecar Envoy proxies running alongside your service workloads.

Ingress gateways define an entry point into your Istio mesh for incoming traffic.

In order to demonstrate the function and use of ingress gateways, we’ll create an application and create a new ingress gateway, and then verify its function.

Let’s deploy a small HTTP echo application on our cluster in the default namespace. We will then use an ingress gateway to expose the provided service. The application YAML file is located under ~/calisti-tutorial-sample-code/calisti/smm/echo.yaml and shown here:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo
  labels:
    k8s-app: echo
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: echo
  template:
    metadata:
      labels:
        k8s-app: echo
    spec:
      terminationGracePeriodSeconds: 2
      containers:
      - name: echo-service
        image: k8s.gcr.io/echoserver:1.10
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: echo
  labels:
    k8s-app: echo
  namespace: default
spec:
  ports:
  - name: http
    port: 80
    targetPort: 8080
  selector:
    k8s-app: echo

We can apply this application to our cluster in the default namespace (as defined by the YAML manifest) using kubectl apply -f ~/calisti-tutorial-sample-code/calisti/smm/echo.yaml.

We’ll now create a new ingress gateway using the IstioMeshGateway resource. Calisti creates a new ingress gateway deployment and a corresponding service, and automatically labels them with the gateway-name and gateway-type labels and their corresponding values.

IstioMeshGateway is a custom Istio operator as defined by SMM that allows you to easily set up multiple gateways in a cluster. To create the new gateway, we’ll apply the YAML file defined in ~/calisti-tutorial-sample-code/calisti/smm/meshgw.yaml, which is given here:

apiVersion: servicemesh.cisco.com/v1alpha1
kind: IstioMeshGateway
metadata:
  name: demo-gw
spec:
  istioControlPlane:
    name: cp-v115x
    namespace: istio-system
  runAsRoot: false
  service:
    ports:
      - name: tcp-status-port
        port: 15021
        protocol: TCP
        targetPort: 15021
      - name: http
        port: 80
        protocol: TCP
        targetPort: 8080
    type: LoadBalancer
  type: ingress

We’ll now apply that manifest to our cluster using kubectl apply -f ~/calisti-tutorial-sample-code/calisti/smm/meshgw.yaml.

Get the IP address of the gateway. (It might take some seconds to become available.)

kubectl -n default get istiomeshgateways demo-gw

The output should look like:

ubuntu@ip-172-31-14-56:~$ kubectl -n default get istiomeshgateways demo-gw
NAME      TYPE      SERVICE TYPE   STATUS        INGRESS IPS        ERROR   AGE   CONTROL PLANE
demo-gw   ingress   LoadBalancer   Available   ["172.18.250.3"]           31s   {"name":"cp-v115x","namespace":"istio-system"}

We’ll now create the Gateway and VirtualService resources to configure listening ports on the matching gateway deployment. A VirtualService defines a set of traffic routing rules to apply when a host is addressed. Each routing rule defines matching criteria for traffic of a specific protocol. If the traffic is matched, then it is sent to a named destination service.

The hosts fields should point to the external hostname of the service. (For testing purposes, we are using nip.io, which is a domain name that provides a wildcard DNS for any IP address.)

The gateway VirtualService is defined by the file located at ~/calisti-tutorial-sample-code/calisti/smm/gw_vs.yaml.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: echo
  namespace: default
spec:
  selector:
    gateway-name: demo-gw
    gateway-type: ingress
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "echo.172.18.250.3.nip.io"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: echo
  namespace: default
spec:
  hosts:
  - "echo.172.18.250.3.nip.io"
  gateways:
  - echo
  http:
  - route:
    - destination:
        port:
          number: 80
        host: echo.default.svc.cluster.local

We can now apply this manifest using kubectl.

kubectl apply -f ~/calisti-tutorial-sample-code/calisti/smm/gw_vs.yaml

Now that the application, the ingress gateway, and the required services are added to the cluster, we can now access the service on the external address using cURL from the CLI of our EC2 instance.

curl -i echo.172.18.250.3.nip.io

The command is making a GET request to echo.172.18.250.3.nip.io and should be able to successfully reach the echo service.

The expected result should be similar to:

HTTP/1.1 200 OK
date: Fri, 02 Jun 2023 05:54:33 GMT
content-type: text/plain
server: istio-envoy
x-envoy-upstream-service-time: 0
transfer-encoding: chunked



Hostname: echo-856cf44f94-znvv6

Pod Information:
        -no pod information available-

Server values:
        server_version=nginx: 1.13.3 - lua: 10008

Request Information:
        client_address=192.168.99.5
        method=GET
        real path=/
        query=
        request_version=1.1
        request_scheme=http
        request_uri=http://echo.172.18.250.3.nip.io:8080/

Request Headers:
        accept=*/*
        host=echo.172.18.250.3.nip.io
        user-agent=curl/7.68.0
        x-b3-sampled=0
        x-b3-spanid=6ec6a9d84ba35244
        x-b3-traceid=8262617ebc6111826ec6a9d84ba35244
        x-envoy-attempt-count=1
        x-envoy-decorator-operation=echo.default.svc.cluster.local:80/*
        x-envoy-internal=true
        x-envoy-peer-metadata=ChQKDkFQUF9DT05UQUlORVJTEgIaAAoaCgpDTFVTVEVSX0lEEgwaCmtpbmQtZGVtbzEKHgoMSU5TVEFOQ0VfSVBTEg4aDDE5Mi4xNjguOTkuNQoZCg1JU1RJT19WRVJTSU9OEggaBjEuMTUuMwrbAgoGTEFCRUxTEtACKs0CChkKDGdhdGV3YXktbmFtZRIJGgdkZW1vLWd3ChkKDGdhdGV3YXktdHlwZRIJGgdpbmdyZXNzCicKDGlzdGlvLmlvL3JldhIXGhVjcC12MTE1eC5pc3Rpby1zeXN0ZW0KIQoRcG9kLXRlbXBsYXRlLWhhc2gSDBoKNWRiODc5YzVmNQoeCgdyZWxlYXNlEhMaEWlzdGlvLW1lc2hnYXRld2F5CiwKH3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLW5hbWUSCRoHZGVtby1ndwovCiNzZXJ2aWNlLmlzdGlvLmlvL2Nhbm9uaWNhbC1yZXZpc2lvbhIIGgZsYXRlc3QKIQoXc2lkZWNhci5pc3Rpby5pby9pbmplY3QSBhoEdHJ1ZQonChl0b3BvbG9neS5pc3Rpby5pby9uZXR3b3JrEgoaCG5ldHdvcmsxChIKB01FU0hfSUQSBxoFbWVzaDEKIgoETkFNRRIaGhhkZW1vLWd3LTVkYjg3OWM1ZjUtdGc1cm4KFgoJTkFNRVNQQUNFEgkaB2RlZmF1bHQKSwoFT1dORVISQhpAa3ViZXJuZXRlczovL2FwaXMvYXBwcy92MS9uYW1lc3BhY2VzL2RlZmF1bHQvZGVwbG95bWVudHMvZGVtby1ndwoXChFQTEFURk9STV9NRVRBREFUQRICKgAKGgoNV09SS0xPQURfTkFNRRIJGgdkZW1vLWd3
        x-envoy-peer-metadata-id=router~192.168.99.5~demo-gw-5db879c5f5-tg5rn.default~default.svc.cluster.local
        x-forwarded-for=172.18.0.2
        x-forwarded-proto=http
        x-request-id=e33fd2a7-4805-4854-9548-c26484b9c273

Request Body:
        -no body in request-

Istio service mesh primarily provides its features to Kubernetes-based resources. However, in some cases, it makes sense to integrate external compute nodes like bare-metal machines or VMs into the mesh:

• Temporary integration: Used as a stopgap during a migration of the non-k8s native workloads into the mesh, providing temporary access to the machine’s network resources until the migration is done

• Long-term integration: When it is impractical to migrate the given workload into Kubernetes because of its size (for example, huge bare-metal machines are required) or when the workload is stateful and it is hard to support within Kubernetes

SMM provides support for both use cases, building on top of Istio’s support for VMs. We will demonstrate this by creating a new “compute” node (another container, but external to k8s) running the analytics microservice and integrate it into the service mesh running within the cluster.

VM Access to Your Cluster

Istio can work in two distinct ways when it comes to network connectivity:

• If the VM has no direct connection to the pod’s IP addresses, it can rely on a mesh expansion gateway and use a different network for its workload group.

• If the VM can access the pod’s IP addresses directly, then it can use the same network as Istio.

The following example will cover the “same network” approach because it will be performed using a container with access to the backend Docker network.

Check Service Mesh Network and Deploy WorkloadGroup

Check which network the existing Istio control planes are attached to:

kubectl get istiocontrolplanes -A

The output should be similar to the following:

ubuntu@ip-172-31-14-56:~/calisti$ kubectl get istiocontrolplanes -A
NAMESPACE      NAME            MODE     NETWORK    STATUS      MESH EXPANSION   EXPANSION GW IPS   ERROR   AGE
istio-system   cp-v115x        ACTIVE   network1   Available   true             ["172.18.250.1"]           5d21h
istio-system   sdm-icp-v115x   ACTIVE   network1   Available                                               5d21h

From the output above, we can determine that Istio is using the network1 network. When we create a WorkloadGroup, we’ll need to reference this network within the manifest to ensure that the same network is used between the VM and the cluster.

The WorkloadGroup definition is provided by ~/calisti-tutorial-sample-code/calisti/smm/workload-analytics.yaml.

  apiVersion: networking.istio.io/v1alpha3
  kind: WorkloadGroup
  metadata:
    labels:
      app: analytics
      version: v0
    name: analytics-v0
    namespace: smm-demo
  spec:
    metadata:
      labels:
        app: analytics
        version: v0
    probe:
      httpGet:
        path: /
        host: 127.0.0.1
        port: 8080
        scheme: HTTP
    template:
      network: network1
      ports:
        http: 8080
        grpc: 8082
        tcp: 8083
      serviceAccount: default

We can apply that manifest using kubectl.

kubectl apply -f ~/calisti-tutorial-sample-code/calisti/smm/workload-analytics.yaml

Mutual TLS settings

After Istio is started on the VM, Istio takes over the service ports defined in the WorkloadGroup resource. Depending on the settings, it will also start enforcing mTLS on those service ports.

If external (non-mesh) services communicate with the VM, we need to ensure that communication without encryption is permitted on the service ports. To do so, create a PeerAuthentication object in the smm-demo namespace. Make sure that the matchLabels selector only matches the WorkloadGroup, and not any other Kubernetes deployment, to avoid permitting unencrypted communication where it’s not needed. In the following example, the matchLabels selector includes both the app and the version labels.

In order to allow the VM to accept connections outside of the service mesh, we configure a peer authentication with permissive mode. This is done in the ~/calisti-tutorial-sample-code/calisti/smm/permissive-mtls.yaml file, which is also given here:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: analytics
  namespace: smm-demo
spec:
  mtls:
    mode: PERMISSIVE
  selector:
    matchLabels:
      app: analytics
      version: v0

We can now apply this using kubectl.

kubectl apply -f ~/calisti-tutorial-sample-code/calisti/smm/permissive-mtls.yaml

Note that for the current example, setting mTLS to permissive is optional because the requests come from within the cluster.

Deploy the VM as Container

We’ll now begin the build of our workload VM (container), which will include building a standard Ubuntu container, adding the analytics service, installing SMM within the VM, and then reconciling and adding this new service to Istio. Once this is done, we’ll illustrate how SMM can be used to direct traffic between workloads as needed.

Begin by running a systemd Ubuntu container as simulated VM. This build command is included in the sample code directory under ~/calisti-tutorial-sample-code/calisti/smm/vm-build.sh.

docker run -d --rm --privileged --cgroupns=host --name systemd-ubuntu --network=kind --tmpfs /tmp --tmpfs /run --tmpfs /run/lock -v /sys/fs/cgroup:/sys/fs/cgroup jrei/systemd-ubuntu
docker exec -t systemd-ubuntu bash -c "apt-get -y update && apt-get install -y curl iptables iproute2 sudo"
VM_NAME=systemd-ubuntu

If you execute the script, you’ll see the Docker and apt processes run.

bash ~/calisti-tutorial-sample-code/calisti/smm/vm-build.sh

You’ll also need to export an environment variable, because these are not persistent outside of the process that executes the Bash script.

VM_NAME=systemd-ubuntu
echo $VM_NAME

Before we can register the VM, the workload must already be running on the VM. The following instructions start an example HTTP server workload on the VM to simulate the analytics service.

Open a Bash session on the simulated VM, which can be done by using the following command:

docker exec -it ${VM_NAME} bash

This should move you to the root shell of the running container. From here, it will be indicated which steps will need to be done from within the container or which are to be done from the shell of the EC2 instance.

Next, we’ll start a fake analytics service inside the VM. Copy and paste the following commands into the container shell.

Note: This task is to be done within the container shell.

# inside container
mkdir /empty_dir
cat <<EOF > /lib/systemd/system/smm-demo-analytics.service
[Unit]
Description=SMM demo app - analytics
After=network-online.target

[Service]
ExecStart=/usr/bin/bash -c "cd /empty_dir && /usr/bin/python3 -m http.server 8080"
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF
ln -s /lib/systemd/system/smm-demo-analytics.service /etc/systemd/system/multi-user.target.wants/
systemctl start smm-demo-analytics.service

Next, ensure that the pkg download cache is in the same filesystem as the target.

Note: This task is to be done within the container shell.

# still inside container
mkdir /tmp/smm-agent
rm -rf /var/cache/smm-agent
ln -s /tmp/smm-agent /var/cache/smm-agent
exit

The exit command at the end of the command string should return you to the shell of your EC2 instance.

Install the SMM Agent on the VM

SMM takes an automation-friendly approach to managing the VMs by providing an agent that runs on the machine. This component enables SMM to provide the same observability features for VMs as for native Kubernetes workloads, such as topology view, service/workload overview, integrated tracing, or traffic tapping.

The agent continuously maintains the configuration of the machine so that any change in the upstream cluster is reflected in its configuration. This behavior ensures that if the mesh expansion gateway IP addresses change, the machine retains the connectivity to the mesh.

To attach the VM to the mesh, you’ll need the following information:

• The URL of the dashboard

• The namespace and name of the WorkloadGroup

• The bearer token of the service account referenced in the .spec.template.serviceAccount of the WorkloadGroup

• (Optional) The IP address that the clusters in the service mesh can use to access the VM. If this is the same as the IP that the public internet sees for the VM, then SMM detects the VM’s IP automatically.

Note: For the following steps, please copy and paste the command strings directly to the shell of your EC2 instance. Each of the echo commands is there to ensure that you receive valid output for that new environment variable.

Let’s start by gathering the IP address of the VM on the Docker network …

NODEIP=$(docker exec -t ${VM_NAME} bash -c "ip a show dev eth0" | awk '$1 == "inet" {gsub(/\/.*$/, "", $2); print $2}')
echo $NODEIP

… and gathering the ingress gateway IP for Calisti:

INGRESS_IP=$(kubectl get svc smm-ingressgateway-external -n smm-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo $INGRESS_IP

Get the bearer token for the smm-demo namespace’s service account:

SA_NAMESPACE=smm-demo
SA_SERVICEACCOUNT=default
SA_SECRET_NAME=$(kubectl describe serviceaccount default -n smm-demo | grep Tokens | awk '{print $2}')
BEARER_TOKEN=$(kubectl get secret -n $SA_NAMESPACE ${SA_SECRET_NAME} -o json | jq -r '.data.token | @base64d')
echo $SA_SECRET_NAME
echo $BEARER_TOKEN

Once we have these values, we can begin the task of installing the smm-agent and reconciling it with the service mesh within our cluster.

Let’s start by installing the smm-agent in the VM.

docker exec -t ${VM_NAME} bash -c "curl http://${INGRESS_IP}/get/smm-agent | bash"

You should receive output similar to the following:

ubuntu@ip-172-31-14-56:~/calisti$ docker exec -t ${VM_NAME} bash -c "curl http://${INGRESS_IP}/get/smm-agent | bash"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1821  100  1821    0     0  42348      0 --:--:-- --:--:-- --:--:-- 43357
Detecting host properties:
- OS: linux
- CPU: amd64
- Packager: deb
- SMM Base URL: http://172.18.250.2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 20.8M    0 20.8M    0     0   464M      0 --:--:-- --:--:-- --:--:--  453M
Selecting previously unselected package smm-agent.
(Reading database ... 9297 files and directories currently installed.)
Preparing to unpack /tmp/tmp.AGfPmEKWl0 ...
Unpacking smm-agent (1.12.0~SNAPSHOT-6f777daeb) ...
Setting up smm-agent (1.12.0~SNAPSHOT-6f777daeb) ...
Created symlink /etc/systemd/system/multi-user.target.wants/smm-agent.service -> /lib/systemd/system/smm-agent.service.
✓ dashboard url set url=http://172.18.250.2

We can verify that the VM has set the dashboard location to the URL of the ingress gateway for the cluster.

Next, we’ll set up and configure the smm-agent on the VM.

docker exec -t ${VM_NAME} bash -c "smm-agent set workload-group smm-demo analytics-v0"
docker exec -t ${VM_NAME} bash -c "smm-agent set node-ip ${NODEIP}"
docker exec -t ${VM_NAME} bash -c "smm-agent set bearer-token ${BEARER_TOKEN}"

The output should be similar to the following:

ubuntu@ip-172-31-14-56:~/calisti$ docker exec -t ${VM_NAME} bash -c "smm-agent set workload-group smm-demo analytics-v0"
✓ target workload group set namespace=smm-demo, name=analytics-v0
ubuntu@ip-172-31-14-56:~/calisti$ docker exec -t ${VM_NAME} bash -c "smm-agent set node-ip ${NODEIP}"
✓ node-ip is set ip=172.18.0.7
ubuntu@ip-172-31-14-56:~/calisti$ docker exec -t ${VM_NAME} bash -c "smm-agent set bearer-token ${BEARER_TOKEN}"
✓ bearer token set

Now that the workload is started on the VM (HTTP server) and smm-agent is configured, we can attach the VM to the mesh. To do so, we need to run a reconciliation on this host. This step will:

• Configure and start Istio so the VM becomes part of the mesh.
• Ensure that the cluster configuration is properly set.
• Start smm-agent in the background so that the system is always up to date.

To reconcile the agent, run the following command:

docker exec -t ${VM_NAME} bash -c "smm-agent reconcile"

There will be many lines of output as smm-agent does the above tasks, but the output should look similar to the following:

ubuntu@ip-172-31-14-56:~/calisti$ docker exec -t ${VM_NAME} bash -c "smm-agent reconcile"
✓ reconciling host operating system
✓ uuid not present in config file, new uuid is generated uuid=0327b100-a008-4364-a2b3-7d1777bb5b9a
✓ configuration loaded config=/etc/smm/agent.yaml
...
✓ wait-for-registration ❯ workloadentry is registered successfully
✓ changes were made to the host operating system
✓ reconciled host operating system

Once the reconciliation is complete, return to the Calisti dashboard, navigate to the topology view, and verify that the VM is visible. You should see another node in the analytics service, the v0 that we created as part of this process. It should also have a blue icon on the workload. If you click the analytics service, it will center in your screen, and you can verify the traffic loads between the v0 and v1 service (in terms of RPS). By default, traffic is load-balanced between the different application versions. Please wait until the number of requests settles at a stable distribution between v0 (VM) and v1 (pod).

VM to Kubernetes Traffic Migrations

One of the common use cases of a service mesh is migrating an existing workload to the mesh (and Kubernetes). In order to accomplish this, the following main steps need to be completed:

• Add the VM to the mesh so that the original workload that is running in the VM is available in the mesh. (We already accomplished this in the previous set of steps.)
• Configure traffic shifting that will allow you to route traffic from the VM to the Kubernetes workload.
• Add the Kubernetes workload that will replace the VM.
• Shift traffic to Kubernetes gradually and test that the Kubernetes workload works properly, even under high load.
• Remove the VM when you have successfully completed the migration.

In order to perform the traffic shifting, we’ll start by creating a DestinationRule for the analytics service. This is defined by the file ~/calisti-tutorials-sample-code/calisti/lab/analyticsDR.yaml, and it creates the destinations to which traffic can be distributed.

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: analytics-destination
  namespace: smm-demo
  labels:
    istio.io/rev: cp-v115x.istio-system
spec:
  host: analytics
  subsets:
    - name: v0
      labels:
        version: v0
    - name: v1
      labels:
        version: v1

Now we can apply this using kubectl apply -f ~/calisti-tutorial-sample-code/calisti/smm/analyticsDR.yaml.

Using Calisti, we can easily weight each destination to receive more or less traffic, depending on the use case. To perform this, we need to create a VirtualService that matches the destinations created earlier, and apply weights for traffic to those destinations. In our configuration, defined in ~/calisti-tutorial-sample-code/calisti/smm/analyticsRD.yaml, we will apply a 10 percent weight to v0 (VM) and 90 percent to v1 (k8s).

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: analytics-route
  namespace: smm-demo
  labels:
    istio.io/rev: cp-v115x.istio-system
spec:
  hosts:
    - analytics
  http:
    - route:
        - destination:
            host: analytics
            subset: v0
          weight: 10
        - destination:
            host: analytics
            subset: v1
          weight: 90

We can apply this configuration using kubectl apply -f ~/calisti-tutorial-sample-code/calisti/smm/analyticsRD.yaml.

After applying the configuration and viewing the RPS within the topology view, you should see the traffic starting to adapt according to what was configured.

The final step is to ensure that we destroy the EC2 instance so that we are not charged for unintentional usage. This is done by exiting the SSH session to the instance and performing a destroy action with Terraform.

exit
terraform destroy -auto-approve

Congratulations! You have successfully completed this tutorial on exploring Calisti SMM and the features it provides for your Kubernetes environment.

Learn More

• Cisco Learning Network DevNet Certifications Community
• DevNet Certification Training Videos
• DevNet Automation Bootcamp