Managing a Pulsar cluster with Terraform

In this guide we will be using quantummetric Terraform provider with a pre-made Terraform manifest to create a tenant, namespace, and topic within an existing Pulsar cluster.

Prerequisites

Terraform Cli 0.10+

Steps

Start a standalone broker

1

./bin/pulsar-daemon start standalone

1

docker run -d -p 6650:6650 -p 8080:8080 --name pulsar apachepulsar/pulsar:latest bin/pulsar standalone

1
2
3
4
5

kubectl apply -f https:https://www.pulsar-neighborhood.iomanifests/standalone-k8s.yaml

# Start a port forward
kubectl -n pulsar port-forward service/pulsar-broker 6650:6650
kubectl -n pulsar port-forward service/pulsar-broker 8080:8080

Get the deployment manifest

This manifest uses the pulsar_cluster, pulsar_tenant, pulsar_namespace, and pulsar_topic providers to chain together a topic creation. You can go much further than with these providers! Read more about the providers here.

1

wget https:https://www.pulsar-neighborhood.iomanifests/terraform.tf

Deploy the manifest

1
2

terraform init
terraform apply

Send a message to the Broker

Otherwise known as producing a message. Pulsar will create a new topic (if it doesn’t already exist) named “tf-topic”.

1

./bin/pulsar-client --url "pulsar://localhost:6650/tf-tenant/tf-namespace/" produce "tf-topic" --messages "Hello there"

There will be quite a bit of feedback output to the screen. Upon a successful publish, the last line should be…

[main] INFO org.apache.pulsar.client.cli.PulsarClientTool - 1 messages successfully produced

Get messages from the Broker

Otherwise known as consuming messages. Similar to producing, if the topic does not exists Pulsar will create it.

1

./bin/pulsar-client --url "pulsar://localhost:6650/tf-tenant/tf-namespace/" consume "tf-topic" -s "my-subscription"

There will be quite a bit of feedback output to the screen. Upon a successful connection, the last line should be…

[main] INFO org.apache.pulsar.client.cli.PulsarClientTool - 1 messages successfully consumed

Summary

Pretty easy eh? We created a (quick) pulsar instance, applied a Terraform config, and proved the new topic by producing/consuming messages. The config was a bare minimum example that created a new tenant, namespace, and topic. Now it’s time to get a little deeper in to what these providers are really do. Read more about all their features here.