/
Stage 7 - Upgrade Kafka (3.2.1 -> 3.7.0)

Stage 7 - Upgrade Kafka (3.2.1 -> 3.7.0)

Objective

Upgrade Kafka from the current version 3.2.1 to version 3.7.0. For detailed information from the Apache Kafka project, refer to the Upgrading section of the Kafka documentation.

Prerequisites

  1. syndeia-cloud-3.6-SP2*_cassandra_zookeeper_kafka_setup.zip package is available.

  2. Ensure previous stages have been completed without error.

Steps

  1. Run the Apache Kafka pre-setup script. This will download Kafka 3.7.0 and upgrade it, or will upgrade the already downloaded Kafka 3.7.0 :

    ./syndeia-cloud-3.6_kafka_pre-setup.bash
[jdoe@new-server bin]$ ./syndeia-cloud-3.6_kafka_pre-setup.bash + shopt -s extglob + shopt -s globstar ++ readlink -f ./syndeia-cloud-3.6_kafka_pre-setup.bash + SC_version_parsed=/home/jdoe/syndeia-cloud-3.6-SP2.2024-09-11_cassandra_zookeeper_kafka_setup/bin/syndeia-cloud-3.6_kafka_pre-setup.bash + SC_version_parsed=/home/jdoe/syndeia-cloud-3.6-SP2.2024-09-11 + SC_version_parsed=3.6-SP2.2024-09-11 + SC_version=3.6-SP2.2024-09-11 + service_name=kafka + Scala_ver=2.13 + Kafka_build_ver=3.7.0 ++ uname + OS=Linux + [[ Linux =~ CYGWIN ]] + systemctl=systemctl + sudo=sudo + ln='ln -nfs' + Kafka_upgrade=false + [[ -e /opt/kafka-current ]] ++ readlink -f /opt/kafka-current + Kafka_build_ver_current=/opt/kafka_2.13-3.2.1 + Kafka_build_ver_current=2.13-3.2.1 + Kafka_build_ver_current=3.2.1 + [[ ! 3.2.1 == \3\.\7\.\0 ]] + Kafka_upgrade=true + umask 0002 + pushd . ~/syndeia-cloud-3.6-SP2.2024-09-11_cassandra_zookeeper_kafka_setup/bin ~/syndeia-cloud-3.6-SP2.2024-09-11_cassandra_zookeeper_kafka_setup/bin + cd /home/jdoe/ + [[ '' == \-\d ]] + [[ Linux =~ CYGWIN ]] + [[ true == \t\r\u\e ]] + [[ -e /etc/systemd/system/kafka.service ]] ++ sudo systemctl is-active --quiet kafka + [[ ! -n '' ]] + sudo systemctl start kafka + printf test ./syndeia-cloud-3.6_kafka_pre-setup.bash: connect: Connection refused ./syndeia-cloud-3.6_kafka_pre-setup.bash: line 132: /dev/tcp/localhost/9092: Connection refused + sleep 2 + printf test ./syndeia-cloud-3.6_kafka_pre-setup.bash: connect: Connection refused ./syndeia-cloud-3.6_kafka_pre-setup.bash: line 132: /dev/tcp/localhost/9092: Connection refused + sleep 2 + printf test + pushd . ~ ~ ~/syndeia-cloud-3.6-SP2.2024-09-11_cassandra_zookeeper_kafka_setup/bin + cd /opt/kafka-current/bin + sudo -u kafka ./kafka-get-offsets.sh --broker-list localhost:9092 --topic store-StoreEvent store-StoreEvent:0:409 + sudo -u kafka ./kafka-get-offsets.sh --broker-list localhost:9092 --topic auth-AuthEvent auth-AuthEvent:0:23 + read -n 1 -s -r -p '[Note the metrics above and press any key to continue or hit ^C to abort]' [Note the metrics above and press any key to continue or hit ^C to abort]+ printf '\n' + echo '' + popd ~ ~/syndeia-cloud-3.6-SP2.2024-09-11_cassandra_zookeeper_kafka_setup/bin + sudo systemctl stop kafka + [[ -e /opt/kafka- ]] + [[ ! -e kafka_2.13-3.7.0.tgz ]] + wget https://archive.apache.org/dist/kafka/3.7.0/kafka_2.13-3.7.0.tgz --2024-09-16 11:50:41-- https://archive.apache.org/dist/kafka/3.7.0/kafka_2.13-3.7.0.tgz Resolving archive.apache.org (archive.apache.org)... 2a01:4f9:1a:a084::2, 65.108.204.189 Connecting to archive.apache.org (archive.apache.org)|2a01:4f9:1a:a084::2|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 119028138 (114M) [application/x-gzip] Saving to: ‘kafka_2.13-3.7.0.tgz’ kafka_2.13-3.7.0.tgz 100%[=========================================================================>] 113.51M 11.7MB/s in 14s 2024-09-16 11:50:56 (8.18 MB/s) - ‘kafka_2.13-3.7.0.tgz’ saved [119028138/119028138] ++ which tar ++ alias ++ eval declare -f +++ declare -f ++ /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot tar + [[ ! -n /usr/bin/tar ]] + sudo tar -xvzf kafka_2.13-3.7.0.tgz -C /opt/ kafka_2.13-3.7.0/ kafka_2.13-3.7.0/LICENSE kafka_2.13-3.7.0/NOTICE kafka_2.13-3.7.0/bin/ kafka_2.13-3.7.0/bin/kafka-delete-records.sh kafka_2.13-3.7.0/bin/trogdor.sh kafka_2.13-3.7.0/bin/kafka-jmx.sh kafka_2.13-3.7.0/bin/connect-mirror-maker.sh kafka_2.13-3.7.0/bin/kafka-console-consumer.sh kafka_2.13-3.7.0/bin/kafka-consumer-perf-test.sh kafka_2.13-3.7.0/bin/kafka-log-dirs.sh kafka_2.13-3.7.0/bin/kafka-metadata-quorum.sh kafka_2.13-3.7.0/bin/zookeeper-server-stop.sh kafka_2.13-3.7.0/bin/kafka-verifiable-consumer.sh kafka_2.13-3.7.0/bin/kafka-features.sh kafka_2.13-3.7.0/bin/kafka-acls.sh kafka_2.13-3.7.0/bin/zookeeper-server-start.sh kafka_2.13-3.7.0/bin/kafka-server-stop.sh kafka_2.13-3.7.0/bin/kafka-configs.sh kafka_2.13-3.7.0/bin/kafka-reassign-partitions.sh kafka_2.13-3.7.0/bin/connect-plugin-path.sh kafka_2.13-3.7.0/bin/kafka-leader-election.sh kafka_2.13-3.7.0/bin/kafka-producer-perf-test.sh kafka_2.13-3.7.0/bin/kafka-transactions.sh kafka_2.13-3.7.0/bin/kafka-topics.sh kafka_2.13-3.7.0/bin/connect-standalone.sh kafka_2.13-3.7.0/bin/kafka-e2e-latency.sh kafka_2.13-3.7.0/bin/kafka-metadata-shell.sh kafka_2.13-3.7.0/bin/kafka-get-offsets.sh kafka_2.13-3.7.0/bin/kafka-dump-log.sh kafka_2.13-3.7.0/bin/kafka-broker-api-versions.sh kafka_2.13-3.7.0/bin/kafka-consumer-groups.sh kafka_2.13-3.7.0/bin/connect-distributed.sh kafka_2.13-3.7.0/bin/kafka-delegation-tokens.sh kafka_2.13-3.7.0/bin/kafka-run-class.sh kafka_2.13-3.7.0/bin/kafka-replica-verification.sh kafka_2.13-3.7.0/bin/kafka-console-producer.sh kafka_2.13-3.7.0/bin/zookeeper-shell.sh kafka_2.13-3.7.0/bin/kafka-client-metrics.sh kafka_2.13-3.7.0/bin/kafka-cluster.sh kafka_2.13-3.7.0/bin/windows/ kafka_2.13-3.7.0/bin/windows/kafka-e2e-latency.bat kafka_2.13-3.7.0/bin/windows/kafka-log-dirs.bat kafka_2.13-3.7.0/bin/windows/zookeeper-server-stop.bat kafka_2.13-3.7.0/bin/windows/connect-distributed.bat kafka_2.13-3.7.0/bin/windows/kafka-configs.bat kafka_2.13-3.7.0/bin/windows/kafka-console-producer.bat kafka_2.13-3.7.0/bin/windows/kafka-delete-records.bat kafka_2.13-3.7.0/bin/windows/kafka-topics.bat kafka_2.13-3.7.0/bin/windows/kafka-dump-log.bat kafka_2.13-3.7.0/bin/windows/kafka-console-consumer.bat kafka_2.13-3.7.0/bin/windows/kafka-server-start.bat kafka_2.13-3.7.0/bin/windows/kafka-consumer-groups.bat kafka_2.13-3.7.0/bin/windows/kafka-mirror-maker.bat kafka_2.13-3.7.0/bin/windows/kafka-features.bat kafka_2.13-3.7.0/bin/windows/kafka-reassign-partitions.bat kafka_2.13-3.7.0/bin/windows/kafka-client-metrics.bat kafka_2.13-3.7.0/bin/windows/kafka-cluster.bat kafka_2.13-3.7.0/bin/windows/kafka-producer-perf-test.bat kafka_2.13-3.7.0/bin/windows/kafka-jmx.bat kafka_2.13-3.7.0/bin/windows/zookeeper-server-start.bat kafka_2.13-3.7.0/bin/windows/kafka-server-stop.bat kafka_2.13-3.7.0/bin/windows/kafka-replica-verification.bat kafka_2.13-3.7.0/bin/windows/kafka-run-class.bat kafka_2.13-3.7.0/bin/windows/kafka-acls.bat kafka_2.13-3.7.0/bin/windows/kafka-delegation-tokens.bat kafka_2.13-3.7.0/bin/windows/kafka-broker-api-versions.bat kafka_2.13-3.7.0/bin/windows/kafka-metadata-quorum.bat kafka_2.13-3.7.0/bin/windows/kafka-transactions.bat kafka_2.13-3.7.0/bin/windows/kafka-storage.bat kafka_2.13-3.7.0/bin/windows/kafka-leader-election.bat kafka_2.13-3.7.0/bin/windows/kafka-streams-application-reset.bat kafka_2.13-3.7.0/bin/windows/kafka-get-offsets.bat kafka_2.13-3.7.0/bin/windows/connect-plugin-path.bat kafka_2.13-3.7.0/bin/windows/zookeeper-shell.bat kafka_2.13-3.7.0/bin/windows/connect-standalone.bat kafka_2.13-3.7.0/bin/windows/kafka-consumer-perf-test.bat kafka_2.13-3.7.0/bin/kafka-verifiable-producer.sh kafka_2.13-3.7.0/bin/kafka-server-start.sh kafka_2.13-3.7.0/bin/kafka-mirror-maker.sh kafka_2.13-3.7.0/bin/kafka-storage.sh kafka_2.13-3.7.0/bin/kafka-streams-application-reset.sh kafka_2.13-3.7.0/bin/zookeeper-security-migration.sh kafka_2.13-3.7.0/config/ kafka_2.13-3.7.0/config/consumer.properties kafka_2.13-3.7.0/config/connect-mirror-maker.properties kafka_2.13-3.7.0/config/zookeeper.properties kafka_2.13-3.7.0/config/server.properties kafka_2.13-3.7.0/config/producer.properties kafka_2.13-3.7.0/config/trogdor.conf kafka_2.13-3.7.0/config/connect-console-sink.properties kafka_2.13-3.7.0/config/connect-log4j.properties kafka_2.13-3.7.0/config/connect-standalone.properties kafka_2.13-3.7.0/config/connect-file-source.properties kafka_2.13-3.7.0/config/connect-console-source.properties kafka_2.13-3.7.0/config/connect-distributed.properties kafka_2.13-3.7.0/config/tools-log4j.properties kafka_2.13-3.7.0/config/connect-file-sink.properties kafka_2.13-3.7.0/config/kraft/ kafka_2.13-3.7.0/config/kraft/server.properties kafka_2.13-3.7.0/config/kraft/broker.properties kafka_2.13-3.7.0/config/kraft/controller.properties kafka_2.13-3.7.0/config/log4j.properties kafka_2.13-3.7.0/licenses/ kafka_2.13-3.7.0/licenses/pcollections-MIT kafka_2.13-3.7.0/licenses/jopt-simple-MIT kafka_2.13-3.7.0/licenses/protobuf-java-BSD-3-clause kafka_2.13-3.7.0/licenses/DWTFYWTPL kafka_2.13-3.7.0/licenses/eclipse-distribution-license-1.0 kafka_2.13-3.7.0/licenses/slf4j-MIT kafka_2.13-3.7.0/licenses/jline-BSD-3-clause kafka_2.13-3.7.0/licenses/jsr305-BSD-3-clause kafka_2.13-3.7.0/licenses/eclipse-public-license-2.0 kafka_2.13-3.7.0/licenses/paranamer-BSD-3-clause kafka_2.13-3.7.0/licenses/argparse-MIT kafka_2.13-3.7.0/licenses/zstd-jni-BSD-2-clause kafka_2.13-3.7.0/licenses/checker-qual-MIT kafka_2.13-3.7.0/licenses/CDDL+GPL-1.1 kafka_2.13-3.7.0/libs/ kafka_2.13-3.7.0/libs/kafka-group-coordinator-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-metadata-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-storage-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-storage-api-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-raft-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-server-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-server-common-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-tools-api-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-clients-3.7.0.jar kafka_2.13-3.7.0/libs/jackson-dataformat-csv-2.16.0.jar kafka_2.13-3.7.0/libs/jackson-datatype-jdk8-2.16.0.jar kafka_2.13-3.7.0/libs/jackson-databind-2.16.0.jar kafka_2.13-3.7.0/libs/jackson-annotations-2.16.0.jar kafka_2.13-3.7.0/libs/jackson-core-2.16.0.jar kafka_2.13-3.7.0/libs/jackson-module-scala_2.13-2.16.0.jar kafka_2.13-3.7.0/libs/scala-collection-compat_2.13-2.10.0.jar kafka_2.13-3.7.0/libs/scala-java8-compat_2.13-1.0.2.jar kafka_2.13-3.7.0/libs/scala-logging_2.13-3.9.4.jar kafka_2.13-3.7.0/libs/scala-reflect-2.13.12.jar kafka_2.13-3.7.0/libs/scala-library-2.13.12.jar kafka_2.13-3.7.0/libs/argparse4j-0.7.0.jar kafka_2.13-3.7.0/libs/commons-validator-1.7.jar kafka_2.13-3.7.0/libs/jopt-simple-5.0.4.jar kafka_2.13-3.7.0/libs/jose4j-0.9.4.jar kafka_2.13-3.7.0/libs/metrics-core-2.2.0.jar kafka_2.13-3.7.0/libs/metrics-core-4.1.12.1.jar kafka_2.13-3.7.0/libs/zookeeper-3.8.3.jar kafka_2.13-3.7.0/libs/slf4j-api-1.7.36.jar kafka_2.13-3.7.0/libs/commons-cli-1.4.jar kafka_2.13-3.7.0/libs/commons-beanutils-1.9.4.jar kafka_2.13-3.7.0/libs/commons-logging-1.2.jar kafka_2.13-3.7.0/libs/commons-collections-3.2.2.jar kafka_2.13-3.7.0/libs/commons-digester-2.1.jar kafka_2.13-3.7.0/libs/paranamer-2.8.jar kafka_2.13-3.7.0/libs/zookeeper-jute-3.8.3.jar kafka_2.13-3.7.0/libs/audience-annotations-0.12.0.jar kafka_2.13-3.7.0/libs/netty-handler-4.1.100.Final.jar kafka_2.13-3.7.0/libs/netty-transport-native-epoll-4.1.100.Final.jar kafka_2.13-3.7.0/libs/netty-transport-classes-epoll-4.1.100.Final.jar kafka_2.13-3.7.0/libs/netty-transport-native-unix-common-4.1.100.Final.jar kafka_2.13-3.7.0/libs/netty-codec-4.1.100.Final.jar kafka_2.13-3.7.0/libs/netty-transport-4.1.100.Final.jar kafka_2.13-3.7.0/libs/netty-resolver-4.1.100.Final.jar kafka_2.13-3.7.0/libs/netty-buffer-4.1.100.Final.jar kafka_2.13-3.7.0/libs/netty-common-4.1.100.Final.jar kafka_2.13-3.7.0/libs/commons-io-2.11.0.jar kafka_2.13-3.7.0/libs/zstd-jni-1.5.5-6.jar kafka_2.13-3.7.0/libs/lz4-java-1.8.0.jar kafka_2.13-3.7.0/libs/snappy-java-1.1.10.5.jar kafka_2.13-3.7.0/libs/opentelemetry-proto-1.0.0-alpha.jar kafka_2.13-3.7.0/libs/pcollections-4.0.1.jar kafka_2.13-3.7.0/libs/caffeine-2.9.3.jar kafka_2.13-3.7.0/libs/protobuf-java-3.23.4.jar kafka_2.13-3.7.0/libs/checker-qual-3.19.0.jar kafka_2.13-3.7.0/libs/error_prone_annotations-2.10.0.jar kafka_2.13-3.7.0/libs/kafka_2.13-3.7.0.jar kafka_2.13-3.7.0/site-docs/ kafka_2.13-3.7.0/site-docs/kafka_2.13-3.7.0-site-docs.tgz kafka_2.13-3.7.0/libs/kafka-tools-3.7.0.jar kafka_2.13-3.7.0/libs/connect-runtime-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-log4j-appender-3.7.0.jar kafka_2.13-3.7.0/libs/connect-json-3.7.0.jar kafka_2.13-3.7.0/libs/connect-transforms-3.7.0.jar kafka_2.13-3.7.0/libs/connect-api-3.7.0.jar kafka_2.13-3.7.0/libs/reflections-0.10.2.jar kafka_2.13-3.7.0/libs/slf4j-reload4j-1.7.36.jar kafka_2.13-3.7.0/libs/reload4j-1.2.25.jar kafka_2.13-3.7.0/libs/jackson-jaxrs-base-2.16.0.jar kafka_2.13-3.7.0/libs/jackson-module-jaxb-annotations-2.16.0.jar kafka_2.13-3.7.0/libs/jackson-jaxrs-json-provider-2.16.0.jar kafka_2.13-3.7.0/libs/jakarta.xml.bind-api-2.3.3.jar kafka_2.13-3.7.0/libs/jakarta.activation-api-1.2.2.jar kafka_2.13-3.7.0/libs/jersey-container-servlet-2.39.1.jar kafka_2.13-3.7.0/libs/jersey-hk2-2.39.1.jar kafka_2.13-3.7.0/libs/jaxb-api-2.3.1.jar kafka_2.13-3.7.0/libs/activation-1.1.1.jar kafka_2.13-3.7.0/libs/jetty-servlet-9.4.53.v20231009.jar kafka_2.13-3.7.0/libs/jetty-security-9.4.53.v20231009.jar kafka_2.13-3.7.0/libs/jetty-server-9.4.53.v20231009.jar kafka_2.13-3.7.0/libs/jetty-servlets-9.4.53.v20231009.jar kafka_2.13-3.7.0/libs/jetty-client-9.4.53.v20231009.jar kafka_2.13-3.7.0/libs/maven-artifact-3.8.8.jar kafka_2.13-3.7.0/libs/swagger-annotations-2.2.8.jar kafka_2.13-3.7.0/libs/javax.ws.rs-api-2.1.1.jar kafka_2.13-3.7.0/libs/jersey-container-servlet-core-2.39.1.jar kafka_2.13-3.7.0/libs/jersey-server-2.39.1.jar kafka_2.13-3.7.0/libs/jersey-client-2.39.1.jar kafka_2.13-3.7.0/libs/jersey-common-2.39.1.jar kafka_2.13-3.7.0/libs/jakarta.ws.rs-api-2.1.6.jar kafka_2.13-3.7.0/libs/hk2-locator-2.6.1.jar kafka_2.13-3.7.0/libs/javassist-3.29.2-GA.jar kafka_2.13-3.7.0/libs/javax.activation-api-1.2.0.jar kafka_2.13-3.7.0/libs/javax.servlet-api-3.1.0.jar kafka_2.13-3.7.0/libs/jetty-http-9.4.53.v20231009.jar kafka_2.13-3.7.0/libs/jetty-io-9.4.53.v20231009.jar kafka_2.13-3.7.0/libs/jetty-util-ajax-9.4.53.v20231009.jar kafka_2.13-3.7.0/libs/jetty-continuation-9.4.53.v20231009.jar kafka_2.13-3.7.0/libs/jetty-util-9.4.53.v20231009.jar kafka_2.13-3.7.0/libs/jsr305-3.0.2.jar kafka_2.13-3.7.0/libs/plexus-utils-3.3.1.jar kafka_2.13-3.7.0/libs/commons-lang3-3.8.1.jar kafka_2.13-3.7.0/libs/hk2-api-2.6.1.jar kafka_2.13-3.7.0/libs/hk2-utils-2.6.1.jar kafka_2.13-3.7.0/libs/jakarta.inject-2.6.1.jar kafka_2.13-3.7.0/libs/jakarta.annotation-api-1.3.5.jar kafka_2.13-3.7.0/libs/osgi-resource-locator-1.0.3.jar kafka_2.13-3.7.0/libs/jakarta.validation-api-2.0.2.jar kafka_2.13-3.7.0/libs/aopalliance-repackaged-2.6.1.jar kafka_2.13-3.7.0/libs/trogdor-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-shell-3.7.0.jar kafka_2.13-3.7.0/libs/jline-3.22.0.jar kafka_2.13-3.7.0/libs/connect-file-3.7.0.jar kafka_2.13-3.7.0/libs/connect-basic-auth-extension-3.7.0.jar kafka_2.13-3.7.0/libs/javax.annotation-api-1.3.2.jar kafka_2.13-3.7.0/libs/connect-mirror-3.7.0.jar kafka_2.13-3.7.0/libs/connect-mirror-client-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-streams-3.7.0.jar kafka_2.13-3.7.0/libs/rocksdbjni-7.9.2.jar kafka_2.13-3.7.0/libs/kafka-streams-scala_2.13-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-streams-test-utils-3.7.0.jar kafka_2.13-3.7.0/libs/kafka-streams-examples-3.7.0.jar + sudo groupadd --system kafka-zookeeper groupadd: group 'kafka-zookeeper' already exists + sudo useradd --system --groups kafka-zookeeper kafka useradd: user 'kafka' already exists + [[ true == \t\r\u\e ]] + sudo cp /opt/kafka_2.13-3.7.0/config/server.properties /opt/kafka_2.13-3.7.0/config/server.properties.build + sudo cp /opt/kafka-current/config/server.properties /opt/kafka_2.13-3.7.0/config/. + sudo cp -r /opt/kafka-current/logs /opt/kafka_2.13-3.7.0/. + echo '############################# Upgrading Kafka #############################' + sudo tee -a /opt/kafka_2.13-3.7.0/config/server.properties ############################# Upgrading Kafka ############################# + echo inter.broker.protocol.version=3.2-IV0 + sudo tee -a /opt/kafka_2.13-3.7.0/config/server.properties inter.broker.protocol.version=3.2-IV0 + sudo chown -R kafka:kafka-zookeeper /opt/kafka_2.13-3.7.0 + sudo ln -nfs /opt/kafka_2.13-3.7.0 /opt/kafka-current + sudo chown -h kafka:kafka-zookeeper /opt/kafka-current + sudo cp syndeia-cloud-3.6-SP2.2024-09-11_cassandra_zookeeper_kafka_setup/conf/init/systemd/kafka.service /etc/systemd/system/. + sudo systemctl daemon-reload + sudo systemctl enable kafka + sudo systemctl start kafka + [[ true == \t\r\u\e ]] + printf test ./syndeia-cloud-3.6_kafka_pre-setup.bash: connect: Connection refused ./syndeia-cloud-3.6_kafka_pre-setup.bash: line 190: /dev/tcp/localhost/9092: Connection refused + sleep 2 + printf test ./syndeia-cloud-3.6_kafka_pre-setup.bash: connect: Connection refused ./syndeia-cloud-3.6_kafka_pre-setup.bash: line 190: /dev/tcp/localhost/9092: Connection refused + sleep 2 + printf test + pushd . ~ ~ ~/syndeia-cloud-3.6-SP2.2024-09-11_cassandra_zookeeper_kafka_setup/bin + cd /opt/kafka-current/bin + sudo ./kafka-get-offsets.sh --broker-list localhost:9092 --topic store-StoreEvent store-StoreEvent:0:409 + sudo ./kafka-get-offsets.sh --broker-list localhost:9092 --topic auth-AuthEvent auth-AuthEvent:0:23 + read -n 1 -s -r -p '[Compare the metrics above with what was seen earlier, they should match, if so, press any key to continue or ^C to cancel]' [Compare the metrics above with what was seen earlier, they should match, if so, press any key to continue or ^C to cancel]+ printf '\n' + echo '' + popd ~ ~/syndeia-cloud-3.6-SP2.2024-09-11_cassandra_zookeeper_kafka_setup/bin + sudo sed -i s/inter.broker.protocol.version=3.2-IV0/inter.broker.protocol.version=3.7-IV4/ /opt/kafka_2.13-3.7.0/config/server.properties + [[ Linux =~ CYGWIN ]] + sudo systemctl restart kafka

Note, during this stage, the current (old) Kafka + Zookeeper services will be started and you will be prompted to note down the event metric counts from the store & auth services. Please make sure these #s match before and after the upgrade.

Verification

  1. Ensure the store and auth service event metric counts match before and after the upgrade.

  2. Verify Kafka is up and functioning by creating a test topic, a producer with test events, and a consumer that replays them:

    1. Create a topic if not already created before (may already have been during prior setup- if so, skip):

      /opt/kafka-current/bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9092

    2. Create a producer:

      /opt/kafka-current/bin/kafka-console-producer.sh --topic quickstart-events --bootstrap-server localhost:9092

    3. You will see an empty prompt where you can now type some test event messages, type something like the following below (hit CTRL-C to exit)

      SC 3.6-SP2_Test event1 SC 3.6-SP2_Test event2

    4. Start a consumer listener:

      /opt/kafka-current/bin/kafka-console-consumer.sh --topic quickstart-events --from-beginning --bootstrap-server localhost:9092

      You should see:

      SC 3.6-SP2_Test event1 SC 3.6-SP2_Test event2

      (Hit CTRL-C to exit)