| Table of Contents | ||||
|---|---|---|---|---|
|
Single Node Setup Instructions
1. Ensure you have the syndeia-cloud-3.56-SP1_cassandra_zookeeper_kafka_setup.zip (or latest service pack) downloaded to your home directory from the download/license instructions sent out by our team.
Note: the .ZIP will pre-create a separate folder for its contents when extracted so there is no need to pre-create a separate folder for it.
2. Ensure you satisfy Zookeeper's pre-requisites, ie: have (Open|Oracle)JDK/JRE, memory, HD space, etc. (see https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_systemReq for more details).
3. If using a firewall, ensure the following port is accessible (consult your local network admin if required): TCP port 2181 (this is the port to listen to for client connections).
...
Download, Install, Configure & Run Apache Zookeeper
1. Download Zookeeper Download Zookeeper 3.68.3 from 4 from https://archive.apache.org/dist/zookeeper/zookeeper-3.68.34/apache-zookeeper-3.68.34-bin.tar.gz (ie: wget https://archive.apache.org/dist/zookeeper/zookeeper-3.68.34/apache-zookeeper-3.68.34-bin.tar.gz)
2. Use tar to extract the .tar.gz file to /opt/ and create a logs folder in it, ie: ZK_build_ver=3.68.34; sudo tar -xvzf apache-zookeeper-${ZK_build_ver}.tar.gz -C /opt/ ; sudo mv /opt/{apache-,}zookeeper-${ZK_build_ver} ; sudo mkdir {/opt/ ; where zookeeper-${ZK_build_ver}/logs,/var/lib/zookeeper}ZK_build_ver = the version you downloaded, ex: 3.68.34.
3. Create/update a symlink to the current version, ie: sudo ln -nfs /opt/zookeeper-${ZK_build_ver} /opt/zookeeper-current
...
7. Create the configuration file /etc/zookeeper/conf/zoo.cfg & paste the configuration below, ie: sudo mkdir -p /etc/zookeeper/conf/ && sudo cp /opt/zookeeper-${ZK_build_ver}/conf/zoo_sample.cfg /etc/zookeeper/conf/zoo.cfg (to use as a template) and edit it, pasting in the below to replace it:
| Code Block | ||||
|---|---|---|---|---|
| ||||
# http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. dataDir=/var/lib/zookeeper # Place the dataLogDir to a separate physical disc for better performance # dataLogDir=/disk2/zookeeper # the port at which the clients will connect clientPort=2181 # specify all zookeeper servers # The fist port is used by followers to connect to the leader # The second one is used for leader election #server.1=zookeeperServer1.mydomain.tld:2888:3888 #server.2=zookeeperServer2.mydomain.tld:2888:3888 #server.3=zookeeperServer3.mydomain.tld:2888:3888 # To avoid seeks ZooKeeper allocates space in the transaction log file in # blocks of preAllocSize kilobytes. The default block size is 64M. One reason # for changing the size of the blocks is to reduce the block size if snapshots # are taken more often. (Also, see snapCount). #preAllocSize=65536 # Clients can submit requests faster than ZooKeeper can process them, # especially if there are a lot of clients. To prevent ZooKeeper from running # out of memory due to queued requests, ZooKeeper will throttle clients so that # there is no more than globalOutstandingLimit outstanding requests in the # system. The default limit is 1,000.ZooKeeper logs transactions to a # transaction log. After snapCount transactions are written to a log file a # snapshot is started and a new transaction log file is started. The default # snapCount is 10,000. #snapCount=1000 # If this option is defined, requests will be will logged to a trace file named # traceFile.year.month.day. #traceFile= # Leader accepts client connections. Default value is "yes". The leader machine # coordinates updates. For higher update throughput at thes slight expense of # read throughput the leader can be configured to not accept clients and focus # on coordination. #leaderServes=yes |
Note: In particular, please pay attention to the value of dataDir=/var/lib/zookeeper and ensure the zookeeper user and kafka-zookeeper group have access to the directory.
Note: For a quick start, the above settings will get you up and running, however for any multi-node deployment scenarios you will need to specify server.n, where n = server # and create a /var/lib/zookeeper/myid file on each server specifying each server's id (see steps 4-5 of https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_zkMulitServerSetup for more details)
8. Start the Zookeeper service, ie: /opt/zookeeper>-<release_ver/bin/zkServer.sh start /etc/zookeeper/conf/zoo.cfg
Note, Apache Zookeeper doesn't include a native systemd .service file by default. While systemd will dynamically create one at runtime via its SysV compatibility module, you may wish to create one yourself to exercise better control over the various service parameters. For your convenience we have created a systemd zookeeper.service file (included in the syndeia-cloud-3.56-SP1_cassandra_zookeeper_kafka_setup.zip download) . To use this, copy it to to /etc/systemd/system, reload systemd units, enable zookeeper to start on boot and start the service, ie: sudo cp <service_file_download_dir>/zookeeper.service /etc/systemd/system/. && sudo systemctl daemon-reload && sudo systemctl enable zookeeper && sudo systemctl start zookeeper
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
$ sudo systemctl status zookeeper ● zookeeper.service Loaded: loaded (/etc/systemd/system/zookeeper.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-04-05 14:04:23 EDT; 4h 16min ago Process: 21740 ExecStart=/opt/zookeeper-3.68.34/bin/zkServer.sh start /etc/zookeeper/conf/zoo.cfg (code=exited, status=0/SUCCESS) Main PID: 21747 (java) CGroup: /system.slice/zookeeper.service └─21747 /usr/bin/java -Dzookeeper.log.dir=/opt/zookeeper-current/logs -Dzookeeper.root.logger=INFO,ROLLINGFILE -cp /etc/zookeeper/conf:/usr/share/java/jline.jar:/usr/share/java/log4j-1. Apr 05 14:04:22 zookeeperServer1.mydomain.tld systemd[1]: Starting zookeeper.service... Apr 05 14:04:22 zookeeperServer1.mydomain.tld zkServer.sh[21740]: ZooKeeper JMX enabled by default Apr 05 14:04:22 zookeeperServer1.mydomain.tld zkServer.sh[21740]: Using config: /etc/zookeeper/conf/zoo.cfg Apr 05 14:04:23 zookeeperServer1.mydomain.tld systemd[1]: Started zookeeper.service. |
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
[...] 2019-04-05 14:04:23,046 - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /etc/zookeeper/conf/zoo.cfg 2019-04-05 14:04:23,051 - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2019-04-05 14:04:23,051 - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2019-04-05 14:04:23,051 - INFO [main:DatadirCleanupManager@101] - Purge task is not scheduled. 2019-04-05 14:04:23,052 - WARN [main:QuorumPeerMain@113] - Either no config or no quorum defined in config, running in standalone mode 2019-04-05 14:04:23,066 - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /etc/zookeeper/conf/zoo.cfg 2019-04-05 14:04:23,067 - INFO [main:ZooKeeperServerMain@95] - Starting server 2019-04-05 14:04:23,095 - INFO [main:Environment@100] - Server environment:zookeeper.version=3.68.34-6401e4ad2087061bc6b9f80dec2d69f2e3c8660a, built on 04/08/2021 16:35 GMT (org.apache.zookeeper.ZooKeeper) 2019-04-05 14:04:23,095 - INFO [main:Environment@100] - Server environment:host.name=zookeeperServer1.mydomain.tld 2019-04-05 14:04:23,095 - INFO [main:Environment@100] - Server environment:java.version=1.8.0_332 2019-04-05 14:04:23,095 - INFO [main:Environment@100] - Server environment:java.vendor=Oracle Corporation 2019-04-05 14:04:23,095 - INFO [main:Environment@100] - Server environment:java.home=/usr/lib/jvm/java-8-oracle/jre 2019-04-05 14:04:23,096 - INFO [main:Environment@100] - Server environment:java.class.path=/etc/zookeeper/conf:/usr/share/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share/java/xer cesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/zookeeper.jar 2019-04-05 14:04:23,096 - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-05 14:04:23,096 - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp 2019-04-05 14:04:23,096 - INFO [main:Environment@100] - Server environment:java.compiler=<NA> 2019-04-05 14:04:23,097 - INFO [main:Environment@100] - Server environment:os.name=Linux 2019-04-05 14:04:23,098 - INFO [main:Environment@100] - Server environment:os.arch=amd64 2019-04-05 14:04:23,098 - INFO [main:Environment@100] - Server environment:os.version=4.18.16-x86_64-linode118 2019-04-05 14:04:23,098 - INFO [main:Environment@100] - Server environment:user.name=zookeeper 2019-04-05 14:04:23,098 - INFO [main:Environment@100] - Server environment:user.home=/var/lib/zookeeper 2019-04-05 14:04:23,098 - INFO [main:Environment@100] - Server environment:user.dir=/ 2019-04-05 14:04:23,107 - INFO [main:ZooKeeperServer@787] - tickTime set to 2000 2019-04-05 14:04:23,110 - INFO [main:ZooKeeperServer@796] - minSessionTimeout set to -1 2019-04-05 14:04:23,110 - INFO [main:ZooKeeperServer@805] - maxSessionTimeout set to -1 2019-04-05 14:04:23,124 - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181 |
...