[Kafka] zookeeper 및 kafka 설치, Cluster 설정

kafka 설치

wget http://apache.mirror.cdnetworks.com/kafka/2.6.0/kafka_2.13-2.6.0.tgz

압축해제는 아래 url 참고해주세요.

halfstorage.tistory.com/71

[Linux] tar 명령어 tar, tgz(tar.gz) 압축해제

Zookeeper 설정({kafka설치경로}/config/zookeeper.properties)

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
# 
#    http://www.apache.org/licenses/LICENSE-2.0
# 
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# the directory where the snapshot is stored.
dataDir=/data/zookeeper
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0
# Disable the adminserver by default to avoid port conflicts.
# Set the port to something non-conflicting if choosing to enable this
admin.enableServer=false
# admin.serverPort=8080
##
# 팔로워가 리더와 초기에 연결하는 시간에 대한 타임아웃
initLimit=5
# 팔로워가 리더와 동기화 하는데에 대한 타임아웃
syncLimit=2
 
# cluster할 서버 ip 설정
server.1=10.100.1.11:2888:3888
server.2=10.100.1.12:2888:3888

- 추가된 설정은 initLimit, syncLimit, server1, 2입니다.

# 팔로워가 리더와 초기에 연결하는 시간에 대한 타임아웃

initLimit=5

# 팔로워가 리더와 동기화 하는데에 대한 타임아웃

syncLimit=2

 

# cluster할 서버 ip 설정

server.1=10.100.1.11:2888:3888

server.2=10.100.1.12:2888:3888

 

- cluster 설정할 서버의 dataDir(/data/zookeeper)에 서버 ID를 지정해야합니다.

# 1번 서버(10.100.1.11)
echo 1 > /data/zookeeper/myid
 
# 2번 서버(10.100.1.12)
echo 2 > /data/zookeeper/myid

 

kafka 설정({kafka설치경로}/config/server.properties)

############################# Server Basics #############################
 
# The id of the broker. This must be set to a unique integer for each broker.
broker.id=1
 
############################# Broker config #############################
# Allow automatic topic creation on the broker when subscribing to or assigning a topic.
# A topic being subscribed to will be automatically created only if the broker allows for it using `auto.create.topics.enable` broker configuration.
# This configuration must be set to `false` when using brokers older than 0.11.0
allow.auto.create.topics=false

- 변경 및 추가사항만 기재했습니다.

위에 zookeeper 설정 시 생성한 myid로 broker.id를 지정 하고

브로커 토픽 자동 생성을 false로 설정합니다.