docker-compose + kafka 정말 이해가 되지 않습니다. 도움 한 번 부탁드려요.

Question

도커에 주키퍼+카프카+프로듀서 서버를 하나에 컨테이너에 올리고, docker 외부에서 컨슈머 클라이언트를 실행하여 메세지가 잘 나오는지 보는 테스트를 하고있었습니다.# docker-compose version: '3' services: zookeeper: hostname: zookeeper container_name: zookeeper image: confluentinc/cp-zookeeper:latest ports: - "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 kafka1: image: confluentinc/cp-kafka:latest container_name: kafka ports: - "9092:9092" environment: KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181" KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 depends_on: - zookeeper kafka-ui: image: provectuslabs/kafka-ui container_name: kafka-ui ports: - "8090:8080" restart: always environment: - KAFKA_CLUSTERS_0_NAME=local - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka1:29092 - KAFKA_CLUSTERS_0_ZOOKEEPER=zookeeper:2181 test-server: build: . # container_name: test-server ports: - "1234:1234" depends_on: - kafka1// 카프카 프로듀서 설정 코드 public static Properties getKafkaProducerProperties() { Properties properties = new Properties(); properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "kafka1:29092"); properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); return properties; }kafka1에 container_name: kafka 라고 되어있는 부분이 보이실겁니다.저게 kafka이외에 다른 단어(kafka1, kafka-1 등등)라면 프로듀서가 브로커로 토픽을 보내지 못합니다. 타임 아웃이 나더군요.오직 kafka라고 써놔야지만 동작합니다... 도대체 왜그럴까요? 정말 이해가 되지 않습니다.포트 문제인가, 리스너 문제인가 싶어 삽질을 몇날 몇일을 했지만 결국 저게 원인이더군요.아무래도 제 시야가 좁아진 것 같습니다... 도움 부탁드려요.

킹스바흐 · Accepted Answer

올려주신 설정 기준으로 간단히 테스트해봤을땐 test-server 에서 붙어야 정상인데요 이상하네요version: '3' services: zookeeper: hostname: zookeeper container_name: zookeeper image: confluentinc/cp-zookeeper:latest ports: - "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 kafka1: image: confluentinc/cp-kafka:latest container_name: kafka1 ports: - "9092:9092" environment: KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181" KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 depends_on: - zookeeper kafka-ui: image: provectuslabs/kafka-ui container_name: kafka-ui ports: - "8090:8080" restart: always environment: - KAFKA_CLUSTERS_0_NAME=local - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka1:29092 - KAFKA_CLUSTERS_0_ZOOKEEPER=zookeeper:2181 always-on: image: ubuntu:latest container_name: always-on command: tail -f /dev/null restart: always

캐티 · Answer

이게 머나요. json 설정에 넣은 아이디 값은 컴포즈에서 사용하는 용도이고 도커가 가져가서 사용하는 값이 아니겠쩌.컴포즈를 사용하면 편하다 하더라도 실질적으로 도커 네트워크에 기반하여 컨테이너 이름을 연결해주는 거여서 망했쩌.

킹스바흐 · Answer

대댓글은 이미지 첨부가 안돼서 새로 답니다어드민에서 브로커 인식 잘 되는거보면 컴포즈 설정은 문제가 없어보여요사용한 컴포즈 설정 첨부합니다version: '3' services: zookeeper: hostname: zookeeper container_name: zookeeper image: confluentinc/cp-zookeeper:latest ports: - "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 kafka1: image: confluentinc/cp-kafka:latest container_name: kafka1 ports: - "9092:9092" environment: KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181" KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:19092,PLAINTEXT_HOST://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 depends_on: - zookeeper kafka-ui: image: provectuslabs/kafka-ui container_name: kafka-ui ports: - "8090:8080" restart: always environment: - KAFKA_CLUSTERS_0_NAME=local - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka1:19092 - KAFKA_CLUSTERS_0_ZOOKEEPER=zookeeper:2181 always-on: image: ubuntu:latest container_name: always-on command: tail -f /dev/null restart: always

킹스바흐 · Answer

로컬호스트 이슈일 수 있으니 네트워크를 명시해보세요 이래도 안되면~~ 제 PC에서는 되는데요 밖에 없네요 ㅠ.ㅠ이만 시간이 늦어 가보겠습니다 건승하세요version: '3' services: zookeeper: hostname: zookeeper container_name: zookeeper image: confluentinc/cp-zookeeper:latest ports: - "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 networks: - kafka-network kafka1: image: confluentinc/cp-kafka:latest container_name: kafka1 ports: - "9092:9092" environment: KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181" KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:19092,PLAINTEXT_HOST://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 depends_on: - zookeeper networks: - kafka-network kafka-ui: image: provectuslabs/kafka-ui container_name: kafka-ui ports: - "8090:8080" restart: always environment: - KAFKA_CLUSTERS_0_NAME=local - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka1:19092 - KAFKA_CLUSTERS_0_ZOOKEEPER=zookeeper:2181 networks: - kafka-network always-on: image: ubuntu:latest container_name: always-on command: tail -f /dev/null restart: always networks: - kafka-network networks: kafka-network: driver: bridge

docker-compose + kafka 정말 이해가 되지 않습니다. 도움 한 번 부탁드려요.

4개의 답변

#kafka 목록