Step-by-step guide: Installing Kafka from scratch on Linux

1. Preparation

Before starting the installation, you need to ensure that your system meets the following requirements:

• Operating system: Ubuntu 16.04 or higher
• Memory: At least 4GB
• Hard disk space: At least 500GB
• Java: Version 8 or higher
• ZooKeeper: Version 3.4 or higher

2. Install Java

1. Use the following command to update the system package list:

sudo apt-get update

2. Install Java:

sudo apt-get install openjdk-8-jdk

3. Verify whether Java has been successfully installed:

java -version

3. Install ZooKeeper

1. Download ZooKeeper:

wget https://mirrors.estointernet.in/apache/zookeeper/zookeeper-3.6.3/zookeeper-3.6.3.tar.gz

2. Unzip ZooKeeper:

tar -xzvf zookeeper-3.6.3.tar.gz

3. Move ZooKeeper to the installation directory:

sudo mv zookeeper-3.6.3 /opt/zookeeper

4. Create ZooKeeper users and groups:

sudo groupadd zookeeper
sudo useradd -g zookeeper zookeeper

5. Grant permissions to the ZooKeeper user and group on the installation directory:

sudo chown -R zookeeper:zookeeper /opt/zookeeper

6. Edit the ZooKeeper configuration file:

sudo nano /opt/zookeeper/conf/zoo.cfg

7. In the configuration file , change the values ​​of the dataDir and clientPort options to /var/lib/zookeeper and 2181 respectively:

dataDir=/var/lib/zookeeper
clientPort=2181

8. Create ZooKeeper data directory:

sudo mkdir -p /var/lib/zookeeper
sudo chown -R zookeeper:zookeeper /var/lib/zookeeper

9. Start ZooKeeper:

sudo /opt/zookeeper/bin/zkServer.sh start

10. Verify that ZooKeeper has started successfully:

sudo netstat -plnt | grep 2181

4. Install Kafka

1. Download Kafka:

wget https://mirrors.estointernet.in/apache/kafka/2.8.1/kafka_2.13-2.8.1.tgz

2. Unzip Kafka:

tar -xzvf kafka_2.13-2.8.1.tgz

3. Move Kafka to the installation directory:

sudo mv kafka_2.13-2.8.1 /opt/kafka

4. Create Kafka user and group:

sudo groupadd kafka
sudo useradd -g kafka kafka

5. Grant Kafka user and group to the installation directory Permissions:

sudo chown -R kafka:kafka /opt/kafka

6. Edit Kafka configuration file:

sudo nano /opt/kafka/config/server.properties

7. In the configuration file, change the values ​​of the following options to:

broker.id=0
listeners=PLAINTEXT://:9092
zookeeper.connect=localhost:2181

8. Create Kafka data directory:

sudo mkdir -p /var/lib/kafka
sudo chown -R kafka:kafka /var/lib/kafka

9. Start Kafka:

sudo /opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties

10. Verify whether Kafka has been started successfully ：

sudo netstat -plnt | grep 9092

5. Test Kafka

1. Create a topic:

kafka-topics --create --topic test --partitions 1 --replication-factor 1

2. Send a message to the topic:

kafka-console-producer --topic test --message "Hello, world!"

3. Receive messages from the topic:

kafka-console-consumer --topic test --from-beginning

6. Summary

You have successfully installed Kafka under Linux. Now, you can start building distributed systems using Kafka.

The above is the detailed content of Step-by-step guide: Installing Kafka from scratch on Linux. For more information, please follow other related articles on the PHP Chinese website!