Haɗuwa Kafka Streams da Node.js

Haɗin kai Kafka Streams tare da Node.js aikace-aikacen hanya ce mai ƙarfi don sarrafawa da bincika bayanai kai tsaye daga Apache Kafka a cikin Node.js yanayi. Kafka Streams yana ba ku damar gina sarrafa bayanai na lokaci-lokaci da haɗa su cikin Node.js aikace-aikacenku ba tare da matsala ba. Ga takamaiman jagora kan yadda ake cimma wannan:

Mataki 1: Shigar Kafka Streams da KafkaJS

Da farko, kuna buƙatar shigar Kafka Streams da KafkaJS don haɗawa Kafka cikin aikace-aikacen ku Node.js. Kuna iya amfani da npm don shigar da waɗannan fakiti:

npm install kafka-streams kafkajs

Mataki na 2: Ƙirƙiri a Kafka Stream

Ƙirƙiri Kafka Stream a cikin Node.js aikace-aikacenku ta amfani da Kafka Streams API. Ga babban misali na ƙirƙira Kafka Stream don sarrafa bayanai daga ɗayan topic kuma fitar da sakamakon zuwa wani topic:

const { KafkaStreams } = require('kafka-streams');  
const { Kafka } = require('kafkajs');  
  
const kafka = new Kafka({  
  clientId: 'your-client-id',  
  brokers: ['broker1:port1', 'broker2:port2'],  
});  
  
const kafkaStreams = new KafkaStreams({  
  kafka,  
  logLevel: 2, // Level 2 for debug logs  
});  
  
const streamConfig = {  
  'group.id': 'your-group-id',  
  'metadata.broker.list': 'broker1:port1,broker2:port2',  
  'enable.auto.commit': false,  
  'socket.keepalive.enable': true,  
};  
  
const stream = kafkaStreams.getKStream(streamConfig);  
  
stream  
  .from('input-topic')  
  .filter(record => record.value && record.value.length > 0)  
  .map(record =>({  
    key: record.key,  
    value: record.value.toUpperCase(),  
  }))  
  .to('output-topic');  
  
kafkaStreams.start();  

Mataki 3: Tsara Bayanai

A cikin misalin da ke sama, mun ƙirƙiri Kafka Stream don sauraron bayanai daga input-topic, sannan mu sarrafa bayanan ta hanyar mayar da su duka zuwa manyan haruffa kuma mu tura sakamakon zuwa output-topic.

Mataki 4: Guda Application

A ƙarshe, kuna buƙatar gudanar da Node.js aikace-aikacen ku don fara sarrafa bayanai daga Kafka Streams.

Lura cewa a cikin misalin da ke sama, kuna buƙatar maye gurbin dabi'u kamar your-client-id, broker1:port1, your-group-id, input-topic da output-topic  takamaiman bayanan aikinku.

Haɗin kai Kafka Streams tare da Node.js aikace-aikacen yana ba ku damar sassauƙa da ƙarfi gina ƙarfin sarrafa bayanai na lokaci-lokaci.