企业🤖AI智能体构建引擎,智能编排和调试,一键部署,支持私有化部署方案 广告
* 技术选型 ![](https://img.kancloud.cn/4c/27/4c27cf580fa94d2a5cdf920b642f52c1_503x313.png) ![](https://img.kancloud.cn/51/ad/51ad5c5ab24714fd7ae7c21172e77426_476x69.png) * 修改配置文件,首先是机器A * `exec-memory-avro`,agent、source、sink、channel都改了名字。 ``` # example.conf: A single-node Flume configuration # Name the components on this agent exec-memory-avro.sources = exec-source exec-memory-avro.sinks = avro-sink exec-memory-avro.channels = memory-channel # Describe/configure the source exec-memory-avro.sources.exec-source.type = exec # 监控文件的更新 exec-memory-avro.sources.exec-source.command = tail -F /home/bizzbee/data/data.log exec-memory-avro.sources.exec-source.shell = /bin/sh -c # Describe the sink exec-memory-avro.sinks.avro-sink.type = avro # 输出到本机,当然也可以到别的机器 exec-memory-avro.sinks.avro-sink.hostname = spark # 本机的44444端口 exec-memory-avro.sinks.avro-sink.port = 44444 # Use a channel which buffers events in memory exec-memory-avro.channels.memory-channel.type = memory # Bind the source and sink to the channel exec-memory-avro.sources.exec-source.channels = memory-channel exec-memory-avro.sinks.avro-sink.channel = memory-channel ``` *再创建一个配置文件,是B机器上的flume配置(本次实验实际上还是本机)`avro-memory-logger.conf` ``` # Name the components on this agent # i avro-memory-logger.sources = avro-source avro-memory-logger.sinks = logger-sink avro-memory-logger.channels = memory-channel # Describe/configure the source avro-memory-logger.sources.avro-source.type = avro avro-memory-logger.sources.avro-source.bind = spark avro-memory-logger.sources.avro-source.port = 44444 # Describe the sink avro-memory-logger.sinks.logger-sink.type = logger # Use a channel which buffers events in memory avro-memory-logger.channels.memory-channel.type = memory # Bind the source and sink to the channel avro-memory-logger.sources.avro-source.channels = memory-channel avro-memory-logger.sinks.logger-sink.channel = memory-channel ``` * 启动,先启动B机器上的flume ``` flume-ng agent --name avro-memory-logger --conf $FLUME_HOME/conf --conf-file $FLUME_HOME/conf/avro-memory-logger.conf -Dflume.root.logger=INFO,console ``` * 然后启动A机器上的 ``` flume-ng agent --name exec-memory-avro --conf $FLUME_HOME/conf --conf-file $FLUME_HOME/conf/exec-memory-avro.conf -Dflume.root.logger=INFO,console ``` * 然后同样向data.log里面添加内容,会在B看到输出。 ![](https://img.kancloud.cn/04/88/04886c87b34a5f3a27ce0887e1790f21_904x464.png)