Pulsar-SQL搭建

环境

• version:pulsar 2.7.1
• 做个Pulsar SQL做个demo查询
• java 8

Pulsar-SQL搭建

node1/2/3配置config.properties

node1节点（coordinator节点）

vim ${
    project.root}/conf/presto/config.properties

node.id=1
node.environment=test
http-server.http.port=8081
coordinator=true
node-scheduler.include-coordinator=true
discovery-server.enabled=true
discovery.uri=http://node1:8081
presto.version=testversionexchange.http-client.max-connections=1000
exchange.http-client.max-connections-per-server=1000
exchange.http-client.connect-timeout=1m
exchange.http-client.idle-timeout=1mscheduler.http-client.max-connections=1000
scheduler.http-client.max-connections-per-server=1000
scheduler.http-client.connect-timeout=1m
scheduler.http-client.idle-timeout=1mquery.client.timeout=5m
query.min-expire-age=30m

node2、3节点（worker节点）

vim ${
    project.root}/conf/presto/config.properties

node.id=2（3）
node.environment=test
http-server.http.port=8081#discovery-server.enabled=true
discovery.uri=http://th199:8081exchange.http-client.max-connections=1000
exchange.http-client.max-connections-per-server=1000
exchange.http-client.connect-timeout=1m
exchange.http-client.idle-timeout=1m#scheduler.http-client.max-connections=1000
#scheduler.http-client.max-connections-per-server=1000
#scheduler.http-client.connect-timeout=1m
#scheduler.http-client.idle-timeout=1mquery.client.timeout=5m
query.min-expire-age=30mpresto.version=testversion
coordinator=false
#node-scheduler.include-coordinator=true

注意
worker节点必须注意吧该屏蔽的屏蔽掉，否则启动报错如下

2021-04-06T20:43:36.728+0800    INFO    main    Bootstrap       transaction.max-finishing-concurrency                                   1                                                                          1                                                                          Maximum parallelism for committing or aborting a transaction
2021-04-06T20:43:36.728+0800    WARN    main    Bootstrap       UNUSED PROPERTIES
2021-04-06T20:43:36.728+0800    WARN    main    Bootstrap       scheduler.http-client.connect-timeout
2021-04-06T20:43:36.728+0800    WARN    main    Bootstrap       scheduler.http-client.idle-timeout
2021-04-06T20:43:36.728+0800    WARN    main    Bootstrap       scheduler.http-client.max-connections
2021-04-06T20:43:36.728+0800    WARN    main    Bootstrap       scheduler.http-client.max-connections-per-server
2021-04-06T20:43:36.728+0800    WARN    main    Bootstrap       
2021-04-06T20:43:37.098+0800    ERROR   main    io.prestosql.server.PrestoServer        Unable to create injector, see the following errors:1) Configuration property 'scheduler.http-client.connect-timeout' was not used2) Configuration property 'scheduler.http-client.idle-timeout' was not used3) Configuration property 'scheduler.http-client.max-connections' was not used4) Configuration property 'scheduler.http-client.max-connections-per-server' was not used4 errors
com.google.inject.CreationException: Unable to create injector, see the following errors:1) Configuration property 'scheduler.http-client.connect-timeout' was not used2) Configuration property 'scheduler.http-client.idle-timeout' was not used3) Configuration property 'scheduler.http-client.max-connections' was not used4) Configuration property 'scheduler.http-client.max-connections-per-server' was not used4 errorsat com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:554)at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:161)at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:108)at com.google.inject.Guice.createInjector(Guice.java:87)at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:262)at io.prestosql.server.PrestoServer.run(PrestoServer.java:111)at io.prestosql.$gen.Presto_332____20210406_124335_1.run(Unknown Source)at io.prestosql.server.PrestoServer.main(PrestoServer.java:72)

node1/2/3配置pulsar.properties

vim ${
    project.root}/conf/presto/catalog/pulsar.properties

# the url of Pulsar broker service
pulsar.broker-service-url=http://node1:8080,node2:8080,node3:8080
# URI of Zookeeper cluster
pulsar.zookeeper-uri=node1:8001,node2:8001,node3:8001

node1/2/3启动sql-worker

./bin sql-worker start

node1/2/3验证是否启动成功

./bin/pulsar sql

Pulsar-SQL 查询主题消息

select * from pulsar."public/default"."test";

当然这个主题未分区，且消息体都是胡乱发的，不是结构化数据
具体可以给pulsar发送结构化的POJO对象，然后就可以查询到该对象的结构化数据了，可以参考https://mp.weixin.qq.com/s?__biz=MzUyMjkzMjA1Ng%3D%3D&idx=1&mid=2247485104&scene=21&sn=5992d6579c8866c1413d0b6fa17ecfe9#wechat_redirect

例如