企业🤖AI智能体构建引擎,智能编排和调试,一键部署,支持私有化部署方案 广告
# Storm UI REST API Storm UI 守护程序提供了一个 REST API, 允许您与 Storm 集群进行交互, 其中包括检索 metrics(度量)数据和配置信息, 启动或停止 topologies(拓扑)的管理操作。 ## 数据格式 该 REST API 返回 JSON 响应并支持 JSONP. 客户端可以传回一个回调查询参数, 以在回调函数中包装 JSON。 **REST API allows CORS by default.** ## 使用 UI REST API _Note(注意): 建议忽略 JSON 响应中的 undocumented elements(未记录元素),因为未来版本的 Storm 可能不再_ _支持这些元素._ ## REST API Base URL REST API 是 Storm 的 UI 守护进程(由 "storm ui" 启动)的一部分,因此与 Storm UI 在同一主机和端口上运行(UI 守护程序通常与 Nimbus 守护程序在同一主机上运行). 通过 `ui.port` 来配置 port(端口), 它默认设置为 8080(请参阅 [defaults.yaml](conf/defaults.yaml)). 该 API 的 base URL 将是: ``` http://<ui-host>:<ui-port>/api/v1/... ``` 您可以使用 `curl` 这样的工具来操作 REST API: ``` # 请求集群配置 # Note(注意): 我们假设 ui.port 配置的默认值是 8080. $ curl http://<ui-host>:8080/api/v1/cluster/configuration ``` ## 在安全的环境中模仿用户 在安全环境中, 经过身份验证的用户可以模拟另一个用户. 为了模拟用户, 调用者必须通过 `doAsUser` 参数或 header, 其值设置为用户需要执行该 request 请求. 请看 SECURITY.MD 以了解有关如何设置模拟 ACL 和授权的更多信息. 其余的 API 使用与 nimbus 使用的相同的配置和 acls. 示例: ``` 1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1425844354\?doAsUser=testUSer1 2\. curl 'http://localhost:8080/api/v1/topology/wordcount-1-1425844354/activate' -X POST -H 'doAsUser:testUSer1' ``` ## GET Operations ### /api/v1/cluster/configuration (GET) Returns the cluster configuration. Sample response (does not include all the data fields): ``` { "dev.zookeeper.path": "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs": null, "topology.builtin.metrics.bucket.size.secs": 60, "topology.fall.back.on.java.serialization": true, "topology.max.error.report.per.interval": 5, "zmq.linger.millis": 5000, "topology.skip.missing.kryo.registrations": false, "storm.messaging.netty.client_worker_threads": 1, "ui.childopts": "-Xmx768m", "storm.zookeeper.session.timeout": 20000, "nimbus.reassign": true, "topology.trident.batch.emit.interval.millis": 500, "storm.messaging.netty.flush.check.interval.ms": 10, "nimbus.monitor.freq.secs": 10, "logviewer.childopts": "-Xmx128m", "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size": 1024, } ``` ### /api/v1/cluster/summary (GET) Returns cluster summary information such as nimbus uptime or number of supervisors. Response fields: | Field | Value | Description | | --- | --- | --- | | stormVersion | String | Storm version | | supervisors | Integer | Number of supervisors running | | topologies | Integer | Number of topologies running | | slotsTotal | Integer | Total number of available worker slots | | slotsUsed | Integer | Number of worker slots used | | slotsFree | Integer | Number of worker slots available | | executorsTotal | Integer | Total number of executors | | tasksTotal | Integer | Total tasks | | schedulerDisplayResource | Boolean | Whether to display scheduler resource information | | totalMem | Double | The total amount of memory in the cluster in MB | | totalCpu | Double | The total amount of CPU in the cluster | | availMem | Double | The amount of available memory in the cluster in MB | | availCpu | Double | The amount of available cpu in the cluster | | memAssignedPercentUtil | Double | The percent utilization of assigned memory resources in cluster | | cpuAssignedPercentUtil | Double | The percent utilization of assigned CPU resources in cluster | Sample response: ``` { "stormVersion": "0.9.2-incubating-SNAPSHOT", "supervisors": 1, "slotsTotal": 4, "slotsUsed": 3, "slotsFree": 1, "executorsTotal": 28, "tasksTotal": 28, "schedulerDisplayResource": true, "totalMem": 4096.0, "totalCpu": 400.0, "availMem": 1024.0, "availCPU": 250.0, "memAssignedPercentUtil": 75.0, "cpuAssignedPercentUtil": 37.5 } ``` ### /api/v1/supervisor/summary (GET) Returns summary information for all supervisors. Response fields: | Field | Value | Description | | --- | --- | --- | | id | String | Supervisor's id | | host | String | Supervisor's host name | | uptime | String | Shows how long the supervisor is running | | uptimeSeconds | Integer | Shows how long the supervisor is running in seconds | | slotsTotal | Integer | Total number of available worker slots for this supervisor | | slotsUsed | Integer | Number of worker slots used on this supervisor | | schedulerDisplayResource | Boolean | Whether to display scheduler resource information | | totalMem | Double | Total memory capacity on this supervisor | | totalCpu | Double | Total CPU capacity on this supervisor | | usedMem | Double | Used memory capacity on this supervisor | | usedCpu | Double | Used CPU capacity on this supervisor | Sample response: ``` { "supervisors": [ { "id": "0b879808-2a26-442b-8f7d-23101e0c3696", "host": "10.11.1.7", "uptime": "5m 58s", "uptimeSeconds": 358, "slotsTotal": 4, "slotsUsed": 3, "totalMem": 3000, "totalCpu": 400, "usedMem": 1280, "usedCPU": 160 } ], "schedulerDisplayResource": true } ``` ### /api/v1/nimbus/summary (GET) Returns summary information for all nimbus hosts. Response fields: | Field | Value | Description | | --- | --- | --- | | host | String | Nimbus' host name | | port | int | Nimbus' port number | | status | String | Possible values are Leader, Not a Leader, Dead | | nimbusUpTime | String | Shows since how long the nimbus has been running | | nimbusUpTimeSeconds | String | Shows since how long the nimbus has been running in seconds | | nimbusLogLink | String | Logviewer url to view the nimbus.log | | version | String | Version of storm this nimbus host is running | Sample response: ``` { "nimbuses":[ { "host":"192.168.202.1", "port":6627, "nimbusLogLink":"http:\/\/192.168.202.1:8000\/log?file=nimbus.log", "status":Leader, "version":"0.10.0-SNAPSHOT", "nimbusUpTime":"3m 33s", "nimbusUpTimeSeconds":"213" } ] } ``` ### /api/v1/history/summary (GET) Returns a list of all running topologies' IDs submitted by the current user. Response fields: | Field | Value | Description | | --- | --- | --- | | topo-history | List | List of Topologies' IDs | Sample response: ``` { "topo-history":[ "wc6-1-1446571009", "wc8-2-1446587178" ] } ``` ### /api/v1/supervisor (GET) Returns summary for a supervisor by id, or all supervisors running on a host. Examples: ``` 1\. By host: http://ui-daemon-host-name:8080/api/v1/supervisor?host=supervisor-daemon-host-name 2\. By id: http://ui-daemon-host-name:8080/api/v1/supervisor?id=f5449110-1daa-43e2-89e3-69917b16dec9-192.168.1.1 ``` Request parameters: | Parameter | Value | Description | | --- | --- | --- | | id | String. Supervisor id | If specified, respond with the supervisor and worker stats with id. Note that when id is specified, the host argument is ignored. | | host | String. Host name | If specified, respond with all supervisors and worker stats in the host (normally just one) | | sys | String. Values 1 or 0\. Default value 0 | Controls including sys stats part of the response | Response fields: | Field | Value | Description | | --- | --- | --- | | supervisors | Array | Array of supervisor summaries | | workers | Array | Array of worker summaries | | schedulerDisplayResource | Boolean | Whether to display scheduler resource information | Each supervisor is defined by: | Field | Value | Description | | --- | --- | --- | | id | String | Supervisor's id | | host | String | Supervisor's host name | | uptime | String | Shows how long the supervisor is running | | uptimeSeconds | Integer | Shows how long the supervisor is running in seconds | | slotsTotal | Integer | Total number of worker slots for this supervisor | | slotsUsed | Integer | Number of worker slots used on this supervisor | | schedulerDisplayResource | Boolean | Whether to display scheduler resource information | | totalMem | Double | Total memory capacity on this supervisor | | totalCpu | Double | Total CPU capacity on this supervisor | | usedMem | Double | Used memory capacity on this supervisor | | usedCpu | Double | Used CPU capacity on this supervisor | Each worker is defined by: | Field | Value | Description | | --- | --- | --- | | supervisorId | String | Supervisor's id | | host | String | Worker's host name | | port | Integer | Worker's port | | topologyId | String | Topology Id | | topologyName | String | Topology Name | | executorsTotal | Integer | Number of executors used by the topology in this worker | | assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) | | assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) | | assignedCpu | Number | Assigned CPU by Scheduler (%) | | componentNumTasks | Dictionary | Components -&gt; # of executing tasks | | uptime | String | Shows how long the worker is running | | uptimeSeconds | Integer | Shows how long the worker is running in seconds | | workerLogLink | String | Link to worker log viewer page | Sample response: ``` { "supervisors": [{ "totalMem": 4096.0, "host":"192.168.10.237", "id":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212", "uptime":"7m 8s", "totalCpu":400.0, "usedCpu":495.0, "usedMem":3432.0, "slotsUsed":2, "version":"0.10.1", "slotsTotal":4, "uptimeSeconds":428 }], "schedulerDisplayResource":true, "workers":[{ "topologyName":"ras", "topologyId":"ras-4-1460229987", "host":"192.168.10.237", "supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212", "assignedMemOnHeap":704.0, "uptime":"2m 47s", "uptimeSeconds":167, "port":6707, "workerLogLink":"http:\/\/192.168.10.237:8000\/log?file=ras-4-1460229987%2F6707%2Fworker.log", "componentNumTasks": { "word":5 }, "executorsTotal":8, "assignedCpu":130.0, "assignedMemOffHeap":80.0 }, { "topologyName":"ras", "topologyId":"ras-4-1460229987", "host":"192.168.10.237", "supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212", "assignedMemOnHeap":904.0, "uptime":"2m 53s", "port":6706, "workerLogLink":"http:\/\/192.168.10.237:8000\/log?file=ras-4-1460229987%2F6706%2Fworker.log", "componentNumTasks":{ "exclaim2":2, "exclaim1":3, "word":5 }, "executorsTotal":10, "uptimeSeconds":173, "assignedCpu":165.0, "assignedMemOffHeap":80.0 }] } ``` ### /api/v1/topology/summary (GET) Returns summary information for all topologies. Response fields: | Field | Value | Description | | --- | --- | --- | | id | String | Topology Id | | name | String | Topology Name | | status | String | Topology Status | | uptime | String | Shows how long the topology is running | | uptimeSeconds | Integer | Shows how long the topology is running in seconds | | tasksTotal | Integer | Total number of tasks for this topology | | workersTotal | Integer | Number of workers used for this topology | | executorsTotal | Integer | Number of executors used for this topology | | replicationCount | Integer | Number of nimbus hosts on which this topology code is replicated | | requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) | | requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) | | requestedTotalMem | Double | Requested Total Memory by User (MB) | | requestedCpu | Double | Requested CPU by User (%) | | assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) | | assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) | | assignedTotalMem | Double | Assigned Total Memory by Scheduler (MB) | | assignedCpu | Double | Assigned CPU by Scheduler (%) | | schedulerDisplayResource | Boolean | Whether to display scheduler resource information | Sample response: ``` { "topologies": [ { "id": "WordCount3-1-1402960825", "name": "WordCount3", "status": "ACTIVE", "uptime": "6m 5s", "uptimeSeconds": 365, "tasksTotal": 28, "workersTotal": 3, "executorsTotal": 28, "replicationCount": 1, "requestedMemOnHeap": 640, "requestedMemOffHeap": 128, "requestedTotalMem": 768, "requestedCpu": 80, "assignedMemOnHeap": 640, "assignedMemOffHeap": 128, "assignedTotalMem": 768, "assignedCpu": 80 } ], "schedulerDisplayResource": true } ``` ### /api/v1/topology-workers/:id (GET) Returns the worker' information (host and port) for a topology. Response fields: | Field | Value | Description | | --- | --- | --- | | hostPortList | List | Workers' information for a topology | | name | Integer | Logviewer Port | Sample response: ``` { "hostPortList":[ { "host":"192.168.202.2", "port":6701 }, { "host":"192.168.202.2", "port":6702 }, { "host":"192.168.202.3", "port":6700 } ], "logviewerPort":8000 } ``` ### /api/v1/topology/:id (GET) Returns topology information and statistics. Substitute id with topology id. Request parameters: | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | window | String. Default value :all-time | Window duration for metrics in seconds | | sys | String. Values 1 or 0\. Default value 0 | Controls including sys stats part of the response | Response fields: | Field | Value | Description | | --- | --- | --- | | id | String | Topology Id | | name | String | Topology Name | | uptime | String | How long the topology has been running | | uptimeSeconds | Integer | How long the topology has been running in seconds | | status | String | Current status of the topology, e.g. "ACTIVE" | | tasksTotal | Integer | Total number of tasks for this topology | | workersTotal | Integer | Number of workers used for this topology | | executorsTotal | Integer | Number of executors used for this topology | | msgTimeout | Integer | Number of seconds a tuple has before the spout considers it failed | | windowHint | String | window param value in "hh mm ss" format. Default value is "All Time" | | schedulerDisplayResource | Boolean | Whether to display scheduler resource information | | replicationCount | Integer | Number of nimbus hosts on which this topology code is replicated | | debug | Boolean | If debug is enabled for the topology | | samplingPct | Double | Controls downsampling of events before they are sent to event log (percentage) | | assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) | | assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) | | assignedTotalMem | Double | Assigned Off-Heap + On-Heap Memory by Scheduler(MB) | | assignedCpu | Double | Assigned CPU by Scheduler(%) | | requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) | | requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) | | requestedCpu | Double | Requested CPU by User (%) | | topologyStats | Array | Array of all the topology related stats per time window | | topologyStats.windowPretty | String | Duration passed in HH:MM:SS format | | topologyStats.window | String | User requested time window for metrics | | topologyStats.emitted | Long | Number of messages emitted in given window | | topologyStats.trasferred | Long | Number messages transferred in given window | | topologyStats.completeLatency | String (double value returned in String format) | Total latency for processing the message | | topologyStats.acked | Long | Number of messages acked in given window | | topologyStats.failed | Long | Number of messages failed in given window | | workers | Array | Array of workers in topology | | workers.supervisorId | String | Supervisor's id | | workers.host | String | Worker's host name | | workers.port | Integer | Worker's port | | workers.topologyId | String | Topology Id | | workers.topologyName | String | Topology Name | | workers.executorsTotal | Integer | Number of executors used by the topology in this worker | | workers.assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) | | workers.assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) | | workers.assignedCpu | Number | Assigned CPU by Scheduler (%) | | workers.componentNumTasks | Dictionary | Components -&gt; # of executing tasks | | workers.uptime | String | Shows how long the worker is running | | workers.uptimeSeconds | Integer | Shows how long the worker is running in seconds | | workers.workerLogLink | String | Link to worker log viewer page | | spouts | Array | Array of all the spout components in the topology | | spouts.spoutId | String | Spout id | | spouts.executors | Integer | Number of executors for the spout | | spouts.emitted | Long | Number of messages emitted in given window | | spouts.completeLatency | String (double value returned in String format) | Total latency for processing the message | | spouts.transferred | Long | Total number of messages transferred in given window | | spouts.tasks | Integer | Total number of tasks for the spout | | spouts.lastError | String | Shows the last error happened in a spout | | spouts.errorLapsedSecs | Integer | Number of seconds elapsed since that last error happened in a spout | | spouts.errorWorkerLogLink | String | Link to the worker log that reported the exception | | spouts.acked | Long | Number of messages acked | | spouts.failed | Long | Number of messages failed | | spouts.requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) | | spouts.requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) | | spouts.requestedCpu | Double | Requested CPU by User (%) | | bolts | Array | Array of bolt components in the topology | | bolts.boltId | String | Bolt id | | bolts.capacity | String (double value returned in String format) | This value indicates number of messages executed * average execute latency / time window | | bolts.processLatency | String (double value returned in String format) | Average time of the bolt to ack a message after it was received | | bolts.executeLatency | String (double value returned in String format) | Average time to run the execute method of the bolt | | bolts.executors | Integer | Number of executor tasks in the bolt component | | bolts.tasks | Integer | Number of instances of bolt | | bolts.acked | Long | Number of tuples acked by the bolt | | bolts.failed | Long | Number of tuples failed by the bolt | | bolts.lastError | String | Shows the last error occurred in the bolt | | bolts.errorLapsedSecs | Integer | Number of seconds elapsed since that last error happened in a bolt | | bolts.errorWorkerLogLink | String | Link to the worker log that reported the exception | | bolts.emitted | Long | Number of tuples emitted | | bolts.requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) | | bolts.requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) | | bolts.requestedCpu | Double | Requested CPU by User (%) | Examples: ``` 1\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825 2\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?sys=1 3\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?window=600 ``` Sample response: ``` { "name": "WordCount3", "id": "WordCount3-1-1402960825", "workersTotal": 3, "window": "600", "status": "ACTIVE", "tasksTotal": 28, "executorsTotal": 28, "uptime": "29m 19s", "uptimeSeconds": 1759, "msgTimeout": 30, "windowHint": "10m 0s", "schedulerDisplayResource": true, "topologyStats": [ { "windowPretty": "10m 0s", "window": "600", "emitted": 397960, "transferred": 213380, "completeLatency": "0.000", "acked": 213460, "failed": 0 }, { "windowPretty": "3h 0m 0s", "window": "10800", "emitted": 1190260, "transferred": 638260, "completeLatency": "0.000", "acked": 638280, "failed": 0 }, { "windowPretty": "1d 0h 0m 0s", "window": "86400", "emitted": 1190260, "transferred": 638260, "completeLatency": "0.000", "acked": 638280, "failed": 0 }, { "windowPretty": "All time", "window": ":all-time", "emitted": 1190260, "transferred": 638260, "completeLatency": "0.000", "acked": 638280, "failed": 0 } ], "workers":[{ "topologyName":"WordCount3", "topologyId":"WordCount3-1-1402960825", "host":"192.168.10.237", "supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212", "uptime":"2m 47s", "uptimeSeconds":167, "port":6707, "workerLogLink":"http:\/\/192.168.10.237:8000\/log?file=WordCount3-1-1402960825%2F6707%2Fworker.log", "componentNumTasks": { "spout":5 }, "executorsTotal":8, "assignedMemOnHeap":704.0, "assignedCpu":130.0, "assignedMemOffHeap":80.0 }], "spouts": [ { "executors": 5, "emitted": 28880, "completeLatency": "0.000", "transferred": 28880, "acked": 0, "spoutId": "spout", "tasks": 5, "lastError": "", "errorLapsedSecs": null, "failed": 0 } ], "bolts": [ { "executors": 12, "emitted": 184580, "transferred": 0, "acked": 184640, "executeLatency": "0.048", "tasks": 12, "executed": 184620, "processLatency": "0.043", "boltId": "count", "lastError": "", "errorLapsedSecs": null, "capacity": "0.003", "failed": 0 }, { "executors": 8, "emitted": 184500, "transferred": 184500, "acked": 28820, "executeLatency": "0.024", "tasks": 8, "executed": 28780, "processLatency": "2.112", "boltId": "split", "lastError": "", "errorLapsedSecs": null, "capacity": "0.000", "failed": 0 } ], "configuration": { "storm.id": "WordCount3-1-1402960825", "dev.zookeeper.path": "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs": null, "topology.builtin.metrics.bucket.size.secs": 60, "topology.fall.back.on.java.serialization": true, "topology.max.error.report.per.interval": 5, "zmq.linger.millis": 5000, "topology.skip.missing.kryo.registrations": false, "storm.messaging.netty.client_worker_threads": 1, "ui.childopts": "-Xmx768m", "storm.zookeeper.session.timeout": 20000, "nimbus.reassign": true, "topology.trident.batch.emit.interval.millis": 500, "storm.messaging.netty.flush.check.interval.ms": 10, "nimbus.monitor.freq.secs": 10, "logviewer.childopts": "-Xmx128m", "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size": 1024, "storm.local.dir": "storm-local", "storm.messaging.netty.buffer_size": 5242880, "supervisor.worker.start.timeout.secs": 120, "topology.enable.message.timeouts": true, "nimbus.cleanup.inbox.freq.secs": 600, "nimbus.inbox.jar.expiration.secs": 3600, "drpc.worker.threads": 64, "topology.worker.shared.thread.pool.size": 4, "nimbus.seeds": [ "hw10843.local" ], "storm.messaging.netty.min_wait_ms": 100, "storm.zookeeper.port": 2181, "transactional.zookeeper.port": null, "topology.executor.receive.buffer.size": 1024, "transactional.zookeeper.servers": null, "storm.zookeeper.root": "/storm", "storm.zookeeper.retry.intervalceiling.millis": 30000, "supervisor.enable": true, "storm.messaging.netty.server_worker_threads": 1 }, "replicationCount": 1 } ``` ### /api/v1/topology/:id/metrics Returns detailed metrics for topology. It shows metrics per component, which are aggregated by stream. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | window | String. Default value :all-time | window duration for metrics in seconds | | sys | String. Values 1 or 0\. Default value 0 | Controls including sys stats part of the response | Response fields: | Field | Value | Description | | --- | --- | --- | | window | String. Default value ":all-time" | window duration for metrics in seconds | | | windowHint | String | | spouts | Array | Array of all the spout components in the topology | | spouts.id | String | Spout id | | spouts.emitted | Array | Array of all the output streams this spout emits messages | | spouts.emitted.stream_id | String | Stream id for this stream | | spouts.emitted.value | Long | Number of messages emitted in given window | | spouts.transferred | Array | Array of all the output streams this spout transfers messages | | spouts.transferred.stream_id | String | Stream id for this stream | | spouts.transferred.value | Long | Number messages transferred in given window | | spouts.acked | Array | Array of all the output streams this spout receives ack of messages | | spouts.acked.stream_id | String | Stream id for this stream | | spouts.acked.value | Long | Number of messages acked in given window | | spouts.failed | Array | Array of all the output streams this spout receives fail of messages | | spouts.failed.stream_id | String | Stream id for this stream | | spouts.failed.value | Long | Number of messages failed in given window | | spouts.complete_ms_avg | Array | Array of all the output streams this spout receives ack of messages | | spouts.complete_ms_avg.stream_id | String | Stream id for this stream | | spouts.complete_ms_avg.value | String (double value returned in String format) | Total latency for processing the message | | bolts | Array | Array of all the bolt components in the topology | | bolts.id | String | Bolt id | | bolts.emitted | Array | Array of all the output streams this bolt emits messages | | bolts.emitted.stream_id | String | Stream id for this stream | | bolts.emitted.value | Long | Number of messages emitted in given window | | bolts.transferred | Array | Array of all the output streams this bolt transfers messages | | bolts.transferred.stream_id | String | Stream id for this stream | | bolts.transferred.value | Long | Number messages transferred in given window | | bolts.acked | Array | Array of all the input streams this bolt acknowledges of messages | | bolts.acked.component_id | String | Component id for this stream | | bolts.acked.stream_id | String | Stream id for this stream | | bolts.acked.value | Long | Number of messages acked in given window | | bolts.failed | Array | Array of all the input streams this bolt receives fail of messages | | bolts.failed.component_id | String | Component id for this stream | | bolts.failed.stream_id | String | Stream id for this stream | | bolts.failed.value | Long | Number of messages failed in given window | | bolts.process_ms_avg | Array | Array of all the input streams this spout acks messages | | bolts.process_ms_avg.component_id | String | Component id for this stream | | bolts.process_ms_avg.stream_id | String | Stream id for this stream | | bolts.process_ms_avg.value | String (double value returned in String format) | Average time of the bolt to ack a message after it was received | | bolts.executed | Array | Array of all the input streams this bolt executes messages | | bolts.executed.component_id | String | Component id for this stream | | bolts.executed.stream_id | String | Stream id for this stream | | bolts.executed.value | Long | Number of messages executed in given window | | bolts.executed_ms_avg | Array | Array of all the output streams this spout receives ack of messages | | bolts.executed_ms_avg.component_id | String | Component id for this stream | | bolts.executed_ms_avg.stream_id | String | Stream id for this stream | | bolts.executed_ms_avg.value | String (double value returned in String format) | Average time to run the execute method of the bolt | Examples: ``` 1\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics 1\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics?sys=1 2\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics?window=600 ``` Sample response: ``` { "window":":all-time", "window-hint":"All time", "spouts":[ { "id":"spout", "emitted":[ { "stream_id":"__metrics", "value":20 }, { "stream_id":"default", "value":17350280 }, { "stream_id":"__ack_init", "value":17328160 }, { "stream_id":"__system", "value":20 } ], "transferred":[ { "stream_id":"__metrics", "value":20 }, { "stream_id":"default", "value":17350280 }, { "stream_id":"__ack_init", "value":17328160 }, { "stream_id":"__system", "value":0 } ], "acked":[ { "stream_id":"default", "value":17339180 } ], "failed":[ ], "complete_ms_avg":[ { "stream_id":"default", "value":"920.497" } ] } ], "bolts":[ { "id":"count", "emitted":[ { "stream_id":"__metrics", "value":120 }, { "stream_id":"default", "value":190748180 }, { "stream_id":"__ack_ack", "value":190718100 }, { "stream_id":"__system", "value":20 } ], "transferred":[ { "stream_id":"__metrics", "value":120 }, { "stream_id":"default", "value":0 }, { "stream_id":"__ack_ack", "value":190718100 }, { "stream_id":"__system", "value":0 } ], "acked":[ { "component_id":"split", "stream_id":"default", "value":190733160 } ], "failed":[ ], "process_ms_avg":[ { "component_id":"split", "stream_id":"default", "value":"0.004" } ], "executed":[ { "component_id":"split", "stream_id":"default", "value":190733140 } ], "executed_ms_avg":[ { "component_id":"split", "stream_id":"default", "value":"0.005" } ] }, { "id":"split", "emitted":[ { "stream_id":"__metrics", "value":60 }, { "stream_id":"default", "value":190754740 }, { "stream_id":"__ack_ack", "value":17317580 }, { "stream_id":"__system", "value":20 } ], "transferred":[ { "stream_id":"__metrics", "value":60 }, { "stream_id":"default", "value":190754740 }, { "stream_id":"__ack_ack", "value":17317580 }, { "stream_id":"__system", "value":0 } ], "acked":[ { "component_id":"spout", "stream_id":"default", "value":17339180 } ], "failed":[ ], "process_ms_avg":[ { "component_id":"spout", "stream_id":"default", "value":"0.051" } ], "executed":[ { "component_id":"spout", "stream_id":"default", "value":17339240 } ], "executed_ms_avg":[ { "component_id":"spout", "stream_id":"default", "value":"0.052" } ] } ] } ``` ### /api/v1/topology/:id/component/:component (GET) Returns detailed metrics and executor information | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | component | String (required) | Component Id | | window | String. Default value :all-time | window duration for metrics in seconds | | sys | String. Values 1 or 0\. Default value 0 | controls including sys stats part of the response | Response fields: | Field | Value | Description | | --- | --- | --- | | user | String | Topology owner | | id | String | Component id | | encodedId | String | URL encoded component id | | name | String | Topology name | | executors | Integer | Number of executor tasks in the component | | tasks | Integer | Number of instances of component | | requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) | | requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) | | requestedCpu | Double | Requested CPU by User (%) | | schedulerDisplayResource | Boolean | Whether to display scheduler resource information | | topologyId | String | Topology id | | topologyStatus | String | Topology status | | encodedTopologyId | String | URL encoded topology id | | window | String. Default value "All Time" | window duration for metrics in seconds | | componentType | String | component type: SPOUT or BOLT | | windowHint | String | window param value in "hh mm ss" format. Default value is "All Time" | | debug | Boolean | If debug is enabled for the component | | samplingPct | Double | Controls downsampling of events before they are sent to event log (percentage) | | eventLogLink | String | URL viewer link to event log (debug mode) | | profilingAndDebuggingCapable | Boolean | true if there is support for Profiling and Debugging Actions | | profileActionEnabled | Boolean | true if worker profiling (Java Flight Recorder) is enabled | | profilerActive | Array | Array of currently active Profiler Actions | | componentErrors | Array of Errors | List of component errors | | componentErrors.errorTime | Long | Timestamp when the exception occurred (Prior to 0.11.0, this field was named 'time'.) | | componentErrors.errorHost | String | host name for the error | | componentErrors.errorPort | String | port for the error | | componentErrors.error | String | Shows the error happened in a component | | componentErrors.errorLapsedSecs | Integer | Number of seconds elapsed since the error happened in a component | | componentErrors.errorWorkerLogLink | String | Link to the worker log that reported the exception | | spoutSummary | Array | (only for spouts) Array of component stats, one element per window. | | spoutSummary.windowPretty | String | Duration passed in HH:MM:SS format | | spoutSummary.window | String | window duration for metrics in seconds | | spoutSummary.emitted | Long | Number of messages emitted in given window | | spoutSummary.completeLatency | String (double value returned in String format) | Total latency for processing the message | | spoutSummary.transferred | Long | Total number of messages transferred in given window | | spoutSummary.acked | Long | Number of messages acked | | spoutSummary.failed | Long | Number of messages failed | | boltStats | Array | (only for bolts) Array of component stats, one element per window. | | boltStats.windowPretty | String | Duration passed in HH:MM:SS format | | boltStats.window | String | window duration for metrics in seconds | | boltStats.transferred | Long | Total number of messages transferred in given window | | boltStats.processLatency | String (double value returned in String format) | Average time of the bolt to ack a message after it was received | | boltStats.acked | Long | Number of messages acked | | boltStats.failed | Long | Number of messages failed | | inputStats | Array | (only for bolts) Array of input stats | | inputStats.component | String | Component id | | inputStats.encodedComponentId | String | URL encoded component id | | inputStats.executeLatency | Long | The average time a tuple spends in the execute method | | inputStats.processLatency | Long | The average time it takes to ack a tuple after it is first received | | inputStats.executed | Long | The number of incoming tuples processed | | inputStats.acked | Long | Number of messages acked | | inputStats.failed | Long | Number of messages failed | | inputStats.stream | String | The name of the tuple stream given in the topology, or "default" if none specified | | outputStats | Array | Array of output stats | | outputStats.transferred | Long | Number of tuples emitted that sent to one ore more bolts | | outputStats.emitted | Long | Number of tuples emitted | | outputStats.stream | String | The name of the tuple stream given in the topology, or "default" if none specified | Examples: ``` 1\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout 2\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?sys=1 3\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?window=600 ``` Sample response: ``` { "name": "WordCount3", "id": "spout", "componentType": "spout", "windowHint": "10m 0s", "executors": 5, "componentErrors":[{"errorTime": 1406006074000, "errorHost": "10.11.1.70", "errorPort": 6701, "errorWorkerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log", "errorLapsedSecs": 16, "error": "java.lang.RuntimeException: java.lang.StringIndexOutOfBoundsException: Some Error\n\tat org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)\n\tat org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)\n\tat org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)\n\tat backtype...more.." }], "topologyId": "WordCount3-1-1402960825", "tasks": 5, "window": "600", "profilerActive": [ { "host": "10.11.1.70", "port": "6701", "dumplink":"http:\/\/10.11.1.70:8000\/dumps\/ex-1-1452718803\/10.11.1.70%3A6701", "timestamp":"576328" } ], "profilingAndDebuggingCapable": true, "profileActionEnabled": true, "spoutSummary": [ { "windowPretty": "10m 0s", "window": "600", "emitted": 28500, "transferred": 28460, "completeLatency": "0.000", "acked": 0, "failed": 0 }, { "windowPretty": "3h 0m 0s", "window": "10800", "emitted": 127640, "transferred": 127440, "completeLatency": "0.000", "acked": 0, "failed": 0 }, { "windowPretty": "1d 0h 0m 0s", "window": "86400", "emitted": 127640, "transferred": 127440, "completeLatency": "0.000", "acked": 0, "failed": 0 }, { "windowPretty": "All time", "window": ":all-time", "emitted": 127640, "transferred": 127440, "completeLatency": "0.000", "acked": 0, "failed": 0 } ], "outputStats": [ { "stream": "__metrics", "emitted": 40, "transferred": 0, "completeLatency": "0", "acked": 0, "failed": 0 }, { "stream": "default", "emitted": 28460, "transferred": 28460, "completeLatency": "0", "acked": 0, "failed": 0 } ], "executorStats": [ { "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log", "emitted": 5720, "port": 6701, "completeLatency": "0.000", "transferred": 5720, "host": "10.11.1.7", "acked": 0, "uptime": "43m 4s", "uptimeSeconds": 2584, "id": "[24-24]", "failed": 0 }, { "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6703.log", "emitted": 5700, "port": 6703, "completeLatency": "0.000", "transferred": 5700, "host": "10.11.1.7", "acked": 0, "uptime": "42m 57s", "uptimeSeconds": 2577, "id": "[25-25]", "failed": 0 }, { "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6702.log", "emitted": 5700, "port": 6702, "completeLatency": "0.000", "transferred": 5680, "host": "10.11.1.7", "acked": 0, "uptime": "42m 57s", "uptimeSeconds": 2577, "id": "[26-26]", "failed": 0 }, { "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log", "emitted": 5700, "port": 6701, "completeLatency": "0.000", "transferred": 5680, "host": "10.11.1.7", "acked": 0, "uptime": "43m 4s", "uptimeSeconds": 2584, "id": "[27-27]", "failed": 0 }, { "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6703.log", "emitted": 5680, "port": 6703, "completeLatency": "0.000", "transferred": 5680, "host": "10.11.1.7", "acked": 0, "uptime": "42m 57s", "uptimeSeconds": 2577, "id": "[28-28]", "failed": 0 } ] } ``` ## Profiling and Debugging GET Operations ### /api/v1/topology/:id/profiling/start/:host-port/:timeout (GET) Request to start profiler on worker with timeout. Returns status and link to profiler artifacts for worker. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | host-port | String (required) | Worker Id | | timeout | String (required) | Time out for profiler to stop in minutes | Response fields: | Field | Value | Description | | --- | --- | --- | | id | String | Worker id | | status | String | Response Status | | timeout | String | Requested timeout | | dumplink | String | Link to logviewer URL for worker profiler documents. | Examples: ``` 1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/10 2\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/5 3\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/20 ``` Sample response: ``` { "status": "ok", "id": "10.11.1.7:6701", "timeout": "10", "dumplink": "http:\/\/10.11.1.7:8000\/dumps\/wordcount-1-1446614150\/10.11.1.7%3A6701" } ``` ### /api/v1/topology/:id/profiling/dumpprofile/:host-port (GET) Request to dump profiler recording on worker. Returns status and worker id for the request. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | host-port | String (required) | Worker Id | Response fields: | Field | Value | Description | | --- | --- | --- | | id | String | Worker id | | status | String | Response Status | Examples: ``` 1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpprofile/10.11.1.7:6701 ``` Sample response: ``` { "status": "ok", "id": "10.11.1.7:6701", } ``` ### /api/v1/topology/:id/profiling/stop/:host-port (GET) Request to stop profiler on worker. Returns status and worker id for the request. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | host-port | String (required) | Worker Id | Response fields: | Field | Value | Description | | --- | --- | --- | | id | String | Worker id | | status | String | Response Status | Examples: ``` 1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/stop/10.11.1.7:6701 ``` Sample response: ``` { "status": "ok", "id": "10.11.1.7:6701", } ``` ### /api/v1/topology/:id/profiling/dumpjstack/:host-port (GET) Request to dump jstack on worker. Returns status and worker id for the request. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | host-port | String (required) | Worker Id | Response fields: | Field | Value | Description | | --- | --- | --- | | id | String | Worker id | | status | String | Response Status | Examples: ``` 1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpjstack/10.11.1.7:6701 ``` Sample response: ``` { "status": "ok", "id": "10.11.1.7:6701", } ``` ### /api/v1/topology/:id/profiling/dumpheap/:host-port (GET) Request to dump heap (jmap) on worker. Returns status and worker id for the request. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | host-port | String (required) | Worker Id | Response fields: | Field | Value | Description | | --- | --- | --- | | id | String | Worker id | | status | String | Response Status | Examples: ``` 1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpheap/10.11.1.7:6701 ``` Sample response: ``` { "status": "ok", "id": "10.11.1.7:6701", } ``` ### /api/v1/topology/:id/profiling/restartworker/:host-port (GET) Request to request the worker. Returns status and worker id for the request. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | host-port | String (required) | Worker Id | Response fields: | Field | Value | Description | | --- | --- | --- | | id | String | Worker id | | status | String | Response Status | Examples: ``` 1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/restartworker/10.11.1.7:6701 ``` Sample response: ``` { "status": "ok", "id": "10.11.1.7:6701", } ``` ## POST Operations ### /api/v1/topology/:id/activate (POST) Activates a topology. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | Sample Response: ``` {"topologyOperation":"activate","topologyId":"wordcount-1-1420308665","status":"success"} ``` ### /api/v1/topology/:id/deactivate (POST) Deactivates a topology. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | Sample Response: ``` {"topologyOperation":"deactivate","topologyId":"wordcount-1-1420308665","status":"success"} ``` ### /api/v1/topology/:id/rebalance/:wait-time (POST) Rebalances a topology. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | wait-time | String (required) | Wait time before rebalance happens | | rebalanceOptions | Json (optional) | topology rebalance options | Sample rebalanceOptions json: ``` {"rebalanceOptions" : {"numWorkers" : 2, "executors" : {"spout" :4, "count" : 10}}, "callback" : "foo"} ``` Examples: ``` curl -i -b ~/cookiejar.txt -c ~/cookiejar.txt -X POST -H "Content-Type: application/json" -d '{"rebalanceOptions": {"numWorkers": 2, "executors": { "spout" : "5", "split": 7, "count": 5 }}, "callback":"foo"}' http://localhost:8080/api/v1/topology/wordcount-1-1420308665/rebalance/0 ``` Sample Response: ``` {"topologyOperation":"rebalance","topologyId":"wordcount-1-1420308665","status":"success"} ``` ### /api/v1/topology/:id/kill/:wait-time (POST) Kills a topology. | Parameter | Value | Description | | --- | --- | --- | | id | String (required) | Topology Id | | wait-time | String (required) | Wait time before rebalance happens | Caution: Small wait times (0-5 seconds) may increase the probability of triggering the bug reported in [STORM-112](https://issues.apache.org/jira/browse/STORM-112), which may result in broker Supervisor daemons. Sample Response: ``` {"topologyOperation":"kill","topologyId":"wordcount-1-1420308665","status":"success"} ``` ## API errors The API returns 500 HTTP status codes in case of any errors. Sample response: ``` { "error": "Internal Server Error", "errorMessage": "java.lang.NullPointerException\n\tat clojure.core$name.invoke(core.clj:1505)\n\tat org.apache.storm.ui.core$component_page.invoke(core.clj:752)\n\tat org.apache.storm.ui.core$fn__7766.invoke(core.clj:782)\n\tat compojure.core$make_route$fn__5755.invoke(core.clj:93)\n\tat compojure.core$if_route$fn__5743.invoke(core.clj:39)\n\tat compojure.core$if_method$fn__5736.invoke(core.clj:24)\n\tat compojure.core$routing$fn__5761.invoke(core.clj:106)\n\tat clojure.core$some.invoke(core.clj:2443)\n\tat compojure.core$routing.doInvoke(core.clj:106)\n\tat clojure.lang.RestFn.applyTo(RestFn.java:139)\n\tat clojure.core$apply.invoke(core.clj:619)\n\tat compojure.core$routes$fn__5765.invoke(core.clj:111)\n\tat ring.middleware.reload$wrap_reload$fn__6880.invoke(reload.clj:14)\n\tat org.apache.storm.ui.core$catch_errors$fn__7800.invoke(core.clj:836)\n\tat ring.middleware.keyword_params$wrap_keyword_params$fn__6319.invoke(keyword_params.clj:27)\n\tat ring.middleware.nested_params$wrap_nested_params$fn__6358.invoke(nested_params.clj:65)\n\tat ring.middleware.params$wrap_params$fn__6291.invoke(params.clj:55)\n\tat ring.middleware.multipart_params$wrap_multipart_params$fn__6386.invoke(multipart_params.clj:103)\n\tat ring.middleware.flash$wrap_flash$fn__6675.invoke(flash.clj:14)\n\tat ring.middleware.session$wrap_session$fn__6664.invoke(session.clj:43)\n\tat ring.middleware.cookies$wrap_cookies$fn__6595.invoke(cookies.clj:160)\n\tat ring.adapter.jetty$proxy_handler$fn__6112.invoke(jetty.clj:16)\n\tat ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown Source)\n\tat org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)\n\tat org.mortbay.jetty.Server.handle(Server.java:326)\n\tat org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)\n\tat org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)\n\tat org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)\n\tat org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)\n\tat org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)\n\tat org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)\n\tat org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)\n" } ```