title: Storm UI REST API layout: documentation documentation: true
Storm UI 守护程序提供了一个 REST API, 允许您与 Storm 集群进行交互, 其中包括检索 metrics（度量）数据和配置信息, 启动或停止 topologies（拓扑）的管理操作。
数据格式
该 REST API 返回 JSON 响应并支持 JSONP. 客户端可以传回一个回调查询参数, 以在回调函数中包装 JSON。 REST API allows CORS by default.
使用 UI REST API
Note（注意）: 建议忽略 JSON 响应中的 undocumented elements（未记录元素），因为未来版本的 Storm 可能不再 支持这些元素.
REST API Base URL
REST API 是 Storm 的 UI 守护进程（由 "storm ui" 启动）的一部分，因此与 Storm UI 在同一主机和端口上运行（UI 守护程序通常与 Nimbus 守护程序在同一主机上运行）. 通过 ui.port 来配置 port（端口）, 它默认设置为 8080（请参阅 defaults.yaml）.
该 API 的 base URL 将是:
http://<ui-host>:<ui-port>/api/v1/...
您可以使用 curl 这样的工具来操作 REST API:
# 请求集群配置
# Note（注意）: 我们假设 ui.port 配置的默认值是 8080.
$ curl http://<ui-host>:8080/api/v1/cluster/configuration
在安全的环境中模仿用户
在安全环境中, 经过身份验证的用户可以模拟另一个用户. 为了模拟用户, 调用者必须通过 doAsUser 参数或 header, 其值设置为用户需要执行该 request 请求. 请看 SECURITY.MD 以了解有关如何设置模拟 ACL 和授权的更多信息. 其余的 API 使用与 nimbus 使用的相同的配置和 acls.
示例:
1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1425844354\?doAsUser=testUSer1
2. curl 'http://localhost:8080/api/v1/topology/wordcount-1-1425844354/activate' -X POST -H 'doAsUser:testUSer1'
GET Operations
/api/v1/cluster/configuration (GET)
Returns the cluster configuration.
Sample response (does not include all the data fields):
{
"dev.zookeeper.path": "../../../../tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs": null,
"topology.builtin.metrics.bucket.size.secs": 60,
"topology.fall.back.on.java.serialization": true,
"topology.max.error.report.per.interval": 5,
"zmq.linger.millis": 5000,
"topology.skip.missing.kryo.registrations": false,
"storm.messaging.netty.client_worker_threads": 1,
"ui.childopts": "-Xmx768m",
"storm.zookeeper.session.timeout": 20000,
"nimbus.reassign": true,
"topology.trident.batch.emit.interval.millis": 500,
"storm.messaging.netty.flush.check.interval.ms": 10,
"nimbus.monitor.freq.secs": 10,
"logviewer.childopts": "-Xmx128m",
"java.library.path": "../../../../usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size": 1024,
}
/api/v1/cluster/summary (GET)
Returns cluster summary information such as nimbus uptime or number of supervisors.
Response fields:
| Field | Value | Description |
|---|---|---|
| stormVersion | String | Storm version |
| supervisors | Integer | Number of supervisors running |
| topologies | Integer | Number of topologies running |
| slotsTotal | Integer | Total number of available worker slots |
| slotsUsed | Integer | Number of worker slots used |
| slotsFree | Integer | Number of worker slots available |
| executorsTotal | Integer | Total number of executors |
| tasksTotal | Integer | Total tasks |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
| totalMem | Double | The total amount of memory in the cluster in MB |
| totalCpu | Double | The total amount of CPU in the cluster |
| availMem | Double | The amount of available memory in the cluster in MB |
| availCpu | Double | The amount of available cpu in the cluster |
| memAssignedPercentUtil | Double | The percent utilization of assigned memory resources in cluster |
| cpuAssignedPercentUtil | Double | The percent utilization of assigned CPU resources in cluster |
Sample response:
{
"stormVersion": "0.9.2-incubating-SNAPSHOT",
"supervisors": 1,
"slotsTotal": 4,
"slotsUsed": 3,
"slotsFree": 1,
"executorsTotal": 28,
"tasksTotal": 28,
"schedulerDisplayResource": true,
"totalMem": 4096.0,
"totalCpu": 400.0,
"availMem": 1024.0,
"availCPU": 250.0,
"memAssignedPercentUtil": 75.0,
"cpuAssignedPercentUtil": 37.5
}
/api/v1/supervisor/summary (GET)
Returns summary information for all supervisors.
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Supervisor's id |
| host | String | Supervisor's host name |
| uptime | String | Shows how long the supervisor is running |
| uptimeSeconds | Integer | Shows how long the supervisor is running in seconds |
| slotsTotal | Integer | Total number of available worker slots for this supervisor |
| slotsUsed | Integer | Number of worker slots used on this supervisor |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
| totalMem | Double | Total memory capacity on this supervisor |
| totalCpu | Double | Total CPU capacity on this supervisor |
| usedMem | Double | Used memory capacity on this supervisor |
| usedCpu | Double | Used CPU capacity on this supervisor |
Sample response:
{
"supervisors": [
{
"id": "0b879808-2a26-442b-8f7d-23101e0c3696",
"host": "10.11.1.7",
"uptime": "5m 58s",
"uptimeSeconds": 358,
"slotsTotal": 4,
"slotsUsed": 3,
"totalMem": 3000,
"totalCpu": 400,
"usedMem": 1280,
"usedCPU": 160
}
],
"schedulerDisplayResource": true
}
/api/v1/nimbus/summary (GET)
Returns summary information for all nimbus hosts.
Response fields:
| Field | Value | Description |
|---|---|---|
| host | String | Nimbus' host name |
| port | int | Nimbus' port number |
| status | String | Possible values are Leader, Not a Leader, Dead |
| nimbusUpTime | String | Shows since how long the nimbus has been running |
| nimbusUpTimeSeconds | String | Shows since how long the nimbus has been running in seconds |
| nimbusLogLink | String | Logviewer url to view the nimbus.log |
| version | String | Version of storm this nimbus host is running |
Sample response:
{
"nimbuses":[
{
"host":"192.168.202.1",
"port":6627,
"nimbusLogLink":"http:\/\/192.168.202.1:8000\/log?file=nimbus.log",
"status":Leader,
"version":"0.10.0-SNAPSHOT",
"nimbusUpTime":"3m 33s",
"nimbusUpTimeSeconds":"213"
}
]
}
/api/v1/history/summary (GET)
Returns a list of all running topologies' IDs submitted by the current user.
Response fields:
| Field | Value | Description |
|---|---|---|
| topo-history | List | List of Topologies' IDs |
Sample response:
{
"topo-history":[
"wc6-1-1446571009",
"wc8-2-1446587178"
]
}
/api/v1/supervisor (GET)
Returns summary for a supervisor by id, or all supervisors running on a host.
Examples:
1. By host: http://ui-daemon-host-name:8080/api/v1/supervisor?host=supervisor-daemon-host-name
2. By id: http://ui-daemon-host-name:8080/api/v1/supervisor?id=f5449110-1daa-43e2-89e3-69917b16dec9-192.168.1.1
Request parameters:
| Parameter | Value | Description |
|---|---|---|
| id | String. Supervisor id | If specified, respond with the supervisor and worker stats with id. Note that when id is specified, the host argument is ignored. |
| host | String. Host name | If specified, respond with all supervisors and worker stats in the host (normally just one) |
| sys | String. Values 1 or 0. Default value 0 | Controls including sys stats part of the response |
Response fields:
| Field | Value | Description |
|---|---|---|
| supervisors | Array | Array of supervisor summaries |
| workers | Array | Array of worker summaries |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
Each supervisor is defined by:
| Field | Value | Description |
|---|---|---|
| id | String | Supervisor's id |
| host | String | Supervisor's host name |
| uptime | String | Shows how long the supervisor is running |
| uptimeSeconds | Integer | Shows how long the supervisor is running in seconds |
| slotsTotal | Integer | Total number of worker slots for this supervisor |
| slotsUsed | Integer | Number of worker slots used on this supervisor |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
| totalMem | Double | Total memory capacity on this supervisor |
| totalCpu | Double | Total CPU capacity on this supervisor |
| usedMem | Double | Used memory capacity on this supervisor |
| usedCpu | Double | Used CPU capacity on this supervisor |
Each worker is defined by:
| Field | Value | Description |
|---|---|---|
| supervisorId | String | Supervisor's id |
| host | String | Worker's host name |
| port | Integer | Worker's port |
| topologyId | String | Topology Id |
| topologyName | String | Topology Name |
| executorsTotal | Integer | Number of executors used by the topology in this worker |
| assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) |
| assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) |
| assignedCpu | Number | Assigned CPU by Scheduler (%) |
| componentNumTasks | Dictionary | Components -> # of executing tasks |
| uptime | String | Shows how long the worker is running |
| uptimeSeconds | Integer | Shows how long the worker is running in seconds |
| workerLogLink | String | Link to worker log viewer page |
Sample response:
{
"supervisors": [{
"totalMem": 4096.0,
"host":"192.168.10.237",
"id":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212",
"uptime":"7m 8s",
"totalCpu":400.0,
"usedCpu":495.0,
"usedMem":3432.0,
"slotsUsed":2,
"version":"0.10.1",
"slotsTotal":4,
"uptimeSeconds":428
}],
"schedulerDisplayResource":true,
"workers":[{
"topologyName":"ras",
"topologyId":"ras-4-1460229987",
"host":"192.168.10.237",
"supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212",
"assignedMemOnHeap":704.0,
"uptime":"2m 47s",
"uptimeSeconds":167,
"port":6707,
"workerLogLink":"http:\/\/192.168.10.237:8000\/log?file=ras-4-1460229987%2F6707%2Fworker.log",
"componentNumTasks": {
"word":5
},
"executorsTotal":8,
"assignedCpu":130.0,
"assignedMemOffHeap":80.0
},
{
"topologyName":"ras",
"topologyId":"ras-4-1460229987",
"host":"192.168.10.237",
"supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212",
"assignedMemOnHeap":904.0,
"uptime":"2m 53s",
"port":6706,
"workerLogLink":"http:\/\/192.168.10.237:8000\/log?file=ras-4-1460229987%2F6706%2Fworker.log",
"componentNumTasks":{
"exclaim2":2,
"exclaim1":3,
"word":5
},
"executorsTotal":10,
"uptimeSeconds":173,
"assignedCpu":165.0,
"assignedMemOffHeap":80.0
}]
}
/api/v1/topology/summary (GET)
Returns summary information for all topologies.
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Topology Id |
| name | String | Topology Name |
| status | String | Topology Status |
| uptime | String | Shows how long the topology is running |
| uptimeSeconds | Integer | Shows how long the topology is running in seconds |
| tasksTotal | Integer | Total number of tasks for this topology |
| workersTotal | Integer | Number of workers used for this topology |
| executorsTotal | Integer | Number of executors used for this topology |
| replicationCount | Integer | Number of nimbus hosts on which this topology code is replicated |
| requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) |
| requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) |
| requestedTotalMem | Double | Requested Total Memory by User (MB) |
| requestedCpu | Double | Requested CPU by User (%) |
| assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) |
| assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) |
| assignedTotalMem | Double | Assigned Total Memory by Scheduler (MB) |
| assignedCpu | Double | Assigned CPU by Scheduler (%) |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
Sample response:
{
"topologies": [
{
"id": "WordCount3-1-1402960825",
"name": "WordCount3",
"status": "ACTIVE",
"uptime": "6m 5s",
"uptimeSeconds": 365,
"tasksTotal": 28,
"workersTotal": 3,
"executorsTotal": 28,
"replicationCount": 1,
"requestedMemOnHeap": 640,
"requestedMemOffHeap": 128,
"requestedTotalMem": 768,
"requestedCpu": 80,
"assignedMemOnHeap": 640,
"assignedMemOffHeap": 128,
"assignedTotalMem": 768,
"assignedCpu": 80
}
],
"schedulerDisplayResource": true
}
/api/v1/topology-workers/:id (GET)
Returns the worker' information (host and port) for a topology.
Response fields:
| Field | Value | Description |
|---|---|---|
| hostPortList | List | Workers' information for a topology |
| name | Integer | Logviewer Port |
Sample response:
{
"hostPortList":[
{
"host":"192.168.202.2",
"port":6701
},
{
"host":"192.168.202.2",
"port":6702
},
{
"host":"192.168.202.3",
"port":6700
}
],
"logviewerPort":8000
}
/api/v1/topology/:id (GET)
Returns topology information and statistics. Substitute id with topology id.
Request parameters:
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| window | String. Default value :all-time | Window duration for metrics in seconds |
| sys | String. Values 1 or 0. Default value 0 | Controls including sys stats part of the response |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Topology Id |
| name | String | Topology Name |
| uptime | String | How long the topology has been running |
| uptimeSeconds | Integer | How long the topology has been running in seconds |
| status | String | Current status of the topology, e.g. "ACTIVE" |
| tasksTotal | Integer | Total number of tasks for this topology |
| workersTotal | Integer | Number of workers used for this topology |
| executorsTotal | Integer | Number of executors used for this topology |
| msgTimeout | Integer | Number of seconds a tuple has before the spout considers it failed |
| windowHint | String | window param value in "hh mm ss" format. Default value is "All Time" |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
| replicationCount | Integer | Number of nimbus hosts on which this topology code is replicated |
| debug | Boolean | If debug is enabled for the topology |
| samplingPct | Double | Controls downsampling of events before they are sent to event log (percentage) |
| assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) |
| assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) |
| assignedTotalMem | Double | Assigned Off-Heap + On-Heap Memory by Scheduler(MB) |
| assignedCpu | Double | Assigned CPU by Scheduler(%) |
| requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) |
| requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) |
| requestedCpu | Double | Requested CPU by User (%) |
| topologyStats | Array | Array of all the topology related stats per time window |
| topologyStats.windowPretty | String | Duration passed in HH:MM:SS format |
| topologyStats.window | String | User requested time window for metrics |
| topologyStats.emitted | Long | Number of messages emitted in given window |
| topologyStats.trasferred | Long | Number messages transferred in given window |
| topologyStats.completeLatency | String (double value returned in String format) | Total latency for processing the message |
| topologyStats.acked | Long | Number of messages acked in given window |
| topologyStats.failed | Long | Number of messages failed in given window |
| workers | Array | Array of workers in topology |
| workers.supervisorId | String | Supervisor's id |
| workers.host | String | Worker's host name |
| workers.port | Integer | Worker's port |
| workers.topologyId | String | Topology Id |
| workers.topologyName | String | Topology Name |
| workers.executorsTotal | Integer | Number of executors used by the topology in this worker |
| workers.assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) |
| workers.assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) |
| workers.assignedCpu | Number | Assigned CPU by Scheduler (%) |
| workers.componentNumTasks | Dictionary | Components -> # of executing tasks |
| workers.uptime | String | Shows how long the worker is running |
| workers.uptimeSeconds | Integer | Shows how long the worker is running in seconds |
| workers.workerLogLink | String | Link to worker log viewer page |
| spouts | Array | Array of all the spout components in the topology |
| spouts.spoutId | String | Spout id |
| spouts.executors | Integer | Number of executors for the spout |
| spouts.emitted | Long | Number of messages emitted in given window |
| spouts.completeLatency | String (double value returned in String format) | Total latency for processing the message |
| spouts.transferred | Long | Total number of messages transferred in given window |
| spouts.tasks | Integer | Total number of tasks for the spout |
| spouts.lastError | String | Shows the last error happened in a spout |
| spouts.errorLapsedSecs | Integer | Number of seconds elapsed since that last error happened in a spout |
| spouts.errorWorkerLogLink | String | Link to the worker log that reported the exception |
| spouts.acked | Long | Number of messages acked |
| spouts.failed | Long | Number of messages failed |
| spouts.requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) |
| spouts.requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) |
| spouts.requestedCpu | Double | Requested CPU by User (%) |
| bolts | Array | Array of bolt components in the topology |
| bolts.boltId | String | Bolt id |
| bolts.capacity | String (double value returned in String format) | This value indicates number of messages executed * average execute latency / time window |
| bolts.processLatency | String (double value returned in String format) | Average time of the bolt to ack a message after it was received |
| bolts.executeLatency | String (double value returned in String format) | Average time to run the execute method of the bolt |
| bolts.executors | Integer | Number of executor tasks in the bolt component |
| bolts.tasks | Integer | Number of instances of bolt |
| bolts.acked | Long | Number of tuples acked by the bolt |
| bolts.failed | Long | Number of tuples failed by the bolt |
| bolts.lastError | String | Shows the last error occurred in the bolt |
| bolts.errorLapsedSecs | Integer | Number of seconds elapsed since that last error happened in a bolt |
| bolts.errorWorkerLogLink | String | Link to the worker log that reported the exception |
| bolts.emitted | Long | Number of tuples emitted |
| bolts.requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) |
| bolts.requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) |
| bolts.requestedCpu | Double | Requested CPU by User (%) |
Examples:
1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825
2. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?sys=1
3. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?window=600
Sample response:
{
"name": "WordCount3",
"id": "WordCount3-1-1402960825",
"workersTotal": 3,
"window": "600",
"status": "ACTIVE",
"tasksTotal": 28,
"executorsTotal": 28,
"uptime": "29m 19s",
"uptimeSeconds": 1759,
"msgTimeout": 30,
"windowHint": "10m 0s",
"schedulerDisplayResource": true,
"topologyStats": [
{
"windowPretty": "10m 0s",
"window": "600",
"emitted": 397960,
"transferred": 213380,
"completeLatency": "0.000",
"acked": 213460,
"failed": 0
},
{
"windowPretty": "3h 0m 0s",
"window": "10800",
"emitted": 1190260,
"transferred": 638260,
"completeLatency": "0.000",
"acked": 638280,
"failed": 0
},
{
"windowPretty": "1d 0h 0m 0s",
"window": "86400",
"emitted": 1190260,
"transferred": 638260,
"completeLatency": "0.000",
"acked": 638280,
"failed": 0
},
{
"windowPretty": "All time",
"window": ":all-time",
"emitted": 1190260,
"transferred": 638260,
"completeLatency": "0.000",
"acked": 638280,
"failed": 0
}
],
"workers":[{
"topologyName":"WordCount3",
"topologyId":"WordCount3-1-1402960825",
"host":"192.168.10.237",
"supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212",
"uptime":"2m 47s",
"uptimeSeconds":167,
"port":6707,
"workerLogLink":"http:\/\/192.168.10.237:8000\/log?file=WordCount3-1-1402960825%2F6707%2Fworker.log",
"componentNumTasks": {
"spout":5
},
"executorsTotal":8,
"assignedMemOnHeap":704.0,
"assignedCpu":130.0,
"assignedMemOffHeap":80.0
}],
"spouts": [
{
"executors": 5,
"emitted": 28880,
"completeLatency": "0.000",
"transferred": 28880,
"acked": 0,
"spoutId": "spout",
"tasks": 5,
"lastError": "",
"errorLapsedSecs": null,
"failed": 0
}
],
"bolts": [
{
"executors": 12,
"emitted": 184580,
"transferred": 0,
"acked": 184640,
"executeLatency": "0.048",
"tasks": 12,
"executed": 184620,
"processLatency": "0.043",
"boltId": "count",
"lastError": "",
"errorLapsedSecs": null,
"capacity": "0.003",
"failed": 0
},
{
"executors": 8,
"emitted": 184500,
"transferred": 184500,
"acked": 28820,
"executeLatency": "0.024",
"tasks": 8,
"executed": 28780,
"processLatency": "2.112",
"boltId": "split",
"lastError": "",
"errorLapsedSecs": null,
"capacity": "0.000",
"failed": 0
}
],
"configuration": {
"storm.id": "WordCount3-1-1402960825",
"dev.zookeeper.path": "../../../../tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs": null,
"topology.builtin.metrics.bucket.size.secs": 60,
"topology.fall.back.on.java.serialization": true,
"topology.max.error.report.per.interval": 5,
"zmq.linger.millis": 5000,
"topology.skip.missing.kryo.registrations": false,
"storm.messaging.netty.client_worker_threads": 1,
"ui.childopts": "-Xmx768m",
"storm.zookeeper.session.timeout": 20000,
"nimbus.reassign": true,
"topology.trident.batch.emit.interval.millis": 500,
"storm.messaging.netty.flush.check.interval.ms": 10,
"nimbus.monitor.freq.secs": 10,
"logviewer.childopts": "-Xmx128m",
"java.library.path": "../../../../usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size": 1024,
"storm.local.dir": "storm-local",
"storm.messaging.netty.buffer_size": 5242880,
"supervisor.worker.start.timeout.secs": 120,
"topology.enable.message.timeouts": true,
"nimbus.cleanup.inbox.freq.secs": 600,
"nimbus.inbox.jar.expiration.secs": 3600,
"drpc.worker.threads": 64,
"topology.worker.shared.thread.pool.size": 4,
"nimbus.seeds": [
"hw10843.local"
],
"storm.messaging.netty.min_wait_ms": 100,
"storm.zookeeper.port": 2181,
"transactional.zookeeper.port": null,
"topology.executor.receive.buffer.size": 1024,
"transactional.zookeeper.servers": null,
"storm.zookeeper.root": "../../../../storm",
"storm.zookeeper.retry.intervalceiling.millis": 30000,
"supervisor.enable": true,
"storm.messaging.netty.server_worker_threads": 1
},
"replicationCount": 1
}
/api/v1/topology/:id/metrics
Returns detailed metrics for topology. It shows metrics per component, which are aggregated by stream.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| window | String. Default value :all-time | window duration for metrics in seconds |
| sys | String. Values 1 or 0. Default value 0 | Controls including sys stats part of the response |
Response fields:
| Field | Value | Description |
|---|---|---|
| window | String. Default value ":all-time" | window duration for metrics in seconds |
| windowHint | String | window param value in "hh mm ss" format. Default value is "All Time" |
| spouts | Array | Array of all the spout components in the topology |
| spouts.id | String | Spout id |
| spouts.emitted | Array | Array of all the output streams this spout emits messages |
| spouts.emitted.stream_id | String | Stream id for this stream |
| spouts.emitted.value | Long | Number of messages emitted in given window |
| spouts.transferred | Array | Array of all the output streams this spout transfers messages |
| spouts.transferred.stream_id | String | Stream id for this stream |
| spouts.transferred.value | Long | Number messages transferred in given window |
| spouts.acked | Array | Array of all the output streams this spout receives ack of messages |
| spouts.acked.stream_id | String | Stream id for this stream |
| spouts.acked.value | Long | Number of messages acked in given window |
| spouts.failed | Array | Array of all the output streams this spout receives fail of messages |
| spouts.failed.stream_id | String | Stream id for this stream |
| spouts.failed.value | Long | Number of messages failed in given window |
| spouts.complete_ms_avg | Array | Array of all the output streams this spout receives ack of messages |
| spouts.complete_ms_avg.stream_id | String | Stream id for this stream |
| spouts.complete_ms_avg.value | String (double value returned in String format) | Total latency for processing the message |
| bolts | Array | Array of all the bolt components in the topology |
| bolts.id | String | Bolt id |
| bolts.emitted | Array | Array of all the output streams this bolt emits messages |
| bolts.emitted.stream_id | String | Stream id for this stream |
| bolts.emitted.value | Long | Number of messages emitted in given window |
| bolts.transferred | Array | Array of all the output streams this bolt transfers messages |
| bolts.transferred.stream_id | String | Stream id for this stream |
| bolts.transferred.value | Long | Number messages transferred in given window |
| bolts.acked | Array | Array of all the input streams this bolt acknowledges of messages |
| bolts.acked.component_id | String | Component id for this stream |
| bolts.acked.stream_id | String | Stream id for this stream |
| bolts.acked.value | Long | Number of messages acked in given window |
| bolts.failed | Array | Array of all the input streams this bolt receives fail of messages |
| bolts.failed.component_id | String | Component id for this stream |
| bolts.failed.stream_id | String | Stream id for this stream |
| bolts.failed.value | Long | Number of messages failed in given window |
| bolts.process_ms_avg | Array | Array of all the input streams this spout acks messages |
| bolts.process_ms_avg.component_id | String | Component id for this stream |
| bolts.process_ms_avg.stream_id | String | Stream id for this stream |
| bolts.process_ms_avg.value | String (double value returned in String format) | Average time of the bolt to ack a message after it was received |
| bolts.executed | Array | Array of all the input streams this bolt executes messages |
| bolts.executed.component_id | String | Component id for this stream |
| bolts.executed.stream_id | String | Stream id for this stream |
| bolts.executed.value | Long | Number of messages executed in given window |
| bolts.executed_ms_avg | Array | Array of all the output streams this spout receives ack of messages |
| bolts.executed_ms_avg.component_id | String | Component id for this stream |
| bolts.executed_ms_avg.stream_id | String | Stream id for this stream |
| bolts.executed_ms_avg.value | String (double value returned in String format) | Average time to run the execute method of the bolt |
Examples:
1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics
1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics?sys=1
2. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics?window=600
Sample response:
{
"window":":all-time",
"window-hint":"All time",
"spouts":[
{
"id":"spout",
"emitted":[
{
"stream_id":"__metrics",
"value":20
},
{
"stream_id":"default",
"value":17350280
},
{
"stream_id":"__ack_init",
"value":17328160
},
{
"stream_id":"__system",
"value":20
}
],
"transferred":[
{
"stream_id":"__metrics",
"value":20
},
{
"stream_id":"default",
"value":17350280
},
{
"stream_id":"__ack_init",
"value":17328160
},
{
"stream_id":"__system",
"value":0
}
],
"acked":[
{
"stream_id":"default",
"value":17339180
}
],
"failed":[
],
"complete_ms_avg":[
{
"stream_id":"default",
"value":"920.497"
}
]
}
],
"bolts":[
{
"id":"count",
"emitted":[
{
"stream_id":"__metrics",
"value":120
},
{
"stream_id":"default",
"value":190748180
},
{
"stream_id":"__ack_ack",
"value":190718100
},
{
"stream_id":"__system",
"value":20
}
],
"transferred":[
{
"stream_id":"__metrics",
"value":120
},
{
"stream_id":"default",
"value":0
},
{
"stream_id":"__ack_ack",
"value":190718100
},
{
"stream_id":"__system",
"value":0
}
],
"acked":[
{
"component_id":"split",
"stream_id":"default",
"value":190733160
}
],
"failed":[
],
"process_ms_avg":[
{
"component_id":"split",
"stream_id":"default",
"value":"0.004"
}
],
"executed":[
{
"component_id":"split",
"stream_id":"default",
"value":190733140
}
],
"executed_ms_avg":[
{
"component_id":"split",
"stream_id":"default",
"value":"0.005"
}
]
},
{
"id":"split",
"emitted":[
{
"stream_id":"__metrics",
"value":60
},
{
"stream_id":"default",
"value":190754740
},
{
"stream_id":"__ack_ack",
"value":17317580
},
{
"stream_id":"__system",
"value":20
}
],
"transferred":[
{
"stream_id":"__metrics",
"value":60
},
{
"stream_id":"default",
"value":190754740
},
{
"stream_id":"__ack_ack",
"value":17317580
},
{
"stream_id":"__system",
"value":0
}
],
"acked":[
{
"component_id":"spout",
"stream_id":"default",
"value":17339180
}
],
"failed":[
],
"process_ms_avg":[
{
"component_id":"spout",
"stream_id":"default",
"value":"0.051"
}
],
"executed":[
{
"component_id":"spout",
"stream_id":"default",
"value":17339240
}
],
"executed_ms_avg":[
{
"component_id":"spout",
"stream_id":"default",
"value":"0.052"
}
]
}
]
}
/api/v1/topology/:id/component/:component (GET)
Returns detailed metrics and executor information
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| component | String (required) | Component Id |
| window | String. Default value :all-time | window duration for metrics in seconds |
| sys | String. Values 1 or 0. Default value 0 | controls including sys stats part of the response |
Response fields:
| Field | Value | Description |
|---|---|---|
| user | String | Topology owner |
| id | String | Component id |
| encodedId | String | URL encoded component id |
| name | String | Topology name |
| executors | Integer | Number of executor tasks in the component |
| tasks | Integer | Number of instances of component |
| requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) |
| requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) |
| requestedCpu | Double | Requested CPU by User (%) |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
| topologyId | String | Topology id |
| topologyStatus | String | Topology status |
| encodedTopologyId | String | URL encoded topology id |
| window | String. Default value "All Time" | window duration for metrics in seconds |
| componentType | String | component type: SPOUT or BOLT |
| windowHint | String | window param value in "hh mm ss" format. Default value is "All Time" |
| debug | Boolean | If debug is enabled for the component |
| samplingPct | Double | Controls downsampling of events before they are sent to event log (percentage) |
| eventLogLink | String | URL viewer link to event log (debug mode) |
| profilingAndDebuggingCapable | Boolean | true if there is support for Profiling and Debugging Actions |
| profileActionEnabled | Boolean | true if worker profiling (Java Flight Recorder) is enabled |
| profilerActive | Array | Array of currently active Profiler Actions |
| componentErrors | Array of Errors | List of component errors |
| componentErrors.errorTime | Long | Timestamp when the exception occurred (Prior to 0.11.0, this field was named 'time'.) |
| componentErrors.errorHost | String | host name for the error |
| componentErrors.errorPort | String | port for the error |
| componentErrors.error | String | Shows the error happened in a component |
| componentErrors.errorLapsedSecs | Integer | Number of seconds elapsed since the error happened in a component |
| componentErrors.errorWorkerLogLink | String | Link to the worker log that reported the exception |
| spoutSummary | Array | (only for spouts) Array of component stats, one element per window. |
| spoutSummary.windowPretty | String | Duration passed in HH:MM:SS format |
| spoutSummary.window | String | window duration for metrics in seconds |
| spoutSummary.emitted | Long | Number of messages emitted in given window |
| spoutSummary.completeLatency | String (double value returned in String format) | Total latency for processing the message |
| spoutSummary.transferred | Long | Total number of messages transferred in given window |
| spoutSummary.acked | Long | Number of messages acked |
| spoutSummary.failed | Long | Number of messages failed |
| boltStats | Array | (only for bolts) Array of component stats, one element per window. |
| boltStats.windowPretty | String | Duration passed in HH:MM:SS format |
| boltStats.window | String | window duration for metrics in seconds |
| boltStats.transferred | Long | Total number of messages transferred in given window |
| boltStats.processLatency | String (double value returned in String format) | Average time of the bolt to ack a message after it was received |
| boltStats.acked | Long | Number of messages acked |
| boltStats.failed | Long | Number of messages failed |
| inputStats | Array | (only for bolts) Array of input stats |
| inputStats.component | String | Component id |
| inputStats.encodedComponentId | String | URL encoded component id |
| inputStats.executeLatency | Long | The average time a tuple spends in the execute method |
| inputStats.processLatency | Long | The average time it takes to ack a tuple after it is first received |
| inputStats.executed | Long | The number of incoming tuples processed |
| inputStats.acked | Long | Number of messages acked |
| inputStats.failed | Long | Number of messages failed |
| inputStats.stream | String | The name of the tuple stream given in the topology, or "default" if none specified |
| outputStats | Array | Array of output stats |
| outputStats.transferred | Long | Number of tuples emitted that sent to one ore more bolts |
| outputStats.emitted | Long | Number of tuples emitted |
| outputStats.stream | String | The name of the tuple stream given in the topology, or "default" if none specified |
Examples:
1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout
2. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?sys=1
3. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?window=600
Sample response:
{
"name": "WordCount3",
"id": "spout",
"componentType": "spout",
"windowHint": "10m 0s",
"executors": 5,
"componentErrors":[{"errorTime": 1406006074000,
"errorHost": "10.11.1.70",
"errorPort": 6701,
"errorWorkerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log",
"errorLapsedSecs": 16,
"error": "java.lang.RuntimeException: java.lang.StringIndexOutOfBoundsException: Some Error\n\tat org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)\n\tat org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)\n\tat org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)\n\tat backtype...more.."
}],
"topologyId": "WordCount3-1-1402960825",
"tasks": 5,
"window": "600",
"profilerActive": [
{
"host": "10.11.1.70",
"port": "6701",
"dumplink":"http:\/\/10.11.1.70:8000\/dumps\/ex-1-1452718803\/10.11.1.70%3A6701",
"timestamp":"576328"
}
],
"profilingAndDebuggingCapable": true,
"profileActionEnabled": true,
"spoutSummary": [
{
"windowPretty": "10m 0s",
"window": "600",
"emitted": 28500,
"transferred": 28460,
"completeLatency": "0.000",
"acked": 0,
"failed": 0
},
{
"windowPretty": "3h 0m 0s",
"window": "10800",
"emitted": 127640,
"transferred": 127440,
"completeLatency": "0.000",
"acked": 0,
"failed": 0
},
{
"windowPretty": "1d 0h 0m 0s",
"window": "86400",
"emitted": 127640,
"transferred": 127440,
"completeLatency": "0.000",
"acked": 0,
"failed": 0
},
{
"windowPretty": "All time",
"window": ":all-time",
"emitted": 127640,
"transferred": 127440,
"completeLatency": "0.000",
"acked": 0,
"failed": 0
}
],
"outputStats": [
{
"stream": "__metrics",
"emitted": 40,
"transferred": 0,
"completeLatency": "0",
"acked": 0,
"failed": 0
},
{
"stream": "default",
"emitted": 28460,
"transferred": 28460,
"completeLatency": "0",
"acked": 0,
"failed": 0
}
],
"executorStats": [
{
"workerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log",
"emitted": 5720,
"port": 6701,
"completeLatency": "0.000",
"transferred": 5720,
"host": "10.11.1.7",
"acked": 0,
"uptime": "43m 4s",
"uptimeSeconds": 2584,
"id": "[24-24]",
"failed": 0
},
{
"workerLogLink": "http://10.11.1.7:8000/log?file=worker-6703.log",
"emitted": 5700,
"port": 6703,
"completeLatency": "0.000",
"transferred": 5700,
"host": "10.11.1.7",
"acked": 0,
"uptime": "42m 57s",
"uptimeSeconds": 2577,
"id": "[25-25]",
"failed": 0
},
{
"workerLogLink": "http://10.11.1.7:8000/log?file=worker-6702.log",
"emitted": 5700,
"port": 6702,
"completeLatency": "0.000",
"transferred": 5680,
"host": "10.11.1.7",
"acked": 0,
"uptime": "42m 57s",
"uptimeSeconds": 2577,
"id": "[26-26]",
"failed": 0
},
{
"workerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log",
"emitted": 5700,
"port": 6701,
"completeLatency": "0.000",
"transferred": 5680,
"host": "10.11.1.7",
"acked": 0,
"uptime": "43m 4s",
"uptimeSeconds": 2584,
"id": "[27-27]",
"failed": 0
},
{
"workerLogLink": "http://10.11.1.7:8000/log?file=worker-6703.log",
"emitted": 5680,
"port": 6703,
"completeLatency": "0.000",
"transferred": 5680,
"host": "10.11.1.7",
"acked": 0,
"uptime": "42m 57s",
"uptimeSeconds": 2577,
"id": "[28-28]",
"failed": 0
}
]
}
Profiling and Debugging GET Operations
/api/v1/topology/:id/profiling/start/:host-port/:timeout (GET)
Request to start profiler on worker with timeout. Returns status and link to profiler artifacts for worker.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
| timeout | String (required) | Time out for profiler to stop in minutes |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
| timeout | String | Requested timeout |
| dumplink | String | Link to logviewer URL for worker profiler documents. |
Examples:
1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/10
2. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/5
3. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/20
Sample response:
{
"status": "ok",
"id": "10.11.1.7:6701",
"timeout": "10",
"dumplink": "http:\/\/10.11.1.7:8000\/dumps\/wordcount-1-1446614150\/10.11.1.7%3A6701"
}
/api/v1/topology/:id/profiling/dumpprofile/:host-port (GET)
Request to dump profiler recording on worker. Returns status and worker id for the request.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
Examples:
1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpprofile/10.11.1.7:6701
Sample response:
{
"status": "ok",
"id": "10.11.1.7:6701",
}
/api/v1/topology/:id/profiling/stop/:host-port (GET)
Request to stop profiler on worker. Returns status and worker id for the request.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
Examples:
1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/stop/10.11.1.7:6701
Sample response:
{
"status": "ok",
"id": "10.11.1.7:6701",
}
/api/v1/topology/:id/profiling/dumpjstack/:host-port (GET)
Request to dump jstack on worker. Returns status and worker id for the request.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
Examples:
1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpjstack/10.11.1.7:6701
Sample response:
{
"status": "ok",
"id": "10.11.1.7:6701",
}
/api/v1/topology/:id/profiling/dumpheap/:host-port (GET)
Request to dump heap (jmap) on worker. Returns status and worker id for the request.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
Examples:
1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpheap/10.11.1.7:6701
Sample response:
{
"status": "ok",
"id": "10.11.1.7:6701",
}
/api/v1/topology/:id/profiling/restartworker/:host-port (GET)
Request to request the worker. Returns status and worker id for the request.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
Examples:
1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/restartworker/10.11.1.7:6701
Sample response:
{
"status": "ok",
"id": "10.11.1.7:6701",
}
POST Operations
/api/v1/topology/:id/activate (POST)
Activates a topology.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
Sample Response:
{"topologyOperation":"activate","topologyId":"wordcount-1-1420308665","status":"success"}
/api/v1/topology/:id/deactivate (POST)
Deactivates a topology.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
Sample Response:
{"topologyOperation":"deactivate","topologyId":"wordcount-1-1420308665","status":"success"}
/api/v1/topology/:id/rebalance/:wait-time (POST)
Rebalances a topology.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| wait-time | String (required) | Wait time before rebalance happens |
| rebalanceOptions | Json (optional) | topology rebalance options |
Sample rebalanceOptions json:
{"rebalanceOptions" : {"numWorkers" : 2, "executors" : {"spout" :4, "count" : 10}}, "callback" : "foo"}
Examples:
curl -i -b ~/cookiejar.txt -c ~/cookiejar.txt -X POST
-H "Content-Type: application/json"
-d '{"rebalanceOptions": {"numWorkers": 2, "executors": { "spout" : "5", "split": 7, "count": 5 }}, "callback":"foo"}'
http://localhost:8080/api/v1/topology/wordcount-1-1420308665/rebalance/0
Sample Response:
{"topologyOperation":"rebalance","topologyId":"wordcount-1-1420308665","status":"success"}
/api/v1/topology/:id/kill/:wait-time (POST)
Kills a topology.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| wait-time | String (required) | Wait time before rebalance happens |
Caution: Small wait times (0-5 seconds) may increase the probability of triggering the bug reported in STORM-112, which may result in broker Supervisor daemons.
Sample Response:
{"topologyOperation":"kill","topologyId":"wordcount-1-1420308665","status":"success"}
API errors
The API returns 500 HTTP status codes in case of any errors.
Sample response:
{
"error": "Internal Server Error",
"errorMessage": "java.lang.NullPointerException\n\tat clojure.core$name.invoke(core.clj:1505)\n\tat org.apache.storm.ui.core$component_page.invoke(core.clj:752)\n\tat org.apache.storm.ui.core$fn__7766.invoke(core.clj:782)\n\tat compojure.core$make_route$fn__5755.invoke(core.clj:93)\n\tat compojure.core$if_route$fn__5743.invoke(core.clj:39)\n\tat compojure.core$if_method$fn__5736.invoke(core.clj:24)\n\tat compojure.core$routing$fn__5761.invoke(core.clj:106)\n\tat clojure.core$some.invoke(core.clj:2443)\n\tat compojure.core$routing.doInvoke(core.clj:106)\n\tat clojure.lang.RestFn.applyTo(RestFn.java:139)\n\tat clojure.core$apply.invoke(core.clj:619)\n\tat compojure.core$routes$fn__5765.invoke(core.clj:111)\n\tat ring.middleware.reload$wrap_reload$fn__6880.invoke(reload.clj:14)\n\tat org.apache.storm.ui.core$catch_errors$fn__7800.invoke(core.clj:836)\n\tat ring.middleware.keyword_params$wrap_keyword_params$fn__6319.invoke(keyword_params.clj:27)\n\tat ring.middleware.nested_params$wrap_nested_params$fn__6358.invoke(nested_params.clj:65)\n\tat ring.middleware.params$wrap_params$fn__6291.invoke(params.clj:55)\n\tat ring.middleware.multipart_params$wrap_multipart_params$fn__6386.invoke(multipart_params.clj:103)\n\tat ring.middleware.flash$wrap_flash$fn__6675.invoke(flash.clj:14)\n\tat ring.middleware.session$wrap_session$fn__6664.invoke(session.clj:43)\n\tat ring.middleware.cookies$wrap_cookies$fn__6595.invoke(cookies.clj:160)\n\tat ring.adapter.jetty$proxy_handler$fn__6112.invoke(jetty.clj:16)\n\tat ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown Source)\n\tat org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)\n\tat org.mortbay.jetty.Server.handle(Server.java:326)\n\tat org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)\n\tat org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)\n\tat org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)\n\tat org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)\n\tat org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)\n\tat org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)\n\tat org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)\n"
}
书籍推荐
