当前位置: 代码迷 >> 综合 >> Dr.Elephant Rest API -【Dr.Elephant系列文章-7】
  详细解决方案

Dr.Elephant Rest API -【Dr.Elephant系列文章-7】

热度:99   发布时间:2023-12-07 22:49:11.0

有的情况下,用户可能不能访问Dr.Elephant的UI界面,但是需要和Dr.Elephant进行交互来获得Dr.Elephant的一些分析结果。Dr.Elephant提供了Rest API来满足用户的这个需求。

根据ID获得应用信息

根据任务的ID,可以获得任务的详细信息以及启发式算法的分析结果。

URI

http://<dr-elephant-host:port>/rest/job

请求参数

parameter

description

id

The app or job id to search

返回体

parameter

description

id

The application id

name

The name of the application

username

User who submitted the application

queueName

The queue the application was submitted to

startTime

The time in which application started

finishTime

The time in which application finished

trackingUrl

The Job history page of the app

jobType

Type of the job e.g. pig

severity

Aggregate severity of all the heuristics. Ranges from 0(LOW) to 4(CRITICAL)

score

The application score which is the sum of heuristic scores

workFlowDepth

The application depth in the scheduled flow. Depth starts from 0

scheduler

The scheduler which triggered the application

jobName

The name of the job in the flow to which this app belongs

jobExecId

A unique reference to a specific execution of the job/action(job in the workflow).

flowExecId

A unique reference to a specific flow execution.

jobDefId

A unique reference to the job in the entire flow independent of the execution.

flowDefId

A unique reference to the entire flow independent of any execution.

jobExecUrl

A url to the job execution on the schedule

flowExecUrl

A url to the flow execution on the scheduler

jobDefUrl

A url to the job definition on the scheduler

flowDefUrl

A url to the flow definition on the scheduler

yarnAppHeuristicResults

Detailed results on individual heuristics

根据执行ID获得应用信息

根据任务的执行ID或者Azkaban的执行URL,可以获得所有的任务信息。

URI

http://<dr-elephant-host:port>/rest/jobexec

请求参数

parameter

description

id

The job execution id or job execution url(azkaban) to search

返回结果

返回结果包含了这个任务的所有子任务和作业。这个返回结果序列表格可以参考上面的“根据ID获得应用信息”部分

根据任务流ID来获取应用

根据任务流的ID或者Azkaban的任务执行url可以获得任务流的详细信息。

URI

http://<dr-elephant-host:port>/rest/flowexec

请求参数

parameter

description

id

The flow execution id or flow execution url(azkaban) to search

返回结果

返回结果包含了这个任务流的所有子任务和作业。这个返回结果序列表格可以参考上面的“根据ID获得应用信息”部分

搜索任务

根据一定的参数可以搜索任务和任务流。Rest API支持通过所有的参数来进行搜索。

URI

http://<dr-elephant-host:port>/rest/search

请求参数

parameter

description

id

The job/app execution id to search. If this is provided, no other parameter is valid

flow-exec-id

The flow execution id or url(azkaban) to search, if this is provided, no other parameter is valid

username

To filter jobs by the user

severity

To filter all jobs based on severity

job-type

The type of jobs to search

analysis

The heuristic name. This goes together with severity.

finished-time-begin

The start range of finish time. (Milliseconds since epoch)

finished-time-end

The end range of finish time. (Milliseconds since epoch)

started-time-begin

The start range of start time. (Milliseconds since epoch)

started-time-end

The end range of start time. (Milliseconds since epoch)


返回结果

返回结果包含了这个任务流的所有任务。这个返回结果序列表格可以参考上面的“根据ID获得应用信息”部分

比较任务的执行

给定两个任务的ID,会返回两个任务比较的结果

URI

http://<dr-elephant-host:port>/rest/compare

请求参数

parameter

description

flow-exec-id1

Flow execution id or url(azkaban) of first flow

flow-exec-id2

Flow execution id or url(azkaban) of second flow


返回结果

结果的格式如下:

job_def_id1: {

    flow_exec_id1 : {

        app1_details,

        app2_details

    },

    flow_exec_id2 : {

        app1_details,

        app2_details

    }

},

job_def_id2: {

    flow_exec_id1 : {

        app1_details,

        app2_details,

        app3_details

    },

    flow_exec_id2 : {

        app1_details,

        app2_details

        app3_details

    }

}

}

app1_details的结果是应用的详细返回结果,这个返回结果序列表格可以参考上面的“根据ID获得应用信息”部分


作者简介:屈世超,对高并发系统设计开发感兴趣,现专注于大数据开发工作。曾任职小米科技公司服务端后台开发工程师,现担任EverString数据平台组高级开发工程师。