Executors

The executor field provides different execution methods for each step. These executors are responsible for executing the commands or scripts specified in the command or script field of the step. Below are the available executors and their use cases.

In the examples directory, you can find a collection of sample DAGs that demonstrate how to use executors.

Running Docker Containers

Note: It requires Docker daemon running on the host.

The docker executor allows us to run Docker containers instead of bare commands. This can be useful for running commands in isolated environments or for reproducibility purposes.

In the example below, it pulls and runs Deno’s docker image and prints ‘Hello World’.

steps:
  - name: deno_hello_world
    executor:
      type: docker
      config:
        image: "denoland/deno:1.10.3"
        autoRemove: true
    command: run https://examples.deno.land/hello-world.ts

Example Log output:

Configuring Container Volumes, Environment Variables, and More

You can config the Docker container (e.g., volumes, env, etc) by passing more detailed options.

For example:

steps:
  - name: deno_hello_world
    executor:
      type: docker
      config:
        image: "denoland/deno:1.10.3"
        container:
          volumes:
            /app:/app:
          env:
            - FOO=BAR
        host:
          autoRemove: true
    command: run https://examples.deno.land/hello-world.ts

See the Docker’s API documentation for all available options.

• For container, see ContainerConfig.

• For host, see HostConfig.

Running Containers on the Host’s Docker Environment

If you are running blackdagger using a container, you need the setup below.

1. Run a socat conainer with the command below.

docker run -v /var/run/docker.sock:/var/run/docker.sock -p 2376:2375 bobrik/socat TCP4-LISTEN:2375,fork,reuseaddr UNIX-CONNECT:/var/run/docker.sock

2. Then you can set the DOCKER_HOST environment as follows.

env:
  - DOCKER_HOST : "tcp://host.docker.internal:2376"
steps:
  - name: deno_hello_world
    executor:
      type: docker
      config:
        image: "denoland/deno:1.10.3"
        autoRemove: true
    command: run https://examples.deno.land/hello-world.ts

For more details, see this page.

Making HTTP Requests

The http executor allows us to make an arbitrary HTTP request. This can be useful for interacting with web services or APIs.

steps:
  - name: send POST request
    command: POST https://foo.bar.com
    executor:
      type: http
      config:
        timeout: 10,
        headers:
          Authorization: "Bearer $TOKEN"
        silent: true # If silent is true, it outputs response body only.
        query:
          key: "value"
        body: "post body"

Sending Email

The mail executor can be used to send email. This can be useful for sending notifications or alerts.

Example:

smtp:
  host: "smtp.foo.bar"
  port: "587"
  username: "<username>"
  password: "<password>"

params: RECIPIENT=XXX

steps:
  - name: step1
    executor:
      type: mail
      config:
        to: <to address>
        from: <from address>
        subject: "Exciting New Features Now Available"
        message: |
          Hello [RECIPIENT],

          We hope you're enjoying your experience with MyApp!
          We're thrilled to announce that [] v2.0 is now available,
          and we've added some fantastic new features based on your
          valuable feedback.

          Thank you for choosing MyApp and for your continued support.
          We look forward to hearing from you and providing you with
          an even better MyApp experience.

          Best regards,

Executing jq Command

The jq executor can be used to transform, query, and format JSON. This can be useful for working with JSON data in pipelines or for data processing.

Query Example

steps:
  - name: run query
    executor: jq
    command: '{(.id): .["10"].b}'
    script: |
      {"id": "sample", "10": {"b": 42}}

Output:

{
    "sample": 42
}

Formatting JSON

steps:
  - name: format json
    executor: jq
    script: |
      {"id": "sample", "10": {"b": 42}}

Output:

{
    "10": {
        "b": 42
    },
    "id": "sample"
}

Command Execution over SSH

The ssh executor allows us to execute commands on remote hosts over SSH. Instead of key, you can use password field and enter your password of your remote host.

steps:
  - name: step1
    executor:
      type: ssh
      config:
        user: blackdagger
        ip: XXX.XXX.XXX.XXX
        port: 22
        key: /Users/blackdagger/.ssh/private.pem
    command: /usr/sbin/ifconfig

To run multiple commands at the same time, use the example below:

steps:
  - name: step1
    executor:
      type: ssh
      config:
        user: blackdagger
        ip: XXX.XXX.XXX.XXX
        port: 22
        key: /Users/blackdagger/.ssh/private.pem
    script: |
      echo "Executing multiple commands:"
      cp /etc/resolv.conf /tmp/x
      cat /tmp/x