# Config

This is the reference of the agola config file format.

# Config file formats

The accepted syntaxes are (checked in this order):

  • jsonnet (.agola/config.jsonnet)
  • yaml (.agola/config.yml)
  • json (.agola/config.json)

The config file is quite simple with just some syntactic sugar for easier definitions of some task steps.

For basic configs you would prefer to use the yml syntax but for more complex configs we suggest to use the jsonnet syntax (see the examples).

Instead of adding some templating facilities to yaml that will always leave some open unsupported cases and add much more complexity we choosed to push the use of jsonnet .

# Jsonnet config format

jsonnet helps writing a parametrizable and easy to maintain config. See the below matrix build example.

Generating a config with jsonnet keeps the principle of reproducible runs since a jsonnet file will always produce the same output.

jsonnet config can also receive a configuration context the can be used to dynamically generate the final config. Many things can also be done without this context variables but instead using When conditions and the provided agola environment variables.

To use this configuration context you should use a main jsonnet function:

function(ctx) {
	// use your context here
}

The ctx is an object that contains these fields:

Name Description
ref_type The ref type. Can be branch, tag, pull_request.
ref The git ref (i.e. refs/heads/master, refs/tags/v0.1.0).
commit_sha The pull request id of this run. Will be an empty string if the run hasn't been triggered by a git branch push event.
branch The git commit sha.
tag The git branch. Will be an empty string if the run hasn't been triggered by a git branch push event.
pull_request_id The git tag. Will be an empty string if the run hasn't been triggered by a git tag push event.

# Custom config file generation

You're also free to use you own way to generate the config file. Just commit the final result to git.

# Agola provided Environment Variables

Every agola task step will have some environment variables populated by Agola that can be useful in some tasks steps (they are also used by the clone step).

Name Description
CI Set as true. Useful for scripts that need to know if they're running inside a CI/CD system.
AGOLA_REPOSITORY_URL The remote git repository url.
AGOLA_GIT_REF_TYPE The ref type. Can be branch, tag, pull_request.
AGOLA_GIT_REF The git ref (i.e. refs/heads/master, refs/tags/v0.1.0).
AGOLA_GIT_COMMITSHA The git commit sha.
AGOLA_GIT_BRANCH The git branch. Will be an empty string if the run hasn't been triggered by a git branch push event.
AGOLA_GIT_TAG The git tag. Will be an empty string if the run hasn't been triggered by a git tag push event.
AGOLA_PULL_REQUEST_ID The pull request id of this run. Will be an empty string if the run hasn't been triggered by a git branch push event.

# Global

Option Type Description
version String Config version
runs List: Run Runs definitions

# Version

The config file version. Currently only v0.

# Runs

# Run

Option Type Description
name String Run name
tasks List: Task Run tasks
when When Conditions to match to execute this run. If the conditions aren't met then the run is skipped
docker_registries_auth Map: RegistryHost(String) => DockerRegistryAuth(DockerRegistryAuth) Docker registries authentication data

# Task

Option Type Description
name String Task name
runtime Runtime Runtime definition
steps List: Step Steps definitions
environment Map: EnvVarName(String) => EnvVarValue(Value) Environment variables to set
working_dir String The working dir where the steps will be executed (default: ~/project)
shell String Shell to use (defaults to /bin/sh -e)
user String The user id or username to use when executing the task steps
ignore_failure Boolean Don't mark the run as failed if this task is failed
approval Boolean If true the task must be approved before it can start
depends List: Depend List of task dependencies and conditions
when When Conditions to match to execute this task. If the conditions aren't met then the task is skipped
docker_registries_auth Map: RegistryHost(String) => DockerRegistryAuth(DockerRegistryAuth) Docker registries authentication data
# Depend
Option Type Description
task String Task name
conditions List: Condition List of conditions to satisfy. If no condition is satisfied the task will be skipped. Possible conditions are: on_success, on_failure, on_skipped

It's also available a short form for specifying a dependency (simpler when using yaml):

  depends:
    - task 01 # depends on task 01 with default condition on_success
    - task 02: # depends on task 02 with conditions on_failure, on_skipped
      - on_failure
      - on_skipped
# Step
Option Type Description
Type String Step type

There are different types of steps:

  • clone: clones and checkouts (to the right commit sha) code
  • run: executes e command
  • save_to_workspace: save in the run workspace the specified files/dirs
  • restore_workspace: restores the run workspace in the specified dir
  • save_cache: save some contents in a project cache key.
  • restore_cache: restores contents from a project cache.

Every step can have a short form (simpler when using yaml) or a normal form:

  • Short form:
   $stepname: $stepdefinition

Examples:

   run:
     name: "run step"
     command: "echo something"
  • Normal form:
   type: $steptype
   $otherstep fields

Examples:

   type: run
   name: "run step"
   command: "echo something"
# clone
  • Short form:
clone:

clone clones and checkouts (to the right commit sha) code.

  • Long form:
Option Type Description
depth Int Clones with --depth <n> parameter (defaults to nil)
recurse_submodules Bool Clones with --recurse-submodules parameter (defaults to false)
# run
  • Short form:
run: command

The command must be a one line command In this case the run step name will be the same of the command trimmed at the maximum name size.

  • Long form:
Option Type Description
name String Step name. Required when the command is multiline. If not provided will be the same of the command trimmed at the maximum name size
command String Command to run
environment Map: EnvVar Name(String) => EnvVar Value(String) Environment variables to set
working_dir String The working dir where the steps will be executed
shell String Shell to use (defaults to /bin/sh -e)
tty Bool Use a pseudo terminal for command execution (defaults to true)
# save_to_workspace
Option Type Description
contents List: content Contents to save in the workspace

content

Option Type Description
source_dir String Source dir where to take the files to save
dest_dir String Workspace destination dir where to save the files
paths List: String Contents to save in the workspace. Every path accepts the ** (doublestar) notation to match all the subchilds
# restore_workspace
Option Type Description
dest_dir String Destination dir where to restore the workspace
# save_cache
Option Type Description
contents List: content Contents to save in the workspace

content

Option Type Description
key String Cache key to save. It's a template to dynamically generate cache keys based on runtime data. See caching
source_dir String Source dir where to take the files to save
dest_dir String Workspace destination dir where to save the files
paths List: String Contents to save in the workspace, if empty all the files inside source_dir will be saved. Every path accepts the ** (doublestar) notation to match all the subchilds
# restore_cache
Option Type Description
keys List of String List of cache key to restore. Every entry is a template to dynamically generate cache keys based on runtime data. See caching. They'll tried in order until a match by prefix of an existing cache key is found, the newest matching key will be restored
dest_dir String Destination dir where to restore the cache

# Runtime

Option Type Description
type String Runtime Type (currently only pod is supported)
arch String Architecture (valid architectures are: 386 amd64 arm arm64)
containers List: Container A list of containers, the first container will be the one where the tasks steps will be executed. Other containers may be defined to provide services needed for the task (a database etc...)
# Container
Option Type Description
image String Image to use
environment Map: EnvVarName(String) => EnvVarValue(Value) Environment variables to set
working_dir String Working dir where the entrypoint will be executed (also used for steps in a task)
user String The user id or username to use when executing the entrypoint or the task run steps
volumes List: Volume A list of volumes (currently only tmpfs is supported)
privileged Bool Give extended privileges to this container (defaults to false). Executors config must have allowPrivilegedContainers set to true
# Volume

Containers can mount one or more volumes (currently only tmpfs is supported).

Option Type Description
path String Mountpoint
tmpfs TmpFS Volume tmpfs volume

Example of single volume with size limit:

  - image: image01
    volumes:
      - path: /mnt/tmpfs
        tmpfs:
          size: 1Gi

Example with two volumes, the former with a size limit and the latter without:

  - image: image01
    volumes:
      - path: /mnt/vol1
        tmpfs:
          size: 512Mi
      - path: /mnt/vol2
        tmpfs: {}
# TmpFS Volume
Option Type Description
size String Filesystem size

# Additional types

# When

When represent a set of conditions to match.

Option Type Description
branch String, List of String or map with keys include/exclude Match a branch with the specified conditions
tag String, List of String or map with keys include/exclude Match a tag with the specified conditions
ref String, List of String or map with keys include/exclude Match a ref with the specified conditions

By default, each option not defined is ignored. If you define an exclude instead, you have to define an include to which the excludes will be applied.

The value provided to branch/tag/ref can be different:

  • A string
  • A list of strings
  • A map with keys include and exclude and values string or list of string (like above). In this way the branch/tag/ref must be included and not excluded

The provided strings can be a simple string (the value must match that string) or a regular expression (when enclosed in / or #).

# Example

  runs:
    - name: run01
      tasks:
        - name: task01
          when:
            branch: master
            tag:
              - v1.0
              - /v1\..*/
            ref:
              include: master
              exclude: [ '#/refs/pull/.*#' , /refs/heads/branch01 ]

The task task01 will be executed only if:

  • we are on a branch with value master
  • we are on a tag with value v1.0 or that matches the regexp v1\..* (i.e v1.0, v1.2 but not v10 or v2)
  • we are on a ref that match the regexp /refs/pull/.* or the value "/refs/heads/branch01`

# Value

A value that can be defined as a string or from a project variable

# As a string

password: yoursecretpassword

# As a project variable

password:
  from_variable: yoursecretpassword

# Docker Registry Auth

Option Type Description
type String Registry authentication type: basic (default) or encodeauth only default is supported)
username Value Registry username. Can be a string or from a variable
password Value Registry password. Can be a string or from a variable
auth Value Represents and encoded auth string like the one used in the docker config.json. Can be a string or from a variable

# Examples

# yaml config

version: v0

runs:
  - name: agola example go run
    tasks:
      - name: build go 1.12
        runtime:
          type: pod
          arch: amd64
          containers:
            - image: golang:1.12-stretch
        environment:
          GO111MODULE: "on"
        steps:
          - clone:
          - restore_cache:
              keys:
                - cache-sum-{{ md5sum "go.sum" }}
                - cache-date-
              ## golang image sets GOPATH to /go
              dest_dir: /go/pkg/mod/cache
          ## This will create a binary called `agola-example-go` under ./bin
          - run:
              name: build the program
              command: go build .
          ## Copy the built binary to the workspace
          - save_to_workspace:
              contents:
                - source_dir: .
                  dest_dir: /bin/
                  paths:
                    - agola-example-go
          - save_cache:
              key: cache-sum-{{ md5sum "go.sum" }}
              contents:
                - source_dir: /go/pkg/mod/cache
          - save_cache:
              key: cache-date-{{ year }}-{{ month }}-{{ day }}
              contents:
                - source_dir: /go/pkg/mod/cache
      - name: run
        ## This task will run the built binary in the parent task and leverages the workspace capabilities
        runtime:
          type: pod
          arch: amd64
          containers:
            - image: debian:stretch
        steps:
          - restore_workspace:
              dest_dir: .
          - run: ./bin/agola-example-go
        depends:
          - build go 1.12

# jsonnet config

This is an example jsonnet config the creates a build matrix between go version and architecture.

It generates 4 "go build" with the different combinations.

local go_runtime(version, arch) = {
  type: 'pod',
  arch: arch,
  containers: [
    { image: 'golang:' + version + '-stretch' },
  ],
};

local task_build_go(version, arch) = {
  name: 'build go ' + version + ' ' + arch,
  runtime: go_runtime(version, arch),
  working_dir: '/go/src/github.com/sorintlab/agola',
  environment: {
    GO111MODULE: 'on',
  },
  steps: [
    { type: 'clone' },
    { type: 'restore_cache', keys: ['cache-sum-{{ md5sum "go.sum" }}', 'cache-date-'], dest_dir: '/go/pkg/mod/cache' },
    { type: 'run', name: 'build the program', command: 'go build .' },
    { type: 'save_to_workspace', contents: [{ source_dir: '.', dest_dir: '/bin/', paths: ['agola-example-go'] }] },
    { type: 'save_cache', key: 'cache-sum-{{ md5sum "go.sum" }}', contents: [{ source_dir: '/go/pkg/mod/cache' }] },
    { type: 'save_cache', key: 'cache-date-{{ year }}-{{ month }}-{{ day }}', contents: [{ source_dir: '/go/pkg/mod/cache' }] },
  ],
};

function(ctx)
{
  runs: [
    {
      name: 'agola build/test',
      tasks: [
        task_build_go(version, arch)
        for version in ['1.11', '1.12']
        for arch in ['amd64', 'arm64']
      ] + [
        {
          name: 'run',
          runtime: {
            type: 'pod',
            arch: 'amd64',
            containers: [
              { image: 'debian:stretch' },
            ],
          },
          steps: [
            { type: 'restore_workspace', dest_dir: '.' },
            { type: 'run', command: './bin/agola-example-go' },
          ],
          depends: [
            'build go 1.12 amd64',
          ],
        },
      ],
    },
  ],
}