diff --git a/.gitignore b/.gitignore index 9b1838b..65aa277 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,3 @@ README.pdf jenkins_jobs.ini - +INSTALL.pdf diff --git a/INSTALL.md b/INSTALL.md new file mode 100644 index 0000000..74549ac --- /dev/null +++ b/INSTALL.md @@ -0,0 +1,65 @@ +# Install Jenkins jobs builder + +You can either make a virtualenv and install jjb under that environment or just + +`pip install --user jenkins-jobs-builder` + +Then, clone this repo and edit `jenkins_jobs.ini`, which contains all the information +about your Jenkins instance. + +# Adding a new project + +Add a yaml file for the project under `jobs/projects/example.yaml`. +The project file might look like this: + +```yaml +- project: + name: zeus + repo: https://github.com/grnet/zeus + staging-env: https://zeus-testing.grnet.gr/zeus/ + domain: zeus-testing.grnet.gr + production-env: https://zeus.grnet.gr/apella/ + jobs: + - 'static-tools': + bandit_extra_args: '-s B101,B102' + - 'dynamic-tools': + nmap_extra_args: '-sV --reason --script=http-security-headers' +``` + +# Adding a new tool + +Add a new folder under `tools/example-tool/` containing all files needed +to build the docker image with the entrypoint being the tool itself. +After that, you need to push it to a docker registry, accesible by all jenkins +slaves, since jobs might run in any of them. + +Then add a job template for the tool under `jobs/tools-templates/`. This job template +might look like this: + +```yaml +- job-template: + name: '{name}-nmap' + description: "Nmap scanner" + concurrent: true + domain: + builders: + - shell: | + #!/bin/bash -e + docker pull linosgian/nmap + docker run --rm linosgian/nmap + triggers: + - timed: '@hourly' + +``` + +After that, you can add it to any of the `dynamic` or `static` +job groups and then it will automatically run for all projects. + +Lastly, you need to update Jenkins' state by generating all new Jenkins jobs +(and update the existing ones). + +All you have to do is: +`jenkins-jobs --conf jenkins_jobs.ini update jobs/projects/:jobs/tools-templates` + +Or if you want to delete the old jobs: +`jenkins-jobs --conf jenkins_jobs.ini update jobs/projects/:jobs/tools-templates --delete-old` diff --git a/README.md b/README.md index 1a10988..f63b7fc 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,51 @@ # Security-tools This repo holds all containerized security tools that we want to run against our web applications + +## Jenkins Jobs Builder + +### Description + +Jenkins jobs builder or JJB is an [Openstack](https://www.openstack.org/) tool that takes +simple descriptions of Jenkins jobs in YAML or JSON and uses them to configure Jenkins. +This helps keeping jobs in a human readable format in a version control system, and to make changes and auditing easier. + +### Project structure + +Under *jobs/* you can find the *projects* and *tools-templates*. + +**projects** contains all active projects that we currently audit regularly, and so we run several tools against. + +**tools-templates** contains all tools' templates and two important *job-groups*: + +- The first job-group is the *dynamic-tools* group that contains all dynamic analysis tools, e.g. nmap, ZAP +- The second job-group is the *static-tools* group that contains all static analysis tools, e.g. bandit + +All other tools templates are tool specific, for example, the template for nmap follows: +```yaml +- job-template: + name: '{name}-nmap' + description: "Nmap scanner" + concurrent: true + domain: + builders: + - shell: | + #!/bin/bash -e + docker pull linosgian/nmap + mkdir -p /tmp/nmap + docker run --rm linosgian/nmap {domain} {nmap_extra_args} | tee /tmp/nmap/{name} + triggers: + - timed: '@hourly' +``` + +The above sets several default value for name, description etc. Next we declare the "builder", which is +how we will run the nmap inside the container. And lastly, how often will the job be triggered. In this +example we run it ~hourly~. + + +## Tools + +Under *tools/* you will find every containerized tool that is available for use in the +jenkinks jobs builder (jjb) templates. Hence in the example above, we pull *linosgian/nmap* from +docker's public repository, this docker image generated by the Dockerfile under *tools/nmap/Dockerfile* +