Commit Graph

15 Commits

Author SHA1 Message Date
Lukas aad03dc285
fix: getUIDandGID is able to resolve non-existing users and groups (#2106)
* fix: getUIDandGID is able to resolve non-existing users and groups

A common pattern in dockerfiles is to provide a plain uid and gid number, which doesn't neccesarily exist inside the os.

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* test: add chown dockerfile

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore: format

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore: add comment

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* tests: fix chown dockerfile

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* refactor: split up getIdsFromUsernameAndGroup func

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix: implement raw uid logic for LookupUser

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* test: add dockerfiles for integration test

* fix: lookup user error message

* test: add dockerfiles for non-existing user testcase

* fix: forgot error check

* tests: fix syscall credentials test

* chore: add debug output for copy command

* tests: set specific gid for integration dockerfile

* tests: fix syscall credentials test

github runner had the exact uid that i was testing on, so the groups were not empty

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* tests: fix test script

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore: apply golangci lint checks

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix: reset file ownership in createFile if not root owned

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore: logrus.Debugf missed format variable

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore(test-script): remove go html coverage

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* test(k8s): increase wait timeout

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
2022-07-12 10:21:37 -04:00
Lukas 8710ce3311
fix(Dockerfile): use temporary busybox mount to create /kaniko directory (#2155)
* ci(minikube): always install latest binary and run as root

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore: remove docker unmask because minikube handles this

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* debug: view run folder and user ids

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* debug: log minikube if start fails

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* debug: start cri-docker before starting minikube cluster

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* Revert "debug: start cri-docker before starting minikube cluster"

This reverts commit 8923941358.

* debug: include verbose minikube output

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix(minikube): cri-dockerd move command

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore(minikube): copy minikube profile from root to user

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix(minikube): change dirs to $HOME directory in config.json

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix(minikube): enable registry on start

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore: run as non root

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* cleanup

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* docs(minikube-script): update

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* deploy: create kaniko.tar

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* deploy: create kaniko folder in dockerfiles with 777 permissions via mount

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore(Dockerfile): remove ARG

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* docs(Dockerfile): add comment to kaniko dir creation

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix(Dockerfile): mount whole busybox image

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix(Dockerfile): use musl busybox

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore(Dockerfile): use musl busybox in debug image

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* typo

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
2022-07-08 09:34:31 -04:00
Lukas 4338275220
ci(setup-minikube): use cri-dockerd (#2149)
* fix(setup-minikube): use cri-dockerd

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix: add fix-broken flag to apt

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix: use focal deb file

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* chore: rerun apt-get update before installing

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix: install containerd with apt

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix: install runc with apt

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* debug: systemd docker service masked

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix: unmask docker service

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix: use binary and copy systemd unit files from repo

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix: remove port-forward of registry

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
2022-06-27 09:50:16 -04:00
Lukas 679c71c907
refactor: simpler local integration tests (#2110)
* tests(integration): add fake gcs server and use sdk instead of gsutil

* tests(integration): add flag to run tests for a subset of dockerfiles

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* tests(integration): conditionally create gcs client

* refactor: create package for gcs bucket interaction

* tests(integration): use util.Tar for integration tarball creation

* refactor: create TarballOfDirectory func

* chore: add dockerignore for faster builds

* docs: add docs for dockerfile pattern

* fix: issue during personal review

* chore: cleanup

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix(integration-tests): remove default bucket

Signed-off-by: Lukas Hoehl <ludi.origin@gmail.com>
2022-06-14 13:38:01 -04:00
Jason Hall 59c2d2d08f
Collapse integration test workflows into one config (#1855)
* Collapse integration test workflows into one config

Remove last traces of Travis, including env vars and `make travis-setup`
and badges in README

* fix lint

* make integration tests pass on PRs
2021-12-22 20:04:40 -05:00
Jason Hall b1b6962726
Share the Go build cache when building in Dockerfiles (#1853)
On my machine this makes image rebuilds go from ~5m to 1.5s.

This also required setting DOCKER_BUILDKIT=1 in integration test image
builds.

I also took the opportunity to bump the Go version used in those tests
to Go 1.17 to match the image build processes, and tidied up the
workflow files a bit too (renaming for consistency, typos, whitespace,
etc.)
2021-12-22 12:43:12 -05:00
Matt Moore 49f0154b3f
Fix the e2e K8s test (#1842)
* Debugging the integration testing.

It seems to be consistently failing (also fix a typo in the file/leg name).

I'm disabling `-v` for `go test` because it interleaves the test output in ways that make it painful to read a single failing test's output when `t.Parallel` is involved.

* Try swapping registry:2 for the registry add-on

* Drop logf and make Logger private
2021-12-17 14:43:38 -05:00
wwade 82fc94d930
git: accept explicit commit hash for git context (#1765)
* git: accept explicit commit hash for git context

When checking out code from non-github repositories, the typical
assumptions may not be valid, e.g. that the only interesting
non-branch commits have ref names starting with refs/pull. A specific
example is fetching an un-merged commit from a gerrit repository by
commit hash.

This change just looks at the second part of the git context path and
checks if it's a SHA commit hash, and if so, will fetch and check out
this commit after cloning the repository.

Sample context argument:

    https://github.repo/project#e1772f228e06d15facdf175e5385e265b57068c0

* ci: fix test script to recognize any non-zero exit as an error

hack/linter.sh didn't properly install golangci-lint in hack/bin as I
already have another version of golangci-lint on my PATH, but then it
failed to execute because it was looking for it specifically in
hack/bin.

When the executable is not found, the exit code is 127 instead of 1,
and so test.sh ignored the error.

Two fixes:

1. `test.sh`:
  - Use `if (script) ...` instead of assigning / checking a result
    variable to determine if each validation script passed or failed.

2. `hack/linter.sh`:
  - Instead of checking for golangci-lint on the path, just
    specifically check for an executable file (`test -x`) in the
    expected location.

Co-authored-by: Wade Carpenter <wwade@users.noreply.github.com>
2021-10-19 19:27:15 -07:00
Tejal Desai 1da17b6e9b
chore: add workflows for pr tests (#1766)
* chore: add workflows for pr tests

* fix unit tests

* fix formatting

* chore: fix gobuild

* change minikube script

* chore: fix lint install script

* chore: ignore and fix tests

* fix lint and run gofmt

* lint fixes

* k8s executor image only

* fix Makefile

* fix travis env variables

* more info on k8s tests

* fix travis run

* fix

* fix

* fix

* fix log

* some more changes

* increase timeout

* delete travis.yml and fix multiple copy tests

* fix registry mirror

* fix lint

* add concurency

* last attemot to fix k8 integrations

* diff id for diff workflows
2021-10-18 23:22:44 -07:00
Tejal Desai ffd35dbe12
Mutli-arch support (#1531)
* initial commit

* remove bazel jobs

* fix arch

* more fixes after testing and code review comments

* fix build platform

* add individual cloudbuild.yaml as its taking 45 mins for a cloud build trigger

* add buildx plugin

* add more debugging

* update busybox version to fix CVE-2018-1000500

* fix

* lint + more debug

* fix

* fix

* fix

* remove images from cloudbiuld

* move CI job back to docker

* one more fix

* lets see

* bring it back

* move CI job back to docker

* remove aerg from top

* live restart config

* remove live restore as minikube setup failed

* add --force-systemd

* add --force-systemd and docker driver none

* change the --run flag

* docker info and some logs removed

* fix docker command

* upgrade version for buildx to 0.5.1

* remove docker service from travis.yml and add systemd cgroup config

* move the docker config up

* move them back to docker build

* fix

* fix all dockerfiles

* fix warmer

* fix

* rm bazel jobs

* add more logs

* fix debug

Co-authored-by: ankitm123 <ankitmohapatra123@gmail.com>
2021-01-25 17:49:19 -08:00
Tejal Desai f8bf02ae3c rm benchmark 2020-06-06 20:43:35 -07:00
Tejal Desai 5d013626fc benchmark project added 2020-05-25 21:30:07 -07:00
Tejal Desai 54c2a7abf0
Add conntrack to minikube setup script. 2020-03-27 10:03:37 -07:00
Sam Stoelinga 882793b755 Add K8s integration tests
- Use minikube for deploying a lightweight K8s on Travis CI
- Build and push Kaniko image to local docker registry
- Build dockerfiles with context on both docker and K8s and check
  the diff between the 2.
2020-03-11 11:49:22 -07:00
Sam Stoelinga cdc8c60c52 Move testing scripts to scripts directory
Since we recently started adding more scripts it makes sense to move to
a separate directory. This aligns with a common practice across golang
projects: https://github.com/golang-standards/project-layout
2020-03-07 13:59:57 -08:00