Commit Graph

56 Commits

Author SHA1 Message Date
Bob Du 16ed6b2428
replace Azure/azure-storage-blob-go => Azure/azure-sdk-for-go/sdk/storage/azblob (#2945)
Signed-off-by: BobDu <i@bobdu.cc>
2024-01-16 10:45:40 -08:00
Manish Giri 7bfc73c3ad
fix: Remove references to deprecated io/ioutil pkg (#2867)
* Update benchmark_test

* Updae tar.go

* Update further refs

* Commit next set of replacements

* Reverting changes in vendor folder

* Update integreation_with_context_test.go

* Update k8s_test.go

* Update remaining usages

* Replace conflicting usage of fs local variable
2023-11-28 13:02:53 -08:00
Bob Du d54caaaa02
use aws-sdk-go-v2 (#2550)
Signed-off-by: BobDu <i@bobdu.cc>
2023-06-07 14:54:30 -07:00
Aaron Prindle 0b53835c0f
fix: use debian buster to fix tests using no longer supported stretch which had broken apt-get urls (#2485) 2023-04-27 22:05:37 -04:00
Joël Pepper 14ea7c4071
Fix Integration tests (#2425)
* fix(ci): Bump golangci-lint to 1.51.1

* chore(lint): fix gofmt and goimport issues

* chore(lint): fix linter issues

- Adapted error comparison according to linter recommendation
- Disabled noctx linting for http request where canceling makes no sense
- Disabled nilerror linting where nil error is returned on purpose
- Disabled makezero linter where slice is explicitly deepcopied

* chore(ci): Update go version in tests workflows

* fix(ci): Allow boilerplate years from 2000-2099

Previously the regex only allowed the copyright notice to contain the
years 2018,2019,2020,2021, or 2022. This commit widens to regex to
20\d\d allowing any year in the range [2000-2099]

* feat(ci): Replace minikube with k3s for intregration tests

The existing setup for minikube is very complicated, replicating most of
the setup steps for a full kubernetes cluster in an only partially
supported minikube configuration (driver=none). Furthermore the existing
setup has been broken for sometime, likely, at least in part due to the
changes to CNI and CRI in recent kubernetes versions.

Since what we actually need is only a running Kubernetes cluster on the
node and access to a registry on localhost:5000, we can switch the
extremely complicated minikube setup for a lightweight cluster using
k3s. Minikube came with a default addon for running a registry on every
node, but the same is not the case for k3s, instead we make use of the
package helm controller and its HelmChart CR to deploy twuni/docker-registry.helm
and expose it on localhost using the integrated LoadBalancer controller.

* fix(test-684): pin base container version

The dockerfile for the regression test connected to issue 684 used a
rolling tag as base image, making it flaky and fail since it was
introduced.

This commit pins the base image to the digest of bionic-20200219, which,
based on the date of the commit that introduced to the dockerfile would
be the most newest ubuntu build and likely what the "rolling" tag
resolved to back then. Since this also an image from the pre-oci days of
ubuntu, this circumvents a bug in container-diff as well
(https://github.com/GoogleContainerTools/container-diff/issues/389)
2023-03-21 12:30:54 -04:00
almg80 f9aaa9fca7
(fix):Pass full context to S3 context handler (#2221)
On version 1.9.0 context fetch from S3 is broken since `url.Parse` requires full path: 90e426ba3f/pkg/util/bucket/bucket_util.go (L77)
Currently on a --context=s3://my-bucket/my-path/context.tar.gz we would pass only my-bucket/my-path/context.tar.gz which breaks context fetch

Closes #2200
2022-09-08 13:08:51 -04:00
Lukas 679c71c907
refactor: simpler local integration tests (#2110)
* tests(integration): add fake gcs server and use sdk instead of gsutil

* tests(integration): add flag to run tests for a subset of dockerfiles

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* tests(integration): conditionally create gcs client

* refactor: create package for gcs bucket interaction

* tests(integration): use util.Tar for integration tarball creation

* refactor: create TarballOfDirectory func

* chore: add dockerignore for faster builds

* docs: add docs for dockerfile pattern

* fix: issue during personal review

* chore: cleanup

Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>

* fix(integration-tests): remove default bucket

Signed-off-by: Lukas Hoehl <ludi.origin@gmail.com>
2022-06-14 13:38:01 -04:00
Jack d4cf49077a
feat: kaniko dir config option (#1997)
* remove configurables from constants

* add configurables to config init

* add kaniko dir flag

* cleanup pkg constants

* replace buildcontext constants pkg occurrences with config

* add KanikoDir to KanikoOptions

* replace executor constants pkg occurrences with config

* remove redundant KanikoDir assignment

* replace constants to config for IntermediateStagesDir

* fix imports

* add default kaniko directory into constants

* add check for kanikoDir on use

* update init to use default path constant

* update executor kanikoDir check

Co-authored-by: Jason Hall <jasonhall@redhat.com>

* alter checkKanikoDir parameter

* add TestKanikoDir func

* update error handling style

Co-authored-by: Jason Hall <jasonhall@redhat.com>

* remove shorthand flag usage from test

Co-authored-by: Jason Hall <jasonhall@redhat.com>

* add docstring to integration test

Co-authored-by: Jason Hall <jasonhall@redhat.com>

* remove shorthand flag from kaniko-dir

Co-authored-by: Jason Hall <jasonhall@redhat.com>
2022-03-31 15:10:23 -04:00
Andrei Kvapil 2ea368dde8
tar: read directly from stdin (#1728)
* tar: run directly from stdin

* export UnTar function
2021-12-26 07:09:26 -05:00
Guillaume Calmettes d2f3e896cd
Fix regression: can fetch branches and tags references without specifying commit hashes for private git repository used as context (#1823)
* [FIX] can fetch tags and branches refs without specifying commit hashes

* [ENH] more precise comment
2021-12-24 15:13:45 -05:00
ygelfand b3ec974cfc
include auth when fetching specific specs (#1796)
Signed-off-by: ygelfand <yuri@shlitz.com>
2021-12-23 11:36:19 -05:00
wwade 82fc94d930
git: accept explicit commit hash for git context (#1765)
* git: accept explicit commit hash for git context

When checking out code from non-github repositories, the typical
assumptions may not be valid, e.g. that the only interesting
non-branch commits have ref names starting with refs/pull. A specific
example is fetching an un-merged commit from a gerrit repository by
commit hash.

This change just looks at the second part of the git context path and
checks if it's a SHA commit hash, and if so, will fetch and check out
this commit after cloning the repository.

Sample context argument:

    https://github.repo/project#e1772f228e06d15facdf175e5385e265b57068c0

* ci: fix test script to recognize any non-zero exit as an error

hack/linter.sh didn't properly install golangci-lint in hack/bin as I
already have another version of golangci-lint on my PATH, but then it
failed to execute because it was looking for it specifically in
hack/bin.

When the executable is not found, the exit code is 127 instead of 1,
and so test.sh ignored the error.

Two fixes:

1. `test.sh`:
  - Use `if (script) ...` instead of assigning / checking a result
    variable to determine if each validation script passed or failed.

2. `hack/linter.sh`:
  - Instead of checking for golangci-lint on the path, just
    specifically check for an executable file (`test -x`) in the
    expected location.

Co-authored-by: Wade Carpenter <wwade@users.noreply.github.com>
2021-10-19 19:27:15 -07:00
Theofilos Papapanagiotou 24fd3b2ab0
Fetch a github pull request (#1543) 2021-01-22 11:43:39 -08:00
Fabrice 454910f613
feat: Add https tar.gz remote source for context (#1519)
* feat: Add https tar.gz remote source for context

* Add license header

* add license header

Co-authored-by: Tejal Desai <tejal29@gmail.com>
2020-12-10 22:55:02 -08:00
Tejal Desai 94b2b6b391 gofmt 2020-08-13 14:03:38 -07:00
Tejal Desai 1f456866eb
Merge branch 'master' into opts 2020-08-13 10:43:14 -07:00
Tejal Desai 1bf66ef435
Merge pull request #1153 from why-xn/master
Checkout a specific git commit
2020-08-13 10:41:26 -07:00
Carlos Alexandro Becker 008f8e890c fix: lint issues
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2020-07-29 23:32:40 -03:00
Carlos Alexandro Becker fdcc40683e fix: lint issues
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2020-07-29 23:26:08 -03:00
Carlos Alexandro Becker ed04106af4 Merge remote-tracking branch 'upstream/master' into git 2020-07-29 23:24:17 -03:00
Tejal Desai a47a78edae
Merge pull request #1319 from caarlos0/go-git
feat: upgrade go-git
2020-07-28 21:32:16 -07:00
Carlos Alexandro Becker 57c2df4d97
feat: new git flags
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2020-06-15 17:24:04 -03:00
Carlos Alexandro Becker ab42a919a5
feat: recurse submodules
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2020-06-15 12:01:25 -03:00
Carlos Alexandro Becker 7b8879c03f
feat: upgrade go-git
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2020-06-15 11:58:04 -03:00
Carlos Alexandro Becker 13e6bdf533
fix: improve tests
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2020-06-15 11:20:27 -03:00
Carlos Alexandro Becker 8322cbe3b6
feat: accept GIT_TOKEN
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2020-06-15 11:16:50 -03:00
Tejal Desai 6ab97e2b5f ADD GCB benchmark code 2020-06-05 12:46:47 -07:00
Yoriyasu Yano 6e615a8ff9 Add ability to set git auth token using environment variables
Currently the only way to set the git auth is by including it in the context
git URL. This can be problematic for certain environments such as ECS where the
command args for are stored in logs. Introduces updates to allow setting the
authentication info using the environment variables `GIT_USERNAME` and
`GIT_PASSWORD`.
2020-05-11 10:59:48 -05:00
Tejal Desai fe0500f8e6
Merge pull request #1196 from yw-liu/master
Add http support for git repository context
2020-05-03 20:53:43 -07:00
Tejal Desai 8a780be748
Merge pull request #1139 from JordanGoasdoue/allow-injecting-context-tar-gz
feat: allow injecting through stdin tar.gz on kaniko
2020-05-01 10:24:05 -07:00
yw-liu 73eb47c752 replace string literal with constant 2020-04-17 09:07:38 +08:00
yw-liu 2e90173238 fix boilerplate 2020-04-17 00:37:12 +08:00
yw-liu c9fc6b5bcf fix golint problem 2020-04-17 00:21:48 +08:00
yw-liu 0fc311a8b7 make string var as constant 2020-04-17 00:00:24 +08:00
yw-liu 7912e4c87b modify unit-test 2020-04-16 23:48:43 +08:00
yw-liu d8b8e811dd modify code format and unit-test 2020-04-16 23:44:37 +08:00
yw-liu 4f8d074e00
add unit-test 2020-04-16 22:21:09 +08:00
yw-liu 34a6ec250f
add http support for git pull
usage: set the GIT_PULL_METHOD env var to http or https for starting the container
2020-04-14 20:25:20 +08:00
Shihab Hasan 99b08059c4 Checkout a specific git commit 2020-03-23 12:12:52 +06:00
Jordan Goasdoué d08a767454 feat: allow injecting through stdin tar.gz on kaniko 2020-03-18 12:29:49 +01:00
Tejal Desai a173bf020b
Merge pull request #1115 from JordanGoasdoue/allow-build-context-from-local-targz
feat: add support of local '.tar.gz' file inside the kaniko container
2020-03-17 15:24:10 -07:00
Jordan GOASDOUE fda7ed4f74 feat: add support of local '.tar.gz' file inside the kaniko container 2020-03-12 22:51:30 +01:00
Yuheng Zhang 8eef58939b fix: credential typo 2020-03-12 14:31:15 +08:00
yangtaokm 45c43a2c89 refactory the code and add CreateTargetTarfile in fs_util.go 2019-10-22 16:44:51 +08:00
Tao Yang a86ffc8c57
Apply suggestions from code review
fix formatting based on review

Co-Authored-By: Tejal Desai <tejal29@gmail.com>
2019-10-21 10:19:32 +08:00
yangtaokm 3e6a24b152 Initial commit for Azure Blob Storage SupportX 2019-10-13 20:52:19 +08:00
priyawadhwa 8acab90dec
Merge pull request #698 from takmatsu/s3-endpoint
Add support for S3 custom endpoint
2019-09-13 09:32:25 -04:00
Carlos Alexandro Becker f578b09846
fix: remove single branch option
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2019-06-22 12:17:46 -03:00
Carlos Alexandro Becker c45e05f668
clean: remove debug msg
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2019-06-22 11:45:42 -03:00
Carlos Alexandro Becker f0b9ad3a57
feat: support specifying branch for cloning
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2019-06-22 11:45:07 -03:00