2abe109eb2
Environment variables should be replaced in URLs in ADD commands. ( #580 )
...
We were previously explicitly skipping this for some reason, but Docker
seems to expand these in URLs so we should too.
2019-02-25 10:42:34 -06:00
1d079e683e
README.me: update Buildah description ( #586 )
...
Use the latest description of Buildah from upstream.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-02-25 09:52:53 -06:00
4b7e2b3a06
Update the cache warmer to also save manifests. ( #576 )
2019-02-21 14:09:22 -06:00
94d7db884b
README.md: update Buildah description ( #573 )
...
The description of Buildah is a bit outdated, most importantly Buildah
does not require root privileges (anymore). Also provide a more
detailed description copied from github.com/containers/buildah.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-02-21 11:11:28 -06:00
378a3f9573
Look for manifests in the local cache next to the full images. ( #570 )
...
Calculating a manifest from a v1.tarball is very expensive. We can
store those locally as well, and use them if they exist.
This should eventually be replaced with oci layout support once that exists
in ggcr.
2019-02-19 13:54:41 -06:00
faadb2af86
Log "Skipping paths under..." to debug ( #571 )
...
This reduces noise in the log output, since it isn't terribly useful to most end users.
2019-02-19 13:54:26 -06:00
f197a06ee9
Merge pull request #569 from frelon/master
...
Fix typo in error message
2019-02-19 11:43:11 -08:00
7c3947b08d
Make the run_in_docker script support caching. ( #564 )
2019-02-19 09:48:29 -06:00
1b54bda32e
Fix typo
2019-02-19 16:43:20 +01:00
e8564f0d28
Add missing tests for bucket util ( #565 )
...
* Add CheckDeepEqual test util
* Fix for srcContext with slash suffix
* Add tests for bucket_util.go
2019-02-19 09:05:23 -06:00
4feed0ff35
Stop storing a separate cache hash. ( #560 )
...
This is unrequired, mtimes should be taken into account during caching.
2019-02-14 12:24:43 -06:00
8d78db4842
Refactor snapshotting ( #561 )
2019-02-14 12:14:28 -06:00
15e70d4142
Speed up workdir by always returning an empty filelist (rather than a nil one). ( #557 )
...
Nil indicates that a full snapshot is required. Empty indicates that nothing should be snapshotted.
2019-02-13 11:55:08 -06:00
877abd30ed
Refactor whitelist handling. ( #559 )
...
Also speed up stage deletion.
2019-02-13 11:17:56 -06:00
8179c47f0d
Refactor the build loop to fetch stagebuilders earlier. ( #558 )
...
This will help with optimizations.
2019-02-12 20:43:22 -06:00
114a0859b8
Cut 0.9.0 release. ( #556 )
2019-02-08 16:42:33 -06:00
af50bcec69
let's return errors when warming a cache ( #553 )
...
* let's return errors when warming a cache
* Fix lint error in warm.go
2019-02-08 15:14:53 -06:00
9047ccf7cc
This fixes a bug in the interaction between volumes in base images ( #555 )
...
and our snapshot optimizations.
If a previous base image has a volume, the directory is added to the
list of files to snapshot. That directory may not actually exist in the image.
2019-02-08 14:40:37 -06:00
e14b660947
Fix a bug in volume handling for multi-stage images. ( #554 )
...
We were previously not resetting the whitelist between stages, this caused issues with volumes.
2019-02-07 09:01:49 -06:00
82fe355f14
Fix issues with layer caching, noPush and tarPath ( #549 )
...
* Set TarPath to empty when pushing a layer
* Fix issues with layer caching, noPush and tarPath.
- Layer caching should work even when tarPath is specified, so this
commit changes the value of tarPath to empty when caching layers.
- When an image is built with just the tarPath and noPush
is true, we should still create the tarBall (which wasn't happening
before this commit).
* Set no-push to false for cache layers
* Remove extra log
* go-imports fix
2019-02-06 10:16:09 -06:00
9f65174cb8
Generate the config.json from gcloud rather than relying on a hardcoded one. ( #551 )
...
marketplace.gcr.io now requires auth, but our credential helper doesn't know that.
2019-02-04 16:53:59 -06:00
c2514305ef
Fix a bug in snapshotting with multi-stage images. ( #546 )
...
We previously had an optimization that would skip snapshotting mutli-stage images
when in an intermediate stage, until the very end.
This conflicted with another optimization to avoid snapshotting when no files had changed.
2019-01-30 13:57:02 -06:00
63547d700c
Add CHANGELOG for 0.8 ( #545 )
2019-01-29 13:29:56 -06:00
f8f59ea4c6
Add insecure-registry and tls-skip-verify-registry flags ( #537 )
2019-01-29 13:29:47 -06:00
e3bf9fb062
Only log the hour of the benchmark run. ( #544 )
...
Also combine all files into one directory.
2019-01-28 15:53:13 -06:00
4c960709df
Actually fix the paths for benchmark uploading during integration test runs. ( #543 )
2019-01-28 12:52:27 -06:00
1ffae47fdd
Change cache key calculation to be more reproducible. ( #525 )
...
Before we were using the full image digest, but that contains a timestamp. Now
we only use the layers themselves and the image config (env vars, etc.).
Also fix a bug in unpacking the layers themselves. mtimes can change during unpacking,
so set them all once at the end.
2019-01-23 13:46:12 -06:00
a493035287
Log the output from uploading files to GCS. ( #541 )
2019-01-23 12:08:04 -06:00
a4b47a4992
Fix the time format we use to upload to GCS. ( #538 )
2019-01-22 14:23:28 -06:00
fcd1976d3b
Make the Digest calculation faster for locally-cached images. ( #534 )
...
Right now when we find a v1.Tarball in the local disk cache, we
recompute the digest. This is very expensive and redundant, because
we store tarballs by their digest and use that as a key to look them up.
2019-01-22 13:28:21 -06:00
b0e7c0e8cd
Upload benchmarks to GCS. ( #523 )
2019-01-17 16:10:14 -05:00
6e7c91a1cb
Deflake the daemon test ( #532 )
...
There's a race condition here where sometimes we get a layer and othertimes we don't.
2019-01-17 15:30:08 -05:00
503a972dbc
Merge pull request #513 from dcecile/patch-1
...
Fill in missing docs for --insecure
2019-01-14 10:51:19 -08:00
48f9968e5b
Move the benchmark files from /workspace to their own mounted volume in tests. ( #527 )
...
If we don't do this, they can end up in the built images causing test failures.
2019-01-11 16:04:16 -07:00
d9055df304
Make benchmark runs push to a separate namespace. ( #526 )
2019-01-11 11:38:09 -07:00
996bec885f
Refactor our credential fetching to only get k8s credentials if we are on k8s. ( #522 )
2019-01-11 11:36:43 -07:00
170e0a2d94
Add a lot more timing data. ( #518 )
2019-01-10 13:27:55 -07:00
9ab66560db
Simplify snapshotting. ( #517 )
2019-01-09 15:31:02 -07:00
9d10516698
Reorder the Dockerfile for faster iterations. ( #515 )
2019-01-09 10:39:41 -08:00
2255837142
Tighten up the timing around Dockerfile commands. ( #514 )
...
Right now this timing also includes the snapshot time.
2019-01-09 10:34:23 -08:00
ee2baf8025
Set the absolute path of the integration-test.sh script. ( #512 )
...
Kokoro runs with a working directory outside of the repo.
2019-01-09 09:34:01 -08:00
d52691f53d
Fill in missing docs for --insecure
2019-01-08 21:12:24 -05:00
1591f41f22
USER command needs /etc/passwd, thus require the fs to be unpacked ( #508 )
2019-01-08 17:26:54 -08:00
1b9e74334b
Add an entrypoint for kokoro to run benchmarks. ( #511 )
2019-01-08 17:26:16 -08:00
5f6fbfe74f
Add support for timing data in JSON format. ( #510 )
2019-01-08 17:24:47 -08:00
859e63c991
Change the CI cloudbuild to stop tagging latest. ( #505 )
...
Latest should only be changed during releases.
2019-01-03 15:04:04 -06:00
a044e2b6e4
Even faster snapshotting with godirwalk. ( #504 )
...
This switches from filepath.Walk to godirwalk.Walk for even faster snapshotting.
A quick test shows a 40% improvement on the dockerfile_mv_add build.
2019-01-03 13:10:18 -06:00
c3afcc0c7d
Adding capability to get Dockerfile from URL. ( #500 )
...
* Changed to set http or https in Dockerfile path.
* Fixed return value when error.
* Fixed ineffectual error
2019-01-03 10:42:15 -06:00
e3bb8bc71a
Adds COPY --from=previous stage name/number validation ( #491 )
...
* Adds COPY --from=previous stage name/number validation
This fixes an issue in which COPY --from=previous-stage-name would try to download docker image previous-stage-name instead of checking that previous-stage-name could be a named stage.
* Fix linting issues
goimports is implemented as 'gofmt + extras', so this should fix import warnings as well.
* Fix linting issues
Fixes linting issues introduced in the merge
* Fix linting issues.
2019-01-02 11:42:36 -06:00
ddeef6d224
Refactor tests and make them run in parallel. ( #499 )
2018-12-21 12:17:57 -06:00
dlorenc
Valentin Rothberg
Jason Hall
priyawadhwa
Fredrik Lönnegren
Shuhei Kitagawa
James Rawlings
Anurag Goel
Daisuke Taniwaki
Dan Cecile
Myers Carpenter
linuxshokunin
MMeent