From 0342d64bd2b44f5ca907415cab912e0adcdad809 Mon Sep 17 00:00:00 2001 From: Priya Wadhwa Date: Thu, 5 Apr 2018 15:19:40 -0700 Subject: [PATCH 1/4] fixed readme --- README.md | 48 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 38 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index d864403e9..e8f9238dc 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,8 @@ # kaniko -kaniko is a tool to build unpriviliged container images from a Dockerfile. It doesn't depend on a Docker daemon, which enables building container images in environments that can't easily or securely run a Docker daemon, such as a standard Kubernetes cluster. +kaniko is a tool to build unpriviliged container images from a Dockerfile. +kaniko doesn't depend on a Docker daemon and executes each command within a Dockerfile completely in userspace. +This enables building container images in environments that can't easily or securely run a Docker daemon, such as a standard Kubernetes cluster. The majority of Dockerfile commands can be executed with kaniko, but we're still working on supporting the following commands: * VOLUME @@ -10,19 +12,27 @@ The majority of Dockerfile commands can be executed with kaniko, but we're still * ONBUILD * ARG -We're currently in the process of building kaniko, so as of now it isn't production ready. Please let us know if you have any feature requests or find any bugs! +We're currently in the process of building kaniko, so as of now it isn't production ready. +Please let us know if you have any feature requests or find any bugs! ## How does kaniko work? -The kaniko executor image is responsible for building an image from a Dockerfile and pushing it to a registry. Within the executor image, we extract the filesystem of the base image (the FROM image in the Dockerfile). We then execute the commands in the Dockerfile, snapshotting the filesystem in userspace after each one. After each command, we append a layer of changed files to the base image (if there are any) and update image metadata. +The kaniko executor image is responsible for building an image from a Dockerfile and pushing it to a registry. +Within the executor image, we extract the filesystem of the base image (the FROM image in the Dockerfile). +We then execute the commands in the Dockerfile, snapshotting the filesystem in userspace after each one. +After each command, we append a layer of changed files to the base image (if there are any) and update image metadata. ## kaniko Build Contexts -kaniko supports local directories and GCS buckets as build contexts. To specify a local directory, pass in the -`--context` flag as an argument to the executor image. To specify a GCS bucket, pass in the -`--bucket` flag. The GCS bucket should contain a compressed tar of the build context called `context.tar.gz`, which kaniko will unpack and use as the build context. +kaniko supports local directories and GCS buckets as build contexts. To specify a local directory, pass in the `--context` flag as an argument to the executor image. +To specify a GCS bucket, pass in the `--bucket` flag. +The GCS bucket should contain a compressed tar of the build context called `context.tar.gz`, which kaniko will unpack and use as the build context. -To easily create `context.tar.gz`, we can use [skaffold](https://github.com/GoogleCloudPlatform/skaffold). Running the following command within the build context will create `context.tar.gz`, which will contain the Dockerfile and any files it depends on. +To create `context.tar.gz`, run the following command: +```shell +tar -C -zcvf context.tar.gz . +``` +Or, you can use [skaffold](https://github.com/GoogleCloudPlatform/skaffold) to create `context.tar.gz` by running ``` skaffold docker context ``` @@ -59,7 +69,9 @@ Requirements: To run kaniko in a Kubernetes cluster, you will need a standard running Kubernetes cluster and a Kubernetes secret, which contains the auth required to push the final image. -To create the secret, first you will need to create a service account in the Pantheon project you want to push the final image to, with `Storage Admin` permissions. You can download a JSON key for this service account, and rename it `kaniko-secret.json`. To create the secret, run: +To create the secret, first you will need to create a service account in the Pantheon project you want to push the final image to, with `Storage Admin` permissions. +You can download a JSON key for this service account, and rename it `kaniko-secret.json`. +To create the secret, run: ```shell kubectl create secret generic kaniko-secret --from-file= @@ -78,7 +90,9 @@ spec: containers: - name: kaniko image: gcr.io/kaniko-project/executor:latest - args: ["--dockerfile=", "--bucket=", "--destination="] + args: ["--dockerfile=", + "--bucket=", + "--destination="] volumeMounts: - name: kaniko-secret mountPath: /secret @@ -94,6 +108,18 @@ spec: This example pulls the build context from a GCS bucket. To use a local directory build context, you could consider using configMaps to mount in small build contexts. +## Running kaniko in Google Container Builder +To run kaniko in GCB, add it to your build config as a build step: + +```yaml +steps: + - name: gcr.io/kaniko-project/executor:latest + args: ["--dockerfile=", + "--context=", + "--destination="] +``` +kaniko will build and push the final image in this build step. + ## Comparison with Other Tools Similar tools include: @@ -102,7 +128,9 @@ Similar tools include: * [buildah](https://github.com/projectatomic/buildah) * [FTL](https://github.com/GoogleCloudPlatform/runtimes-common/tree/master/ftl) -All of these tools build container images with different approaches. Both kaniko and img build unprivileged images, but they interpret “unprivileged” differently. img builds as a non root user from within the container, while kaniko is run in an unprivileged environment with root access inside the container. +All of these tools build container images with different approaches. +Both kaniko and img build unprivileged images, but they interpret “unprivileged” differently. +img builds as a non root user from within the container, while kaniko is run in an unprivileged environment with root access inside the container. orca-build depends on runC to build images from Dockerfiles; since kaniko doesn't use runC it doesn't require the use of kernel namespacing techniques. From ce2b515d4949082b1db94e10cdc0235c6e3d985b Mon Sep 17 00:00:00 2001 From: Sharif Elgamal Date: Fri, 6 Apr 2018 12:02:57 -0700 Subject: [PATCH 2/4] adding VOLUME command (#62) * adding VOLUME command * proper test project * general fixes * fixing project name * fixing volume unit test * fixing integration test * adding tests * adding util test * fixing test * actually create the volume mounted directory * fix test --- executor/cmd/root.go | 1 + .../dockerfiles/Dockerfile_test_volume | 7 ++ .../dockerfiles/config_test_volume.json | 12 ++++ integration_tests/integration_test_yaml.go | 8 +++ pkg/commands/commands.go | 2 + pkg/commands/volume.go | 71 +++++++++++++++++++ pkg/commands/volume_test.go | 54 ++++++++++++++ pkg/util/fs_util.go | 20 ++++++ 8 files changed, 175 insertions(+) create mode 100644 integration_tests/dockerfiles/Dockerfile_test_volume create mode 100644 integration_tests/dockerfiles/config_test_volume.json create mode 100644 pkg/commands/volume.go create mode 100644 pkg/commands/volume_test.go diff --git a/executor/cmd/root.go b/executor/cmd/root.go index 42ca57e1c..cc6e37815 100644 --- a/executor/cmd/root.go +++ b/executor/cmd/root.go @@ -146,6 +146,7 @@ func execute() error { if err != nil { return err } + util.MoveVolumeWhitelistToWhitelist() if contents == nil { logrus.Info("No files were changed, appending empty layer to config.") sourceImage.AppendConfigHistory(constants.Author, true) diff --git a/integration_tests/dockerfiles/Dockerfile_test_volume b/integration_tests/dockerfiles/Dockerfile_test_volume new file mode 100644 index 000000000..f5a255cfe --- /dev/null +++ b/integration_tests/dockerfiles/Dockerfile_test_volume @@ -0,0 +1,7 @@ +FROM gcr.io/google-appengine/debian9 +RUN mkdir /foo +RUN echo "hello" > /foo/hey +VOLUME /foo/bar /tmp +ENV VOL /baz/bat +VOLUME ["${VOL}"] +RUN echo "hello again" > /tmp/hey diff --git a/integration_tests/dockerfiles/config_test_volume.json b/integration_tests/dockerfiles/config_test_volume.json new file mode 100644 index 000000000..72e958d85 --- /dev/null +++ b/integration_tests/dockerfiles/config_test_volume.json @@ -0,0 +1,12 @@ +[ + { + "Image1": "gcr.io/kbuild-test/docker-test-volume:latest", + "Image2": "gcr.io/kbuild-test/kbuild-test-volume:latest", + "DiffType": "File", + "Diff": { + "Adds": null, + "Dels": null, + "Mods": null + } + } +] diff --git a/integration_tests/integration_test_yaml.go b/integration_tests/integration_test_yaml.go index 3de05f43d..3616d219a 100644 --- a/integration_tests/integration_test_yaml.go +++ b/integration_tests/integration_test_yaml.go @@ -94,6 +94,14 @@ var fileTests = []struct { kbuildContext: buildcontextPath, repo: "test-workdir", }, + { + description: "test volume", + dockerfilePath: "/workspace/integration_tests/dockerfiles/Dockerfile_test_volume", + configPath: "/workspace/integration_tests/dockerfiles/config_test_volume.json", + dockerContext: buildcontextPath, + kbuildContext: buildcontextPath, + repo: "test-volume", + }, { description: "test add", dockerfilePath: "/workspace/integration_tests/dockerfiles/Dockerfile_test_add", diff --git a/pkg/commands/commands.go b/pkg/commands/commands.go index fd040decc..1c56faaa5 100644 --- a/pkg/commands/commands.go +++ b/pkg/commands/commands.go @@ -56,6 +56,8 @@ func GetCommand(cmd instructions.Command, buildcontext string) (DockerCommand, e return &LabelCommand{cmd: c}, nil case *instructions.UserCommand: return &UserCommand{cmd: c}, nil + case *instructions.VolumeCommand: + return &VolumeCommand{cmd: c}, nil } return nil, errors.Errorf("%s is not a supported command", cmd.Name()) } diff --git a/pkg/commands/volume.go b/pkg/commands/volume.go new file mode 100644 index 000000000..d5fd8ad05 --- /dev/null +++ b/pkg/commands/volume.go @@ -0,0 +1,71 @@ +/* +Copyright 2018 Google LLC + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package commands + +import ( + "github.com/GoogleCloudPlatform/k8s-container-builder/pkg/util" + "github.com/containers/image/manifest" + "github.com/docker/docker/builder/dockerfile/instructions" + "github.com/sirupsen/logrus" + "os" + "strings" +) + +type VolumeCommand struct { + cmd *instructions.VolumeCommand + snapshotFiles []string +} + +func (v *VolumeCommand) ExecuteCommand(config *manifest.Schema2Config) error { + logrus.Info("cmd: VOLUME") + volumes := v.cmd.Volumes + resolvedVolumes, err := util.ResolveEnvironmentReplacementList(volumes, config.Env, true) + if err != nil { + return err + } + existingVolumes := config.Volumes + if existingVolumes == nil { + existingVolumes = map[string]struct{}{} + } + for _, volume := range resolvedVolumes { + var x struct{} + existingVolumes[volume] = x + err := util.AddPathToVolumeWhitelist(volume) + if err != nil { + return err + } + + logrus.Infof("Creating directory %s", volume) + if err := os.MkdirAll(volume, 0755); err != nil { + return err + } + + //Check if directory already exists? + v.snapshotFiles = append(v.snapshotFiles, volume) + } + config.Volumes = existingVolumes + + return nil +} + +func (v *VolumeCommand) FilesToSnapshot() []string { + return v.snapshotFiles +} + +func (v *VolumeCommand) CreatedBy() string { + return strings.Join(append([]string{v.cmd.Name()}, v.cmd.Volumes...), " ") +} diff --git a/pkg/commands/volume_test.go b/pkg/commands/volume_test.go new file mode 100644 index 000000000..69bd5b9c4 --- /dev/null +++ b/pkg/commands/volume_test.go @@ -0,0 +1,54 @@ +/* +Copyright 2018 Google LLC + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ +package commands + +import ( + "github.com/GoogleCloudPlatform/k8s-container-builder/testutil" + "github.com/containers/image/manifest" + "github.com/docker/docker/builder/dockerfile/instructions" + "testing" +) + +func TestUpdateVolume(t *testing.T) { + cfg := &manifest.Schema2Config{ + Env: []string{ + "VOLUME=/etc", + }, + Volumes: map[string]struct{}{}, + } + + volumes := []string{ + "/tmp", + "/var/lib", + "$VOLUME", + } + + volumeCmd := &VolumeCommand{ + cmd: &instructions.VolumeCommand{ + Volumes: volumes, + }, + snapshotFiles: []string{}, + } + + expectedVolumes := map[string]struct{}{ + "/tmp": {}, + "/var/lib": {}, + "/etc": {}, + } + + err := volumeCmd.ExecuteCommand(cfg) + testutil.CheckErrorAndDeepEqual(t, false, err, expectedVolumes, cfg.Volumes) +} diff --git a/pkg/util/fs_util.go b/pkg/util/fs_util.go index b3b020c15..a97c75964 100644 --- a/pkg/util/fs_util.go +++ b/pkg/util/fs_util.go @@ -31,6 +31,7 @@ import ( ) var whitelist = []string{"/kbuild"} +var volumeWhitelist = []string{} // ExtractFileSystemFromImage pulls an image and unpacks it to a file system at root func ExtractFileSystemFromImage(img string) error { @@ -156,6 +157,25 @@ func CreateFile(path string, reader io.Reader, perm os.FileMode) error { return dest.Chmod(perm) } +// AddPathToVolumeWhitelist adds the given path to the volume whitelist +// It will get snapshotted when the VOLUME command is run then ignored +// for subsequent commands. +func AddPathToVolumeWhitelist(path string) error { + logrus.Infof("adding %s to volume whitelist", path) + volumeWhitelist = append(volumeWhitelist, path) + return nil +} + +// MoveVolumeWhitelistToWhitelist copies over all directories that were volume mounted +// in this step to be whitelisted for all subsequent docker commands. +func MoveVolumeWhitelistToWhitelist() error { + if len(volumeWhitelist) > 0 { + whitelist = append(whitelist, volumeWhitelist...) + volumeWhitelist = []string{} + } + return nil +} + // DownloadFileToDest downloads the file at rawurl to the given dest for the ADD command // From add command docs: // 1. If is a remote file URL: From ab76d0f541db0fb0a2b1bff7839d00964e587611 Mon Sep 17 00:00:00 2001 From: Priya Wadhwa Date: Fri, 6 Apr 2018 12:10:10 -0700 Subject: [PATCH 3/4] changed kbuild to kaniko --- README.md | 3 ++- integration_tests/dockerfiles/config_test_volume.json | 4 ++-- integration_tests/integration_test_yaml.go | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index e8f9238dc..67c289453 100644 --- a/README.md +++ b/README.md @@ -136,4 +136,5 @@ orca-build depends on runC to build images from Dockerfiles; since kaniko doesn' buildah requires the same root privilges as a Docker daemon does to run, while kaniko runs without any special privileges or permissions. -FTL aims to achieve the fastest possible creation of Docker images for a subset of images. It can be thought of as a special-case "fast path" that can be used in conjunction with the support for general Dockerfiles kaniko provides. +FTL aims to achieve the fastest possible creation of Docker images for a subset of images. +It can be thought of as a special-case "fast path" that can be used in conjunction with the support for general Dockerfiles kaniko provides. diff --git a/integration_tests/dockerfiles/config_test_volume.json b/integration_tests/dockerfiles/config_test_volume.json index 72e958d85..c4ba05ffe 100644 --- a/integration_tests/dockerfiles/config_test_volume.json +++ b/integration_tests/dockerfiles/config_test_volume.json @@ -1,7 +1,7 @@ [ { - "Image1": "gcr.io/kbuild-test/docker-test-volume:latest", - "Image2": "gcr.io/kbuild-test/kbuild-test-volume:latest", + "Image1": "gcr.io/kaniko-test/docker-test-volume:latest", + "Image2": "gcr.io/kaniko-test/kaniko-test-volume:latest", "DiffType": "File", "Diff": { "Adds": null, diff --git a/integration_tests/integration_test_yaml.go b/integration_tests/integration_test_yaml.go index 49ed136c5..f08dab881 100644 --- a/integration_tests/integration_test_yaml.go +++ b/integration_tests/integration_test_yaml.go @@ -99,7 +99,7 @@ var fileTests = []struct { dockerfilePath: "/workspace/integration_tests/dockerfiles/Dockerfile_test_volume", configPath: "/workspace/integration_tests/dockerfiles/config_test_volume.json", dockerContext: buildcontextPath, - kbuildContext: buildcontextPath, + kanikoContext: buildcontextPath, repo: "test-volume", }, { From 885987076ee8b31acfa66f4f0754227d31cfb53d Mon Sep 17 00:00:00 2001 From: Priya Wadhwa Date: Tue, 10 Apr 2018 14:27:10 -0700 Subject: [PATCH 4/4] Change k8s Job to Pod --- README.md | 50 ++++++++++++++++++++++++++------------------------ 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index 67c289453..b6184d978 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,6 @@ kaniko doesn't depend on a Docker daemon and executes each command within a Dock This enables building container images in environments that can't easily or securely run a Docker daemon, such as a standard Kubernetes cluster. The majority of Dockerfile commands can be executed with kaniko, but we're still working on supporting the following commands: -* VOLUME * SHELL * HEALTHCHECK * STOPSIGNAL @@ -77,36 +76,35 @@ To create the secret, run: kubectl create secret generic kaniko-secret --from-file= ``` -The Kubernetes job.yaml should look similar to this, with the args parameters filled in: +The Kubernetes Pod spec should look similar to this, with the args parameters filled in: ```yaml -apiVersion: batch/v1 -kind: Job +apiVersion: v1 +kind: Pod metadata: name: kaniko spec: - template: - spec: - containers: - - name: kaniko - image: gcr.io/kaniko-project/executor:latest - args: ["--dockerfile=", - "--bucket=", - "--destination="] - volumeMounts: - - name: kaniko-secret - mountPath: /secret - env: - - name: GOOGLE_APPLICATION_CREDENTIALS - value: /secret/kaniko-secret.json - restartPolicy: Never - volumes: - - name: kaniko-secret - secret: - secretName: kaniko-secret + containers: + - name: kaniko + image: gcr.io/kaniko-project/executor:latest + args: ["--dockerfile=", + "--bucket=", + "--destination="] + volumeMounts: + - name: kaniko-secret + mountPath: /secret + env: + - name: GOOGLE_APPLICATION_CREDENTIALS + value: /secret/kaniko-secret.json + restartPolicy: Never + volumes: + - name: kaniko-secret + secret: + secretName: kaniko-secret ``` -This example pulls the build context from a GCS bucket. To use a local directory build context, you could consider using configMaps to mount in small build contexts. +This example pulls the build context from a GCS bucket. +To use a local directory build context, you could consider using configMaps to mount in small build contexts. ## Running kaniko in Google Container Builder To run kaniko in GCB, add it to your build config as a build step: @@ -138,3 +136,7 @@ buildah requires the same root privilges as a Docker daemon does to run, while k FTL aims to achieve the fastest possible creation of Docker images for a subset of images. It can be thought of as a special-case "fast path" that can be used in conjunction with the support for general Dockerfiles kaniko provides. + +## Community + +[kaniko-users](https://groups.google.com/forum/#!forum/kaniko-users) Google group