feat: experimental integration with helm-x (#673)

This enhances helmfile so that it can:

- Treat K8s manifests directories and Kustomize projects as charts
- Add adhoc chart dependencies on sync/diff/template without forking or modifying chart(s) (#649)
- Add adhoc patches(JSON Patch or Strategic Merge Patch supported) to be applied to the K8s resources before sync/diff/template, without forking or modifyin chart(s) (#650)

The usage is as outlined in https://github.com/mumoshu/helm-x/tree/master/examples/helmfile.

Add any or all of `dependencies:`, `jsonPatches:` and `strategicMergePatches:` so that it adds additional flags to `helm` calls that is only supported by `helm x`.

```yaml
releases:
- name: kustomize
  chart: ../kustomize
- name: manifests
  chart: ../manifests
- name: foo
  chart: incubator/raw
  dependencies:
  - alias: bar
    chart: incubator/raw
  values:
  - values.yaml
  - bar:
      enabled: true
      resources:
      - apiVersion: v1
        kind: Pod
        metadata:
          name: bar
        spec:
          containers:
          - command:
            - sleep
            - 1000
            image: alpine:3.9.4
            imagePullPolicy: IfNotPresent
            name: bar
  jsonPatches:
  - target:
      version: v1
      kind: Pod
      name: foo
    patch:
    - op: replace
      path: /spec/containers/0/command
      value:
      - sleep
      - "123"
  strategicMergePatches:
  - apiVersion: v1
    kind: Pod
    metadata:
      name: bar
    spec:
      containers:
      - name: bar
        command:
        - sleep
        - "234"
```

You can alternatively provide `source: path/to/patch.yaml` for `jsonPatches` and `strategicMergePatches` items to externalize it. Add `.gotmpl` suffix like you would do for values files for templating.

When running `helmfile` you must point `--helm-binary` to the `helm-x` binary like below:

```
$ helmfile --helm-binary ~/.helm/plugins/helm-x/bin/helm-x --log-level debug apply
```

after installing the [helm-x](https://github.com/mumoshu/helm-x) plugin.

The integration should ideally be automatic. That is, it shouldn't force you to set `--helm-binary`. But I had no other way to not bloat helmfile's codebase to just add this experimental feature.

Resolves #649
Resolves #650

Dockerfile

@@ -22,7 +22,9 @@ RUN wget ${HELM_LOCATION}/${HELM_FILENAME} && \
 
 RUN mkdir -p "$(helm home)/plugins"
 RUN helm plugin install https://github.com/databus23/helm-diff && \
-    helm plugin install https://github.com/futuresimple/helm-secrets
+    helm plugin install https://github.com/futuresimple/helm-secrets && \
+    helm plugin install https://github.com/hypnoglow/helm-s3.git && \
+    helm plugin install https://github.com/aslafy-z/helm-git.git
 
 COPY --from=builder /workspace/helmfile/dist/helmfile_linux_amd64 /usr/local/bin/helmfile
 

README.md

@@ -1,19 +1,19 @@
-# helmfile [![CircleCI](https://circleci.com/gh/roboll/helmfile.svg?style=svg)](https://circleci.com/gh/roboll/helmfile)
+# Helmfile [![CircleCI](https://circleci.com/gh/roboll/helmfile.svg?style=svg)](https://circleci.com/gh/roboll/helmfile)
 
 Deploy Kubernetes Helm Charts
 
 [![Docker Repository on Quay](https://quay.io/repository/roboll/helmfile/status "Docker Repository on Quay")](https://quay.io/repository/roboll/helmfile)
 [![Slack Community #helmfile](https://slack.sweetops.com/badge.svg)](https://slack.sweetops.com)
 
-## status
+## Status
 
 Even though Helmfile is used in production environments [across multiple organizations](USERS.md), it is still in its early stage of development, hence versioned 0.x.
 
 Helmfile complies to Semantic Versioning 2.0.0 in which v0.x means that there could be backward-incompatible changes for every release.
 
-Note that we will try our best to document any backward incompatibility.
+Note that we will try our best to document any backward incompatibility. And in reality, helmfile had no breaking change for a year or so.
 
-## about
+## About
 
 Helmfile is a declarative spec for deploying helm charts. It lets you...
 
@@ -23,20 +23,44 @@ Helmfile is a declarative spec for deploying helm charts. It lets you...
 
 To avoid upgrades for each iteration of `helm`, the `helmfile` executable delegates to `helm` - as a result, `helm` must be installed.
 
-## configuration syntax
+## Highlights
+
+**Declarative**: Write, version-control, apply the desired state file for visibility and reproducibility.
+
+**Modules**: Modularize common patterns of your infrastructure, distribute it via Git, S3, etc. to be reused across the entire company (See [#648](https://github.com/roboll/helmfile/pull/648))
+
+**Versatility**: Manage your cluster consisting of charts, [kustomizations](https://github.com/kubernetes-sigs/kustomize), and directories of Kubernetes resources, turning everything to Helm releases (See [#673](https://github.com/roboll/helmfile/pull/673))
+
+**Patch**: JSON/Strategic-Merge Patch Kubernetes resources before `helm-install`ing, without forking upstream charts (See [#673](https://github.com/roboll/helmfile/pull/673))
+
+## Configuration
 
 **CAUTION**: This documentation is for the development version of Helmfile. If you are looking for the documentation for any of releases, please switch to the corresponding release tag like [v0.31.0](https://github.com/roboll/helmfile/tree/v0.31.0).
 
 The default helmfile is `helmfile.yaml`:
 
 ```yaml
+# Chart repositories used from within this state file
+#
+# Use `helm-s3` and `helm-git` and whatever Helm Downloader plugins
+# to use repositories other than the official repository or one backend by chartmuseum.
 repositories:
-  - name: roboll
-    url: http://roboll.io/charts
-    certFile: optional_client_cert
-    keyFile: optional_client_key
-    username: optional_username
-    password: optional_password
+# To use official "stable" charts a.k.a https://github.com/helm/charts/tree/master/stable
+- name: stable
+  url: https://kubernetes-charts.storage.googleapis.com
+# To use official "incubator" charts a.k.a https://github.com/helm/charts/tree/master/incubator
+- name: incubator
+  url: https://kubernetes-charts-incubator.storage.googleapis.com
+# helm-git powered repository: You can treat any Git repository as a charts repository
+- name: polaris
+  url: git+https://github.com/reactiveops/polaris@deploy/helm?ref=master
+# Advanced configuration: You can setup basic or tls auth
+- name: roboll
+  url: http://roboll.io/charts
+  certFile: optional_client_cert
+  keyFile: optional_client_key
+  username: optional_username
+  password: optional_password
 
 # context: kube-context # this directive is deprecated, please consider using helmDefaults.kubeContext
 
@@ -63,6 +87,9 @@ helmDefaults:
   # path to TLS key file (default "$HELM_HOME/key.pem")
   tlsKey: "path/to/key.pem"
 
+# The desired states of Helm releases.
+#
+# Helmfile runs various helm commands to converge the current state in the live cluster to the desired state defined here.
 releases:
   # Published chart example
   - name: vault                            # name of this release
@@ -71,12 +98,21 @@ releases:
       foo: bar
     chart: roboll/vault-secret-manager     # the chart being installed to create this release, referenced by `repository/chart` syntax
     version: ~1.24.1                       # the semver of the chart. range constraint is supported
-    missingFileHandler: warn # set to either "Error" or "Warn". "Error" instructs helmfile to fail when unable to find a values or secrets file. When "Warn", it prints the file and continues.
+    missingFileHandler: Warn # set to either "Error" or "Warn". "Error" instructs helmfile to fail when unable to find a values or secrets file. When "Warn", it prints the file and continues.
+    # Values files used for rendering the chart
     values:
-      # value files passed via --values
+      # Value files passed via --values
       - vault.yaml
-      # inline values, passed via a temporary values file and --values
+      # Inline values, passed via a temporary values file and --values, so that it doesn't suffer from type issues like --set
       - address: https://vault.example.com
+      # Go template available in inline values and values files.
+      - image:
+          # The end result is more or less YAML. So do `quote` to prevent number-like strings from accidentally parsed into numbers!
+          # See https://github.com/roboll/helmfile/issues/608
+          tag: {{ requiredEnv "IMAGE_TAG" | quote }}
+          # Otherwise:
+          #   tag: "{{ requiredEnv "IMAGE_TAG" }}"
+          #   tag: !!string {{ requiredEnv "IMAGE_TAG" }}
         db:
           username: {{ requiredEnv "DB_USERNAME" }}
           # value taken from environment variable. Quotes are necessary. Will throw an error if the environment variable is not set. $DB_PASSWORD needs to be set in the calling environment ex: export DB_PASSWORD='password1'
@@ -85,6 +121,8 @@ releases:
           # Interpolate environment variable with a fixed string
           domain: {{ requiredEnv "PLATFORM_ID" }}.my-domain.com
           scheme: {{ env "SCHEME" | default "https" }}
+    # Use `values` whenever possible!
+    # `set` translates to helm's `--set key=val`, that is known to suffer from type issues like https://github.com/roboll/helmfile/issues/608
     set:
     # single value loaded from a local file, translates to --set-file foo.config=path/to/file
     - name: foo.config
@@ -100,6 +138,8 @@ releases:
     # will attempt to decrypt it using helm-secrets plugin
     secrets:
       - vault_secret.yaml
+    # verify the chart before upgrading (only works with packaged charts not directories)
+    verify: true
     # wait for k8s resources via --wait. Defaults to `false`
     wait: true
     # time in seconds to wait for any individual Kubernetes operation (like Jobs for hooks, and waits on pod/pvc/svc/deployment readiness) (default 300)
@@ -134,6 +174,80 @@ releases:
     - ./values/{{ requiredEnv "PLATFORM_ENV" }}/config.yaml # Values file taken from path with environment variable. $PLATFORM_ENV must be set in the calling environment.
     wait: true
 
+#
+# Advanced Configuration: Nested States
+#
+helmfiles:
+- # Path to the helmfile state file being processed BEFORE releases in this state file
+  path: path/to/subhelmfile.yaml
+  # Label selector used for filtering releases in the nested state.
+  # For example, `name=prometheus` in this context is equivalent to processing the nested state like
+  #   helmfile -f path/to/subhelmfile.yaml -l name=prometheus sync
+  selectors:
+  - name=prometheus
+  # Override state values
+  values:
+  # Values files merged into the nested state's values
+  - additiona.values.yaml
+  # Inline state values merged into the nested state's values
+  - key1: val1
+- # All the nested state files under `helmfiles:` is processed in the order of definition.
+  # So it can be used for preparation for your main `releases`. An example would be creating CRDs required by `reelases` in the parent state file. 
+  path: path/to/mycrd.helmfile.yaml
+- # Terraform-module-like URL for importing a remote directory and use a file in it as a nested-state file
+  # The nested-state file is locally checked-out along with the remote directory containing it.
+  # Therefore all the local paths in the file are resolved relative to the file
+  path: git::https://github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.40.0
+
+#
+# Advanced Configuration: Environments
+#
+
+# The list of environments managed by helmfile.
+#
+# The default is `environments: {"default": {}}` which implies:
+#
+# - `{{ .Environment.Name }}` evaluates to "default"
+# - `{{ .Environment.Values }}` being empty
+environments:
+  # The "default" environment is available and used when `helmfile` is run without `--environment NAME`.
+  default:
+    # Everything from the values.yaml is available via `{{ .Environment.Values.KEY }}`.
+    # Suppose `{"foo": {"bar": 1}}` contained in the values.yaml below,
+    # `{{ .Environment.Values.foo.bar }}` is evaluated to `1`.
+    values:
+    - environments/default/values.yaml
+    # Each entry in values can be either a file path or inline values.
+    # The below is an example of inline values, which is merged to the `.Environment.Values`
+    - myChartVer: 1.0.0-dev
+  # Any environment other than `default` is used only when `helmfile` is run with `--environment NAME`.
+  # That is, the "production" env below is used when and only when it is run like `helmfile --environment production sync`.
+  production:
+    values:
+    - environment/production/values.yaml
+    - myChartVer: 1.0.0
+    ## `secrets.yaml` is decrypted by `helm-secrets` and available via `{{ .Environment.Secrets.KEY }}`
+    secrets:
+    - environment/production/secrets.yaml
+    # Instructs helmfile to fail when unable to find a environment values file listed under `environments.NAME.values`.
+    #
+    # Possible values are  "Error", "Warn", "Info", "Debug". The default is "Error".
+    #
+    # Use "Warn", "Info", or "Debug" if you want helmfile to not fail when a values file is missing, while just leaving
+    # a message about the missing file at the log-level.
+    missingFileHandler: Error
+
+#
+# Advanced Configuration: Layering
+#
+# Helmfile merges all the "base" state files and this state file before processing.
+#
+# Assuming this state file is named `helmfile.yaml`, all the files are merged in the order of:
+#   environments.yaml <- defaults.yaml <- templates.yaml <- helmfile.yaml
+bases:
+- environments.yaml
+- defaults.yaml
+- templates.yaml
 ```
 
 ## Templating
@@ -151,7 +265,7 @@ Environment variables can be used in most places for templating the helmfile. Cu
 Examples:
 
 ```yaml
-respositories:
+repositories:
 - name: your-private-git-repo-hosted-charts
   url: https://{{ requiredEnv "GITHUB_TOKEN"}}@raw.githubusercontent.com/kmzfs/helm-repo-in-github/master/
 ```
@@ -220,9 +334,10 @@ USAGE:
    helmfile [global options] command [command options] [arguments...]
 
 VERSION:
-   v0.52.0
+   v0.70.0
 
 COMMANDS:
+     deps      update charts based on the contents of requirements.yaml
      repos     sync repositories from state file (helm repo add && helm repo update)
      charts    DEPRECATED: sync releases from state file (helm upgrade --install)
      diff      diff releases from state file against env (helm diff)
@@ -239,6 +354,8 @@ GLOBAL OPTIONS:
    --helm-binary value, -b value           path to helm binary
    --file helmfile.yaml, -f helmfile.yaml  load config from file or directory. defaults to helmfile.yaml or `helmfile.d`(means `helmfile.d/*.yaml`) in this preference
    --environment default, -e default       specify the environment name. defaults to default
+   --state-values-set value                set state values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
+   --state-values-file value               specify state values in a YAML file
    --quiet, -q                             Silence output. Equivalent to log-level warn
    --kube-context value                    Set kubectl context. Uses current context by default
    --log-level value                       Set log level, default info
@@ -247,6 +364,7 @@ GLOBAL OPTIONS:
                                            A release must match all labels in a group in order to be used. Multiple groups can be specified at once.
                                            --selector tier=frontend,tier!=proxy --selector tier=backend. Will match all frontend, non-proxy releases AND all backend releases.
                                            The name of a release can be used as a label. --selector name=myrelease
+   --allow-no-matching-release             Do not exit with an error code if the provided selector has no matching releases.
    --interactive, -i                       Request confirmation before attempting to modify clusters
    --help, -h                              show help
    --version, -v                           print the version
@@ -261,6 +379,20 @@ dependencies of any referenced local charts.
 
 For Helm 2.9+ you can use a username and password to authenticate to a remote repository.
 
+### deps
+
+The `helmfile deps` sub-command locks your helmfile state and local charts dependencies.
+
+It basically runs `helm dependency update` on your helmfile state file and all the referenced local charts, so that you get a "lock" file per each helmfile state or local chart.
+
+All the other `helmfile` sub-commands like `sync` use chart versions recorded in the lock files, so that e.g. untested chart versions won't suddenly get deployed to the production environment.
+
+For example, the lock file for a helmfile state file named `helmfile.1.yaml` will be `helmfile.1.lock`. The lock file for a local chart would be `requirements.lock`, which is the same as `helm`.
+
+It is recommended to version-control all the lock files, so that they can be used in the production deployment pipeline for extra reproducibility.
+
+To bring in chart updates systematically, it would also be a good idea to run `helmfile deps` regularly, test it, and then update the lock files in the version-control system.
+
 ### diff
 
 The `helmfile diff` sub-command executes the [helm-diff](https://github.com/databus23/helm-diff) plugin across all of
@@ -292,6 +424,7 @@ The `helmfile delete` sub-command deletes all the releases defined in the manife
 
 Note that `delete` doesn't purge releases. So `helmfile delete && helmfile sync` results in sync failed due to that releases names are not deleted but preserved for future references. If you really want to remove releases for reuse, add `--purge` flag to run it like `helmfile delete --purge`.
 
+
 ### secrets
 
 The `secrets` parameter in a `helmfile.yaml` causes the [helm-secrets](https://github.com/futuresimple/helm-secrets) plugin to be executed to decrypt the file.
@@ -332,7 +465,7 @@ The `selector` parameter can be specified multiple times. Each parameter is reso
 
 `--selector tier=frontend --selector tier=backend` will select all the charts
 
-In addition to user supplied labels the name, namespace, and chart are available to be used as selectors.  The chart will just be the chart name excluding the repository (Example `stable/filebeat` would be selected using `--selector chart=filebeat`).
+In addition to user supplied labels, the name, the namespace, and the chart are available to be used as selectors.  The chart will just be the chart name excluding the repository (Example `stable/filebeat` would be selected using `--selector chart=filebeat`).
 
 ## Templates
 
@@ -643,6 +776,30 @@ Just run `helmfile sync` inside `myteam/`, and you are done.
 
 All the files are sorted alphabetically per group = array item inside `helmfiles:`, so that you have granular control over ordering, too.
 
+#### selectors
+
+When composing helmfiles you can use selectors from the command line as well as explicit selectors inside the parent helmfile to filter the releases to be used.
+
+```yaml
+helmfiles:
+- apps/*/helmfile.yaml
+- path: apps/a-helmfile.yaml
+  selectors:          # list of selectors
+  - name=prometheus
+  - tier=frontend
+- path: apps/b-helmfile.yaml # no selector, so all releases are used
+selectors: []
+- path: apps/c-helmfile.yaml # parent selector to be used or cli selector for the initial helmfile
+  selectorsInherited: true
+```
+
+* When a selector is specified, only this selector applies and the parents or CLI selectors are ignored.
+* When not selector is specified there are 2 modes for the selector inheritance because we would like to change the current inheritance behavior (see [issue #344](https://github.com/roboll/helmfile/issues/344)  ).
+  * Legacy mode, sub-helmfiles without selectors inherit selectors from their parent helmfile. The initial helmfiles inherit from the command line selectors.
+  * explicit mode, sub-helmfile without selectors do not inherit from their parent or the CLI selector. If you want them to inherit from their parent selector then use `selectorsInherited: true`. To enable this explicit mode you need to set the following environment variable `HELMFILE_EXPERIMENTAL=explicit-selector-inheritance` (see [experimental](#experimental-features)).
+* Using `selector: []` will select all releases regardless of the parent selector or cli for the initial helmfile
+* using `selectorsInherited: true` make the sub-helmfile selects releases with the parent selector or the cli for the initial helmfile. You cannot specify an explicit selector while using `selectorsInherited: true`
+
 ## Importing values from any source
 
 The `exec` template function that is available in `values.yaml.gotmpl` is useful for importing values from any source
@@ -678,12 +835,18 @@ Once `events` are triggered, associated `hooks` are executed, by running the `co
 Currently supported `events` are:
 
 - `prepare`
+- `presync`
+- `postsync`
 - `cleanup`
 
 Hooks associated to `prepare` events are triggered after each release in your helmfile is loaded from YAML, before execution.
 
 Hooks associated to `cleanup` events are triggered after each release is processed.
 
+Hooks associated to `presync` events are triggered before each release is applied to the remote cluster. This is the ideal event to execute any commands that may mutate the cluster state as it will not be run for read-only operations like `lint`, `diff` or `template`.
+
+Hooks associated to `postsync` events are triggered after each release is applied to the remote cluster. This is the ideal event to execute any commands that may mutate the cluster state as it will not be run for read-only operations like `lint`, `diff` or `template`.
+
 The following is an example hook that just prints the contextual information provided to hook:
 
 ```
@@ -805,6 +968,12 @@ Once you download all required charts into your machine, you can run `helmfile c
 It basically run only `helm upgrade --install` with your already-downloaded charts, hence no Internet connection is required.
 See #155 for more information on this topic.
 
+## Experimental features
+Some experimental features may be available for testing in perspective of being (or not) included in a future release.
+Those features are set using the environment variable `HELMFILE_EXPERIMENTAL`. Here is the current experimental feature :
+* `explicit-selector-inheritance` : remove today implicit cli selectors inheritance for composed helmfiles, see [composition selector](#selectors)
+
+If you want to enable all experimental features set the env var to `HELMFILE_EXPERIMENTAL=true`
 
 ## Examples
 

cmd/cmd.go

@@ -1,100 +0,0 @@
-package cmd
-
-import (
-	"fmt"
-	"github.com/roboll/helmfile/helmexec"
-	"github.com/roboll/helmfile/pkg/app"
-	"github.com/roboll/helmfile/state"
-	"github.com/urfave/cli"
-	"go.uber.org/zap"
-	"os/exec"
-	"strings"
-	"syscall"
-)
-
-func VisitAllDesiredStates(c *cli.Context, converge func(*state.HelmState, helmexec.Interface, app.Context) (bool, []error)) error {
-	a, fileOrDir, err := InitAppEntry(c, false)
-	if err != nil {
-		return err
-	}
-
-	ctx := app.NewContext()
-
-	convergeWithHelmBinary := func(st *state.HelmState, helm helmexec.Interface) (bool, []error) {
-		if c.GlobalString("helm-binary") != "" {
-			helm.SetHelmBinary(c.GlobalString("helm-binary"))
-		}
-		return converge(st, helm, ctx)
-	}
-
-	err = a.VisitDesiredStates(fileOrDir, convergeWithHelmBinary)
-
-	return toCliError(err)
-}
-
-func InitAppEntry(c *cli.Context, reverse bool) (*app.App, string, error) {
-	if c.NArg() > 0 {
-		cli.ShowAppHelp(c)
-		return nil, "", fmt.Errorf("err: extraneous arguments: %s", strings.Join(c.Args(), ", "))
-	}
-
-	fileOrDir := c.GlobalString("file")
-	kubeContext := c.GlobalString("kube-context")
-	namespace := c.GlobalString("namespace")
-	selectors := c.GlobalStringSlice("selector")
-	logger := c.App.Metadata["logger"].(*zap.SugaredLogger)
-
-	env := c.GlobalString("environment")
-	if env == "" {
-		env = state.DefaultEnv
-	}
-
-	app := app.Init(&app.App{
-		KubeContext: kubeContext,
-		Logger:      logger,
-		Reverse:     reverse,
-		Env:         env,
-		Namespace:   namespace,
-		Selectors:   selectors,
-	})
-
-	return app, fileOrDir, nil
-}
-
-func FindAndIterateOverDesiredStatesUsingFlagsWithReverse(c *cli.Context, reverse bool, converge func(*state.HelmState, helmexec.Interface, app.Context) []error) error {
-	a, fileOrDir, err := InitAppEntry(c, reverse)
-	if err != nil {
-		return err
-	}
-
-	ctx := app.NewContext()
-
-	convergeWithHelmBinary := func(st *state.HelmState, helm helmexec.Interface) []error {
-		if c.GlobalString("helm-binary") != "" {
-			helm.SetHelmBinary(c.GlobalString("helm-binary"))
-		}
-		return converge(st, helm, ctx)
-	}
-
-	err = a.VisitDesiredStatesWithReleasesFiltered(fileOrDir, convergeWithHelmBinary)
-
-	return toCliError(err)
-}
-
-func toCliError(err error) error {
-	if err != nil {
-		switch e := err.(type) {
-		case *app.NoMatchingHelmfileError:
-			return cli.NewExitError(e.Error(), 2)
-		case *exec.ExitError:
-			// Propagate any non-zero exit status from the external command like `helm` that is failed under the hood
-			status := e.Sys().(syscall.WaitStatus)
-			return cli.NewExitError(e.Error(), status.ExitStatus())
-		case *state.DiffError:
-			return cli.NewExitError(e.Error(), e.Code)
-		default:
-			return cli.NewExitError(e.Error(), 1)
-		}
-	}
-	return err
-}

cmd/deps.go

@@ -1,37 +0,0 @@
-package cmd
-
-import (
-	"github.com/roboll/helmfile/args"
-	"github.com/roboll/helmfile/helmexec"
-	"github.com/roboll/helmfile/pkg/app"
-	"github.com/roboll/helmfile/state"
-	"github.com/urfave/cli"
-)
-
-func Deps(a *app.App) cli.Command {
-	return cli.Command{
-		Name:  "deps",
-		Usage: "update charts based on the contents of requirements.yaml",
-		Flags: []cli.Flag{
-			cli.StringFlag{
-				Name:  "args",
-				Value: "",
-				Usage: "pass args to helm exec",
-			},
-		},
-		Action: func(c *cli.Context) error {
-			return VisitAllDesiredStates(c, func(state *state.HelmState, helm helmexec.Interface, ctx app.Context) (bool, []error) {
-				args := args.GetArgs(c.String("args"), state)
-				if len(args) > 0 {
-					helm.SetExtraArgs(args...)
-				}
-
-				errs := state.UpdateDeps(helm)
-
-				ok := len(errs) == 0
-
-				return ok, errs
-			})
-		},
-	}
-}

docs/writing-helmfile.md

@@ -91,7 +91,9 @@ releases:
 
 See the [issue 428](https://github.com/roboll/helmfile/issues/428) for more context on how this is supposed to work.
 
-## Layering
+## Layering State Files
+
+> See **Layering State Template Files** if you're layering templates.
 
 You may occasionally end up with many helmfiles that shares common parts like which repositories to use, and whichi release to be bundled by default.
 
@@ -100,24 +102,17 @@ Use Layering to extract the common parts into a dedicated *library helmfile*s, s
 Let's assume that your `helmfile.yaml` looks like:
 
 ```
-{ readFile "commons.yaml" }}
----
-{{ readFile "environments.yaml" }}
----
+bases:
+- environments.yaml
+
 releases:
+- name: metricbaet
+  chart: stable/metricbeat
 - name: myapp
   chart: mychart
 ```
 
-Whereas `commons.yaml` contained a monitoring agent:
-
-```yaml
-releases:
-- name: metricbaet
-  chart: stable/metricbeat
-```
-
-And `environments.yaml` contained well-known environments:
+Whereas `environments.yaml` contained well-known environments:
 
 ```yaml
 environments:
@@ -125,23 +120,24 @@ environments:
   production:
 ```
 
-At run time, template expressions in your `helmfile.yaml` are executed:
+At run time, `bases` in your `helmfile.yaml` are evaluated to produce:
 
 ```yaml
-releases:
-- name: metricbaet
-  chart: stable/metricbeat
 ---
+# environments.yaml
 environments:
   development:
   production:
 ---
+# helmfile.yaml
 releases:
 - name: myapp
   chart: mychart
+- name: metricbaet
+  chart: stable/metricbeat
 ```
 
-Resulting YAML documents are merged in the order of occurrence,
+Finally the resulting YAML documents are merged in the order of occurrence,
 so that your `helmfile.yaml` becomes:
 
 ```yaml
@@ -159,3 +155,164 @@ releases:
 Great!
 
 Now, repeat the above steps for each your `helmfile.yaml`, so that all your helmfiles becomes DRY.
+
+Please also see [the discussion in the issue 388](https://github.com/roboll/helmfile/issues/388#issuecomment-491710348) for more advanced layering examples.
+
+## Merging Arrays in Layers
+
+Helmfile doesn't merge arrays across layers. That is, the below example doesn't work as you might have expected:
+
+```yaml
+releases:
+- name: metricbaet
+  chart: stable/metricbeat
+---
+releases:
+- name: myapp
+  chart: mychart
+```
+
+Helmfile overrides the `releases` array with the latest layer so the resulting state file will be:
+
+```yaml
+releases:
+# metricbeat release disappeared! but that's how helmfiel works
+- name: myapp
+  chart: mychart
+```
+
+A work-around is to treat the state file as a go template and use `readFile` template function to import the common part of your state file as a plain text:
+
+`common.yaml`:
+
+```yaml
+templates:
+  metricbeat: &metricbeat
+  - name: metricbeat
+    chart: stable/metricbeat
+```
+
+`helmfile.yaml`:
+
+```yaml
+{{ readFile "common.yaml" }}
+
+releases:
+- <<: *metricbeat
+- name: myapp
+  chart: mychart
+```
+
+## Layering State Template Files
+
+Do you need to make your state file even more DRY?
+
+Turned out layering state files wasn't enough for you?
+
+Helmfile supports an advanced feature that allows you to compose state "template" files to generate the final state to be processed.
+
+In the following example `helmfile.yaml.gotmpl`, each `---` separated part of the file is a go template.
+
+`helmfile.yaml.gotmpl`:
+
+```yaml
+# Part 1: Reused Enviroment Values
+bases:
+  - myenv.yaml
+---
+# Part 2: Reused Defaults
+bases:
+  - mydefaults.yaml
+---
+# Part 3: Dynamic Releases
+releases:
+  - name: test1
+    chart: mychart-{{ .Environment.Values.myname }}
+    values:
+      replicaCount: 1
+      image:
+        repository: "nginx"
+        tag: "latest"
+```
+
+Suppose the `myenv.yaml` and `test.env.yaml` loaded in the first part looks like:
+
+`myenv.yaml`:
+
+```yaml
+environments:
+  test:
+    values:
+      - test.env.yaml
+```
+
+`test.env.yaml`:
+
+```yaml
+kubeContext: test
+wait: false
+cvOnly: false
+myname: "dog"
+```
+
+Where the gotmpl file loaded in the second part looks like:
+
+`mydefaults.yaml.gotmpl`:
+
+```yaml
+helmDefaults:
+  tillerNamespace: kube-system
+  kubeContext: {{ .Environment.Values.kubeContext }}
+  verify: false
+  {{ if .Environment.Values.wait }}
+  wait: true
+  {{ else }}
+  wait: false
+  {{ end }}
+  timeout: 600
+  recreatePods: false
+  force: true
+```
+
+Each go template is rendered in the context where `.Environment.Values` is inherited from the previous part. 
+
+So in `mydefaults.yaml.gotmpl`, both `.Environment.Values.kubeContext` and `.Environment.Values.wait` are valid as they do exist in the environment values inherited from the previous part(=the first part) of your `helmfile.yaml.gotmpl`, and therefore the template is rendered to:
+
+```yaml
+helmDefaults:
+  tillerNamespace: kube-system
+  kubeContext: test
+  verify: false
+  wait: false
+  timeout: 600
+  recreatePods: false
+  force: true
+```
+
+Similarly, the third part of the top-level `helmfile.yaml.gotmpl`, `.Environment.Values.myname` is valid as it is included in the environment values inherited from the previous parts:
+
+```yaml
+# Part 3: Dynamic Releases
+releases:
+  - name: test1
+    chart: mychart-{{ .Environment.Values.myname }}
+    values:
+      replicaCount: 1
+      image:
+        repository: "nginx"
+        tag: "latest"
+````
+
+hence rendered to:
+
+```yaml
+# Part 3: Dynamic Releases
+releases:
+  - name: test1
+    chart: mychart-dog
+    values:
+      replicaCount: 1
+      image:
+        repository: "nginx"
+        tag: "latest"
+```

environment/environment.go

@@ -1,8 +0,0 @@
-package environment
-
-type Environment struct {
-	Name   string
-	Values map[string]interface{}
-}
-
-var EmptyEnvironment Environment

go.mod

@@ -3,18 +3,20 @@ module github.com/roboll/helmfile
 go 1.12
 
 require (
-	github.com/Masterminds/semver v1.4.1 // indirect
-	github.com/Masterminds/sprig v2.15.0+incompatible
+	github.com/Masterminds/semver v1.4.1
+	github.com/Masterminds/sprig v2.16.0+incompatible
 	github.com/aokoli/goutils v1.0.1 // indirect
+	github.com/google/go-cmp v0.3.0
 	github.com/google/uuid v0.0.0-20161128191214-064e2069ce9c // indirect
+	github.com/hashicorp/go-getter v1.3.0
 	github.com/huandu/xstrings v1.0.0 // indirect
 	github.com/imdario/mergo v0.3.6
-	github.com/mattn/go-runewidth v0.0.4 // indirect
+	github.com/pkg/errors v0.8.1 // indirect
 	github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939
 	github.com/urfave/cli v0.0.0-20160620154522-6011f165dc28
 	go.uber.org/atomic v1.3.2 // indirect
 	go.uber.org/multierr v1.1.0 // indirect
 	go.uber.org/zap v1.8.0
-	golang.org/x/crypto v0.0.0-20180403160946-b2aa35443fbc // indirect
 	gopkg.in/yaml.v2 v2.2.1
+	gotest.tools v2.2.0+incompatible
 )

go.sum

@@ -1,29 +1,204 @@
+cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+cloud.google.com/go v0.31.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+cloud.google.com/go v0.36.0 h1:+aCSj7tOo2LODWVEuZDZeGCckdt6MlSF+X/rB3wUiS8=
+cloud.google.com/go v0.36.0/go.mod h1:RUoy9p/M4ge0HzT8L+SDZ8jg+Q6fth0CiBuhFJpSV40=
+dmitri.shuralyov.com/app/changes v0.0.0-20180602232624-0a106ad413e3/go.mod h1:Yl+fi1br7+Rr3LqpNJf1/uxUdtRUV+Tnj0o93V2B9MU=
+dmitri.shuralyov.com/html/belt v0.0.0-20180602232347-f7d459c86be0/go.mod h1:JLBrvjyP0v+ecvNYvCpyZgu5/xkfAUhi6wJj28eUfSU=
+dmitri.shuralyov.com/service/change v0.0.0-20181023043359-a85b471d5412/go.mod h1:a1inKt/atXimZ4Mv927x+r7UpyzRUf4emIoiiSC2TN4=
+dmitri.shuralyov.com/state v0.0.0-20180228185332-28bcc343414c/go.mod h1:0PRwlb0D6DFvNNtx+9ybjezNCa8XF0xaYcETyp6rHWU=
+git.apache.org/thrift.git v0.0.0-20180902110319-2566ecd5d999/go.mod h1:fPE2ZNJGynbRyZ4dJvy6G277gSllfV2HJqblrnkyeyg=
+github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
 github.com/Masterminds/semver v1.4.1 h1:CaDA1wAoM3rj9sAFyyZP37LloExUzxFGYt+DqJ870JA=
 github.com/Masterminds/semver v1.4.1/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF078ddwwvV3Y=
 github.com/Masterminds/sprig v2.15.0+incompatible h1:0gSxPGWS9PAr7U2NsQ2YQg6juRDINkUyuvbb4b2Xm8w=
 github.com/Masterminds/sprig v2.15.0+incompatible/go.mod h1:y6hNFY5UBTIWBxnzTeuNhlNS5hqE0NB0E6fgfo2Br3o=
+github.com/Masterminds/sprig v2.16.0+incompatible h1:QZbMUPxRQ50EKAq3LFMnxddMu88/EUUG3qmxwtDmPsY=
+github.com/Masterminds/sprig v2.16.0+incompatible/go.mod h1:y6hNFY5UBTIWBxnzTeuNhlNS5hqE0NB0E6fgfo2Br3o=
+github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c=
 github.com/aokoli/goutils v1.0.1 h1:7fpzNGoJ3VA8qcrm++XEE1QUe0mIwNeLa02Nwq7RDkg=
 github.com/aokoli/goutils v1.0.1/go.mod h1:SijmP0QR8LtwsmDs8Yii5Z/S4trXFGFC2oO5g9DP+DQ=
+github.com/aws/aws-sdk-go v1.15.78 h1:LaXy6lWR0YK7LKyuU0QWy2ws/LWTPfYV/UgfiBu4tvY=
+github.com/aws/aws-sdk-go v1.15.78/go.mod h1:E3/ieXAlvM0XWO57iftYVDLLvQ824smPP3ATZkfNZeM=
+github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
+github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d h1:xDfNPAt8lFiC1UJrqV3uuy861HCTo708pDMbjHHdCas=
+github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d/go.mod h1:6QX/PXZ00z/TKoufEY6K/a0k6AhaJrQKdFe6OfVXsa4=
+github.com/bradfitz/go-smtpd v0.0.0-20170404230938-deb6d6237625/go.mod h1:HYsPBTaaSFSlLx/70C2HPIMNZpVV8+vt/A+FMnYP11g=
+github.com/cheggaaa/pb v1.0.27/go.mod h1:pQciLPpbU0oxA0h+VJYYLxO+XeDQb5pZijXscXHm81s=
+github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
+github.com/coreos/go-systemd v0.0.0-20181012123002-c6f51f82210d/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
+github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
+github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
+github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
+github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
+github.com/gliderlabs/ssh v0.1.1/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0=
+github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
+github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
+github.com/golang/lint v0.0.0-20180702182130-06c8688daad7/go.mod h1:tluoj9z5200jBnyusfRPU2LqT6J+DAorxEvtC7LHB+E=
+github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM=
+github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
+github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
+github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
+github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
+github.com/google/go-github v17.0.0+incompatible/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ=
+github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
+github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
+github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
 github.com/google/uuid v0.0.0-20161128191214-064e2069ce9c h1:jWtZjFEUE/Bz0IeIhqCnyZ3HG6KRXSntXe4SjtuTH7c=
 github.com/google/uuid v0.0.0-20161128191214-064e2069ce9c/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/googleapis/gax-go v2.0.0+incompatible h1:j0GKcs05QVmm7yesiZq2+9cxHkNK9YM6zKx4D2qucQU=
+github.com/googleapis/gax-go v2.0.0+incompatible/go.mod h1:SFVmujtThgffbyetf+mdk2eWhX2bMyUtNHzFKcPA9HY=
+github.com/googleapis/gax-go/v2 v2.0.3 h1:siORttZ36U2R/WjiJuDz8znElWBiAlO9rVt+mqJt0Cc=
+github.com/googleapis/gax-go/v2 v2.0.3/go.mod h1:LLvjysVCY1JZeum8Z6l8qUty8fiNwE08qbEPm1M08qg=
+github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
+github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=
+github.com/grpc-ecosystem/grpc-gateway v1.5.0/go.mod h1:RSKVYQBd5MCa4OVpNdGskqpgL2+G+NZTnrVHpWWfpdw=
+github.com/hashicorp/go-cleanhttp v0.5.0 h1:wvCrVc9TjDls6+YGAF2hAifE1E5U1+b4tH6KdvN3Gig=
+github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
+github.com/hashicorp/go-getter v1.3.0 h1:pFMSFlI9l5NaeuzkpE3L7BYk9qQ9juTAgXW/H0cqxcU=
+github.com/hashicorp/go-getter v1.3.0/go.mod h1:/O1k/AizTN0QmfEKknCYGvICeyKUDqCYA8vvWtGWDeQ=
+github.com/hashicorp/go-safetemp v1.0.0 h1:2HR189eFNrjHQyENnQMMpCiBAsRxzbTMIgBhEyExpmo=
+github.com/hashicorp/go-safetemp v1.0.0/go.mod h1:oaerMy3BhqiTbVye6QuFhFtIceqFoDHxNAB65b+Rj1I=
+github.com/hashicorp/go-version v1.1.0 h1:bPIoEKD27tNdebFGGxxYwcL4nepeY4j1QP23PFRGzg0=
+github.com/hashicorp/go-version v1.1.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
 github.com/huandu/xstrings v1.0.0 h1:pO2K/gKgKaat5LdpAhxhluX2GPQMaI3W5FUz/I/UnWk=
 github.com/huandu/xstrings v1.0.0/go.mod h1:4qWG/gcEcfX4z/mBDHJ++3ReCw9ibxbsNJbcucJdbSo=
 github.com/imdario/mergo v0.3.6 h1:xTNEAn+kxVO7dTZGu0CegyqKZmoWFI0rF8UxjlB2d28=
 github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
+github.com/jellevandenhooff/dkim v0.0.0-20150330215556-f50fe3d243e1/go.mod h1:E0B/fFc00Y+Rasa88328GlI/XbtyysCtTHZS8h7IrBU=
+github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8 h1:12VvqtR6Aowv3l/EQUlocDHW2Cp4G9WJVH7uyH8QFJE=
+github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
+github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
+github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
+github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/pty v1.1.3/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
+github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
+github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
 github.com/mattn/go-runewidth v0.0.4 h1:2BvfKmzob6Bmd4YsL0zygOqfdFnK7GR4QL06Do4/p7Y=
 github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
+github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
+github.com/microcosm-cc/bluemonday v1.0.1/go.mod h1:hsXNsILzKxV+sX77C5b8FSuKF00vh2OMYv+xgHpAMF4=
+github.com/mitchellh/go-homedir v1.0.0 h1:vKb8ShqSby24Yrqr/yDYkuFz8d0WUjys40rvnGC8aR0=
+github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
+github.com/mitchellh/go-testing-interface v1.0.0 h1:fzU/JVNcaqHQEcVFAKeR41fkiLdIPrefOvVG1VZ96U0=
+github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=
+github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86/go.mod h1:kHJEU3ofeGjhHklVoIGuVj85JJwZ6kWPaJwCIxgnFmo=
+github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab/go.mod h1:Qr6/a/Q4r9LP1IltGz7tA7iOK1WonHEYhu1HRBA7ZiM=
+github.com/openzipkin/zipkin-go v0.1.1/go.mod h1:NtoC/o8u3JlF1lSlyPNswIbeQH9bJTmOf0Erfk+hxe8=
+github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
+github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/prometheus/client_golang v0.8.0/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
+github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
+github.com/prometheus/common v0.0.0-20180801064454-c7de2306084e/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
+github.com/prometheus/procfs v0.0.0-20180725123919-05ee40e3a273/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
+github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
+github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
+github.com/shurcooL/component v0.0.0-20170202220835-f88ec8f54cc4/go.mod h1:XhFIlyj5a1fBNx5aJTbKoIq0mNaPvOagO+HjB3EtxrY=
+github.com/shurcooL/events v0.0.0-20181021180414-410e4ca65f48/go.mod h1:5u70Mqkb5O5cxEA8nxTsgrgLehJeAw6Oc4Ab1c/P1HM=
+github.com/shurcooL/github_flavored_markdown v0.0.0-20181002035957-2122de532470/go.mod h1:2dOwnU2uBioM+SGy2aZoq1f/Sd1l9OkAeAUvjSyvgU0=
+github.com/shurcooL/go v0.0.0-20180423040247-9e1955d9fb6e/go.mod h1:TDJrrUr11Vxrven61rcy3hJMUqaf/CLWYhHNPmT14Lk=
+github.com/shurcooL/go-goon v0.0.0-20170922171312-37c2f522c041/go.mod h1:N5mDOmsrJOB+vfqUK+7DmDyjhSLIIBnXo9lvZJj3MWQ=
+github.com/shurcooL/gofontwoff v0.0.0-20180329035133-29b52fc0a18d/go.mod h1:05UtEgK5zq39gLST6uB0cf3NEHjETfB4Fgr3Gx5R9Vw=
+github.com/shurcooL/gopherjslib v0.0.0-20160914041154-feb6d3990c2c/go.mod h1:8d3azKNyqcHP1GaQE/c6dDgjkgSx2BZ4IoEi4F1reUI=
+github.com/shurcooL/highlight_diff v0.0.0-20170515013008-09bb4053de1b/go.mod h1:ZpfEhSmds4ytuByIcDnOLkTHGUI6KNqRNPDLHDk+mUU=
+github.com/shurcooL/highlight_go v0.0.0-20181028180052-98c3abbbae20/go.mod h1:UDKB5a1T23gOMUJrI+uSuH0VRDStOiUVSjBTRDVBVag=
+github.com/shurcooL/home v0.0.0-20181020052607-80b7ffcb30f9/go.mod h1:+rgNQw2P9ARFAs37qieuu7ohDNQ3gds9msbT2yn85sg=
+github.com/shurcooL/htmlg v0.0.0-20170918183704-d01228ac9e50/go.mod h1:zPn1wHpTIePGnXSHpsVPWEktKXHr6+SS6x/IKRb7cpw=
+github.com/shurcooL/httperror v0.0.0-20170206035902-86b7830d14cc/go.mod h1:aYMfkZ6DWSJPJ6c4Wwz3QtW22G7mf/PEgaB9k/ik5+Y=
+github.com/shurcooL/httpfs v0.0.0-20171119174359-809beceb2371/go.mod h1:ZY1cvUeJuFPAdZ/B6v7RHavJWZn2YPVFQ1OSXhCGOkg=
+github.com/shurcooL/httpgzip v0.0.0-20180522190206-b1c53ac65af9/go.mod h1:919LwcH0M7/W4fcZ0/jy0qGght1GIhqyS/EgWGH2j5Q=
+github.com/shurcooL/issues v0.0.0-20181008053335-6292fdc1e191/go.mod h1:e2qWDig5bLteJ4fwvDAc2NHzqFEthkqn7aOZAOpj+PQ=
+github.com/shurcooL/issuesapp v0.0.0-20180602232740-048589ce2241/go.mod h1:NPpHK2TI7iSaM0buivtFUc9offApnI0Alt/K8hcHy0I=
+github.com/shurcooL/notifications v0.0.0-20181007000457-627ab5aea122/go.mod h1:b5uSkrEVM1jQUspwbixRBhaIjIzL2xazXp6kntxYle0=
+github.com/shurcooL/octicon v0.0.0-20181028054416-fa4f57f9efb2/go.mod h1:eWdoE5JD4R5UVWDucdOPg1g2fqQRq78IQa9zlOV1vpQ=
+github.com/shurcooL/reactions v0.0.0-20181006231557-f2e0b4ca5b82/go.mod h1:TCR1lToEk4d2s07G3XGfz2QrgHXg4RJBvjrOozvoWfk=
+github.com/shurcooL/sanitized_anchor_name v0.0.0-20170918181015-86672fcb3f95/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
+github.com/shurcooL/users v0.0.0-20180125191416-49c67e49c537/go.mod h1:QJTqeLYEDaXHZDBsXlPCDqdhQuJkuw4NOtaxYe3xii4=
+github.com/shurcooL/webdavfs v0.0.0-20170829043945-18c3829fa133/go.mod h1:hKmq5kWdCj2z2KEozexVbfEZIWiTjhE0+UjmZgPqehw=
+github.com/sourcegraph/annotate v0.0.0-20160123013949-f4cad6c6324d/go.mod h1:UdhH50NIW0fCiwBSr0co2m7BnFLdv4fQTgdqdJTHFeE=
+github.com/sourcegraph/syntaxhighlight v0.0.0-20170531221838-bd320f5d308e/go.mod h1:HuIsMU8RRBOtsCgI77wP899iHVBQpCmg4ErYMZB+2IA=
+github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
+github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07/go.mod h1:kDXzergiv9cbyO7IOYJZWg1U88JhDg3PB6klq9Hg2pA=
 github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939 h1:BhIUXV2ySTLrKgh/Hnts+QTQlIbWtomXt3LMdzME0A0=
 github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939/go.mod h1:omGxs4/6hNjxPKUTjmaNkPzehSnNJOJN6pMEbrlYIT4=
+github.com/ulikunitz/xz v0.5.5 h1:pFrO0lVpTBXLpYw+pnLj6TbvHuyjXMfjGeCwSqCVwok=
+github.com/ulikunitz/xz v0.5.5/go.mod h1:2bypXElzHzzJZwzH67Y6wb67pO62Rzfn7BSiF4ABRW8=
 github.com/urfave/cli v0.0.0-20160620154522-6011f165dc28 h1:Yf7/DcGM+61oLBGXQV2Q+ztEGBcOe3EUnbKSOn4fQuE=
 github.com/urfave/cli v0.0.0-20160620154522-6011f165dc28/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
+go.opencensus.io v0.18.0 h1:Mk5rgZcggtbvtAun5aJzAtjKKN/t0R3jJPlWILlv938=
+go.opencensus.io v0.18.0/go.mod h1:vKdFvxhtzZ9onBp9VKHK8z/sRpBMnKAsufL7wlDrCOA=
 go.uber.org/atomic v1.3.2 h1:2Oa65PReHzfn29GpvgsYwloV9AVFHPDk8tYxt2c2tr4=
 go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
 go.uber.org/multierr v1.1.0 h1:HoEmRHQPVSqub6w2z2d2EOVs2fjyFRGyofhKuyDq0QI=
 go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
 go.uber.org/zap v1.8.0 h1:r6Za1Rii8+EGOYRDLvpooNOF6kP3iyDnkpzbw67gCQ8=
 go.uber.org/zap v1.8.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
+go4.org v0.0.0-20180809161055-417644f6feb5/go.mod h1:MkTOUMDaeVYJUOUsaDXIhWPZYa1yOyC1qaOBpL57BhE=
+golang.org/x/build v0.0.0-20190111050920-041ab4dc3f9d/go.mod h1:OWs+y06UdEOHN4y+MfF/py+xQ/tYqIWW03b70/CG9Rw=
 golang.org/x/crypto v0.0.0-20180403160946-b2aa35443fbc h1:Kx1Ke+iCR1aDjbWXgmEQGFxoHtNL49aRZGV7/+jJ41Y=
 golang.org/x/crypto v0.0.0-20180403160946-b2aa35443fbc/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+golang.org/x/crypto v0.0.0-20181030102418-4d3f4d9ffa16 h1:y6ce7gCWtnH+m3dCjzQ1PCuwl28DDIc3VNnvY29DlIA=
+golang.org/x/crypto v0.0.0-20181030102418-4d3f4d9ffa16/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
+golang.org/x/lint v0.0.0-20180702182130-06c8688daad7/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181029044818-c44066c5c816/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181106065722-10aee1819953/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181114220301-adae6a3d119a h1:gOpx8G595UYyvj8UK4+OFyY4rx037g3fmfhe5SasG3U=
+golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
+golang.org/x/oauth2 v0.0.0-20181017192945-9dcd33a902f4/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
+golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890 h1:uESlIz09WIHT2I+pasSXcpLYqYK8wHcdCetU3VuMBJE=
+golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
+golang.org/x/perf v0.0.0-20180704124530-6e6d33e29852/go.mod h1:JLpeXjPJfIyPr5TlbXLkXWLhP8nz10XfvxElABhCtcw=
+golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181029174526-d69651ed3497/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181213200352-4d1cda033e06/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2 h1:z99zHgr7hKfrUcX/KsoJk5FJfjTceCKIp96+biqP4To=
+golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20181030000716-a0a13e073c7b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+google.golang.org/api v0.0.0-20180910000450-7ca32eb868bf/go.mod h1:4mhQ8q/RsB7i+udVvVy5NUi08OU8ZlA0gRVgrF7VFY0=
+google.golang.org/api v0.0.0-20181030000543-1d582fd0359e/go.mod h1:4mhQ8q/RsB7i+udVvVy5NUi08OU8ZlA0gRVgrF7VFY0=
+google.golang.org/api v0.1.0 h1:K6z2u68e86TPdSdefXdzvXgR1zEMa+459vBSfWYAZkI=
+google.golang.org/api v0.1.0/go.mod h1:UGEZY7KEX120AnNLIHFMKIo4obdJhkp2tPbaPlQx13Y=
+google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
+google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
+google.golang.org/appengine v1.3.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
+google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
+google.golang.org/genproto v0.0.0-20180831171423-11092d34479b/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
+google.golang.org/genproto v0.0.0-20181029155118-b69ba1387ce2/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
+google.golang.org/genproto v0.0.0-20181202183823-bd91e49a0898/go.mod h1:7Ep/1NZk928CDR8SjdVbjWNpdIf6nzjE3BTgJDr2Atg=
+google.golang.org/genproto v0.0.0-20190201180003-4b09977fb922 h1:mBVYJnbrXLA/ZCBTCe7PtEgAUP+1bg92qTaFoPHdz+8=
+google.golang.org/genproto v0.0.0-20190201180003-4b09977fb922/go.mod h1:L3J43x8/uS+qIUoksaLKe6OS3nUKxOKuIFz1sl2/jx4=
+google.golang.org/grpc v1.14.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
+google.golang.org/grpc v1.16.0/go.mod h1:0JHn/cJsOMiMfNA9+DeHDlAU7KAAB5GDlYFpa9MZMio=
+google.golang.org/grpc v1.17.0 h1:TRJYBgMclJvGYn2rIMjj+h9KtMt5r1Ij7ODVRIZkwhk=
+google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/cheggaaa/pb.v1 v1.0.27/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw=
+gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
 gopkg.in/yaml.v2 v2.2.1 h1:mUhvW9EsL+naU5Q3cakzfE91YhliOondGd6ZrsDBHQE=
 gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo=
+gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw=
+grpc.go4.org v0.0.0-20170609214715-11d0a25b4919/go.mod h1:77eQGdRu53HpSqPFJFmuJdjuHRquDANNeA4x7B8WQ9o=
+honnef.co/go/tools v0.0.0-20180728063816-88497007e858/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+sourcegraph.com/sourcegraph/go-diff v0.5.0/go.mod h1:kuch7UrkMzY0X+p9CRK03kfuPQ2zzQcaEFbx8wA8rck=
+sourcegraph.com/sqs/pbtypes v0.0.0-20180604144634-d3ebe8f20ae4/go.mod h1:ketZ/q3QxT9HOBeFhu6RdvsftgpsbFHBF5Cas6cDKZ0=

helmexec/runner.go

@@ -1,55 +0,0 @@
-package helmexec
-
-import (
-	"os"
-	"os/exec"
-	"strings"
-)
-
-const (
-	tmpPrefix = "helmfile-"
-	tmpSuffix = "-exec"
-)
-
-// Runner interface for shell commands
-type Runner interface {
-	Execute(cmd string, args []string, env map[string]string) ([]byte, error)
-}
-
-// ShellRunner implemention for shell commands
-type ShellRunner struct {
-	Dir string
-}
-
-// Execute a shell command
-func (shell ShellRunner) Execute(cmd string, args []string, env map[string]string) ([]byte, error) {
-	preparedCmd := exec.Command(cmd, args...)
-	preparedCmd.Dir = shell.Dir
-	preparedCmd.Env = mergeEnv(os.Environ(), env)
-	return preparedCmd.CombinedOutput()
-}
-
-func mergeEnv(orig []string, new map[string]string) []string {
-	wanted := env2map(orig)
-	for k, v := range new {
-		wanted[k] = v
-	}
-	return map2env(wanted)
-}
-
-func map2env(wanted map[string]string) []string {
-	result := []string{}
-	for k, v := range wanted {
-		result = append(result, k+"="+v)
-	}
-	return result
-}
-
-func env2map(env []string) map[string]string {
-	wanted := map[string]string{}
-	for _, cur := range env {
-		pair := strings.SplitN(cur, "=", 2)
-		wanted[pair[0]] = pair[1]
-	}
-	return wanted
-}

main.go

@@ -2,11 +2,10 @@ package main
 
 import (
 	"fmt"
-	"github.com/roboll/helmfile/args"
-	"github.com/roboll/helmfile/cmd"
-	"github.com/roboll/helmfile/helmexec"
 	"github.com/roboll/helmfile/pkg/app"
-	"github.com/roboll/helmfile/state"
+	"github.com/roboll/helmfile/pkg/helmexec"
+	"github.com/roboll/helmfile/pkg/maputil"
+	"github.com/roboll/helmfile/pkg/state"
 	"github.com/urfave/cli"
 	"go.uber.org/zap"
 	"go.uber.org/zap/zapcore"
@@ -30,7 +29,7 @@ func configureLogging(c *cli.Context) error {
 	if err != nil {
 		return err
 	}
-	logger = helmexec.NewLogger(os.Stdout, logLevel)
+	logger = helmexec.NewLogger(os.Stderr, logLevel)
 	if c.App.Metadata == nil {
 		// Auto-initialised in 1.19.0
 		// https://github.com/urfave/cli/blob/master/CHANGELOG.md#1190---2016-11-19
@@ -59,6 +58,14 @@ func main() {
 			Name:  "environment, e",
 			Usage: "specify the environment name. defaults to `default`",
 		},
+		cli.StringSliceFlag{
+			Name:  "state-values-set",
+			Usage: "set state values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)",
+		},
+		cli.StringSliceFlag{
+			Name:  "state-values-file",
+			Usage: "specify state values in a YAML file",
+		},
 		cli.BoolFlag{
 			Name:  "quiet, q",
 			Usage: "Silence output. Equivalent to log-level warn",
@@ -82,6 +89,10 @@ func main() {
 	--selector tier=frontend,tier!=proxy --selector tier=backend. Will match all frontend, non-proxy releases AND all backend releases.
 	The name of a release can be used as a label. --selector name=myrelease`,
 		},
+		cli.BoolFlag{
+			Name:  "allow-no-matching-release",
+			Usage: `Do not exit with an error code if the provided selector has no matching releases.`,
+		},
 		cli.BoolFlag{
 			Name:  "interactive, i",
 			Usage: "Request confirmation before attempting to modify clusters",
@@ -90,6 +101,20 @@ func main() {
 
 	cliApp.Before = configureLogging
 	cliApp.Commands = []cli.Command{
+		{
+			Name:  "deps",
+			Usage: "update charts based on the contents of requirements.yaml",
+			Flags: []cli.Flag{
+				cli.StringFlag{
+					Name:  "args",
+					Value: "",
+					Usage: "pass args to helm exec",
+				},
+			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Deps(c)
+			}),
+		},
 		{
 			Name:  "repos",
 			Usage: "sync repositories from state file (helm repo add && helm repo update)",
@@ -100,20 +125,9 @@ func main() {
 					Usage: "pass args to helm exec",
 				},
 			},
-			Action: func(c *cli.Context) error {
-				return cmd.VisitAllDesiredStates(c, func(state *state.HelmState, helm helmexec.Interface, ctx app.Context) (bool, []error) {
-					args := args.GetArgs(c.String("args"), state)
-					if len(args) > 0 {
-						helm.SetExtraArgs(args...)
-					}
-
-					errs := ctx.SyncReposOnce(state, helm)
-
-					ok := len(state.Repositories) > 0 && len(errs) == 0
-
-					return ok, errs
-				})
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Repos(c)
+			}),
 		},
 		{
 			Name:  "charts",
@@ -134,14 +148,9 @@ func main() {
 					Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
 				},
 			},
-			Action: func(c *cli.Context) error {
-				affectedReleases := state.AffectedReleases{}
-				errs := findAndIterateOverDesiredStatesUsingFlags(c, func(st *state.HelmState, helm helmexec.Interface, _ app.Context) []error {
-					return executeSyncCommand(c, &affectedReleases, st, helm)
-				})
-				affectedReleases.DisplayAffectedReleases(c.App.Metadata["logger"].(*zap.SugaredLogger))
-				return errs
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.DeprecatedSyncCharts(c)
+			}),
 		},
 		{
 			Name:  "diff",
@@ -174,24 +183,9 @@ func main() {
 					Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
 				},
 			},
-			Action: func(c *cli.Context) error {
-				return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface, ctx app.Context) []error {
-					if !c.Bool("skip-deps") {
-						if errs := ctx.SyncReposOnce(state, helm); errs != nil && len(errs) > 0 {
-							return errs
-						}
-						if errs := state.BuildDeps(helm); errs != nil && len(errs) > 0 {
-							return errs
-						}
-					}
-					if errs := state.PrepareRelease(helm, "diff"); errs != nil && len(errs) > 0 {
-						return errs
-					}
-
-					_, errs := executeDiffCommand(c, state, helm, c.Bool("detailed-exitcode"), c.Bool("suppress-secrets"))
-					return errs
-				})
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Diff(c)
+			}),
 		},
 		{
 			Name:  "template",
@@ -216,22 +210,9 @@ func main() {
 					Usage: "skip running `helm repo update` and `helm dependency build`",
 				},
 			},
-			Action: func(c *cli.Context) error {
-				return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface, ctx app.Context) []error {
-					if !c.Bool("skip-deps") {
-						if errs := ctx.SyncReposOnce(state, helm); errs != nil && len(errs) > 0 {
-							return errs
-						}
-						if errs := state.BuildDeps(helm); errs != nil && len(errs) > 0 {
-							return errs
-						}
-					}
-					if errs := state.PrepareRelease(helm, "template"); errs != nil && len(errs) > 0 {
-						return errs
-					}
-					return executeTemplateCommand(c, state, helm)
-				})
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Template(c)
+			}),
 		},
 		{
 			Name:  "lint",
@@ -251,21 +232,14 @@ func main() {
 					Value: 0,
 					Usage: "maximum number of concurrent downloads of release charts",
 				},
+				cli.BoolFlag{
+					Name:  "skip-deps",
+					Usage: "skip running `helm repo update` and `helm dependency build`",
+				},
 			},
-			Action: func(c *cli.Context) error {
-				return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface, ctx app.Context) []error {
-					values := c.StringSlice("values")
-					args := args.GetArgs(c.String("args"), state)
-					workers := c.Int("concurrency")
-					if errs := ctx.SyncReposOnce(state, helm); errs != nil && len(errs) > 0 {
-						return errs
-					}
-					if errs := state.PrepareRelease(helm, "lint"); errs != nil && len(errs) > 0 {
-						return errs
-					}
-					return state.LintReleases(helm, values, args, workers)
-				})
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Lint(c)
+			}),
 		},
 		{
 			Name:  "sync",
@@ -290,25 +264,9 @@ func main() {
 					Usage: "skip running `helm repo update` and `helm dependency build`",
 				},
 			},
-			Action: func(c *cli.Context) error {
-				affectedReleases := state.AffectedReleases{}
-				errs := findAndIterateOverDesiredStatesUsingFlags(c, func(st *state.HelmState, helm helmexec.Interface, ctx app.Context) []error {
-					if !c.Bool("skip-deps") {
-						if errs := ctx.SyncReposOnce(st, helm); errs != nil && len(errs) > 0 {
-							return errs
-						}
-						if errs := st.BuildDeps(helm); errs != nil && len(errs) > 0 {
-							return errs
-						}
-					}
-					if errs := st.PrepareRelease(helm, "sync"); errs != nil && len(errs) > 0 {
-						return errs
-					}
-					return executeSyncCommand(c, &affectedReleases, st, helm)
-				})
-				affectedReleases.DisplayAffectedReleases(c.App.Metadata["logger"].(*zap.SugaredLogger))
-				return errs
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Sync(c)
+			}),
 		},
 		{
 			Name:  "apply",
@@ -337,82 +295,9 @@ func main() {
 					Usage: "skip running `helm repo update` and `helm dependency build`",
 				},
 			},
-			Action: func(c *cli.Context) error {
-				affectedReleases := state.AffectedReleases{}
-				errs := findAndIterateOverDesiredStatesUsingFlags(c, func(st *state.HelmState, helm helmexec.Interface, ctx app.Context) []error {
-					if !c.Bool("skip-deps") {
-						if errs := ctx.SyncReposOnce(st, helm); errs != nil && len(errs) > 0 {
-							return errs
-						}
-						if errs := st.BuildDeps(helm); errs != nil && len(errs) > 0 {
-							return errs
-						}
-					}
-					if errs := st.PrepareRelease(helm, "apply"); errs != nil && len(errs) > 0 {
-						return errs
-					}
-
-					releases, errs := executeDiffCommand(c, st, helm, true, c.Bool("suppress-secrets"))
-
-					releasesToBeDeleted, err := st.DetectReleasesToBeDeleted(helm)
-					if err != nil {
-						errs = append(errs, err)
-					}
-
-					noError := true
-					for _, e := range errs {
-						switch err := e.(type) {
-						case *state.DiffError:
-							noError = noError && err.Code == 2
-						default:
-							noError = false
-						}
-					}
-
-					// sync only when there are changes
-					if noError {
-						if len(releases) == 0 && len(releasesToBeDeleted) == 0 {
-							// TODO better way to get the logger
-							logger := c.App.Metadata["logger"].(*zap.SugaredLogger)
-							logger.Infof("")
-							logger.Infof("No affected releases")
-						} else {
-							names := []string{}
-							for _, r := range releases {
-								names = append(names, fmt.Sprintf("  %s (%s) UPDATED", r.Name, r.Chart))
-							}
-							for _, r := range releasesToBeDeleted {
-								names = append(names, fmt.Sprintf("  %s (%s) DELETED", r.Name, r.Chart))
-							}
-
-							msg := fmt.Sprintf(`Affected releases are:
-%s
-
-Do you really want to apply?
-  Helmfile will apply all your changes, as shown above.
-
-`, strings.Join(names, "\n"))
-							interactive := c.GlobalBool("interactive")
-							if !interactive || interactive && askForConfirmation(msg) {
-								rs := []state.ReleaseSpec{}
-								for _, r := range releases {
-									rs = append(rs, *r)
-								}
-								for _, r := range releasesToBeDeleted {
-									rs = append(rs, *r)
-								}
-
-								st.Releases = rs
-								return executeSyncCommand(c, &affectedReleases, st, helm)
-							}
-						}
-					}
-
-					return errs
-				})
-				affectedReleases.DisplayAffectedReleases(c.App.Metadata["logger"].(*zap.SugaredLogger))
-				return errs
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Apply(c)
+			}),
 		},
 		{
 			Name:  "status",
@@ -429,18 +314,9 @@ Do you really want to apply?
 					Usage: "pass args to helm exec",
 				},
 			},
-			Action: func(c *cli.Context) error {
-				return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface, _ app.Context) []error {
-					workers := c.Int("concurrency")
-
-					args := args.GetArgs(c.String("args"), state)
-					if len(args) > 0 {
-						helm.SetExtraArgs(args...)
-					}
-
-					return state.ReleaseStatuses(helm, workers)
-				})
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Status(c)
+			}),
 		},
 		{
 			Name:  "delete",
@@ -456,37 +332,9 @@ Do you really want to apply?
 					Usage: "purge releases i.e. free release names and histories",
 				},
 			},
-			Action: func(c *cli.Context) error {
-				affectedReleases := state.AffectedReleases{}
-				errs := cmd.FindAndIterateOverDesiredStatesUsingFlagsWithReverse(c, true, func(state *state.HelmState, helm helmexec.Interface, _ app.Context) []error {
-					purge := c.Bool("purge")
-
-					args := args.GetArgs(c.String("args"), state)
-					if len(args) > 0 {
-						helm.SetExtraArgs(args...)
-					}
-
-					names := make([]string, len(state.Releases))
-					for i, r := range state.Releases {
-						names[i] = fmt.Sprintf("  %s (%s)", r.Name, r.Chart)
-					}
-
-					msg := fmt.Sprintf(`Affected releases are:
-%s
-
-Do you really want to delete?
-  Helmfile will delete all your releases, as shown above.
-
-`, strings.Join(names, "\n"))
-					interactive := c.GlobalBool("interactive")
-					if !interactive || interactive && askForConfirmation(msg) {
-						return state.DeleteReleases(&affectedReleases, helm, purge)
-					}
-					return nil
-				})
-				affectedReleases.DisplayAffectedReleases(c.App.Metadata["logger"].(*zap.SugaredLogger))
-				return errs
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Delete(c)
+			}),
 		},
 		{
 			Name:  "destroy",
@@ -498,35 +346,9 @@ Do you really want to delete?
 					Usage: "pass args to helm exec",
 				},
 			},
-			Action: func(c *cli.Context) error {
-				affectedReleases := state.AffectedReleases{}
-				errs := cmd.FindAndIterateOverDesiredStatesUsingFlagsWithReverse(c, true, func(state *state.HelmState, helm helmexec.Interface, _ app.Context) []error {
-					args := args.GetArgs(c.String("args"), state)
-					if len(args) > 0 {
-						helm.SetExtraArgs(args...)
-					}
-
-					names := make([]string, len(state.Releases))
-					for i, r := range state.Releases {
-						names[i] = fmt.Sprintf("  %s (%s)", r.Name, r.Chart)
-					}
-
-					msg := fmt.Sprintf(`Affected releases are:
-%s
-
-Do you really want to delete?
-  Helmfile will delete all your releases, as shown above.
-
-`, strings.Join(names, "\n"))
-					interactive := c.GlobalBool("interactive")
-					if !interactive || interactive && askForConfirmation(msg) {
-						return state.DeleteReleases(&affectedReleases, helm, true)
-					}
-					return nil
-				})
-				affectedReleases.DisplayAffectedReleases(c.App.Metadata["logger"].(*zap.SugaredLogger))
-				return errs
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Destroy(c)
+			}),
 		},
 		{
 			Name:  "test",
@@ -552,20 +374,9 @@ Do you really want to delete?
 					Usage: "maximum number of concurrent helm processes to run, 0 is unlimited",
 				},
 			},
-			Action: func(c *cli.Context) error {
-				return findAndIterateOverDesiredStatesUsingFlags(c, func(state *state.HelmState, helm helmexec.Interface, _ app.Context) []error {
-					cleanup := c.Bool("cleanup")
-					timeout := c.Int("timeout")
-					concurrency := c.Int("concurrency")
-
-					args := args.GetArgs(c.String("args"), state)
-					if len(args) > 0 {
-						helm.SetExtraArgs(args...)
-					}
-
-					return state.TestReleases(helm, cleanup, timeout, concurrency)
-				})
-			},
+			Action: action(func(run *app.App, c configImpl) error {
+				return run.Test(c)
+			}),
 		},
 	}
 
@@ -576,39 +387,164 @@ Do you really want to delete?
 	}
 }
 
-func executeSyncCommand(c *cli.Context, affectedReleases *state.AffectedReleases, state *state.HelmState, helm helmexec.Interface) []error {
-	args := args.GetArgs(c.String("args"), state)
-	if len(args) > 0 {
-		helm.SetExtraArgs(args...)
+type configImpl struct {
+	c *cli.Context
+
+	set map[string]interface{}
+}
+
+func NewUrfaveCliConfigImpl(c *cli.Context) (configImpl, error) {
+	if c.NArg() > 0 {
+		cli.ShowAppHelp(c)
+		return configImpl{}, fmt.Errorf("err: extraneous arguments: %s", strings.Join(c.Args(), ", "))
 	}
 
-	values := c.StringSlice("values")
-	workers := c.Int("concurrency")
-
-	return state.SyncReleases(affectedReleases, helm, values, workers)
-}
-
-func executeTemplateCommand(c *cli.Context, state *state.HelmState, helm helmexec.Interface) []error {
-	args := args.GetArgs(c.String("args"), state)
-	values := c.StringSlice("values")
-	workers := c.Int("concurrency")
-
-	return state.TemplateReleases(helm, values, args, workers)
-}
-
-func executeDiffCommand(c *cli.Context, st *state.HelmState, helm helmexec.Interface, detailedExitCode, suppressSecrets bool) ([]*state.ReleaseSpec, []error) {
-	args := args.GetArgs(c.String("args"), st)
-	if len(args) > 0 {
-		helm.SetExtraArgs(args...)
+	conf := configImpl{
+		c: c,
 	}
 
-	values := c.StringSlice("values")
-	workers := c.Int("concurrency")
-	triggerCleanupEvents := c.Command.HasName("diff")
+	optsSet := c.GlobalStringSlice("state-values-set")
+	if len(optsSet) > 0 {
+		set := map[string]interface{}{}
+		for i := range optsSet {
+			ops := strings.Split(optsSet[i], ",")
+			for j := range ops {
+				op := strings.Split(ops[j], "=")
+				k := strings.Split(op[0], ".")
+				v := op[1]
 
-	return st.DiffReleases(helm, values, workers, detailedExitCode, suppressSecrets, triggerCleanupEvents)
+				set = maputil.Set(set, k, v)
+			}
+		}
+		conf.set = set
+	}
+
+	return conf, nil
 }
 
-func findAndIterateOverDesiredStatesUsingFlags(c *cli.Context, converge func(*state.HelmState, helmexec.Interface, app.Context) []error) error {
-	return cmd.FindAndIterateOverDesiredStatesUsingFlagsWithReverse(c, false, converge)
+func (c configImpl) Values() []string {
+	return c.c.StringSlice("values")
+}
+
+func (c configImpl) Args() string {
+	return c.c.String("args")
+}
+
+func (c configImpl) Concurrency() int {
+	return c.c.Int("concurrency")
+}
+
+func (c configImpl) HasCommandName(name string) bool {
+	return c.c.Command.HasName(name)
+}
+
+// DiffConfig
+
+func (c configImpl) SkipDeps() bool {
+	return c.c.Bool("skip-deps")
+}
+
+func (c configImpl) DetailedExitcode() bool {
+	return c.c.Bool("detailed-exitcode")
+}
+
+func (c configImpl) SuppressSecrets() bool {
+	return c.c.Bool("suppress-secrets")
+}
+
+// DeleteConfig
+
+func (c configImpl) Purge() bool {
+	return c.c.Bool("purge")
+}
+
+// TestConfig
+
+func (c configImpl) Cleanup() bool {
+	return c.c.Bool("cleanup")
+}
+
+func (c configImpl) Timeout() int {
+	return c.c.Int("timeout")
+}
+
+// GlobalConfig
+
+func (c configImpl) HelmBinary() string {
+	return c.c.GlobalString("helm-binary")
+}
+
+func (c configImpl) KubeContext() string {
+	return c.c.GlobalString("kube-context")
+}
+
+func (c configImpl) Namespace() string {
+	return c.c.GlobalString("namespace")
+}
+
+func (c configImpl) FileOrDir() string {
+	return c.c.GlobalString("file")
+}
+
+func (c configImpl) Selectors() []string {
+	return c.c.GlobalStringSlice("selector")
+}
+
+func (c configImpl) Set() map[string]interface{} {
+	return c.set
+}
+
+func (c configImpl) ValuesFiles() []string {
+	return c.c.GlobalStringSlice("state-values-file")
+}
+
+func (c configImpl) Interactive() bool {
+	return c.c.GlobalBool("interactive")
+}
+
+func (c configImpl) Logger() *zap.SugaredLogger {
+	return c.c.App.Metadata["logger"].(*zap.SugaredLogger)
+}
+
+func (c configImpl) Env() string {
+	env := c.c.GlobalString("environment")
+	if env == "" {
+		env = state.DefaultEnv
+	}
+	return env
+}
+
+func action(do func(*app.App, configImpl) error) func(*cli.Context) error {
+	return func(implCtx *cli.Context) error {
+		conf, err := NewUrfaveCliConfigImpl(implCtx)
+		if err != nil {
+			return err
+		}
+
+		a := app.New(conf)
+
+		a.ErrorHandler = func(err error) error {
+			return toCliError(implCtx, err)
+		}
+
+		return do(a, conf)
+	}
+}
+
+func toCliError(c *cli.Context, err error) error {
+	if err != nil {
+		switch e := err.(type) {
+		case *app.NoMatchingHelmfileError:
+			noMatchingExitCode := 3
+			if c.GlobalBool("allow-no-matching-release") {
+				noMatchingExitCode = 0
+			}
+			return cli.NewExitError(e.Error(), noMatchingExitCode)
+		case *app.Error:
+			return cli.NewExitError(e.Error(), e.Code())
+		default:
+			panic(fmt.Errorf("BUG: please file an github issue for this unhandled error: %T: %v", e, e))
+		}
+	}
+	return err
 }

pkg/app/app.go

@@ -2,16 +2,20 @@ package app
 
 import (
 	"fmt"
-	"github.com/roboll/helmfile/helmexec"
-	"github.com/roboll/helmfile/state"
-	"go.uber.org/zap"
+	"github.com/roboll/helmfile/pkg/helmexec"
+	"github.com/roboll/helmfile/pkg/remote"
+	"github.com/roboll/helmfile/pkg/state"
 	"io/ioutil"
 	"log"
 	"os"
 	"os/signal"
+	"strings"
+	"syscall"
+
+	"go.uber.org/zap"
+
 	"path/filepath"
 	"sort"
-	"syscall"
 )
 
 type App struct {
@@ -21,8 +25,17 @@ type App struct {
 	Env         string
 	Namespace   string
 	Selectors   []string
+	HelmBinary  string
+	Args        string
+	ValuesFiles []string
+	Set         map[string]interface{}
+
+	FileOrDir string
+
+	ErrorHandler func(error) error
 
 	readFile          func(string) ([]byte, error)
+	fileExists        func(string) (bool, error)
 	glob              func(string) ([]string, error)
 	abs               func(string) (string, error)
 	fileExistsAt      func(string) bool
@@ -30,6 +43,23 @@ type App struct {
 
 	getwd func() (string, error)
 	chdir func(string) error
+
+	remote *remote.Remote
+}
+
+func New(conf ConfigProvider) *App {
+	return Init(&App{
+		KubeContext: conf.KubeContext(),
+		Logger:      conf.Logger(),
+		Env:         conf.Env(),
+		Namespace:   conf.Namespace(),
+		Selectors:   conf.Selectors(),
+		HelmBinary:  conf.HelmBinary(),
+		Args:        conf.Args(),
+		FileOrDir:   conf.FileOrDir(),
+		ValuesFiles: conf.ValuesFiles(),
+		Set:         conf.Set(),
+	})
 }
 
 func Init(app *App) *App {
@@ -39,10 +69,89 @@ func Init(app *App) *App {
 	app.getwd = os.Getwd
 	app.chdir = os.Chdir
 	app.fileExistsAt = fileExistsAt
+	app.fileExists = fileExists
 	app.directoryExistsAt = directoryExistsAt
 	return app
 }
 
+func (a *App) Deps(c DepsConfigProvider) error {
+	return a.ForEachState(func(run *Run) []error {
+		return run.Deps(c)
+	})
+}
+
+func (a *App) Repos(c ReposConfigProvider) error {
+	return a.ForEachState(func(run *Run) []error {
+		return run.Repos(c)
+	})
+}
+
+func (a *App) reverse() *App {
+	new := *a
+	new.Reverse = true
+	return &new
+}
+
+func (a *App) DeprecatedSyncCharts(c DeprecatedChartsConfigProvider) error {
+	return a.ForEachState(func(run *Run) []error {
+		return run.DeprecatedSyncCharts(c)
+	})
+}
+
+func (a *App) Diff(c DiffConfigProvider) error {
+	return a.ForEachState(func(run *Run) []error {
+		return run.Diff(c)
+	})
+}
+
+func (a *App) Template(c TemplateConfigProvider) error {
+	return a.ForEachState(func(run *Run) []error {
+		return run.Template(c)
+	})
+}
+
+func (a *App) Lint(c LintConfigProvider) error {
+	return a.ForEachState(func(run *Run) []error {
+		return run.Lint(c)
+	})
+}
+
+func (a *App) Sync(c SyncConfigProvider) error {
+	return a.ForEachState(func(run *Run) []error {
+		return run.Sync(c)
+	})
+}
+
+func (a *App) Apply(c ApplyConfigProvider) error {
+	return a.ForEachState(func(run *Run) []error {
+		return run.Apply(c)
+	})
+}
+
+func (a *App) Status(c StatusesConfigProvider) error {
+	return a.ForEachState(func(run *Run) []error {
+		return run.Status(c)
+	})
+}
+
+func (a *App) Delete(c DeleteConfigProvider) error {
+	return a.reverse().ForEachState(func(run *Run) []error {
+		return run.Delete(c)
+	})
+}
+
+func (a *App) Destroy(c DestroyConfigProvider) error {
+	return a.reverse().ForEachState(func(run *Run) []error {
+		return run.Destroy(c)
+	})
+}
+
+func (a *App) Test(c TestConfigProvider) error {
+	return a.ForEachState(func(run *Run) []error {
+		return run.Test(c)
+	})
+}
+
 func (a *App) within(dir string, do func() error) error {
 	if dir == "." {
 		return do()
@@ -78,10 +187,10 @@ func (a *App) within(dir string, do func() error) error {
 	return appErr
 }
 
-func (a *App) visitStateFiles(fileOrDir string, do func(string) error) error {
+func (a *App) visitStateFiles(fileOrDir string, do func(string, string) error) error {
 	desiredStateFiles, err := a.findDesiredStateFiles(fileOrDir)
 	if err != nil {
-		return err
+		return appError("", err)
 	}
 
 	for _, relPath := range desiredStateFiles {
@@ -98,44 +207,72 @@ func (a *App) visitStateFiles(fileOrDir string, do func(string) error) error {
 		a.Logger.Debugf("processing file \"%s\" in directory \"%s\"", file, dir)
 
 		err := a.within(dir, func() error {
-			return do(file)
+			absd, err := a.abs(dir)
+			if err != nil {
+				return err
+			}
+
+			return do(file, absd)
 		})
 		if err != nil {
-			return err
+			return appError(fmt.Sprintf("in %s/%s", dir, file), err)
 		}
 	}
 
 	return nil
 }
 
-func (a *App) VisitDesiredStates(fileOrDir string, converge func(*state.HelmState, helmexec.Interface) (bool, []error)) error {
+func (a *App) loadDesiredStateFromYaml(file string, opts ...LoadOpts) (*state.HelmState, error) {
+	ld := &desiredStateLoader{
+		readFile:   a.readFile,
+		fileExists: a.fileExists,
+		env:        a.Env,
+		namespace:  a.Namespace,
+		logger:     a.Logger,
+		abs:        a.abs,
+
+		Reverse:     a.Reverse,
+		KubeContext: a.KubeContext,
+		glob:        a.glob,
+	}
+
+	var op LoadOpts
+	if len(opts) > 0 {
+		op = opts[0]
+	}
+
+	return ld.Load(file, op)
+}
+
+func (a *App) visitStates(fileOrDir string, defOpts LoadOpts, converge func(*state.HelmState, helmexec.Interface) (bool, []error)) error {
 	noMatchInHelmfiles := true
 
-	err := a.visitStateFiles(fileOrDir, func(f string) error {
-		content, err := a.readFile(f)
-		if err != nil {
-			return err
-		}
-		// render template, in two runs
-		r := &twoPassRenderer{
-			reader:    a.readFile,
-			env:       a.Env,
-			namespace: a.Namespace,
-			filename:  f,
-			logger:    a.Logger,
-			abs:       a.abs,
-		}
-		yamlBuf, err := r.renderTemplate(content)
-		if err != nil {
-			return fmt.Errorf("error during %s parsing: %v", f, err)
+	err := a.visitStateFiles(fileOrDir, func(f, d string) error {
+		opts := defOpts.DeepCopy()
+
+		if opts.CalleePath == "" {
+			opts.CalleePath = f
 		}
 
-		st, err := a.loadDesiredStateFromYaml(
-			yamlBuf.Bytes(),
-			f,
-			a.Namespace,
-			a.Env,
-		)
+		st, err := a.loadDesiredStateFromYaml(f, opts)
+
+		sigs := make(chan os.Signal, 1)
+		signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
+		go func() {
+			sig := <-sigs
+
+			errs := []error{fmt.Errorf("Received [%s] to shutdown ", sig)}
+			_ = context{a, st}.clean(errs)
+			// See http://tldp.org/LDP/abs/html/exitcodes.html
+			switch sig {
+			case syscall.SIGINT:
+				os.Exit(130)
+			case syscall.SIGTERM:
+				os.Exit(143)
+			}
+		}()
+
+		ctx := context{a, st}
 
 		helm := helmexec.New(a.Logger, a.KubeContext)
 
@@ -147,22 +284,34 @@ func (a *App) VisitDesiredStates(fileOrDir string, converge func(*state.HelmStat
 				case *state.UndefinedEnvError:
 					return nil
 				default:
-					return err
+					return ctx.wrapErrs(err)
 				}
 			default:
-				return err
+				return ctx.wrapErrs(err)
 			}
 		}
+		st.Selectors = opts.Selectors
 
 		if len(st.Helmfiles) > 0 {
 			noMatchInSubHelmfiles := true
-			for _, m := range st.Helmfiles {
-				if err := a.VisitDesiredStates(m, converge); err != nil {
+			for i, m := range st.Helmfiles {
+				optsForNestedState := LoadOpts{
+					CalleePath:  filepath.Join(d, f),
+					Environment: m.Environment,
+				}
+				//assign parent selector to sub helm selector in legacy mode or do not inherit in experimental mode
+				if (m.Selectors == nil && !isExplicitSelectorInheritanceEnabled()) || m.SelectorsInherited {
+					optsForNestedState.Selectors = opts.Selectors
+				} else {
+					optsForNestedState.Selectors = m.Selectors
+				}
+
+				if err := a.visitStates(m.Path, optsForNestedState, converge); err != nil {
 					switch err.(type) {
 					case *NoMatchingHelmfileError:
 
 					default:
-						return fmt.Errorf("failed processing %s: %v", m, err)
+						return appError(fmt.Sprintf("in .helmfiles[%d]", i), err)
 					}
 				} else {
 					noMatchInSubHelmfiles = false
@@ -173,11 +322,11 @@ func (a *App) VisitDesiredStates(fileOrDir string, converge func(*state.HelmStat
 
 		templated, tmplErr := st.ExecuteTemplates()
 		if tmplErr != nil {
-			return fmt.Errorf("failed executing release templates in \"%s\": %v", f, tmplErr)
+			return appError(fmt.Sprintf("failed executing release templates in \"%s\"", f), tmplErr)
 		}
 		processed, errs := converge(templated, helm)
 		noMatchInHelmfiles = noMatchInHelmfiles && !processed
-		return clean(templated, errs)
+		return context{a, templated}.clean(errs)
 	})
 
 	if err != nil {
@@ -191,20 +340,77 @@ func (a *App) VisitDesiredStates(fileOrDir string, converge func(*state.HelmStat
 	return nil
 }
 
-func (a *App) VisitDesiredStatesWithReleasesFiltered(fileOrDir string, converge func(*state.HelmState, helmexec.Interface) []error) error {
-	selectors := a.Selectors
+func (a *App) ForEachState(do func(*Run) []error) error {
+	err := a.VisitDesiredStatesWithReleasesFiltered(a.FileOrDir, func(st *state.HelmState, helm helmexec.Interface) []error {
+		ctx := NewContext()
 
-	err := a.VisitDesiredStates(fileOrDir, func(st *state.HelmState, helm helmexec.Interface) (bool, []error) {
-		if len(selectors) > 0 {
-			err := st.FilterReleases(selectors)
+		run := NewRun(st, helm, ctx)
+
+		return do(run)
+	})
+
+	if err != nil && a.ErrorHandler != nil {
+		return a.ErrorHandler(err)
+	}
+
+	return err
+}
+
+func (a *App) VisitDesiredStatesWithReleasesFiltered(fileOrDir string, converge func(*state.HelmState, helmexec.Interface) []error) error {
+	opts := LoadOpts{
+		Selectors: a.Selectors,
+	}
+
+	envvals := []interface{}{}
+
+	if a.ValuesFiles != nil {
+		for i := range a.ValuesFiles {
+			envvals = append(envvals, a.ValuesFiles[i])
+		}
+	}
+
+	if a.Set != nil {
+		envvals = append(envvals, a.Set)
+	}
+
+	if len(envvals) > 0 {
+		opts.Environment.OverrideValues = envvals
+	}
+
+	dir, err := a.getwd()
+	if err != nil {
+		return err
+	}
+
+	getter := &remote.GoGetter{Logger: a.Logger}
+
+	remote := &remote.Remote{
+		Logger:     a.Logger,
+		Home:       dir,
+		Getter:     getter,
+		ReadFile:   a.readFile,
+		DirExists:  a.directoryExistsAt,
+		FileExists: a.fileExistsAt,
+	}
+
+	a.remote = remote
+
+	return a.visitStates(fileOrDir, opts, func(st *state.HelmState, helm helmexec.Interface) (bool, []error) {
+		if len(st.Selectors) > 0 {
+			err := st.FilterReleases()
 			if err != nil {
 				return false, []error{err}
 			}
 		}
 
+		if a.HelmBinary != "" {
+			helm.SetHelmBinary(a.HelmBinary)
+		}
+
 		type Key struct {
 			TillerNamespace, Name string
 		}
+
 		releaseNameCounts := map[Key]int{}
 		for _, r := range st.Releases {
 			tillerNamespace := st.HelmDefaults.TillerNamespace
@@ -225,10 +431,6 @@ func (a *App) VisitDesiredStatesWithReleasesFiltered(fileOrDir string, converge
 
 		return processed, errs
 	})
-	if err != nil {
-		return err
-	}
-	return nil
 }
 
 func (a *App) findStateFilesInAbsPaths(specifiedPath string) ([]string, error) {
@@ -248,6 +450,15 @@ func (a *App) findStateFilesInAbsPaths(specifiedPath string) ([]string, error) {
 }
 
 func (a *App) findDesiredStateFiles(specifiedPath string) ([]string, error) {
+	path, err := a.remote.Locate(specifiedPath)
+	if err != nil {
+		return nil, fmt.Errorf("locate: %v", err)
+	}
+	if specifiedPath != path {
+		a.Logger.Debugf("fetched remote \"%s\" to local cache \"%s\" and loading the latter...", specifiedPath, path)
+	}
+	specifiedPath = path
+
 	var helmfileDir string
 	if specifiedPath != "" {
 		if a.fileExistsAt(specifiedPath) {
@@ -280,11 +491,11 @@ func (a *App) findDesiredStateFiles(specifiedPath string) ([]string, error) {
 		} else if defaultFile != "" {
 			return []string{defaultFile}, nil
 		} else {
-			return []string{}, fmt.Errorf("no state file found. It must be named %s/*.yaml, %s, or %s, or otherwise specified with the --file flag", DefaultHelmfileDirectory, DefaultHelmfile, DeprecatedHelmfile)
+			return []string{}, fmt.Errorf("no state file found. It must be named %s/*.{yaml,yml}, %s, or %s, or otherwise specified with the --file flag", DefaultHelmfileDirectory, DefaultHelmfile, DeprecatedHelmfile)
 		}
 	}
 
-	files, err := a.glob(filepath.Join(helmfileDir, "*.yaml"))
+	files, err := a.glob(filepath.Join(helmfileDir, "*.y*ml"))
 	if err != nil {
 		return []string{}, err
 	}
@@ -299,98 +510,121 @@ func fileExistsAt(path string) bool {
 	return err == nil && fileInfo.Mode().IsRegular()
 }
 
+func fileExists(path string) (bool, error) {
+	_, err := os.Stat(path)
+
+	if err != nil {
+		if os.IsNotExist(err) {
+			return false, nil
+		}
+		return false, err
+	}
+	return true, nil
+}
+
 func directoryExistsAt(path string) bool {
 	fileInfo, err := os.Stat(path)
 	return err == nil && fileInfo.Mode().IsDir()
 }
 
-func (a *App) loadDesiredStateFromYaml(yaml []byte, file string, namespace string, env string) (*state.HelmState, error) {
-	c := state.NewCreator(a.Logger, a.readFile, a.abs)
-	st, err := c.CreateFromYaml(yaml, file, env)
-	if err != nil {
-		return nil, err
-	}
+type Error struct {
+	msg string
 
-	helmfiles := []string{}
-	for _, globPattern := range st.Helmfiles {
-		var absPathPattern string
-		if filepath.IsAbs(globPattern) {
-			absPathPattern = globPattern
-		} else {
-			absPathPattern = st.JoinBase(globPattern)
-		}
-		matches, err := a.glob(absPathPattern)
-		if err != nil {
-			return nil, fmt.Errorf("failed processing %s: %v", globPattern, err)
-		}
-		sort.Strings(matches)
-
-		helmfiles = append(helmfiles, matches...)
-	}
-	st.Helmfiles = helmfiles
-
-	if a.Reverse {
-		rev := func(i, j int) bool {
-			return j < i
-		}
-		sort.Slice(st.Releases, rev)
-		sort.Slice(st.Helmfiles, rev)
-	}
-
-	if a.KubeContext != "" {
-		if st.HelmDefaults.KubeContext != "" {
-			log.Printf("err: Cannot use option --kube-context and set attribute helmDefaults.kubeContext.")
-			os.Exit(1)
-		}
-		st.HelmDefaults.KubeContext = a.KubeContext
-	}
-	if namespace != "" {
-		if st.Namespace != "" {
-			log.Printf("err: Cannot use option --namespace and set attribute namespace.")
-			os.Exit(1)
-		}
-		st.Namespace = namespace
-	}
-
-	sigs := make(chan os.Signal, 1)
-	signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
-	go func() {
-		sig := <-sigs
-
-		errs := []error{fmt.Errorf("Received [%s] to shutdown ", sig)}
-		_ = clean(st, errs)
-		// See http://tldp.org/LDP/abs/html/exitcodes.html
-		switch sig {
-		case syscall.SIGINT:
-			os.Exit(130)
-		case syscall.SIGTERM:
-			os.Exit(143)
-		}
-	}()
-
-	return st, nil
+	Errors []error
 }
 
-func clean(st *state.HelmState, errs []error) error {
+func (e *Error) Error() string {
+	var cause string
+	if e.Errors == nil {
+		return e.msg
+	}
+	if len(e.Errors) == 1 {
+		if e.Errors[0] == nil {
+			panic(fmt.Sprintf("[bug] assertion error: unexpected state: e.Errors: %v", e.Errors))
+		}
+		cause = e.Errors[0].Error()
+	} else {
+		msgs := []string{}
+		for i, err := range e.Errors {
+			if err == nil {
+				continue
+			}
+			msgs = append(msgs, fmt.Sprintf("err %d: %v", i, err.Error()))
+		}
+		cause = fmt.Sprintf("%d errors:\n%s", len(e.Errors), strings.Join(msgs, "\n"))
+	}
+	msg := ""
+	if e.msg != "" {
+		msg = fmt.Sprintf("%s: %s", e.msg, cause)
+	} else {
+		msg = cause
+	}
+	return msg
+}
+
+func (e *Error) Code() int {
+	allDiff := false
+	anyNonZero := false
+	for _, err := range e.Errors {
+		switch ee := err.(type) {
+		case *state.ReleaseError:
+			if anyNonZero {
+				allDiff = allDiff && ee.Code == 2
+			} else {
+				allDiff = ee.Code == 2
+			}
+		case *Error:
+			if anyNonZero {
+				allDiff = allDiff && ee.Code() == 2
+			} else {
+				allDiff = ee.Code() == 2
+			}
+		}
+		anyNonZero = true
+	}
+
+	if anyNonZero {
+		if allDiff {
+			return 2
+		}
+		return 1
+	}
+	panic(fmt.Sprintf("[bug] assertion error: unexpected state: unable to handle errors: %v", e.Errors))
+}
+
+func appError(msg string, err error) error {
+	return &Error{msg, []error{err}}
+}
+
+func (c context) clean(errs []error) error {
 	if errs == nil {
 		errs = []error{}
 	}
 
-	cleanErrs := st.Clean()
+	cleanErrs := c.st.Clean()
 	if cleanErrs != nil {
 		errs = append(errs, cleanErrs...)
 	}
 
+	return c.wrapErrs(errs...)
+}
+
+type context struct {
+	app *App
+	st  *state.HelmState
+}
+
+func (c context) wrapErrs(errs ...error) error {
 	if errs != nil && len(errs) > 0 {
 		for _, err := range errs {
 			switch e := err.(type) {
 			case *state.ReleaseError:
-				fmt.Printf("err: release \"%s\" in \"%s\" failed: %v\n", e.Name, st.FilePath, e)
+				c.app.Logger.Debugf("err: release \"%s\" in \"%s\" failed: %v", e.Name, c.st.FilePath, e)
 			default:
-				fmt.Printf("err: %v\n", e)
+				c.app.Logger.Debugf("err: %v", e)
 			}
 		}
-		return errs[0]
+		return &Error{Errors: errs}
 	}
 	return nil
 }

pkg/app/ask.go

@@ -1,4 +1,4 @@
-package main
+package app
 
 import (
 	"bufio"
@@ -11,7 +11,7 @@ import (
 // Copyright (c) 2017 Roland Singer [roland.singer@desertbit.com]
 //
 // Shamelessly borrowed from @r0l1's awesome work that is available at https://gist.github.com/r0l1/3dcbb0c8f6cfe9c66ab8008f55f8f28b
-func askForConfirmation(s string) bool {
+func AskForConfirmation(s string) bool {
 	reader := bufio.NewReader(os.Stdin)
 
 	for {

pkg/app/config.go

@@ -0,0 +1,130 @@
+package app
+
+import "go.uber.org/zap"
+
+type ConfigProvider interface {
+	Args() string
+	HelmBinary() string
+
+	FileOrDir() string
+	KubeContext() string
+	Namespace() string
+	Selectors() []string
+	Set() map[string]interface{}
+	ValuesFiles() []string
+	Env() string
+
+	loggingConfig
+}
+
+type DeprecatedChartsConfigProvider interface {
+	Values() []string
+
+	concurrencyConfig
+	loggingConfig
+}
+
+type DepsConfigProvider interface {
+	Args() string
+}
+
+type ReposConfigProvider interface {
+	Args() string
+}
+
+type ApplyConfigProvider interface {
+	Args() string
+
+	Values() []string
+	SkipDeps() bool
+
+	SuppressSecrets() bool
+
+	concurrencyConfig
+	interactive
+	loggingConfig
+}
+
+type SyncConfigProvider interface {
+	Args() string
+
+	Values() []string
+	SkipDeps() bool
+
+	concurrencyConfig
+	loggingConfig
+}
+
+type DiffConfigProvider interface {
+	Args() string
+
+	Values() []string
+	SkipDeps() bool
+
+	SuppressSecrets() bool
+
+	DetailedExitcode() bool
+
+	concurrencyConfig
+}
+
+type DeleteConfigProvider interface {
+	Args() string
+
+	Purge() bool
+
+	interactive
+	loggingConfig
+}
+
+type DestroyConfigProvider interface {
+	Args() string
+
+	interactive
+	loggingConfig
+}
+
+type TestConfigProvider interface {
+	Args() string
+
+	Timeout() int
+	Cleanup() bool
+
+	concurrencyConfig
+}
+
+type LintConfigProvider interface {
+	Args() string
+
+	Values() []string
+	SkipDeps() bool
+
+	concurrencyConfig
+}
+
+type TemplateConfigProvider interface {
+	Args() string
+
+	Values() []string
+	SkipDeps() bool
+
+	concurrencyConfig
+}
+
+type StatusesConfigProvider interface {
+	Args() string
+
+	concurrencyConfig
+}
+
+type concurrencyConfig interface {
+	Concurrency() int
+}
+
+type loggingConfig interface {
+	Logger() *zap.SugaredLogger
+}
+
+type interactive interface {
+	Interactive() bool
+}

pkg/app/constants.go

@@ -1,7 +1,22 @@
 package app
 
-const (
-	DefaultHelmfile          = "helmfile.yaml"
-	DeprecatedHelmfile       = "charts.yaml"
-	DefaultHelmfileDirectory = "helmfile.d"
+import (
+	"os"
+	"strings"
 )
+
+const (
+	DefaultHelmfile              = "helmfile.yaml"
+	DeprecatedHelmfile           = "charts.yaml"
+	DefaultHelmfileDirectory     = "helmfile.d"
+	ExperimentalEnvVar           = "HELMFILE_EXPERIMENTAL"         // environment variable for experimental features, expecting "true" lower case
+	ExperimentalSelectorExplicit = "explicit-selector-inheritance" // value to remove default selector inheritance to sub-helmfiles and use the explicit one
+)
+
+func experimentalModeEnabled() bool {
+	return os.Getenv(ExperimentalEnvVar) == "true"
+}
+
+func isExplicitSelectorInheritanceEnabled() bool {
+	return experimentalModeEnabled() || strings.Contains(os.Getenv(ExperimentalEnvVar), ExperimentalSelectorExplicit)
+}

pkg/app/constants_test.go

@@ -0,0 +1,27 @@
+package app
+
+import (
+	"gotest.tools/assert"
+	is "gotest.tools/assert/cmp"
+	"gotest.tools/env"
+	"os"
+	"testing"
+)
+
+func TestIsExplicitSelectorInheritanceEnabled(t *testing.T) {
+	//env var ExperimentalEnvVar is set
+	assert.Assert(t, is.Equal(os.Getenv(ExperimentalEnvVar), ""))
+	assert.Check(t, !isExplicitSelectorInheritanceEnabled())
+
+	//check for env var ExperimentalEnvVar set to true
+	defer env.Patch(t, ExperimentalEnvVar, "true")()
+	assert.Check(t, isExplicitSelectorInheritanceEnabled())
+
+	//check for env var ExperimentalEnvVar set to anything
+	defer env.Patch(t, ExperimentalEnvVar, "foo")()
+	assert.Check(t, !isExplicitSelectorInheritanceEnabled())
+
+	//check for env var ExperimentalEnvVar set to ExperimentalSelectorExplicit
+	defer env.Patch(t, ExperimentalEnvVar, ExperimentalSelectorExplicit)()
+	assert.Check(t, isExplicitSelectorInheritanceEnabled())
+}

pkg/app/context.go

@@ -1,6 +1,8 @@
 package app
 
-import "github.com/roboll/helmfile/state"
+import (
+	"github.com/roboll/helmfile/pkg/state"
+)
 
 type Context struct {
 	updatedRepos map[string]struct{}

pkg/app/desired_state_file_loader.go

@@ -0,0 +1,202 @@
+package app
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"github.com/imdario/mergo"
+	"github.com/roboll/helmfile/pkg/environment"
+	"github.com/roboll/helmfile/pkg/state"
+	"go.uber.org/zap"
+	"path/filepath"
+	"sort"
+)
+
+type desiredStateLoader struct {
+	KubeContext string
+	Reverse     bool
+
+	env       string
+	namespace string
+
+	readFile   func(string) ([]byte, error)
+	fileExists func(string) (bool, error)
+	abs        func(string) (string, error)
+	glob       func(string) ([]string, error)
+
+	logger *zap.SugaredLogger
+}
+
+func (ld *desiredStateLoader) Load(f string, opts LoadOpts) (*state.HelmState, error) {
+	var overrodeEnv *environment.Environment
+
+	args := opts.Environment.OverrideValues
+
+	if len(args) > 0 {
+		if opts.CalleePath == "" {
+			return nil, fmt.Errorf("bug: opts.CalleePath was nil: f=%s, opts=%v", f, opts)
+		}
+		storage := state.NewStorage(opts.CalleePath, ld.logger, ld.glob)
+		envld := state.NewEnvironmentValuesLoader(storage, ld.readFile, ld.logger)
+		handler := state.MissingFileHandlerError
+		vals, err := envld.LoadEnvironmentValues(&handler, args)
+		if err != nil {
+			return nil, err
+		}
+
+		overrodeEnv = &environment.Environment{
+			Name:   ld.env,
+			Values: vals,
+		}
+	}
+
+	st, err := ld.loadFileWithOverrides(nil, overrodeEnv, filepath.Dir(f), filepath.Base(f), true)
+	if err != nil {
+		return nil, err
+	}
+
+	if ld.Reverse {
+		rev := func(i, j int) bool {
+			return j < i
+		}
+		sort.Slice(st.Releases, rev)
+		sort.Slice(st.Helmfiles, rev)
+	}
+
+	if ld.KubeContext != "" {
+		if st.HelmDefaults.KubeContext != "" {
+			return nil, errors.New("err: Cannot use option --kube-context and set attribute helmDefaults.kubeContext.")
+		}
+		st.HelmDefaults.KubeContext = ld.KubeContext
+	}
+
+	if ld.namespace != "" {
+		if st.Namespace != "" {
+			return nil, errors.New("err: Cannot use option --namespace and set attribute namespace.")
+		}
+		st.Namespace = ld.namespace
+	}
+
+	return st, nil
+}
+
+func (ld *desiredStateLoader) loadFile(inheritedEnv *environment.Environment, baseDir, file string, evaluateBases bool) (*state.HelmState, error) {
+	return ld.loadFileWithOverrides(inheritedEnv, nil, baseDir, file, evaluateBases)
+}
+
+func (ld *desiredStateLoader) loadFileWithOverrides(inheritedEnv, overrodeEnv *environment.Environment, baseDir, file string, evaluateBases bool) (*state.HelmState, error) {
+	var f string
+	if filepath.IsAbs(file) {
+		f = file
+	} else {
+		f = filepath.Join(baseDir, file)
+	}
+
+	fileBytes, err := ld.readFile(f)
+	if err != nil {
+		return nil, err
+	}
+
+	ext := filepath.Ext(f)
+
+	var self *state.HelmState
+
+	if !experimentalModeEnabled() || ext == ".gotmpl" {
+		self, err = ld.renderAndLoad(
+			inheritedEnv,
+			overrodeEnv,
+			baseDir,
+			f,
+			fileBytes,
+			evaluateBases,
+		)
+	} else {
+		self, err = ld.load(
+			fileBytes,
+			baseDir,
+			file,
+			evaluateBases,
+			inheritedEnv,
+			overrodeEnv,
+		)
+	}
+
+	return self, err
+}
+
+func (a *desiredStateLoader) underlying() *state.StateCreator {
+	c := state.NewCreator(a.logger, a.readFile, a.fileExists, a.abs, a.glob)
+	c.LoadFile = a.loadFile
+	return c
+}
+
+func (a *desiredStateLoader) load(yaml []byte, baseDir, file string, evaluateBases bool, env, overrodeEnv *environment.Environment) (*state.HelmState, error) {
+	merged, err := env.Merge(overrodeEnv)
+	if err != nil {
+		return nil, err
+	}
+
+	st, err := a.underlying().ParseAndLoad(yaml, baseDir, file, a.env, evaluateBases, merged)
+	if err != nil {
+		return nil, err
+	}
+
+	helmfiles, err := st.ExpandedHelmfiles()
+	if err != nil {
+		return nil, err
+	}
+	st.Helmfiles = helmfiles
+
+	return st, nil
+}
+
+func (ld *desiredStateLoader) renderAndLoad(env, overrodeEnv *environment.Environment, baseDir, filename string, content []byte, evaluateBases bool) (*state.HelmState, error) {
+	parts := bytes.Split(content, []byte("\n---\n"))
+
+	var finalState *state.HelmState
+
+	for i, part := range parts {
+		var yamlBuf *bytes.Buffer
+		var err error
+
+		id := fmt.Sprintf("%s.part.%d", filename, i)
+
+		if env == nil && overrodeEnv == nil {
+			yamlBuf, err = ld.renderTemplatesToYaml(baseDir, id, part)
+			if err != nil {
+				return nil, fmt.Errorf("error during %s parsing: %v", id, err)
+			}
+		} else {
+			yamlBuf, err = ld.renderTemplatesToYamlWithEnv(baseDir, id, part, env, overrodeEnv)
+			if err != nil {
+				return nil, fmt.Errorf("error during %s parsing: %v", id, err)
+			}
+		}
+
+		currentState, err := ld.load(
+			yamlBuf.Bytes(),
+			baseDir,
+			filename,
+			evaluateBases,
+			env,
+			overrodeEnv,
+		)
+		if err != nil {
+			return nil, err
+		}
+
+		if finalState == nil {
+			finalState = currentState
+		} else {
+			if err := mergo.Merge(finalState, currentState, mergo.WithOverride); err != nil {
+				return nil, err
+			}
+		}
+
+		env = &finalState.Env
+
+		ld.logger.Debugf("merged environment: %v", env)
+	}
+
+	return finalState, nil
+}

pkg/app/load_opts.go

@@ -0,0 +1,28 @@
+package app
+
+import (
+	"github.com/roboll/helmfile/pkg/state"
+	"gopkg.in/yaml.v2"
+)
+
+type LoadOpts struct {
+	Selectors   []string
+	Environment state.SubhelmfileEnvironmentSpec
+
+	// CalleePath is the absolute path to the file being loaded
+	CalleePath string
+}
+
+func (o LoadOpts) DeepCopy() LoadOpts {
+	bytes, err := yaml.Marshal(o)
+	if err != nil {
+		panic(err)
+	}
+
+	new := LoadOpts{}
+	if err := yaml.Unmarshal(bytes, &new); err != nil {
+		panic(err)
+	}
+
+	return new
+}

pkg/app/run.go

@@ -0,0 +1,306 @@
+package app
+
+import (
+	"fmt"
+	"github.com/roboll/helmfile/pkg/argparser"
+	"github.com/roboll/helmfile/pkg/helmexec"
+	"github.com/roboll/helmfile/pkg/state"
+	"strings"
+)
+
+type Run struct {
+	state *state.HelmState
+	helm  helmexec.Interface
+	ctx   Context
+
+	Ask func(string) bool
+}
+
+func NewRun(st *state.HelmState, helm helmexec.Interface, ctx Context) *Run {
+	return &Run{state: st, helm: helm, ctx: ctx}
+}
+
+func (r *Run) askForConfirmation(msg string) bool {
+	if r.Ask != nil {
+		return r.Ask(msg)
+	}
+	return AskForConfirmation(msg)
+}
+
+func (r *Run) Deps(c DepsConfigProvider) []error {
+	r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
+
+	if errs := r.ctx.SyncReposOnce(r.state, r.helm); errs != nil && len(errs) > 0 {
+		return errs
+	}
+
+	return r.state.UpdateDeps(r.helm)
+}
+
+func (r *Run) Repos(c ReposConfigProvider) []error {
+	r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
+
+	return r.ctx.SyncReposOnce(r.state, r.helm)
+}
+
+func (r *Run) DeprecatedSyncCharts(c DeprecatedChartsConfigProvider) []error {
+	st := r.state
+	helm := r.helm
+
+	affectedReleases := state.AffectedReleases{}
+	errs := st.SyncReleases(&affectedReleases, helm, c.Values(), c.Concurrency())
+	affectedReleases.DisplayAffectedReleases(c.Logger())
+	return errs
+}
+
+func (r *Run) Status(c StatusesConfigProvider) []error {
+	workers := c.Concurrency()
+
+	r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
+
+	return r.state.ReleaseStatuses(r.helm, workers)
+}
+
+func (r *Run) Delete(c DeleteConfigProvider) []error {
+	affectedReleases := state.AffectedReleases{}
+	purge := c.Purge()
+
+	errs := []error{}
+
+	names := make([]string, len(r.state.Releases))
+	for i, r := range r.state.Releases {
+		names[i] = fmt.Sprintf("  %s (%s)", r.Name, r.Chart)
+	}
+
+	msg := fmt.Sprintf(`Affected releases are:
+%s
+
+Do you really want to delete?
+  Helmfile will delete all your releases, as shown above.
+
+`, strings.Join(names, "\n"))
+	interactive := c.Interactive()
+	if !interactive || interactive && r.askForConfirmation(msg) {
+		r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
+
+		errs = r.state.DeleteReleases(&affectedReleases, r.helm, purge)
+	}
+	affectedReleases.DisplayAffectedReleases(c.Logger())
+	return errs
+}
+
+func (r *Run) Destroy(c DestroyConfigProvider) []error {
+	errs := []error{}
+	affectedReleases := state.AffectedReleases{}
+
+	names := make([]string, len(r.state.Releases))
+	for i, r := range r.state.Releases {
+		names[i] = fmt.Sprintf("  %s (%s)", r.Name, r.Chart)
+	}
+
+	msg := fmt.Sprintf(`Affected releases are:
+%s
+
+Do you really want to delete?
+  Helmfile will delete all your releases, as shown above.
+
+`, strings.Join(names, "\n"))
+	interactive := c.Interactive()
+	if !interactive || interactive && r.askForConfirmation(msg) {
+		r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
+
+		errs = r.state.DeleteReleases(&affectedReleases, r.helm, true)
+	}
+	affectedReleases.DisplayAffectedReleases(c.Logger())
+	return errs
+}
+
+func (r *Run) Apply(c ApplyConfigProvider) []error {
+	st := r.state
+	helm := r.helm
+	ctx := r.ctx
+
+	affectedReleases := state.AffectedReleases{}
+	if !c.SkipDeps() {
+		if errs := ctx.SyncReposOnce(st, helm); errs != nil && len(errs) > 0 {
+			return errs
+		}
+		if errs := st.BuildDeps(helm); errs != nil && len(errs) > 0 {
+			return errs
+		}
+	}
+	if errs := st.PrepareReleases(helm, "apply"); errs != nil && len(errs) > 0 {
+		return errs
+	}
+
+	// helm must be 2.11+ and helm-diff should be provided `--detailed-exitcode` in order for `helmfile apply` to work properly
+	detailedExitCode := true
+
+	releases, errs := st.DiffReleases(helm, c.Values(), c.Concurrency(), detailedExitCode, c.SuppressSecrets(), false)
+
+	releasesToBeDeleted, err := st.DetectReleasesToBeDeleted(helm)
+	if err != nil {
+		errs = append(errs, err)
+	}
+
+	fatalErrs := []error{}
+
+	noError := true
+	for _, e := range errs {
+		switch err := e.(type) {
+		case *state.ReleaseError:
+			if err.Code != 2 {
+				noError = false
+				fatalErrs = append(fatalErrs, e)
+			}
+		default:
+			noError = false
+			fatalErrs = append(fatalErrs, e)
+		}
+	}
+
+	// sync only when there are changes
+	if noError {
+		if len(releases) == 0 && len(releasesToBeDeleted) == 0 {
+			// TODO better way to get the logger
+			logger := c.Logger()
+			logger.Infof("")
+			logger.Infof("No affected releases")
+		} else {
+			names := []string{}
+			for _, r := range releases {
+				names = append(names, fmt.Sprintf("  %s (%s) UPDATED", r.Name, r.Chart))
+			}
+			for _, r := range releasesToBeDeleted {
+				names = append(names, fmt.Sprintf("  %s (%s) DELETED", r.Name, r.Chart))
+			}
+
+			msg := fmt.Sprintf(`Affected releases are:
+%s
+
+Do you really want to apply?
+  Helmfile will apply all your changes, as shown above.
+
+`, strings.Join(names, "\n"))
+			interactive := c.Interactive()
+			if !interactive || interactive && r.askForConfirmation(msg) {
+				rs := []state.ReleaseSpec{}
+				for _, r := range releases {
+					rs = append(rs, *r)
+				}
+				for _, r := range releasesToBeDeleted {
+					rs = append(rs, *r)
+				}
+
+				r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
+
+				st.Releases = rs
+				return st.SyncReleases(&affectedReleases, helm, c.Values(), c.Concurrency())
+			}
+		}
+	}
+
+	affectedReleases.DisplayAffectedReleases(c.Logger())
+	return fatalErrs
+}
+
+func (r *Run) Diff(c DiffConfigProvider) []error {
+	st := r.state
+	helm := r.helm
+	ctx := r.ctx
+
+	if !c.SkipDeps() {
+		if errs := ctx.SyncReposOnce(st, helm); errs != nil && len(errs) > 0 {
+			return errs
+		}
+		if errs := st.BuildDeps(helm); errs != nil && len(errs) > 0 {
+			return errs
+		}
+	}
+	if errs := st.PrepareReleases(helm, "diff"); errs != nil && len(errs) > 0 {
+		return errs
+	}
+
+	r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
+
+	_, errs := st.DiffReleases(helm, c.Values(), c.Concurrency(), c.DetailedExitcode(), c.SuppressSecrets(), true)
+	return errs
+}
+
+func (r *Run) Sync(c SyncConfigProvider) []error {
+	st := r.state
+	helm := r.helm
+	ctx := r.ctx
+
+	affectedReleases := state.AffectedReleases{}
+	if !c.SkipDeps() {
+		if errs := ctx.SyncReposOnce(st, helm); errs != nil && len(errs) > 0 {
+			return errs
+		}
+		if errs := st.BuildDeps(helm); errs != nil && len(errs) > 0 {
+			return errs
+		}
+	}
+	if errs := st.PrepareReleases(helm, "sync"); errs != nil && len(errs) > 0 {
+		return errs
+	}
+
+	r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
+
+	errs := st.SyncReleases(&affectedReleases, helm, c.Values(), c.Concurrency())
+	affectedReleases.DisplayAffectedReleases(c.Logger())
+	return errs
+}
+
+func (r *Run) Template(c TemplateConfigProvider) []error {
+	state := r.state
+	helm := r.helm
+	ctx := r.ctx
+
+	if !c.SkipDeps() {
+		if errs := ctx.SyncReposOnce(state, helm); errs != nil && len(errs) > 0 {
+			return errs
+		}
+		if errs := state.BuildDeps(helm); errs != nil && len(errs) > 0 {
+			return errs
+		}
+	}
+	if errs := state.PrepareReleases(helm, "template"); errs != nil && len(errs) > 0 {
+		return errs
+	}
+
+	args := argparser.GetArgs(c.Args(), state)
+	return state.TemplateReleases(helm, c.Values(), args, c.Concurrency())
+}
+
+func (r *Run) Test(c TestConfigProvider) []error {
+	cleanup := c.Cleanup()
+	timeout := c.Timeout()
+	concurrency := c.Concurrency()
+
+	r.helm.SetExtraArgs(argparser.GetArgs(c.Args(), r.state)...)
+
+	return r.state.TestReleases(r.helm, cleanup, timeout, concurrency)
+}
+
+func (r *Run) Lint(c LintConfigProvider) []error {
+	state := r.state
+	helm := r.helm
+	ctx := r.ctx
+
+	values := c.Values()
+	args := argparser.GetArgs(c.Args(), state)
+	workers := c.Concurrency()
+	if !c.SkipDeps() {
+		if errs := ctx.SyncReposOnce(state, helm); errs != nil && len(errs) > 0 {
+			return errs
+		}
+		if errs := state.BuildDeps(helm); errs != nil && len(errs) > 0 {
+			return errs
+		}
+	}
+	if errs := state.PrepareReleases(helm, "lint"); errs != nil && len(errs) > 0 {
+		return errs
+	}
+	return state.LintReleases(helm, values, args, workers)
+}

pkg/app/two_pass_renderer.go

@@ -3,11 +3,10 @@ package app
 import (
 	"bytes"
 	"fmt"
-	"github.com/roboll/helmfile/environment"
-	"github.com/roboll/helmfile/state"
-	"github.com/roboll/helmfile/tmpl"
-	"go.uber.org/zap"
-	"path/filepath"
+	"github.com/google/go-cmp/cmp"
+	"github.com/roboll/helmfile/pkg/environment"
+	"github.com/roboll/helmfile/pkg/state"
+	"github.com/roboll/helmfile/pkg/tmpl"
 	"strings"
 )
 
@@ -20,61 +19,127 @@ func prependLineNumbers(text string) string {
 	return buf.String()
 }
 
-type twoPassRenderer struct {
-	reader    func(string) ([]byte, error)
-	env       string
-	namespace string
-	filename  string
-	logger    *zap.SugaredLogger
-	abs       func(string) (string, error)
-}
-
-func (r *twoPassRenderer) renderEnvironment(content []byte) environment.Environment {
-	firstPassEnv := environment.Environment{Name: r.env, Values: map[string]interface{}(nil)}
-	tmplData := state.EnvironmentTemplateData{Environment: firstPassEnv, Namespace: r.namespace}
-	firstPassRenderer := tmpl.NewFirstPassRenderer(filepath.Dir(r.filename), tmplData)
+func (r *desiredStateLoader) renderPrestate(firstPassEnv *environment.Environment, baseDir, filename string, content []byte) (*environment.Environment, *state.HelmState) {
+	tmplData := state.EnvironmentTemplateData{
+		Environment: *firstPassEnv,
+		Namespace:   r.namespace,
+		Values:      map[string]interface{}{},
+	}
+	firstPassRenderer := tmpl.NewFirstPassRenderer(baseDir, tmplData)
 
 	// parse as much as we can, tolerate errors, this is a preparse
 	yamlBuf, err := firstPassRenderer.RenderTemplateContentToBuffer(content)
 	if err != nil && r.logger != nil {
-		r.logger.Debugf("first-pass rendering input of \"%s\":\n%s", r.filename, prependLineNumbers(string(content)))
+		r.logger.Debugf("first-pass rendering input of \"%s\":\n%s", filename, prependLineNumbers(string(content)))
 		if yamlBuf == nil { // we have a template syntax error, let the second parse report
 			r.logger.Debugf("template syntax error: %v", err)
-			return firstPassEnv
+			return firstPassEnv, nil
 		}
 	}
-	c := state.NewCreator(r.logger, r.reader, r.abs)
+	yamlData := yamlBuf.String()
+
+	// Work-around for https://github.com/golang/go/issues/24963
+	sanitized := strings.ReplaceAll(yamlData, "<no value>", "")
+
+	if len(yamlData) != len(sanitized) {
+		msg := "replaced <no value>s to workaround https://github.com/golang/go/issues/24963 to address https://github.com/roboll/helmfile/issues/553:\n%s"
+		r.logger.Debugf(msg, cmp.Diff(yamlData, sanitized))
+	}
+
+	c := r.underlying()
 	c.Strict = false
 	// create preliminary state, as we may have an environment. Tolerate errors.
-	prestate, err := c.CreateFromYaml(yamlBuf.Bytes(), r.filename, r.env)
+	prestate, err := c.ParseAndLoad([]byte(sanitized), baseDir, filename, r.env, false, firstPassEnv)
 	if err != nil && r.logger != nil {
 		switch err.(type) {
 		case *state.StateLoadError:
 			r.logger.Infof("could not deduce `environment:` block, configuring only .Environment.Name. error: %v", err)
 		}
-		r.logger.Debugf("error in first-pass rendering: result of \"%s\":\n%s", r.filename, prependLineNumbers(yamlBuf.String()))
+		r.logger.Debugf("error in first-pass rendering: result of \"%s\":\n%s", filename, prependLineNumbers(yamlBuf.String()))
 	}
+
 	if prestate != nil {
-		firstPassEnv = prestate.Env
+		firstPassEnv = &prestate.Env
 	}
-	return firstPassEnv
+
+	return firstPassEnv, prestate
 }
 
-func (r *twoPassRenderer) renderTemplate(content []byte) (*bytes.Buffer, error) {
-	// try a first pass render. This will always succeed, but can produce a limited env
-	firstPassEnv := r.renderEnvironment(content)
+type RenderOpts struct {
+}
 
-	tmplData := state.EnvironmentTemplateData{Environment: firstPassEnv, Namespace: r.namespace}
-	secondPassRenderer := tmpl.NewFileRenderer(r.reader, filepath.Dir(r.filename), tmplData)
+func (r *desiredStateLoader) renderTemplatesToYaml(baseDir, filename string, content []byte) (*bytes.Buffer, error) {
+	env := &environment.Environment{Name: r.env, Values: map[string]interface{}(nil)}
+
+	return r.renderTemplatesToYamlWithEnv(baseDir, filename, content, env, nil)
+}
+
+func (r *desiredStateLoader) renderTemplatesToYamlWithEnv(baseDir, filename string, content []byte, inherited, overrode *environment.Environment) (*bytes.Buffer, error) {
+	return r.twoPassRenderTemplateToYaml(inherited, overrode, baseDir, filename, content)
+}
+
+func (r *desiredStateLoader) twoPassRenderTemplateToYaml(inherited, overrode *environment.Environment, baseDir, filename string, content []byte) (*bytes.Buffer, error) {
+	// try a first pass render. This will always succeed, but can produce a limited env
+	if r.logger != nil {
+		r.logger.Debugf("first-pass rendering starting for \"%s\": inherited=%v, overrode=%v", filename, inherited, overrode)
+	}
+
+	initEnv, err := inherited.Merge(overrode)
+	if err != nil {
+		return nil, err
+	}
+
+	if r.logger != nil {
+		r.logger.Debugf("first-pass uses: %v", initEnv)
+	}
+
+	renderedEnv, prestate := r.renderPrestate(initEnv, baseDir, filename, content)
+
+	if r.logger != nil {
+		r.logger.Debugf("first-pass produced: %v", renderedEnv)
+	}
+
+	finalEnv, err := inherited.Merge(renderedEnv)
+	if err != nil {
+		return nil, err
+	}
+
+	finalEnv, err = finalEnv.Merge(overrode)
+	if err != nil {
+		return nil, err
+	}
+
+	if r.logger != nil {
+		r.logger.Debugf("first-pass rendering result of \"%s\": %v", filename, *finalEnv)
+	}
+
+	vals := map[string]interface{}{}
+	if prestate != nil {
+		prestate.Env = *finalEnv
+		vals, err = prestate.Values()
+		if err != nil {
+			return nil, err
+		}
+	}
+	if prestate != nil {
+		r.logger.Debugf("vals:\n%v\ndefaultVals:%v", vals, prestate.DefaultValues)
+	}
+
+	tmplData := state.EnvironmentTemplateData{
+		Environment: *finalEnv,
+		Namespace:   r.namespace,
+		Values:      vals,
+	}
+	secondPassRenderer := tmpl.NewFileRenderer(r.readFile, baseDir, tmplData)
 	yamlBuf, err := secondPassRenderer.RenderTemplateContentToBuffer(content)
 	if err != nil {
 		if r.logger != nil {
-			r.logger.Debugf("second-pass rendering failed, input of \"%s\":\n%s", r.filename, prependLineNumbers(string(content)))
+			r.logger.Debugf("second-pass rendering failed, input of \"%s\":\n%s", filename, prependLineNumbers(string(content)))
 		}
 		return nil, err
 	}
 	if r.logger != nil {
-		r.logger.Debugf("second-pass rendering result of \"%s\":\n%s", r.filename, prependLineNumbers(yamlBuf.String()))
+		r.logger.Debugf("second-pass rendering result of \"%s\":\n%s", filename, prependLineNumbers(yamlBuf.String()))
 	}
 	return yamlBuf, nil
 }

pkg/app/two_pass_renderer_test.go

@@ -1,26 +1,27 @@
 package app
 
 import (
-	"fmt"
+	"github.com/roboll/helmfile/pkg/helmexec"
+	"github.com/roboll/helmfile/pkg/state"
+	"github.com/roboll/helmfile/pkg/testhelper"
 	"os"
-	"path/filepath"
 	"strings"
 	"testing"
 
-	"github.com/roboll/helmfile/helmexec"
-	"github.com/roboll/helmfile/state"
 	"gopkg.in/yaml.v2"
 )
 
-func makeRenderer(readFile func(string) ([]byte, error), env string) *twoPassRenderer {
-	return &twoPassRenderer{
-		reader:    readFile,
-		env:       env,
-		namespace: "namespace",
-		filename:  "",
-		logger:    helmexec.NewLogger(os.Stdout, "debug"),
-		abs:       filepath.Abs,
-	}
+func makeLoader(files map[string]string, env string) (*desiredStateLoader, *testhelper.TestFs) {
+	testfs := testhelper.NewTestFs(files)
+	return &desiredStateLoader{
+		env:        env,
+		namespace:  "namespace",
+		logger:     helmexec.NewLogger(os.Stdout, "debug"),
+		readFile:   testfs.ReadFile,
+		fileExists: testfs.FileExists,
+		abs:        testfs.Abs,
+		glob:       testfs.Glob,
+	}, testfs
 }
 
 func TestReadFromYaml_MakeEnvironmentHasNoSideEffects(t *testing.T) {
@@ -37,30 +38,21 @@ releases:
   chart: mychart1
 `)
 
-	fileReaderCalls := 0
-	// make a reader that returns a simulated context
-	fileReader := func(filename string) ([]byte, error) {
-		expectedFilename := filepath.Clean("default/values.yaml")
-		if !strings.HasSuffix(filename, expectedFilename) {
-			return nil, fmt.Errorf("unexpected filename: expected=%s, actual=%s", expectedFilename, filename)
-		}
-		fileReaderCalls++
-		if fileReaderCalls == 2 {
-			return []byte("SecondPass"), nil
-		}
-		return []byte(""), nil
+	files := map[string]string{
+		"/path/to/default/values.yaml":       ``,
+		"/path/to/other/default/values.yaml": `SecondPass`,
 	}
 
-	r := makeRenderer(fileReader, "staging")
-	yamlBuf, err := r.renderTemplate(yamlContent)
+	r, testfs := makeLoader(files, "staging")
+	yamlBuf, err := r.renderTemplatesToYaml("", "", yamlContent)
 	if err != nil {
-		t.Errorf("unexpected error: %v", err)
+		t.Fatalf("unexpected error: %v", err)
 	}
 
 	var state state.HelmState
 	err = yaml.Unmarshal(yamlBuf.Bytes(), &state)
 
-	if fileReaderCalls > 2 {
+	if testfs.FileReaderCalls() > 2 {
 		t.Error("reader should be called only twice")
 	}
 
@@ -71,10 +63,10 @@ releases:
 
 func TestReadFromYaml_RenderTemplate(t *testing.T) {
 
-	defaultValuesYaml := []byte(`
+	defaultValuesYaml := `
 releaseName: "hello"
 conditionalReleaseTag: "yes"
-`)
+`
 
 	yamlContent := []byte(`
 environments:
@@ -93,18 +85,13 @@ releases:
 
 `)
 
-	// make a reader that returns a simulated context
-	fileReader := func(filename string) ([]byte, error) {
-		expectedFilename := filepath.Clean("default/values.yaml")
-		if !strings.HasSuffix(filename, expectedFilename) {
-			return nil, fmt.Errorf("unexpected filename: expected=%s, actual=%s", expectedFilename, filename)
-		}
-		return defaultValuesYaml, nil
+	files := map[string]string{
+		"/path/to/default/values.yaml": defaultValuesYaml,
 	}
 
-	r := makeRenderer(fileReader, "staging")
+	r, _ := makeLoader(files, "staging")
 	// test the double rendering
-	yamlBuf, err := r.renderTemplate(yamlContent)
+	yamlBuf, err := r.renderTemplatesToYaml("", "", yamlContent)
 	if err != nil {
 		t.Fatalf("unexpected error: %v", err)
 	}
@@ -130,7 +117,7 @@ releases:
 }
 
 func TestReadFromYaml_RenderTemplateWithValuesReferenceError(t *testing.T) {
-	defaultValuesYaml := []byte("")
+	defaultValuesYaml := ``
 
 	yamlContent := []byte(`
 environments:
@@ -146,14 +133,13 @@ releases:
 {{ end }}
 `)
 
-	// make a reader that returns a simulated context
-	fileReader := func(filename string) ([]byte, error) {
-		return defaultValuesYaml, nil
+	files := map[string]string{
+		"/path/to/default/values.yaml": defaultValuesYaml,
 	}
 
-	r := makeRenderer(fileReader, "staging")
+	r, _ := makeLoader(files, "staging")
 	// test the double rendering
-	_, err := r.renderTemplate(yamlContent)
+	_, err := r.renderTemplatesToYaml("", "", yamlContent)
 
 	if !strings.Contains(err.Error(), "stringTemplate:8") {
 		t.Fatalf("error should contain a stringTemplate error (reference to unknow key) %v", err)
@@ -165,9 +151,9 @@ releases:
 // This does not apply to .gotmpl files, which is a nice side-effect.
 func TestReadFromYaml_RenderTemplateWithGotmpl(t *testing.T) {
 
-	defaultValuesYamlGotmpl := []byte(`
+	defaultValuesYamlGotmpl := `
 releaseName: {{ readFile "nonIgnoredFile" }}
-`)
+`
 
 	yamlContent := []byte(`
 environments:
@@ -183,15 +169,13 @@ releases:
 {{ end }}
 `)
 
-	fileReader := func(filename string) ([]byte, error) {
-		if strings.HasSuffix(filename, "nonIgnoredFile") {
-			return []byte("release-a"), nil
-		}
-		return defaultValuesYamlGotmpl, nil
+	files := map[string]string{
+		"/path/to/nonIgnoredFile":     `release-a`,
+		"/path/to/values.yaml.gotmpl": defaultValuesYamlGotmpl,
 	}
 
-	r := makeRenderer(fileReader, "staging")
-	rendered, _ := r.renderTemplate(yamlContent)
+	r, _ := makeLoader(files, "staging")
+	rendered, _ := r.renderTemplatesToYaml("", "", yamlContent)
 
 	var state state.HelmState
 	yaml.Unmarshal(rendered.Bytes(), &state)
@@ -206,19 +190,15 @@ releases:
 }
 
 func TestReadFromYaml_RenderTemplateWithNamespace(t *testing.T) {
-	defaultValuesYaml := []byte(``)
 	yamlContent := []byte(`releases:
 - name: {{ .Namespace }}-myrelease
   chart: mychart
 `)
 
-	// make a reader that returns a simulated context
-	fileReader := func(filename string) ([]byte, error) {
-		return defaultValuesYaml, nil
-	}
+	files := map[string]string{}
 
-	r := makeRenderer(fileReader, "staging")
-	yamlBuf, err := r.renderTemplate(yamlContent)
+	r, _ := makeLoader(files, "staging")
+	yamlBuf, err := r.renderTemplatesToYaml("", "", yamlContent)
 	if err != nil {
 		t.Fatalf("unexpected error: %v", err)
 	}
@@ -231,7 +211,7 @@ func TestReadFromYaml_RenderTemplateWithNamespace(t *testing.T) {
 	}
 }
 
-func TestReadFromYaml_HelfileShouldBeResilentToTemplateErrors(t *testing.T) {
+func TestReadFromYaml_HelmfileShouldBeResilentToTemplateErrors(t *testing.T) {
 	yamlContent := []byte(`environments:
   staging:
 	production:
@@ -244,12 +224,9 @@ releases:
 {{ end }}
   chart: mychart
 `)
-	fileReader := func(filename string) ([]byte, error) {
-		return yamlContent, nil
-	}
 
-	r := makeRenderer(fileReader, "staging")
-	_, err := r.renderTemplate(yamlContent)
+	r, _ := makeLoader(map[string]string{}, "staging")
+	_, err := r.renderTemplatesToYaml("", "", yamlContent)
 	if err == nil {
 		t.Fatalf("wanted error, none returned")
 	}

pkg/argparser/args.go

@@ -1,10 +1,9 @@
-package args
+package argparser
 
 import (
 	"fmt"
+	"github.com/roboll/helmfile/pkg/state"
 	"strings"
-
-	"github.com/roboll/helmfile/state"
 )
 
 type keyVal struct {

pkg/argparser/args_test.go

@@ -1,10 +1,9 @@
-package args
+package argparser
 
 import (
+	"github.com/roboll/helmfile/pkg/state"
 	"strings"
 	"testing"
-
-	"github.com/roboll/helmfile/state"
 )
 
 func TestGetArgs(t *testing.T) {

pkg/environment/environment.go

@@ -0,0 +1,66 @@
+package environment
+
+import (
+	"github.com/imdario/mergo"
+	"github.com/roboll/helmfile/pkg/maputil"
+	"gopkg.in/yaml.v2"
+)
+
+type Environment struct {
+	Name     string
+	Values   map[string]interface{}
+	Defaults map[string]interface{}
+}
+
+var EmptyEnvironment Environment
+
+func (e Environment) DeepCopy() Environment {
+	valuesBytes, err := yaml.Marshal(e.Values)
+	if err != nil {
+		panic(err)
+	}
+	var values map[string]interface{}
+	if err := yaml.Unmarshal(valuesBytes, &values); err != nil {
+		panic(err)
+	}
+	values, err = maputil.CastKeysToStrings(values)
+	if err != nil {
+		panic(err)
+	}
+
+	defaultsBytes, err := yaml.Marshal(e.Defaults)
+	if err != nil {
+		panic(err)
+	}
+	var defaults map[string]interface{}
+	if err := yaml.Unmarshal(defaultsBytes, &defaults); err != nil {
+		panic(err)
+	}
+	defaults, err = maputil.CastKeysToStrings(defaults)
+	if err != nil {
+		panic(err)
+	}
+
+	return Environment{
+		Name:     e.Name,
+		Values:   values,
+		Defaults: defaults,
+	}
+}
+
+func (e *Environment) Merge(other *Environment) (*Environment, error) {
+	if e == nil {
+		if other != nil {
+			copy := other.DeepCopy()
+			return &copy, nil
+		}
+		return nil, nil
+	}
+	copy := e.DeepCopy()
+	if other != nil {
+		if err := mergo.Merge(&copy, other, mergo.WithOverride); err != nil {
+			return nil, err
+		}
+	}
+	return &copy, nil
+}

pkg/event/bus.go

@@ -2,9 +2,9 @@ package event
 
 import (
 	"fmt"
-	"github.com/roboll/helmfile/environment"
-	"github.com/roboll/helmfile/helmexec"
-	"github.com/roboll/helmfile/tmpl"
+	"github.com/roboll/helmfile/pkg/environment"
+	"github.com/roboll/helmfile/pkg/helmexec"
+	"github.com/roboll/helmfile/pkg/tmpl"
 	"go.uber.org/zap"
 	"os"
 )

pkg/event/bus_test.go

@@ -2,8 +2,8 @@ package event
 
 import (
 	"fmt"
-	"github.com/roboll/helmfile/environment"
-	"github.com/roboll/helmfile/helmexec"
+	"github.com/roboll/helmfile/pkg/environment"
+	"github.com/roboll/helmfile/pkg/helmexec"
 	"os"
 	"testing"
 )

pkg/helmexec/exec.go

@@ -1,11 +1,12 @@
 package helmexec
 
 import (
+	"fmt"
 	"io"
 	"io/ioutil"
 	"os"
+	"path/filepath"
 	"strings"
-
 	"sync"
 
 	"go.uber.org/zap"
@@ -48,7 +49,9 @@ func New(logger *zap.SugaredLogger, kubeContext string) *execer {
 		helmBinary:  command,
 		logger:      logger,
 		kubeContext: kubeContext,
-		runner:      &ShellRunner{},
+		runner: &ShellRunner{
+			logger: logger,
+		},
 	}
 }
 
@@ -71,21 +74,21 @@ func (helm *execer) AddRepo(name, repository, certfile, keyfile, username, passw
 	}
 	helm.logger.Infof("Adding repo %v %v", name, repository)
 	out, err := helm.exec(args, map[string]string{})
-	helm.write(out)
+	helm.info(out)
 	return err
 }
 
 func (helm *execer) UpdateRepo() error {
 	helm.logger.Info("Updating repo")
 	out, err := helm.exec([]string{"repo", "update"}, map[string]string{})
-	helm.write(out)
+	helm.info(out)
 	return err
 }
 
 func (helm *execer) UpdateDeps(chart string) error {
 	helm.logger.Infof("Updating dependency %v", chart)
 	out, err := helm.exec([]string{"dependency", "update", chart}, map[string]string{})
-	helm.write(out)
+	helm.info(out)
 	return err
 }
 
@@ -128,10 +131,14 @@ func (helm *execer) DecryptSecret(context HelmContext, name string, flags ...str
 	helm.decryptionMutex.Lock()
 	defer helm.decryptionMutex.Unlock()
 
-	helm.logger.Infof("Decrypting secret %v", name)
+	absPath, err := filepath.Abs(name)
+	if err != nil {
+		return "", err
+	}
+	helm.logger.Infof("Decrypting secret %v", absPath)
 	preArgs := context.GetTillerlessArgs(helm.helmBinary)
 	env := context.getTillerlessEnv()
-	out, err := helm.exec(append(append(preArgs, "secrets", "dec", name), flags...), env)
+	out, err := helm.exec(append(append(preArgs, "secrets", "dec", absPath), flags...), env)
 	helm.write(out)
 	if err != nil {
 		return "", err
@@ -143,10 +150,16 @@ func (helm *execer) DecryptSecret(context HelmContext, name string, flags ...str
 	}
 	defer tmpFile.Close()
 
+	// HELM_SECRETS_DEC_SUFFIX is used by the helm-secrets plugin to define the output file
+	decSuffix := os.Getenv("HELM_SECRETS_DEC_SUFFIX")
+	if len(decSuffix) == 0 {
+		decSuffix = ".yaml.dec"
+	}
+	decFilename := strings.Replace(absPath, ".yaml", decSuffix, 1)
+
 	// os.Rename seems to results in "cross-device link` errors in some cases
 	// Instead of moving, copy it to the destination temp file as a work-around
 	// See https://github.com/roboll/helmfile/issues/251#issuecomment-417166296f
-	decFilename := name + ".dec"
 	decFile, err := os.Open(decFilename)
 	if err != nil {
 		return "", err
@@ -180,7 +193,26 @@ func (helm *execer) DiffRelease(context HelmContext, name, chart string, flags .
 	preArgs := context.GetTillerlessArgs(helm.helmBinary)
 	env := context.getTillerlessEnv()
 	out, err := helm.exec(append(append(preArgs, "diff", "upgrade", "--allow-unreleased", name, chart), flags...), env)
-	helm.write(out)
+	// Do our best to write STDOUT only when diff existed
+	// Unfortunately, this works only when you run helmfile with `--detailed-exitcode`
+	detailedExitcodeEnabled := false
+	for _, f := range flags {
+		if strings.Contains(f, "detailed-exitcode") {
+			detailedExitcodeEnabled = true
+			break
+		}
+	}
+	if detailedExitcodeEnabled {
+		switch e := err.(type) {
+		case ExitError:
+			if e.ExitStatus() == 2 {
+				helm.write(out)
+				return err
+			}
+		}
+	} else {
+		helm.write(out)
+	}
 	return err
 }
 
@@ -194,7 +226,7 @@ func (helm *execer) Lint(chart string, flags ...string) error {
 func (helm *execer) Fetch(chart string, flags ...string) error {
 	helm.logger.Infof("Fetching %v", chart)
 	out, err := helm.exec(append([]string{"fetch", chart}, flags...), map[string]string{})
-	helm.write(out)
+	helm.info(out)
 	return err
 }
 
@@ -224,12 +256,21 @@ func (helm *execer) exec(args []string, env map[string]string) ([]byte, error) {
 	if helm.kubeContext != "" {
 		cmdargs = append(cmdargs, "--kube-context", helm.kubeContext)
 	}
-	helm.logger.Debugf("exec: %s %s", helm.helmBinary, strings.Join(cmdargs, " "))
-	return helm.runner.Execute(helm.helmBinary, cmdargs, env)
+	cmd := fmt.Sprintf("exec: %s %s", helm.helmBinary, strings.Join(cmdargs, " "))
+	helm.logger.Debug(cmd)
+	bytes, err := helm.runner.Execute(helm.helmBinary, cmdargs, env)
+	helm.logger.Debugf("%s: %s", cmd, bytes)
+	return bytes, err
 }
 
-func (helm *execer) write(out []byte) {
+func (helm *execer) info(out []byte) {
 	if len(out) > 0 {
 		helm.logger.Infof("%s", out)
 	}
 }
+
+func (helm *execer) write(out []byte) {
+	if len(out) > 0 {
+		fmt.Printf("%s\n", out)
+	}
+}

pkg/helmexec/exec_test.go

@@ -2,8 +2,10 @@ package helmexec
 
 import (
 	"bytes"
+	"fmt"
 	"os"
 	"path"
+	"path/filepath"
 	"reflect"
 	"testing"
 
@@ -78,6 +80,7 @@ func Test_AddRepo(t *testing.T) {
 	helm.AddRepo("myRepo", "https://repo.example.com/", "cert.pem", "key.pem", "", "")
 	expected := `Adding repo myRepo https://repo.example.com/
 exec: helm repo add myRepo https://repo.example.com/ --cert-file cert.pem --key-file key.pem --kube-context dev
+exec: helm repo add myRepo https://repo.example.com/ --cert-file cert.pem --key-file key.pem --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -87,6 +90,7 @@ exec: helm repo add myRepo https://repo.example.com/ --cert-file cert.pem --key-
 	helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "", "")
 	expected = `Adding repo myRepo https://repo.example.com/
 exec: helm repo add myRepo https://repo.example.com/ --kube-context dev
+exec: helm repo add myRepo https://repo.example.com/ --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -96,6 +100,7 @@ exec: helm repo add myRepo https://repo.example.com/ --kube-context dev
 	helm.AddRepo("myRepo", "https://repo.example.com/", "", "", "example_user", "example_password")
 	expected = `Adding repo myRepo https://repo.example.com/
 exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password --kube-context dev
+exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -109,6 +114,7 @@ func Test_UpdateRepo(t *testing.T) {
 	helm.UpdateRepo()
 	expected := `Updating repo
 exec: helm repo update --kube-context dev
+exec: helm repo update --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.UpdateRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -122,6 +128,7 @@ func Test_SyncRelease(t *testing.T) {
 	helm.SyncRelease(HelmContext{}, "release", "chart", "--timeout 10", "--wait")
 	expected := `Upgrading chart
 exec: helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev
+exec: helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -131,6 +138,7 @@ exec: helm upgrade --install --reset-values release chart --timeout 10 --wait --
 	helm.SyncRelease(HelmContext{}, "release", "chart")
 	expected = `Upgrading chart
 exec: helm upgrade --install --reset-values release chart --kube-context dev
+exec: helm upgrade --install --reset-values release chart --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -145,6 +153,7 @@ func Test_SyncReleaseTillerless(t *testing.T) {
 		"--timeout 10", "--wait")
 	expected := `Upgrading chart
 exec: helm tiller run foo -- helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev
+exec: helm tiller run foo -- helm upgrade --install --reset-values release chart --timeout 10 --wait --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.SyncRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -158,6 +167,7 @@ func Test_UpdateDeps(t *testing.T) {
 	helm.UpdateDeps("./chart/foo")
 	expected := `Updating dependency ./chart/foo
 exec: helm dependency update ./chart/foo --kube-context dev
+exec: helm dependency update ./chart/foo --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.UpdateDeps()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -168,6 +178,7 @@ exec: helm dependency update ./chart/foo --kube-context dev
 	helm.UpdateDeps("./chart/foo")
 	expected = `Updating dependency ./chart/foo
 exec: helm dependency update ./chart/foo --verify --kube-context dev
+exec: helm dependency update ./chart/foo --verify --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.AddRepo()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -181,6 +192,7 @@ func Test_BuildDeps(t *testing.T) {
 	helm.BuildDeps("./chart/foo")
 	expected := `Building dependency ./chart/foo
 exec: helm dependency build ./chart/foo --kube-context dev
+exec: helm dependency build ./chart/foo --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.BuildDeps()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -191,6 +203,7 @@ exec: helm dependency build ./chart/foo --kube-context dev
 	helm.BuildDeps("./chart/foo")
 	expected = `Building dependency ./chart/foo
 exec: helm dependency build ./chart/foo --verify --kube-context dev
+exec: helm dependency build ./chart/foo --verify --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.BuildDeps()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -202,9 +215,14 @@ func Test_DecryptSecret(t *testing.T) {
 	logger := NewLogger(&buffer, "debug")
 	helm := MockExecer(logger, "dev")
 	helm.DecryptSecret(HelmContext{}, "secretName")
-	expected := `Decrypting secret secretName
-exec: helm secrets dec secretName --kube-context dev
-`
+	cwd, err := filepath.Abs(".")
+	if err != nil {
+		t.Errorf("Error: %v", err)
+	}
+	expected := fmt.Sprintf(`Decrypting secret %s/secretName
+exec: helm secrets dec %s/secretName --kube-context dev
+exec: helm secrets dec %s/secretName --kube-context dev: 
+`, cwd, cwd, cwd)
 	if buffer.String() != expected {
 		t.Errorf("helmexec.DecryptSecret()\nactual = %v\nexpect = %v", buffer.String(), expected)
 	}
@@ -217,6 +235,7 @@ func Test_DiffRelease(t *testing.T) {
 	helm.DiffRelease(HelmContext{}, "release", "chart", "--timeout 10", "--wait")
 	expected := `Comparing release chart
 exec: helm diff upgrade --allow-unreleased release chart --timeout 10 --wait --kube-context dev
+exec: helm diff upgrade --allow-unreleased release chart --timeout 10 --wait --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -226,6 +245,7 @@ exec: helm diff upgrade --allow-unreleased release chart --timeout 10 --wait --k
 	helm.DiffRelease(HelmContext{}, "release", "chart")
 	expected = `Comparing release chart
 exec: helm diff upgrade --allow-unreleased release chart --kube-context dev
+exec: helm diff upgrade --allow-unreleased release chart --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -239,6 +259,7 @@ func Test_DiffReleaseTillerless(t *testing.T) {
 	helm.DiffRelease(HelmContext{Tillerless: true}, "release", "chart", "--timeout 10", "--wait")
 	expected := `Comparing release chart
 exec: helm tiller run -- helm diff upgrade --allow-unreleased release chart --timeout 10 --wait --kube-context dev
+exec: helm tiller run -- helm diff upgrade --allow-unreleased release chart --timeout 10 --wait --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.DiffRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -252,6 +273,7 @@ func Test_DeleteRelease(t *testing.T) {
 	helm.DeleteRelease(HelmContext{}, "release")
 	expected := `Deleting release
 exec: helm delete release --kube-context dev
+exec: helm delete release --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.DeleteRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -264,6 +286,7 @@ func Test_DeleteRelease_Flags(t *testing.T) {
 	helm.DeleteRelease(HelmContext{}, "release", "--purge")
 	expected := `Deleting release
 exec: helm delete release --purge --kube-context dev
+exec: helm delete release --purge --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.DeleteRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -277,6 +300,7 @@ func Test_TestRelease(t *testing.T) {
 	helm.TestRelease(HelmContext{}, "release")
 	expected := `Testing release
 exec: helm test release --kube-context dev
+exec: helm test release --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.TestRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -289,6 +313,7 @@ func Test_TestRelease_Flags(t *testing.T) {
 	helm.TestRelease(HelmContext{}, "release", "--cleanup", "--timeout", "60")
 	expected := `Testing release
 exec: helm test release --cleanup --timeout 60 --kube-context dev
+exec: helm test release --cleanup --timeout 60 --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.TestRelease()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -302,6 +327,7 @@ func Test_ReleaseStatus(t *testing.T) {
 	helm.ReleaseStatus(HelmContext{}, "myRelease")
 	expected := `Getting status myRelease
 exec: helm status myRelease --kube-context dev
+exec: helm status myRelease --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.ReleaseStatus()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -314,7 +340,9 @@ func Test_exec(t *testing.T) {
 	helm := MockExecer(logger, "")
 	env := map[string]string{}
 	helm.exec([]string{"version"}, env)
-	expected := "exec: helm version\n"
+	expected := `exec: helm version
+exec: helm version: 
+`
 	if buffer.String() != expected {
 		t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
 	}
@@ -328,14 +356,18 @@ func Test_exec(t *testing.T) {
 	buffer.Reset()
 	helm = MockExecer(logger, "dev")
 	helm.exec([]string{"diff", "release", "chart", "--timeout 10", "--wait"}, env)
-	expected = "exec: helm diff release chart --timeout 10 --wait --kube-context dev\n"
+	expected = `exec: helm diff release chart --timeout 10 --wait --kube-context dev
+exec: helm diff release chart --timeout 10 --wait --kube-context dev: 
+`
 	if buffer.String() != expected {
 		t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
 	}
 
 	buffer.Reset()
 	helm.exec([]string{"version"}, env)
-	expected = "exec: helm version --kube-context dev\n"
+	expected = `exec: helm version --kube-context dev
+exec: helm version --kube-context dev: 
+`
 	if buffer.String() != expected {
 		t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
 	}
@@ -343,7 +375,9 @@ func Test_exec(t *testing.T) {
 	buffer.Reset()
 	helm.SetExtraArgs("foo")
 	helm.exec([]string{"version"}, env)
-	expected = "exec: helm version foo --kube-context dev\n"
+	expected = `exec: helm version foo --kube-context dev
+exec: helm version foo --kube-context dev: 
+`
 	if buffer.String() != expected {
 		t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
 	}
@@ -352,7 +386,9 @@ func Test_exec(t *testing.T) {
 	helm = MockExecer(logger, "")
 	helm.SetHelmBinary("overwritten")
 	helm.exec([]string{"version"}, env)
-	expected = "exec: overwritten version\n"
+	expected = `exec: overwritten version
+exec: overwritten version: 
+`
 	if buffer.String() != expected {
 		t.Errorf("helmexec.exec()\nactual = %v\nexpect = %v", buffer.String(), expected)
 	}
@@ -365,6 +401,7 @@ func Test_Lint(t *testing.T) {
 	helm.Lint("path/to/chart", "--values", "file.yml")
 	expected := `Linting path/to/chart
 exec: helm lint path/to/chart --values file.yml --kube-context dev
+exec: helm lint path/to/chart --values file.yml --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.Lint()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -378,6 +415,7 @@ func Test_Fetch(t *testing.T) {
 	helm.Fetch("chart", "--version", "1.2.3", "--untar", "--untardir", "/tmp/dir")
 	expected := `Fetching chart
 exec: helm fetch chart --version 1.2.3 --untar --untardir /tmp/dir --kube-context dev
+exec: helm fetch chart --version 1.2.3 --untar --untardir /tmp/dir --kube-context dev: 
 `
 	if buffer.String() != expected {
 		t.Errorf("helmexec.Lint()\nactual = %v\nexpect = %v", buffer.String(), expected)
@@ -387,6 +425,7 @@ exec: helm fetch chart --version 1.2.3 --untar --untardir /tmp/dir --kube-contex
 var logLevelTests = map[string]string{
 	"debug": `Adding repo myRepo https://repo.example.com/
 exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password
+exec: helm repo add myRepo https://repo.example.com/ --username example_user --password example_password: 
 `,
 	"info": `Adding repo myRepo https://repo.example.com/
 `,

pkg/helmexec/exit_error.go

@@ -0,0 +1,36 @@
+package helmexec
+
+import (
+	"fmt"
+	"path/filepath"
+	"strings"
+)
+
+func newExitError(helmCmdPath string, exitStatus int, errorMessage string) ExitError {
+	return ExitError{
+		msg:        fmt.Sprintf("%s exited with status %d:\n%s", filepath.Base(helmCmdPath), exitStatus, indent(strings.TrimSpace(errorMessage))),
+		exitStatus: exitStatus,
+	}
+}
+
+func indent(text string) string {
+	lines := strings.Split(text, "\n")
+	for i := range lines {
+		lines[i] = "  " + lines[i]
+	}
+	return strings.Join(lines, "\n")
+}
+
+// ExitError is created whenever your shell command exits with a non-zero exit status
+type ExitError struct {
+	msg        string
+	exitStatus int
+}
+
+func (e ExitError) Error() string {
+	return e.msg
+}
+
+func (e ExitError) ExitStatus() int {
+	return e.exitStatus
+}

pkg/helmexec/helmexec.go

@@ -20,3 +20,7 @@ type Interface interface {
 	List(context HelmContext, filter string, flags ...string) (string, error)
 	DecryptSecret(context HelmContext, name string, flags ...string) (string, error)
 }
+
+type DependencyUpdater interface {
+	UpdateDeps(chart string) error
+}

pkg/helmexec/runner.go

@@ -0,0 +1,99 @@
+package helmexec
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"go.uber.org/zap"
+	"os"
+	"os/exec"
+	"strings"
+	"syscall"
+)
+
+const (
+	tmpPrefix = "helmfile-"
+	tmpSuffix = "-exec"
+)
+
+// Runner interface for shell commands
+type Runner interface {
+	Execute(cmd string, args []string, env map[string]string) ([]byte, error)
+}
+
+// ShellRunner implemention for shell commands
+type ShellRunner struct {
+	Dir string
+
+	logger *zap.SugaredLogger
+}
+
+// Execute a shell command
+func (shell ShellRunner) Execute(cmd string, args []string, env map[string]string) ([]byte, error) {
+	preparedCmd := exec.Command(cmd, args...)
+	preparedCmd.Dir = shell.Dir
+	preparedCmd.Env = mergeEnv(os.Environ(), env)
+	return combinedOutput(preparedCmd, shell.logger)
+}
+
+func combinedOutput(c *exec.Cmd, logger *zap.SugaredLogger) ([]byte, error) {
+	if c.Stdout != nil {
+		return nil, errors.New("exec: Stdout already set")
+	}
+	if c.Stderr != nil {
+		return nil, errors.New("exec: Stderr already set")
+	}
+	var stdout bytes.Buffer
+	var stderr bytes.Buffer
+	c.Stdout = &stdout
+	c.Stderr = &stderr
+	err := c.Run()
+
+	o := stdout.Bytes()
+	e := stderr.Bytes()
+
+	if err != nil {
+		// TrimSpace is necessary, because otherwise helmfile prints the redundant new-lines after each error like:
+		//
+		//   err: release "envoy2" in "helmfile.yaml" failed: exit status 1: Error: could not find a ready tiller pod
+		//   <redundant new line!>
+		//   err: release "envoy" in "helmfile.yaml" failed: exit status 1: Error: could not find a ready tiller pod
+		switch ee := err.(type) {
+		case *exec.ExitError:
+			// Propagate any non-zero exit status from the external command, rather than throwing it away,
+			// so that helmfile could return its own exit code accordingly
+			waitStatus := ee.Sys().(syscall.WaitStatus)
+			exitStatus := waitStatus.ExitStatus()
+			err = newExitError(c.Path, exitStatus, string(e))
+		default:
+			panic(fmt.Sprintf("unexpected error: %v", err))
+		}
+	}
+
+	return o, err
+}
+
+func mergeEnv(orig []string, new map[string]string) []string {
+	wanted := env2map(orig)
+	for k, v := range new {
+		wanted[k] = v
+	}
+	return map2env(wanted)
+}
+
+func map2env(wanted map[string]string) []string {
+	result := []string{}
+	for k, v := range wanted {
+		result = append(result, k+"="+v)
+	}
+	return result
+}
+
+func env2map(env []string) map[string]string {
+	wanted := map[string]string{}
+	for _, cur := range env {
+		pair := strings.SplitN(cur, "=", 2)
+		wanted[pair[0]] = pair[1]
+	}
+	return wanted
+}

pkg/maputil/maputil.go

@@ -0,0 +1,86 @@
+package maputil
+
+import "fmt"
+
+func CastKeysToStrings(s interface{}) (map[string]interface{}, error) {
+	new := map[string]interface{}{}
+	switch src := s.(type) {
+	case map[interface{}]interface{}:
+		for k, v := range src {
+			var str_k string
+			switch typed_k := k.(type) {
+			case string:
+				str_k = typed_k
+			default:
+				return nil, fmt.Errorf("unexpected type of key in map: expected string, got %T: value=%v, map=%v", typed_k, typed_k, src)
+			}
+
+			casted_v, err := recursivelyStringifyMapKey(v)
+			if err != nil {
+				return nil, err
+			}
+
+			new[str_k] = casted_v
+		}
+	case map[string]interface{}:
+		for k, v := range src {
+			casted_v, err := recursivelyStringifyMapKey(v)
+			if err != nil {
+				return nil, err
+			}
+
+			new[k] = casted_v
+		}
+	}
+	return new, nil
+}
+
+func recursivelyStringifyMapKey(v interface{}) (interface{}, error) {
+	var casted_v interface{}
+	switch typed_v := v.(type) {
+	case map[interface{}]interface{}, map[string]interface{}:
+		tmp, err := CastKeysToStrings(typed_v)
+		if err != nil {
+			return nil, err
+		}
+		casted_v = tmp
+	case []interface{}:
+		a := []interface{}{}
+		for i := range typed_v {
+			res, err := recursivelyStringifyMapKey(typed_v[i])
+			if err != nil {
+				return nil, err
+			}
+			a = append(a, res)
+		}
+		casted_v = a
+	default:
+		casted_v = typed_v
+	}
+	return casted_v, nil
+}
+
+func Set(m map[string]interface{}, key []string, value string) map[string]interface{} {
+	if len(key) == 0 {
+		panic(fmt.Errorf("bug: unexpected length of key: %d", len(key)))
+	}
+
+	k := key[0]
+
+	if len(key) == 1 {
+		m[k] = value
+		return m
+	}
+
+	remain := key[1:]
+
+	nested, ok := m[k]
+	if !ok {
+		new_m := map[string]interface{}{}
+		nested = Set(new_m, remain, value)
+	}
+
+	m[k] = nested
+
+	return m
+}

pkg/maputil/maputil_test.go

@@ -0,0 +1,61 @@
+package maputil
+
+import "testing"
+
+func TestMapUtil_StrKeys(t *testing.T) {
+	m := map[string]interface{}{
+		"a": []interface{}{
+			map[string]interface{}{
+				"b": []interface{}{
+					map[string]interface{}{
+						"c": "C",
+					},
+				},
+			},
+		},
+	}
+
+	r, err := CastKeysToStrings(m)
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+
+	a := r["a"].([]interface{})
+	a0 := a[0].(map[string]interface{})
+	b := a0["b"].([]interface{})
+	b0 := b[0].(map[string]interface{})
+	c := b0["c"]
+
+	if c != "C" {
+		t.Errorf("unexpected c: expected=C, got=%s", c)
+	}
+}
+
+func TestMapUtil_IFKeys(t *testing.T) {
+	m := map[interface{}]interface{}{
+		"a": []interface{}{
+			map[interface{}]interface{}{
+				"b": []interface{}{
+					map[interface{}]interface{}{
+						"c": "C",
+					},
+				},
+			},
+		},
+	}
+
+	r, err := CastKeysToStrings(m)
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+
+	a := r["a"].([]interface{})
+	a0 := a[0].(map[string]interface{})
+	b := a0["b"].([]interface{})
+	b0 := b[0].(map[string]interface{})
+	c := b0["c"]
+
+	if c != "C" {
+		t.Errorf("unexpected c: expected=C, got=%s", c)
+	}
+}

pkg/remote/remote.go

@@ -0,0 +1,246 @@
+package remote
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"github.com/hashicorp/go-getter"
+	"github.com/hashicorp/go-getter/helper/url"
+	"go.uber.org/zap"
+	"gopkg.in/yaml.v2"
+	"path/filepath"
+	"strings"
+)
+
+const DefaultCacheDir = ".helmfile/cache"
+
+type Remote struct {
+	Logger *zap.SugaredLogger
+
+	// Home is the home directory for helmfile. Usually this points to $HOME of the user running helmfile.
+	// Helmfile saves fetched remote files into .helmfile/cache under home
+	Home string
+
+	// Getter is the underlying implementation of getter used for fetching remote files
+	Getter Getter
+
+	// ReadFile is the implementation of the file reader that reads a local file from the specified path.
+	// Inject any implementation of your choice, like an im-memory impl for testing, ioutil.ReadFile for the real-world use.
+	ReadFile   func(string) ([]byte, error)
+	DirExists  func(string) bool
+	FileExists func(string) bool
+}
+
+func (r *Remote) Unmarshal(src string, dst interface{}) error {
+	bytes, err := r.GetBytes(src)
+	if err != nil {
+		return err
+	}
+
+	strs := strings.Split(src, "/")
+	file := strs[len(strs)-1]
+	ext := filepath.Ext(file)
+
+	{
+		r.Logger.Debugf("unmarshalling %s", string(bytes))
+
+		var err error
+		switch ext {
+		case "json":
+			err = json.Unmarshal(bytes, dst)
+		default:
+			err = yaml.Unmarshal(bytes, dst)
+		}
+
+		r.Logger.Debugf("unmarshalled to %v", dst)
+
+		if err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func (r *Remote) GetBytes(goGetterSrc string) ([]byte, error) {
+	f, err := r.Fetch(goGetterSrc)
+	if err != nil {
+		return nil, err
+	}
+
+	bytes, err := r.ReadFile(f)
+	if err != nil {
+		return nil, fmt.Errorf("read file: %v", err)
+	}
+
+	return bytes, nil
+}
+
+// Locate takes an URL to a remote file or a path to a local file.
+// If the argument was an URL, it fetches the remote directory contained within the URL,
+// and returns the path to the file in the fetched directory
+func (r *Remote) Locate(urlOrPath string) (string, error) {
+	fetched, err := r.Fetch(urlOrPath)
+	if err != nil {
+		switch err.(type) {
+		case InvalidURLError:
+			return urlOrPath, nil
+		}
+		return "", err
+	}
+	return fetched, nil
+}
+
+type InvalidURLError struct {
+	err string
+}
+
+func (e InvalidURLError) Error() string {
+	return e.err
+}
+
+type Source struct {
+	Getter, Scheme, Host, Dir, File, RawQuery string
+}
+
+func IsRemote(goGetterSrc string) bool {
+	if _, err := Parse(goGetterSrc); err != nil {
+		return false
+	}
+	return true
+}
+
+func Parse(goGetterSrc string) (*Source, error) {
+	items := strings.Split(goGetterSrc, "::")
+	var getter string
+	switch len(items) {
+	case 2:
+		getter = items[0]
+		goGetterSrc = items[1]
+	}
+
+	u, err := url.Parse(goGetterSrc)
+	if err != nil {
+		return nil, InvalidURLError{err: fmt.Sprintf("parse url: %v", err)}
+	}
+
+	if u.Scheme == "" {
+		return nil, InvalidURLError{err: fmt.Sprintf("parse url: missing scheme - probably this is a local file path? %s", goGetterSrc)}
+	}
+
+	pathComponents := strings.Split(u.Path, "@")
+	if len(pathComponents) != 2 {
+		return nil, fmt.Errorf("invalid src format: it must be `[<getter>::]<scheme>://<host>/<path/to/dir>@<path/to/file>?key1=val1&key2=val2: got %s", goGetterSrc)
+	}
+
+	return &Source{
+		Getter:   getter,
+		Scheme:   u.Scheme,
+		Host:     u.Host,
+		Dir:      pathComponents[0],
+		File:     pathComponents[1],
+		RawQuery: u.RawQuery,
+	}, nil
+}
+
+func (r *Remote) Fetch(goGetterSrc string) (string, error) {
+	u, err := Parse(goGetterSrc)
+	if err != nil {
+		return "", err
+	}
+
+	srcDir := fmt.Sprintf("%s://%s%s", u.Scheme, u.Host, u.Dir)
+	file := u.File
+
+	r.Logger.Debugf("getter: %s", u.Getter)
+	r.Logger.Debugf("scheme: %s", u.Scheme)
+	r.Logger.Debugf("host: %s", u.Host)
+	r.Logger.Debugf("dir: %s", u.Dir)
+	r.Logger.Debugf("file: %s", u.File)
+
+	// This should be shared across variant commands, so that they can share cache for the shared imports
+	cacheBaseDir := DefaultCacheDir
+
+	query := u.RawQuery
+
+	var cacheKey string
+	replacer := strings.NewReplacer(":", "", "//", "_", "/", "_", ".", "_")
+	dirKey := replacer.Replace(srcDir)
+	if len(query) > 0 {
+		paramsKey := strings.Replace(query, "&", "_", -1)
+		cacheKey = fmt.Sprintf("%s.%s", dirKey, paramsKey)
+	} else {
+		cacheKey = dirKey
+	}
+
+	cached := false
+
+	getterDst := filepath.Join(cacheBaseDir, cacheKey)
+
+	cacheDirPath := filepath.Join(r.Home, getterDst)
+
+	r.Logger.Debugf("home: %s", r.Home)
+	r.Logger.Debugf("getter dest: %s", getterDst)
+	r.Logger.Debugf("cached dir: %s", cacheDirPath)
+
+	{
+		if r.FileExists(cacheDirPath) {
+			return "", fmt.Errorf("%s is not directory. please remove it so that variant could use it for dependency caching", getterDst)
+		}
+
+		if r.DirExists(cacheDirPath) {
+			cached = true
+		}
+	}
+
+	if !cached {
+		var getterSrc string
+
+		if len(query) == 0 {
+			getterSrc = srcDir
+		} else {
+			getterSrc = strings.Join([]string{srcDir, query}, "?")
+		}
+
+		if u.Getter != "" {
+			getterSrc = u.Getter + "::" + getterSrc
+		}
+
+		r.Logger.Debugf("downloading %s to %s", getterSrc, getterDst)
+
+		if err := r.Getter.Get(r.Home, getterSrc, getterDst); err != nil {
+			return "", err
+		}
+	}
+
+	return filepath.Join(cacheDirPath, file), nil
+}
+
+type Getter interface {
+	Get(wd, src, dst string) error
+}
+
+type GoGetter struct {
+	Logger *zap.SugaredLogger
+}
+
+func (g *GoGetter) Get(wd, src, dst string) error {
+	ctx := context.Background()
+
+	get := &getter.Client{
+		Ctx:     ctx,
+		Src:     src,
+		Dst:     dst,
+		Pwd:     wd,
+		Mode:    getter.ClientModeDir,
+		Options: []getter.ClientOption{},
+	}
+
+	g.Logger.Debugf("client: %+v", *get)
+
+	if err := get.Get(); err != nil {
+		return fmt.Errorf("get: %v", err)
+	}
+
+	return nil
+}

pkg/remote/remote_test.go

@@ -0,0 +1,94 @@
+package remote
+
+import (
+	"fmt"
+	"github.com/roboll/helmfile/pkg/helmexec"
+	"github.com/roboll/helmfile/pkg/testhelper"
+	"os"
+	"testing"
+)
+
+func TestRemote(t *testing.T) {
+	cleanfs := map[string]string{
+		"path/to/home": "",
+	}
+	cachefs := map[string]string{
+		"path/to/home/.helmfile/cache/https_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml": "foo: bar",
+	}
+
+	type testcase struct {
+		files          map[string]string
+		expectCacheHit bool
+	}
+
+	testcases := []testcase{
+		{files: cleanfs, expectCacheHit: false},
+		{files: cachefs, expectCacheHit: true},
+	}
+
+	for i := range testcases {
+		testcase := testcases[i]
+
+		t.Run(fmt.Sprintf("case %d", i), func(t *testing.T) {
+			testfs := testhelper.NewTestFs(testcase.files)
+
+			hit := true
+
+			get := func(wd, src, dst string) error {
+				if wd != "path/to/home" {
+					return fmt.Errorf("unexpected wd: %s", wd)
+				}
+				if src != "git::https://github.com/cloudposse/helmfiles.git?ref=0.40.0" {
+					return fmt.Errorf("unexpected src: %s", src)
+				}
+
+				hit = false
+
+				return nil
+			}
+
+			getter := &testGetter{
+				get: get,
+			}
+			remote := &Remote{
+				Logger:     helmexec.NewLogger(os.Stderr, "debug"),
+				Home:       "path/to/home",
+				Getter:     getter,
+				ReadFile:   testfs.ReadFile,
+				FileExists: testfs.FileExistsAt,
+				DirExists:  testfs.DirectoryExistsAt,
+			}
+
+			// FYI, go-getter in the `dir` mode accepts URL like the below. So helmfile expects URLs similar to it:
+			//   go-getter -mode dir git::https://github.com/cloudposse/helmfiles.git?ref=0.40.0 gettertest1/b
+
+			// We use `@` to separate dir and the file path. This is a good idea borrowed from helm-git:
+			//   https://github.com/aslafy-z/helm-git
+
+			url := "git::https://github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.40.0"
+			file, err := remote.Locate(url)
+			if err != nil {
+				t.Fatalf("unexpected error: %v", err)
+			}
+
+			if file != "path/to/home/.helmfile/cache/https_github_com_cloudposse_helmfiles_git.ref=0.40.0/releases/kiam.yaml" {
+				t.Errorf("unexpected file located: %s", file)
+			}
+
+			if testcase.expectCacheHit && !hit {
+				t.Errorf("unexpected result: unexpected cache miss")
+			}
+			if !testcase.expectCacheHit && hit {
+				t.Errorf("unexpected result: unexpected cache hit")
+			}
+		})
+	}
+}
+
+type testGetter struct {
+	get func(wd, src, dst string) error
+}
+
+func (t *testGetter) Get(wd, src, dst string) error {
+	return t.get(wd, src, dst)
+}

pkg/state/chart_dependency.go

@@ -0,0 +1,400 @@
+package state
+
+import (
+	"fmt"
+	"github.com/Masterminds/semver"
+	"github.com/roboll/helmfile/pkg/helmexec"
+	"go.uber.org/zap"
+	"gopkg.in/yaml.v2"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+)
+
+type ChartMeta struct {
+	Name string `yaml:"name"`
+}
+
+type unresolvedChartDependency struct {
+	// ChartName identifies the dependant chart. In Helmfile, ChartName for `chart: stable/envoy` would be just `envoy`.
+	// It can't be collided with other charts referenced in the same helmfile spec.
+	// That is, collocating `chart: incubator/foo` and `chart: stable/foo` isn't allowed. Name them differently for a work-around.
+	ChartName string `yaml:"name"`
+	// Repository contains the URL for the helm chart repository that hosts the chart identified by ChartName
+	Repository string `yaml:"repository"`
+	// VersionConstraint is the version constraint of the dependent chart. "*" means the latest version.
+	VersionConstraint string `yaml:"version"`
+}
+
+type ResolvedChartDependency struct {
+	// ChartName identifies the dependant chart. In Helmfile, ChartName for `chart: stable/envoy` would be just `envoy`.
+	// It can't be collided with other charts referenced in the same helmfile spec.
+	// That is, collocating `chart: incubator/foo` and `chart: stable/foo` isn't allowed. Name them differently for a work-around.
+	ChartName string `yaml:"name"`
+	// Repository contains the URL for the helm chart repository that hosts the chart identified by ChartName
+	Repository string `yaml:"repository"`
+	// Version is the version number of the dependent chart.
+	// In the context of helmfile this can be omitted. When omitted, it is considered `*` which results helm/helmfile fetching the latest version.
+	Version string `yaml:"version"`
+}
+
+type UnresolvedDependencies struct {
+	deps map[string][]unresolvedChartDependency
+}
+
+type ChartRequirements struct {
+	UnresolvedDependencies []unresolvedChartDependency `yaml:"dependencies"`
+}
+
+type ChartLockedRequirements struct {
+	ResolvedDependencies []ResolvedChartDependency `yaml:"dependencies"`
+	Digest               string                    `yaml:"digest"`
+	Generated            string                    `yaml:"generated"`
+}
+
+func (d *UnresolvedDependencies) Add(chart, url, versionConstraint string) error {
+	dep := unresolvedChartDependency{
+		ChartName:         chart,
+		Repository:        url,
+		VersionConstraint: versionConstraint,
+	}
+	return d.add(dep)
+}
+
+func (d *UnresolvedDependencies) add(dep unresolvedChartDependency) error {
+	deps := d.deps[dep.ChartName]
+	if deps == nil {
+		deps = []unresolvedChartDependency{dep}
+	} else {
+		deps = append(deps, dep)
+	}
+	d.deps[dep.ChartName] = deps
+	return nil
+}
+
+func (d *UnresolvedDependencies) ToChartRequirements() *ChartRequirements {
+	deps := []unresolvedChartDependency{}
+
+	for _, ds := range d.deps {
+		for _, d := range ds {
+			if d.VersionConstraint == "" {
+				d.VersionConstraint = "*"
+			}
+			deps = append(deps, d)
+		}
+	}
+
+	return &ChartRequirements{UnresolvedDependencies: deps}
+}
+
+type ResolvedDependencies struct {
+	deps map[string][]ResolvedChartDependency
+}
+
+func (d *ResolvedDependencies) add(dep ResolvedChartDependency) error {
+	deps := d.deps[dep.ChartName]
+	if deps == nil {
+		deps = []ResolvedChartDependency{dep}
+	} else {
+		deps = append(deps, dep)
+	}
+	d.deps[dep.ChartName] = deps
+	return nil
+}
+
+func (d *ResolvedDependencies) Get(chart, versionConstraint string) (string, error) {
+	if versionConstraint == "" {
+		versionConstraint = "*"
+	}
+
+	deps, exists := d.deps[chart]
+	if exists {
+		for _, dep := range deps {
+			constraint, err := semver.NewConstraint(versionConstraint)
+			if err != nil {
+				return "", err
+			}
+			version, err := semver.NewVersion(dep.Version)
+			if err != nil {
+				return "", err
+			}
+			if constraint.Check(version) {
+				return dep.Version, nil
+			}
+		}
+	}
+	return "", fmt.Errorf("no resolved dependency found for \"%s\"", chart)
+}
+
+func resolveRemoteChart(repoAndChart string) (string, string, bool) {
+	parts := strings.Split(repoAndChart, "/")
+	if isLocalChart(repoAndChart) {
+		return "", "", false
+	}
+	if len(parts) != 2 {
+		panic(fmt.Sprintf("unsupported format of chart name: %s", repoAndChart))
+	}
+
+	repo := parts[0]
+	chart := parts[1]
+
+	return repo, chart, true
+}
+
+func (st *HelmState) mergeLockedDependencies() (*HelmState, error) {
+	filename, unresolved, err := getUnresolvedDependenciess(st)
+	if err != nil {
+		return nil, err
+	}
+
+	if len(unresolved.deps) == 0 {
+		return st, nil
+	}
+
+	depMan := NewChartDependencyManager(filename, st.logger)
+
+	if st.readFile != nil {
+		depMan.readFile = st.readFile
+	}
+
+	return resolveDependencies(st, depMan, unresolved)
+}
+
+func resolveDependencies(st *HelmState, depMan *chartDependencyManager, unresolved *UnresolvedDependencies) (*HelmState, error) {
+	resolved, lockfileExists, err := depMan.Resolve(unresolved)
+	if err != nil {
+		return nil, fmt.Errorf("unable to resolve %d deps: %v", len(unresolved.deps), err)
+	}
+	if !lockfileExists {
+		return st, nil
+	}
+
+	repoToURL := map[string]string{}
+
+	for _, r := range st.Repositories {
+		repoToURL[r.Name] = r.URL
+	}
+
+	updated := *st
+	for i, r := range updated.Releases {
+		repo, chart, ok := resolveRemoteChart(r.Chart)
+		if !ok {
+			continue
+		}
+
+		_, ok = repoToURL[repo]
+		// Skip this chart from dependency management, as there's no matching `repository` in the helmfile state,
+		// which may imply that this is a local chart within a directory, like `charts/myapp`
+		if !ok {
+			continue
+		}
+
+		ver, err := resolved.Get(chart, r.Version)
+		if err != nil {
+			return nil, err
+		}
+
+		updated.Releases[i].Version = ver
+	}
+
+	return &updated, nil
+}
+
+func (st *HelmState) updateDependenciesInTempDir(shell helmexec.DependencyUpdater, tempDir func(string, string) (string, error)) (*HelmState, error) {
+	filename, unresolved, err := getUnresolvedDependenciess(st)
+	if err != nil {
+		return nil, err
+	}
+
+	if len(unresolved.deps) == 0 {
+		return st, nil
+	}
+
+	d, err := tempDir("", "")
+	if err != nil {
+		return nil, fmt.Errorf("unable to create dir: %v", err)
+	}
+	defer os.RemoveAll(d)
+
+	return updateDependencies(st, shell, unresolved, filename, d)
+}
+
+func getUnresolvedDependenciess(st *HelmState) (string, *UnresolvedDependencies, error) {
+	repoToURL := map[string]string{}
+
+	for _, r := range st.Repositories {
+		repoToURL[r.Name] = r.URL
+	}
+
+	unresolved := &UnresolvedDependencies{deps: map[string][]unresolvedChartDependency{}}
+	//if err := unresolved.Add("stable/envoy", "https://kubernetes-charts.storage.googleapis.com", ""); err != nil {
+	//	panic(err)
+	//}
+
+	for _, r := range st.Releases {
+		repo, chart, ok := resolveRemoteChart(r.Chart)
+		if !ok {
+			continue
+		}
+
+		url, ok := repoToURL[repo]
+		// Skip this chart from dependency management, as there's no matching `repository` in the helmfile state,
+		// which may imply that this is a local chart within a directory, like `charts/myapp`
+		if !ok {
+			continue
+		}
+
+		if err := unresolved.Add(chart, url, r.Version); err != nil {
+			return "", nil, err
+		}
+	}
+
+	filename := filepath.Base(st.FilePath)
+	filename = strings.TrimSuffix(filename, ".gotmpl")
+	filename = strings.TrimSuffix(filename, ".yaml")
+	filename = strings.TrimSuffix(filename, ".yml")
+
+	return filename, unresolved, nil
+}
+
+func updateDependencies(st *HelmState, shell helmexec.DependencyUpdater, unresolved *UnresolvedDependencies, filename, wd string) (*HelmState, error) {
+	depMan := NewChartDependencyManager(filename, st.logger)
+
+	_, err := depMan.Update(shell, wd, unresolved)
+	if err != nil {
+		return nil, fmt.Errorf("unable to resolve %d deps: %v", len(unresolved.deps), err)
+	}
+
+	return resolveDependencies(st, depMan, unresolved)
+}
+
+type chartDependencyManager struct {
+	Name string
+
+	logger *zap.SugaredLogger
+
+	readFile  func(string) ([]byte, error)
+	writeFile func(string, []byte, os.FileMode) error
+}
+
+func NewChartDependencyManager(name string, logger *zap.SugaredLogger) *chartDependencyManager {
+	return &chartDependencyManager{
+		Name:      name,
+		readFile:  ioutil.ReadFile,
+		writeFile: ioutil.WriteFile,
+		logger:    logger,
+	}
+}
+
+func (m *chartDependencyManager) lockFileName() string {
+	return fmt.Sprintf("%s.lock", m.Name)
+}
+
+func (m *chartDependencyManager) Update(shell helmexec.DependencyUpdater, wd string, unresolved *UnresolvedDependencies) (*ResolvedDependencies, error) {
+	// Generate `Chart.yaml` of the temporary local chart
+	if err := m.writeBytes(filepath.Join(wd, "Chart.yaml"), []byte(fmt.Sprintf("name: %s\n", m.Name))); err != nil {
+		return nil, err
+	}
+
+	// Generate `requirements.yaml` of the temporary local chart from the helmfile state
+	reqsContent, err := yaml.Marshal(unresolved.ToChartRequirements())
+	if err != nil {
+		return nil, err
+	}
+	if err := m.writeBytes(filepath.Join(wd, "requirements.yaml"), reqsContent); err != nil {
+		return nil, err
+	}
+
+	// Generate `requirements.lock` of the temporary local chart by coping `<basename>.lock`
+	lockFile := m.lockFileName()
+
+	lockFileContent, err := m.readBytes(lockFile)
+	if err != nil && !os.IsNotExist(err) {
+		return nil, err
+	}
+
+	if lockFileContent != nil {
+		if err := m.writeBytes(filepath.Join(wd, "requirements.lock"), lockFileContent); err != nil {
+			return nil, err
+		}
+	}
+
+	// Update the lock file by running `helm dependency update`
+	if err := shell.UpdateDeps(wd); err != nil {
+		return nil, err
+	}
+
+	updatedLockFileContent, err := m.readBytes(filepath.Join(wd, "requirements.lock"))
+	if err != nil {
+		return nil, err
+	}
+
+	// Sort requirements alphabetically by name.
+	lockedReqs := &ChartLockedRequirements{}
+	if err := yaml.Unmarshal(updatedLockFileContent, lockedReqs); err != nil {
+		return nil, err
+	}
+
+	sort.Slice(lockedReqs.ResolvedDependencies, func(i, j int) bool {
+		return lockedReqs.ResolvedDependencies[i].ChartName < lockedReqs.ResolvedDependencies[j].ChartName
+	})
+
+	updatedLockFileContent, err = yaml.Marshal(lockedReqs)
+
+	if err != nil {
+		return nil, err
+	}
+
+	// Commit the lock file if and only if everything looks ok
+	if err := m.writeBytes(lockFile, updatedLockFileContent); err != nil {
+		return nil, err
+	}
+
+	resolved, _, err := m.Resolve(unresolved)
+	return resolved, err
+}
+
+func (m *chartDependencyManager) Resolve(unresolved *UnresolvedDependencies) (*ResolvedDependencies, bool, error) {
+	updatedLockFileContent, err := m.readBytes(m.lockFileName())
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, false, nil
+		}
+		return nil, false, err
+	}
+
+	// Load resolved dependencies into memory
+	lockedReqs := &ChartLockedRequirements{}
+	if err := yaml.Unmarshal(updatedLockFileContent, lockedReqs); err != nil {
+		return nil, false, err
+	}
+
+	resolved := &ResolvedDependencies{deps: map[string][]ResolvedChartDependency{}}
+	for _, d := range lockedReqs.ResolvedDependencies {
+		if err := resolved.add(d); err != nil {
+			return nil, false, err
+		}
+	}
+
+	return resolved, true, nil
+}
+
+func (m *chartDependencyManager) readBytes(filename string) ([]byte, error) {
+	bytes, err := m.readFile(filename)
+	if err != nil {
+		return nil, err
+	}
+	m.logger.Debugf("readBytes: read from %s:\n%s", filename, bytes)
+	return bytes, nil
+}
+
+func (m *chartDependencyManager) writeBytes(filename string, data []byte) error {
+	err := m.writeFile(filename, data, 0644)
+	if err != nil {
+		return err
+	}
+	m.logger.Debugf("writeBytes: wrote to %s:\n%s", filename, data)
+	return nil
+}

pkg/state/create.go

@@ -0,0 +1,261 @@
+package state
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"github.com/imdario/mergo"
+	"github.com/roboll/helmfile/pkg/environment"
+	"github.com/roboll/helmfile/pkg/helmexec"
+	"go.uber.org/zap"
+	"gopkg.in/yaml.v2"
+	"io"
+	"os"
+)
+
+type StateLoadError struct {
+	msg   string
+	Cause error
+}
+
+func (e *StateLoadError) Error() string {
+	return fmt.Sprintf("%s: %v", e.msg, e.Cause)
+}
+
+type UndefinedEnvError struct {
+	msg string
+}
+
+func (e *UndefinedEnvError) Error() string {
+	return e.msg
+}
+
+type StateCreator struct {
+	logger     *zap.SugaredLogger
+	readFile   func(string) ([]byte, error)
+	fileExists func(string) (bool, error)
+	abs        func(string) (string, error)
+	glob       func(string) ([]string, error)
+
+	Strict bool
+
+	LoadFile func(inheritedEnv *environment.Environment, baseDir, file string, evaluateBases bool) (*HelmState, error)
+}
+
+func NewCreator(logger *zap.SugaredLogger, readFile func(string) ([]byte, error), fileExists func(string) (bool, error), abs func(string) (string, error), glob func(string) ([]string, error)) *StateCreator {
+	return &StateCreator{
+		logger:     logger,
+		readFile:   readFile,
+		fileExists: fileExists,
+		abs:        abs,
+		glob:       glob,
+		Strict:     true,
+	}
+}
+
+// Parse parses YAML into HelmState
+func (c *StateCreator) Parse(content []byte, baseDir, file string) (*HelmState, error) {
+	var state HelmState
+
+	state.FilePath = file
+	state.basePath = baseDir
+
+	decoder := yaml.NewDecoder(bytes.NewReader(content))
+	if !c.Strict {
+		decoder.SetStrict(false)
+	} else {
+		decoder.SetStrict(true)
+	}
+	i := 0
+	for {
+		i++
+
+		var intermediate HelmState
+
+		err := decoder.Decode(&intermediate)
+		if err == io.EOF {
+			break
+		} else if err != nil {
+			return nil, &StateLoadError{fmt.Sprintf("failed to read %s: reading document at index %d", file, i), err}
+		}
+
+		if err := mergo.Merge(&state, &intermediate, mergo.WithAppendSlice); err != nil {
+			return nil, &StateLoadError{fmt.Sprintf("failed to read %s: merging document at index %d", file, i), err}
+		}
+	}
+
+	if len(state.DeprecatedReleases) > 0 {
+		if len(state.Releases) > 0 {
+			return nil, fmt.Errorf("failed to parse %s: you can't specify both `charts` and `releases` sections", file)
+		}
+		state.Releases = state.DeprecatedReleases
+		state.DeprecatedReleases = []ReleaseSpec{}
+	}
+
+	if state.DeprecatedContext != "" && state.HelmDefaults.KubeContext == "" {
+		state.HelmDefaults.KubeContext = state.DeprecatedContext
+	}
+
+	state.logger = c.logger
+
+	state.readFile = c.readFile
+	state.removeFile = os.Remove
+	state.fileExists = c.fileExists
+	state.glob = c.glob
+
+	return &state, nil
+}
+
+// LoadEnvValues loads environment values files relative to the `baseDir`
+func (c *StateCreator) LoadEnvValues(target *HelmState, env string, ctxEnv *environment.Environment) (*HelmState, error) {
+	state := *target
+
+	e, err := state.loadEnvValues(env, ctxEnv, c.readFile, c.glob)
+	if err != nil {
+		return nil, &StateLoadError{fmt.Sprintf("failed to read %s", state.FilePath), err}
+	}
+
+	e.Defaults, err = state.loadValuesEntries(nil, state.DefaultValues)
+	if err != nil {
+		return nil, err
+	}
+
+	state.Env = *e
+
+	return &state, nil
+}
+
+// Parses YAML into HelmState, while loading environment values files relative to the `baseDir`
+func (c *StateCreator) ParseAndLoad(content []byte, baseDir, file string, envName string, evaluateBases bool, envValues *environment.Environment) (*HelmState, error) {
+	state, err := c.Parse(content, baseDir, file)
+	if err != nil {
+		return nil, err
+	}
+
+	if !evaluateBases {
+		if len(state.Bases) > 0 {
+			return nil, errors.New("nested `base` helmfile is unsupported. please submit a feature request if you need this!")
+		}
+	}
+
+	state, err = c.loadBases(envValues, state, baseDir)
+	if err != nil {
+		return nil, err
+	}
+
+	state, err = c.LoadEnvValues(state, envName, envValues)
+	if err != nil {
+		return nil, err
+	}
+
+	state.FilePath = file
+
+	return state, nil
+}
+
+func (c *StateCreator) loadBases(envValues *environment.Environment, st *HelmState, baseDir string) (*HelmState, error) {
+	layers := []*HelmState{}
+	for _, b := range st.Bases {
+		base, err := c.LoadFile(envValues, baseDir, b, false)
+		if err != nil {
+			return nil, err
+		}
+		layers = append(layers, base)
+	}
+	layers = append(layers, st)
+
+	for i := 1; i < len(layers); i++ {
+		if err := mergo.Merge(layers[0], layers[i], mergo.WithAppendSlice); err != nil {
+			return nil, err
+		}
+	}
+
+	return layers[0], nil
+}
+
+func (st *HelmState) loadEnvValues(name string, ctxEnv *environment.Environment, readFile func(string) ([]byte, error), glob func(string) ([]string, error)) (*environment.Environment, error) {
+	envVals := map[string]interface{}{}
+	envSpec, ok := st.Environments[name]
+	if ok {
+		var err error
+		envVals, err = st.loadValuesEntries(envSpec.MissingFileHandler, envSpec.Values)
+		if err != nil {
+			return nil, err
+		}
+
+		if len(envSpec.Secrets) > 0 {
+			helm := helmexec.New(st.logger, "")
+
+			var envSecretFiles []string
+			for _, urlOrPath := range envSpec.Secrets {
+				resolved, skipped, err := st.storage().resolveFile(envSpec.MissingFileHandler, "environment values", urlOrPath)
+				if err != nil {
+					return nil, err
+				}
+				if skipped {
+					continue
+				}
+
+				envSecretFiles = append(envSecretFiles, resolved...)
+			}
+
+			for _, path := range envSecretFiles {
+				// Work-around to allow decrypting environment secrets
+				//
+				// We don't have releases loaded yet and therefore unable to decide whether
+				// helmfile should use helm-tiller to call helm-secrets or not.
+				//
+				// This means that, when you use environment secrets + tillerless setup, you still need a tiller
+				// installed on the cluster, just for decrypting secrets!
+				// Related: https://github.com/futuresimple/helm-secrets/issues/83
+				release := &ReleaseSpec{}
+				flags := st.appendTillerFlags([]string{}, release)
+				decFile, err := helm.DecryptSecret(st.createHelmContext(release, 0), path, flags...)
+				if err != nil {
+					return nil, err
+				}
+				bytes, err := readFile(decFile)
+				if err != nil {
+					return nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", path, err)
+				}
+				m := map[string]interface{}{}
+				if err := yaml.Unmarshal(bytes, &m); err != nil {
+					return nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", path, err)
+				}
+				if err := mergo.Merge(&envVals, &m, mergo.WithOverride); err != nil {
+					return nil, fmt.Errorf("failed to load \"%s\": %v", path, err)
+				}
+			}
+		}
+	} else if ctxEnv == nil && name != DefaultEnv {
+		return nil, &UndefinedEnvError{msg: fmt.Sprintf("environment \"%s\" is not defined", name)}
+	}
+
+	newEnv := &environment.Environment{Name: name, Values: envVals}
+
+	if ctxEnv != nil {
+		intEnv := *ctxEnv
+
+		if err := mergo.Merge(&intEnv, newEnv, mergo.WithOverride); err != nil {
+			return nil, fmt.Errorf("error while merging environment values for \"%s\": %v", name, err)
+		}
+
+		newEnv = &intEnv
+	}
+
+	return newEnv, nil
+}
+
+func (st *HelmState) loadValuesEntries(missingFileHandler *string, entries []interface{}) (map[string]interface{}, error) {
+	envVals := map[string]interface{}{}
+
+	valuesEntries := append([]interface{}{}, entries...)
+	ld := NewEnvironmentValuesLoader(st.storage(), st.readFile, st.logger)
+	var err error
+	envVals, err = ld.LoadEnvironmentValues(missingFileHandler, valuesEntries)
+	if err != nil {
+		return nil, err
+	}
+
+	return envVals, nil
+}

pkg/state/create_test.go

@@ -1,12 +1,27 @@
 package state
 
 import (
-	"fmt"
+	"github.com/roboll/helmfile/pkg/testhelper"
+	"go.uber.org/zap"
+	"io/ioutil"
 	"path/filepath"
 	"reflect"
 	"testing"
+
+	. "gotest.tools/assert"
+	"gotest.tools/assert/cmp"
 )
 
+func createFromYaml(content []byte, file string, env string, logger *zap.SugaredLogger) (*HelmState, error) {
+	c := &StateCreator{
+		logger:   logger,
+		readFile: ioutil.ReadFile,
+		abs:      filepath.Abs,
+		Strict:   true,
+	}
+	return c.ParseAndLoad(content, filepath.Dir(file), file, env, false, nil)
+}
+
 func TestReadFromYaml(t *testing.T) {
 	yamlFile := "example/path/to/yaml/file"
 	yamlContent := []byte(`releases:
@@ -84,23 +99,17 @@ bar: {{ readFile "bar.txt" }}
 
 	expectedValues := `env: production`
 
-	readFile := func(filename string) ([]byte, error) {
-		switch filename {
-		case fooYamlFile:
-			return fooYamlContent, nil
-		case barYamlFile:
-			return barYamlContent, nil
-		case barTextFile:
-			return barTextContent, nil
-		case valuesFile:
-			return valuesContent, nil
-		}
-		return nil, fmt.Errorf("unexpected filename: %s", filename)
-	}
+	testFs := testhelper.NewTestFs(map[string]string{
+		fooYamlFile: string(fooYamlContent),
+		barYamlFile: string(barYamlContent),
+		barTextFile: string(barTextContent),
+		valuesFile:  string(valuesContent),
+	})
+	testFs.Cwd = "/example/path/to"
 
-	state, err := NewCreator(logger, readFile, filepath.Abs).CreateFromYaml(yamlContent, yamlFile, "production")
+	state, err := NewCreator(logger, testFs.ReadFile, testFs.FileExists, testFs.Abs, testFs.Glob).ParseAndLoad(yamlContent, filepath.Dir(yamlFile), yamlFile, "production", false, nil)
 	if err != nil {
-		t.Errorf("unexpected error: %v", err)
+		t.Fatalf("unexpected error: %v", err)
 	}
 
 	actual := state.Env.Values
@@ -247,3 +256,95 @@ func TestReadFromYaml_FilterNegatives(t *testing.T) {
 		}
 	}
 }
+
+func TestReadFromYaml_Helmfiles_Selectors(t *testing.T) {
+	tests := []struct {
+		path      string
+		content   []byte
+		wantErr   bool
+		helmfiles []SubHelmfileSpec
+	}{
+		{
+			path: "working/selector",
+			content: []byte(`helmfiles:
+- simple/helmfile.yaml
+- path: path/prefix/selector.yaml
+  selectors:
+    - name=zorba
+    - foo=bar
+- path: path/prefix/empty/selector.yaml
+  selectors: []
+- path: path/prefix/inherits/selector.yaml
+  selectorsInherited: true
+`),
+			wantErr: false,
+			helmfiles: []SubHelmfileSpec{{Path: "simple/helmfile.yaml", Selectors: nil, SelectorsInherited: false},
+				{Path: "path/prefix/selector.yaml", Selectors: []string{"name=zorba", "foo=bar"}, SelectorsInherited: false},
+				{Path: "path/prefix/empty/selector.yaml", Selectors: []string{}, SelectorsInherited: false},
+				{Path: "path/prefix/inherits/selector.yaml", Selectors: nil, SelectorsInherited: true},
+			},
+		},
+		{
+			path: "failing2/selector",
+			content: []byte(`helmfiles:
+- path: failing2/helmfile.yaml
+    wrongkey:
+`),
+			wantErr: true,
+		},
+		{
+			path: "failing3/selector",
+			content: []byte(`helmfiles:
+- path: failing3/helmfile.yaml
+    selectors: foo
+`),
+			wantErr: true,
+		},
+		{
+			path: "failing4/selector",
+			content: []byte(`helmfiles:
+- path: failing4/helmfile.yaml
+    selectors:
+`),
+			wantErr: true,
+		},
+		{
+			path: "failing4/selector",
+			content: []byte(`helmfiles:
+- path: failing4/helmfile.yaml
+    selectors:
+      - colon: not-authorized
+`),
+			wantErr: true,
+		},
+		{
+			path: "failing6/selector",
+			content: []byte(`helmfiles:
+- selectors:
+    - whatever
+`),
+			wantErr: true,
+		},
+		{
+			path: "failing7/selector",
+			content: []byte(`helmfiles:
+- path: foo/bar
+  selectors:
+  - foo=bar
+  selectorsInherited: true
+`),
+			wantErr: true,
+		},
+	}
+	for _, test := range tests {
+		st, err := createFromYaml(test.content, test.path, DefaultEnv, logger)
+		if err != nil {
+			if test.wantErr {
+				continue
+			} else {
+				t.Error("unexpected error:", err)
+			}
+		}
+		Assert(t, cmp.DeepEqual(st.Helmfiles, test.helmfiles), "for path %v", test.path)
+	}
+}

pkg/state/environment.go

@@ -0,0 +1,15 @@
+package state
+
+type EnvironmentSpec struct {
+	Values  []interface{} `yaml:"values"`
+	Secrets []string      `yaml:"secrets"`
+
+	// MissingFileHandler instructs helmfile to fail when unable to find a environment values file listed
+	// under `environments.NAME.values`.
+	//
+	// Possible values are  "Error", "Warn", "Info", "Debug". The default is "Error".
+	//
+	// Use "Warn", "Info", or "Debug" if you want helmfile to not fail when a values file is missing, while just leaving
+	// a message about the missing file at the log-level.
+	MissingFileHandler *string `yaml:"missingFileHandler"`
+}

pkg/state/envvals_loader.go

@@ -0,0 +1,83 @@
+package state
+
+import (
+	"fmt"
+	"github.com/imdario/mergo"
+	"github.com/roboll/helmfile/pkg/environment"
+	"github.com/roboll/helmfile/pkg/maputil"
+	"github.com/roboll/helmfile/pkg/tmpl"
+	"go.uber.org/zap"
+	"gopkg.in/yaml.v2"
+	"path/filepath"
+)
+
+type EnvironmentValuesLoader struct {
+	storage *Storage
+
+	readFile func(string) ([]byte, error)
+
+	logger *zap.SugaredLogger
+}
+
+func NewEnvironmentValuesLoader(storage *Storage, readFile func(string) ([]byte, error), logger *zap.SugaredLogger) *EnvironmentValuesLoader {
+	return &EnvironmentValuesLoader{
+		storage:  storage,
+		readFile: readFile,
+		logger:   logger,
+	}
+}
+
+func (ld *EnvironmentValuesLoader) LoadEnvironmentValues(missingFileHandler *string, valuesEntries []interface{}) (map[string]interface{}, error) {
+	result := map[string]interface{}{}
+
+	for _, entry := range valuesEntries {
+		maps := []interface{}{}
+
+		switch strOrMap := entry.(type) {
+		case string:
+			urlOrPath := strOrMap
+			files, skipped, err := ld.storage.resolveFile(missingFileHandler, "environment values", urlOrPath)
+			if err != nil {
+				return nil, err
+			}
+			if skipped {
+				continue
+			}
+
+			for _, f := range files {
+				tmplData := EnvironmentTemplateData{environment.EmptyEnvironment, "", map[string]interface{}{}}
+				r := tmpl.NewFileRenderer(ld.readFile, filepath.Dir(f), tmplData)
+				bytes, err := r.RenderToBytes(f)
+				if err != nil {
+					return nil, fmt.Errorf("failed to load environment values file \"%s\": %v", f, err)
+				}
+				m := map[string]interface{}{}
+				if err := yaml.Unmarshal(bytes, &m); err != nil {
+					return nil, fmt.Errorf("failed to load environment values file \"%s\": %v", f, err)
+				}
+				maps = append(maps, m)
+				if ld.logger != nil {
+					ld.logger.Debugf("envvals_loader: loaded %s:%v", strOrMap, m)
+				}
+			}
+		case map[interface{}]interface{}:
+			maps = append(maps, strOrMap)
+		default:
+			return nil, fmt.Errorf("unexpected type of value: value=%v, type=%T", strOrMap, strOrMap)
+		}
+		for _, m := range maps {
+			// All the nested map key should be string. Otherwise we get strange errors due to that
+			// mergo or reflect is unable to merge map[interface{}]interface{} with map[string]interface{} or vice versa.
+			// See https://github.com/roboll/helmfile/issues/677
+			vals, err := maputil.CastKeysToStrings(m)
+			if err != nil {
+				return nil, err
+			}
+			if err := mergo.Merge(&result, &vals, mergo.WithOverride); err != nil {
+				return nil, fmt.Errorf("failed to merge %v: %v", m, err)
+			}
+		}
+	}
+
+	return result, nil
+}

pkg/state/helmx.go

@@ -0,0 +1,51 @@
+package state
+
+type Dependency struct {
+	Chart   string `yaml:"chart"`
+	Version string `yaml:"version"`
+	Alias   string `yaml:"alias"`
+}
+
+func (st *HelmState) appendHelmXFlags(flags []string, release *ReleaseSpec) ([]string, error) {
+	for _, d := range release.Dependencies {
+		var dep string
+		if d.Alias != "" {
+			dep += d.Alias + "="
+		}
+		dep += d.Chart
+		if d.Version != "" {
+			dep += ":" + d.Version
+		}
+		flags = append(flags, "--dependency", dep)
+	}
+
+	jsonPatches := release.JSONPatches
+	if len(jsonPatches) > 0 {
+		generatedFiles, err := st.generateTemporaryValuesFiles(jsonPatches, release.MissingFileHandler)
+		if err != nil {
+			return nil, err
+		}
+
+		for _, f := range generatedFiles {
+			flags = append(flags, "--json-patch", f)
+		}
+
+		release.generatedValues = append(release.generatedValues, generatedFiles...)
+	}
+
+	strategicMergePatches := release.StrategicMergePatches
+	if len(strategicMergePatches) > 0 {
+		generatedFiles, err := st.generateTemporaryValuesFiles(strategicMergePatches, release.MissingFileHandler)
+		if err != nil {
+			return nil, err
+		}
+
+		for _, f := range generatedFiles {
+			flags = append(flags, "--strategic-merge-patch", f)
+		}
+
+		release.generatedValues = append(release.generatedValues, generatedFiles...)
+	}
+
+	return flags, nil
+}

pkg/state/release.go

@@ -2,7 +2,8 @@ package state
 
 import (
 	"fmt"
-	"github.com/roboll/helmfile/tmpl"
+
+	"github.com/roboll/helmfile/pkg/tmpl"
 	"gopkg.in/yaml.v2"
 )
 
@@ -15,6 +16,14 @@ func (r ReleaseSpec) ExecuteTemplateExpressions(renderer *tmpl.FileRenderer) (*R
 		return nil, fmt.Errorf("failed executing template expressions in release \"%s\": %v", r.Name, err)
 	}
 
+	{
+		ts := result.Name
+		result.Name, err = renderer.RenderTemplateContentToString([]byte(ts))
+		if err != nil {
+			return nil, fmt.Errorf("failed executing template expressions in release \"%s\".name = \"%s\": %v", r.Name, ts, err)
+		}
+	}
+
 	{
 		ts := result.Chart
 		result.Chart, err = renderer.RenderTemplateContentToString([]byte(ts))
@@ -31,6 +40,14 @@ func (r ReleaseSpec) ExecuteTemplateExpressions(renderer *tmpl.FileRenderer) (*R
 		}
 	}
 
+	{
+		ts := result.Version
+		result.Version, err = renderer.RenderTemplateContentToString([]byte(ts))
+		if err != nil {
+			return nil, fmt.Errorf("failed executing template expressions in release \"%s\".version = \"%s\": %v", r.Name, ts, err)
+		}
+	}
+
 	for i, t := range result.Values {
 		switch ts := t.(type) {
 		case string:

pkg/state/release_error.go

@@ -0,0 +1,21 @@
+package state
+
+import (
+	"fmt"
+)
+
+const ReleaseErrorCodeFailure = 1
+
+type ReleaseError struct {
+	*ReleaseSpec
+	err  error
+	Code int
+}
+
+func (e *ReleaseError) Error() string {
+	return fmt.Sprintf("failed processing release %s: %v", e.Name, e.err.Error())
+}
+
+func newReleaseError(release *ReleaseSpec, err error) *ReleaseError {
+	return &ReleaseError{release, err, ReleaseErrorCodeFailure}
+}

pkg/state/state.go

@@ -11,34 +11,38 @@ import (
 	"strconv"
 	"strings"
 
-	"github.com/roboll/helmfile/helmexec"
+	"github.com/roboll/helmfile/pkg/environment"
+	"github.com/roboll/helmfile/pkg/event"
+	"github.com/roboll/helmfile/pkg/helmexec"
+	"github.com/roboll/helmfile/pkg/remote"
+	"github.com/roboll/helmfile/pkg/tmpl"
 
 	"regexp"
 
-	"os/exec"
-	"syscall"
-
-	"github.com/roboll/helmfile/environment"
-	"github.com/roboll/helmfile/event"
-	"github.com/roboll/helmfile/tmpl"
 	"github.com/tatsushid/go-prettytable"
 	"go.uber.org/zap"
 	"gopkg.in/yaml.v2"
-	"net/url"
 )
 
 // HelmState structure for the helmfile
 type HelmState struct {
-	basePath           string
-	Environments       map[string]EnvironmentSpec
-	FilePath           string
-	HelmDefaults       HelmSpec         `yaml:"helmDefaults"`
-	Helmfiles          []string         `yaml:"helmfiles"`
-	DeprecatedContext  string           `yaml:"context"`
-	DeprecatedReleases []ReleaseSpec    `yaml:"charts"`
-	Namespace          string           `yaml:"namespace"`
-	Repositories       []RepositorySpec `yaml:"repositories"`
-	Releases           []ReleaseSpec    `yaml:"releases"`
+	basePath string
+	FilePath string
+
+	// DefaultValues is the default values to be overrode by environment values and command-line overrides
+	DefaultValues []interface{} `yaml:"values"`
+
+	Environments map[string]EnvironmentSpec `yaml:"environments"`
+
+	Bases              []string          `yaml:"bases"`
+	HelmDefaults       HelmSpec          `yaml:"helmDefaults"`
+	Helmfiles          []SubHelmfileSpec `yaml:"helmfiles"`
+	DeprecatedContext  string            `yaml:"context"`
+	DeprecatedReleases []ReleaseSpec     `yaml:"charts"`
+	Namespace          string            `yaml:"namespace"`
+	Repositories       []RepositorySpec  `yaml:"repositories"`
+	Releases           []ReleaseSpec     `yaml:"releases"`
+	Selectors          []string
 
 	Templates map[string]TemplateSpec `yaml:"templates"`
 
@@ -50,10 +54,25 @@ type HelmState struct {
 
 	removeFile func(string) error
 	fileExists func(string) (bool, error)
+	glob       func(string) ([]string, error)
+	tempDir    func(string, string) (string, error)
 
 	runner helmexec.Runner
 }
 
+// SubHelmfileSpec defines the subhelmfile path and options
+type SubHelmfileSpec struct {
+	Path               string   //path or glob pattern for the sub helmfiles
+	Selectors          []string //chosen selectors for the sub helmfiles
+	SelectorsInherited bool     //do the sub helmfiles inherits from parent selectors
+
+	Environment SubhelmfileEnvironmentSpec
+}
+
+type SubhelmfileEnvironmentSpec struct {
+	OverrideValues []interface{} `yaml:"values"`
+}
+
 // HelmSpec to defines helmDefault values
 type HelmSpec struct {
 	KubeContext     string   `yaml:"kubeContext"`
@@ -139,6 +158,11 @@ type ReleaseSpec struct {
 	TLSKey    string `yaml:"tlsKey"`
 	TLSCert   string `yaml:"tlsCert"`
 
+	// These settings requires helm-x integration to work
+	Dependencies          []Dependency  `yaml:"dependencies"`
+	JSONPatches           []interface{} `yaml:"jsonPatches"`
+	StrategicMergePatches []interface{} `yaml:"strategicMergePatches"`
+
 	// generatedValues are values that need cleaned up on exit
 	generatedValues []string
 	//version of the chart that has really been installed cause desired version may be fuzzy (~2.0.0)
@@ -162,6 +186,11 @@ type AffectedReleases struct {
 
 const DefaultEnv = "default"
 
+const MissingFileHandlerError = "Error"
+const MissingFileHandlerInfo = "Info"
+const MissingFileHandlerWarn = "Warn"
+const MissingFileHandlerDebug = "Debug"
+
 func (st *HelmState) applyDefaultsTo(spec *ReleaseSpec) {
 	if st.Namespace != "" {
 		spec.Namespace = st.Namespace
@@ -193,15 +222,6 @@ func (st *HelmState) SyncRepos(helm RepoUpdater) []error {
 	return nil
 }
 
-type ReleaseError struct {
-	*ReleaseSpec
-	underlying error
-}
-
-func (e *ReleaseError) Error() string {
-	return e.underlying.Error()
-}
-
 type syncResult struct {
 	errors []*ReleaseError
 }
@@ -239,9 +259,20 @@ func (st *HelmState) prepareSyncReleases(helm helmexec.Interface, additionalValu
 			for release := range jobs {
 				st.applyDefaultsTo(release)
 
+				// If `installed: false`, the only potential operation on this release would be uninstalling.
+				// We skip generating values files in that case, because for an uninstall with `helm delete`, we don't need to those.
+				// The values files are for `helm upgrade -f values.yaml` calls that happens when the release has `installed: true`.
+				// This logic addresses:
+				// - https://github.com/roboll/helmfile/issues/519
+				// - https://github.com/roboll/helmfile/issues/616
+				if !release.Desired() {
+					results <- syncPrepareResult{release: release, flags: []string{}, errors: []*ReleaseError{}}
+					continue
+				}
+
 				flags, flagsErr := st.flagsForUpgrade(helm, release, workerIndex)
 				if flagsErr != nil {
-					results <- syncPrepareResult{errors: []*ReleaseError{&ReleaseError{release, flagsErr}}}
+					results <- syncPrepareResult{errors: []*ReleaseError{newReleaseError(release, flagsErr)}}
 					continue
 				}
 
@@ -249,14 +280,14 @@ func (st *HelmState) prepareSyncReleases(helm helmexec.Interface, additionalValu
 				for _, value := range additionalValues {
 					valfile, err := filepath.Abs(value)
 					if err != nil {
-						errs = append(errs, &ReleaseError{release, err})
+						errs = append(errs, newReleaseError(release, err))
 					}
 
 					ok, err := st.fileExists(valfile)
 					if err != nil {
-						errs = append(errs, &ReleaseError{release, err})
+						errs = append(errs, newReleaseError(release, err))
 					} else if !ok {
-						errs = append(errs, &ReleaseError{release, fmt.Errorf("file does not exist: %s", valfile)})
+						errs = append(errs, newReleaseError(release, fmt.Errorf("file does not exist: %s", valfile)))
 					}
 					flags = append(flags, "--values", valfile)
 				}
@@ -298,7 +329,9 @@ func (st *HelmState) isReleaseInstalled(context helmexec.HelmContext, helm helme
 
 func (st *HelmState) DetectReleasesToBeDeleted(helm helmexec.Interface) ([]*ReleaseSpec, error) {
 	detected := []*ReleaseSpec{}
-	for _, release := range st.Releases {
+	for i := range st.Releases {
+		release := st.Releases[i]
+
 		if !release.Desired() {
 			installed, err := st.isReleaseInstalled(st.createHelmContext(&release, 0), helm, release)
 			if err != nil {
@@ -340,27 +373,32 @@ func (st *HelmState) SyncReleases(affectedReleases *AffectedReleases, helm helme
 				chart := normalizeChart(st.basePath, release.Chart)
 				var relErr *ReleaseError
 				context := st.createHelmContext(release, workerIndex)
-				if !release.Desired() {
+
+				if _, err := st.triggerPresyncEvent(release, "sync"); err != nil {
+					relErr = newReleaseError(release, err)
+				} else if !release.Desired() {
 					installed, err := st.isReleaseInstalled(context, helm, *release)
 					if err != nil {
-						relErr = &ReleaseError{release, err}
+						relErr = newReleaseError(release, err)
 					} else if installed {
 						if err := helm.DeleteRelease(context, release.Name, "--purge"); err != nil {
 							affectedReleases.Failed = append(affectedReleases.Failed, release)
-							relErr = &ReleaseError{release, err}
+							relErr = newReleaseError(release, err)
 						} else {
 							affectedReleases.Deleted = append(affectedReleases.Deleted, release)
 						}
 					}
 				} else if err := helm.SyncRelease(context, release.Name, chart, flags...); err != nil {
 					affectedReleases.Failed = append(affectedReleases.Failed, release)
-					relErr = &ReleaseError{release, err}
+					relErr = newReleaseError(release, err)
 				} else {
 					affectedReleases.Upgraded = append(affectedReleases.Upgraded, release)
 					installedVersion, err := st.getDeployedVersion(context, helm, release)
-					//err is not really impacting so just ignors it
-					st.logger.Debugf("getting deployed release version failed:%v", err)
-					release.installedVersion = installedVersion
+					if err != nil { //err is not really impacting so just log it
+						st.logger.Debugf("getting deployed release version failed:%v", err)
+					} else {
+						release.installedVersion = installedVersion
+					}
 				}
 
 				if relErr == nil {
@@ -369,7 +407,11 @@ func (st *HelmState) SyncReleases(affectedReleases *AffectedReleases, helm helme
 					results <- syncResult{errors: []*ReleaseError{relErr}}
 				}
 
-				if _, err := st.triggerCleanupEvent(prep.release, "sync"); err != nil {
+				if _, err := st.triggerPostsyncEvent(release, "sync"); err != nil {
+					st.logger.Warnf("warn: %v\n", err)
+				}
+
+				if _, err := st.triggerCleanupEvent(release, "sync"); err != nil {
 					st.logger.Warnf("warn: %v\n", err)
 				}
 			}
@@ -398,7 +440,8 @@ func (st *HelmState) getDeployedVersion(context helmexec.HelmContext, helm helme
 	//retrieve the version
 	if out, err := helm.List(context, "^"+release.Name+"$", st.tillerFlags(release)...); err == nil {
 		chartName := filepath.Base(release.Chart)
-		pat := regexp.MustCompile(chartName + "-(.*?)\\s")
+		//the regexp without escapes : .*\s.*\s.*\s.*\schartName-(.*?)\s
+		pat := regexp.MustCompile(".*\\s.*\\s.*\\s.*\\s" + chartName + "-(.*?)\\s")
 		versions := pat.FindStringSubmatch(out)
 		if len(versions) > 0 {
 			return versions[1], nil
@@ -446,6 +489,10 @@ func (st *HelmState) downloadCharts(helm helmexec.Interface, dir string, concurr
 						chartPath = path.Join(dir, release.Name, "latest", release.Chart)
 					}
 
+					if st.isDevelopment(release) {
+						fetchFlags = append(fetchFlags, "--devel")
+					}
+
 					// only fetch chart if it is not already fetched
 					if _, err := os.Stat(chartPath); os.IsNotExist(err) {
 						fetchFlags = append(fetchFlags, "--untar", "--untardir", chartPath)
@@ -459,7 +506,6 @@ func (st *HelmState) downloadCharts(helm helmexec.Interface, dir string, concurr
 						chartPath = filepath.Dir(fullChartPath)
 					}
 				}
-
 				results <- &downloadResults{release.Name, chartPath}
 			}
 		},
@@ -479,6 +525,9 @@ func (st *HelmState) downloadCharts(helm helmexec.Interface, dir string, concurr
 
 // TemplateReleases wrapper for executing helm template on the releases
 func (st *HelmState) TemplateReleases(helm helmexec.Interface, additionalValues []string, args []string, workerLimit int) []error {
+	// Reset the extra args if already set, not to break `helm fetch` by adding the args intended for `lint`
+	helm.SetExtraArgs()
+
 	errs := []error{}
 	// Create tmp directory and bail immediately if it fails
 	dir, err := ioutil.TempDir("", "")
@@ -499,7 +548,9 @@ func (st *HelmState) TemplateReleases(helm helmexec.Interface, additionalValues
 		helm.SetExtraArgs(args...)
 	}
 
-	for _, release := range st.Releases {
+	for i := range st.Releases {
+		release := st.Releases[i]
+
 		if !release.Desired() {
 			continue
 		}
@@ -540,6 +591,9 @@ func (st *HelmState) TemplateReleases(helm helmexec.Interface, additionalValues
 
 // LintReleases wrapper for executing helm lint on the releases
 func (st *HelmState) LintReleases(helm helmexec.Interface, additionalValues []string, args []string, workerLimit int) []error {
+	// Reset the extra args if already set, not to break `helm fetch` by adding the args intended for `lint`
+	helm.SetExtraArgs()
+
 	errs := []error{}
 	// Create tmp directory and bail immediately if it fails
 	dir, err := ioutil.TempDir("", "")
@@ -559,7 +613,9 @@ func (st *HelmState) LintReleases(helm helmexec.Interface, additionalValues []st
 		helm.SetExtraArgs(args...)
 	}
 
-	for _, release := range st.Releases {
+	for i := range st.Releases {
+		release := st.Releases[i]
+
 		if !release.Desired() {
 			continue
 		}
@@ -598,18 +654,8 @@ func (st *HelmState) LintReleases(helm helmexec.Interface, additionalValues []st
 	return nil
 }
 
-type DiffError struct {
-	*ReleaseSpec
-	err  error
-	Code int
-}
-
-func (e *DiffError) Error() string {
-	return e.err.Error()
-}
-
 type diffResult struct {
-	err *DiffError
+	err *ReleaseError
 }
 
 type diffPrepareResult struct {
@@ -677,7 +723,7 @@ func (st *HelmState) prepareDiffReleases(helm helmexec.Interface, additionalValu
 				if len(errs) > 0 {
 					rsErrs := make([]*ReleaseError, len(errs))
 					for i, e := range errs {
-						rsErrs[i] = &ReleaseError{release, e}
+						rsErrs[i] = newReleaseError(release, e)
 					}
 					results <- diffPrepareResult{errors: rsErrs}
 				} else {
@@ -748,12 +794,11 @@ func (st *HelmState) DiffReleases(helm helmexec.Interface, additionalValues []st
 				release := prep.release
 				if err := helm.DiffRelease(st.createHelmContext(release, workerIndex), release.Name, normalizeChart(st.basePath, release.Chart), flags...); err != nil {
 					switch e := err.(type) {
-					case *exec.ExitError:
+					case helmexec.ExitError:
 						// Propagate any non-zero exit status from the external command like `helm` that is failed under the hood
-						status := e.Sys().(syscall.WaitStatus)
-						results <- diffResult{&DiffError{release, err, status.ExitStatus()}}
+						results <- diffResult{&ReleaseError{release, err, e.ExitStatus()}}
 					default:
-						results <- diffResult{&DiffError{release, err, 0}}
+						results <- diffResult{&ReleaseError{release, err, 0}}
 					}
 				} else {
 					// diff succeeded, found no changes
@@ -866,11 +911,11 @@ func (st *HelmState) Clean() []error {
 }
 
 // FilterReleases allows for the execution of helm commands against a subset of the releases in the helmfile.
-func (st *HelmState) FilterReleases(labels []string) error {
+func (st *HelmState) FilterReleases() error {
 	var filteredReleases []ReleaseSpec
 	releaseSet := map[string][]ReleaseSpec{}
 	filters := []ReleaseFilter{}
-	for _, label := range labels {
+	for _, label := range st.Selectors {
 		f, err := ParseLabels(label)
 		if err != nil {
 			return err
@@ -902,22 +947,32 @@ func (st *HelmState) FilterReleases(labels []string) error {
 	}
 	st.Releases = filteredReleases
 	numFound := len(filteredReleases)
-	st.logger.Debugf("%d release(s) matching %s found in %s\n", numFound, strings.Join(labels, ","), st.FilePath)
+	st.logger.Debugf("%d release(s) matching %s found in %s\n", numFound, strings.Join(st.Selectors, ","), st.FilePath)
 	return nil
 }
 
-func (st *HelmState) PrepareRelease(helm helmexec.Interface, helmfileCommand string) []error {
+func (st *HelmState) PrepareReleases(helm helmexec.Interface, helmfileCommand string) []error {
 	errs := []error{}
 
-	for _, release := range st.Releases {
+	for i := range st.Releases {
+		release := st.Releases[i]
+
 		if _, err := st.triggerPrepareEvent(&release, helmfileCommand); err != nil {
-			errs = append(errs, &ReleaseError{&release, err})
+			errs = append(errs, newReleaseError(&release, err))
 			continue
 		}
 	}
 	if len(errs) != 0 {
 		return errs
 	}
+
+	updated, err := st.ResolveDeps()
+	if err != nil {
+		return []error{err}
+	}
+
+	*st = *updated
+
 	return nil
 }
 
@@ -929,6 +984,14 @@ func (st *HelmState) triggerCleanupEvent(r *ReleaseSpec, helmfileCommand string)
 	return st.triggerReleaseEvent("cleanup", r, helmfileCommand)
 }
 
+func (st *HelmState) triggerPresyncEvent(r *ReleaseSpec, helmfileCommand string) (bool, error) {
+	return st.triggerReleaseEvent("presync", r, helmfileCommand)
+}
+
+func (st *HelmState) triggerPostsyncEvent(r *ReleaseSpec, helmfileCommand string) (bool, error) {
+	return st.triggerReleaseEvent("postsync", r, helmfileCommand)
+}
+
 func (st *HelmState) triggerReleaseEvent(evt string, r *ReleaseSpec, helmfileCmd string) (bool, error) {
 	bus := &event.Bus{
 		Hooks:         r.Hooks,
@@ -946,6 +1009,11 @@ func (st *HelmState) triggerReleaseEvent(evt string, r *ReleaseSpec, helmfileCmd
 	return bus.Trigger(evt, data)
 }
 
+// ResolveDeps returns a copy of this helmfile state with the concrete chart version numbers filled in for remote chart dependencies
+func (st *HelmState) ResolveDeps() (*HelmState, error) {
+	return st.mergeLockedDependencies()
+}
+
 // UpdateDeps wrapper for updating dependencies on the releases
 func (st *HelmState) UpdateDeps(helm helmexec.Interface) []error {
 	errs := []error{}
@@ -957,6 +1025,18 @@ func (st *HelmState) UpdateDeps(helm helmexec.Interface) []error {
 			}
 		}
 	}
+
+	if len(errs) == 0 {
+		tempDir := st.tempDir
+		if tempDir == nil {
+			tempDir = ioutil.TempDir
+		}
+		_, err := st.updateDependenciesInTempDir(helm, tempDir)
+		if err != nil {
+			errs = append(errs, fmt.Errorf("unable to update deps: %v", err))
+		}
+	}
+
 	if len(errs) != 0 {
 		return errs
 	}
@@ -980,21 +1060,6 @@ func (st *HelmState) BuildDeps(helm helmexec.Interface) []error {
 	return nil
 }
 
-// JoinBase returns an absolute path in the form basePath/relative
-func (st *HelmState) JoinBase(relPath string) string {
-	return filepath.Join(st.basePath, relPath)
-}
-
-// normalizes relative path to absolute one
-func (st *HelmState) normalizePath(path string) string {
-	u, _ := url.Parse(path)
-	if u.Scheme != "" || filepath.IsAbs(path) {
-		return path
-	} else {
-		return st.JoinBase(path)
-	}
-}
-
 // normalizeChart allows for the distinction between a file path reference and repository references.
 // - Any single (or double character) followed by a `/` will be considered a local file reference and
 // 	 be constructed relative to the `base path`.
@@ -1008,7 +1073,12 @@ func normalizeChart(basePath, chart string) string {
 
 func isLocalChart(chart string) bool {
 	regex, _ := regexp.Compile("^[.]?./")
-	return regex.MatchString(chart)
+	matched := regex.MatchString(chart)
+	if matched {
+		return true
+	}
+
+	return chart == "" || chart[0] == '/' || len(strings.Split(chart, "/")) != 2
 }
 
 func pathExists(chart string) bool {
@@ -1024,7 +1094,10 @@ func chartNameWithoutRepository(chart string) string {
 // find "Chart.yaml"
 func findChartDirectory(topLevelDir string) (string, error) {
 	var files []string
-	filepath.Walk(topLevelDir, func(path string, f os.FileInfo, _ error) error {
+	filepath.Walk(topLevelDir, func(path string, f os.FileInfo, err error) error {
+		if err != nil {
+			return fmt.Errorf("error walking through %s: %v", path, err)
+		}
 		if !f.IsDir() {
 			r, err := regexp.MatchString("Chart.yaml", f.Name())
 			if err == nil && r {
@@ -1130,6 +1203,12 @@ func (st *HelmState) flagsForUpgrade(helm helmexec.Interface, release *ReleaseSp
 
 	flags = st.appendTillerFlags(flags, release)
 
+	var err error
+	flags, err = st.appendHelmXFlags(flags, release)
+	if err != nil {
+		return nil, err
+	}
+
 	common, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
 	if err != nil {
 		return nil, err
@@ -1141,6 +1220,13 @@ func (st *HelmState) flagsForTemplate(helm helmexec.Interface, release *ReleaseS
 	flags := []string{
 		"--name", release.Name,
 	}
+
+	var err error
+	flags, err = st.appendHelmXFlags(flags, release)
+	if err != nil {
+		return nil, err
+	}
+
 	common, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
 	if err != nil {
 		return nil, err
@@ -1160,6 +1246,12 @@ func (st *HelmState) flagsForDiff(helm helmexec.Interface, release *ReleaseSpec,
 
 	flags = st.appendTillerFlags(flags, release)
 
+	var err error
+	flags, err = st.appendHelmXFlags(flags, release)
+	if err != nil {
+		return nil, err
+	}
+
 	common, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
 	if err != nil {
 		return nil, err
@@ -1177,41 +1269,77 @@ func (st *HelmState) isDevelopment(release *ReleaseSpec) bool {
 }
 
 func (st *HelmState) flagsForLint(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, error) {
-	return st.namespaceAndValuesFlags(helm, release, workerIndex)
+	flags, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
+	if err != nil {
+		return nil, err
+	}
+
+	flags, err = st.appendHelmXFlags(flags, release)
+	if err != nil {
+		return nil, err
+	}
+
+	return flags, nil
 }
 
 func (st *HelmState) RenderValuesFileToBytes(path string) ([]byte, error) {
-	r := tmpl.NewFileRenderer(st.readFile, filepath.Dir(path), st.envTemplateData())
+	r := tmpl.NewFileRenderer(st.readFile, filepath.Dir(path), st.valuesFileTemplateData())
 	return r.RenderToBytes(path)
 }
 
+func (st *HelmState) storage() *Storage {
+	return &Storage{
+		FilePath: st.FilePath,
+		basePath: st.basePath,
+		glob:     st.glob,
+		logger:   st.logger,
+	}
+}
+
+func (st *HelmState) ExpandedHelmfiles() ([]SubHelmfileSpec, error) {
+	helmfiles := []SubHelmfileSpec{}
+	for _, hf := range st.Helmfiles {
+		if remote.IsRemote(hf.Path) {
+			helmfiles = append(helmfiles, hf)
+			continue
+		}
+
+		matches, err := st.storage().ExpandPaths(hf.Path)
+		if err != nil {
+			return nil, err
+		}
+		if len(matches) == 0 {
+			return nil, fmt.Errorf("no file matching %s found", hf.Path)
+		}
+		for _, match := range matches {
+			newHelmfile := hf
+			newHelmfile.Path = match
+			helmfiles = append(helmfiles, newHelmfile)
+		}
+	}
+
+	return helmfiles, nil
+}
+
 func (st *HelmState) generateTemporaryValuesFiles(values []interface{}, missingFileHandler *string) ([]string, error) {
 	generatedFiles := []string{}
 
 	for _, value := range values {
 		switch typedValue := value.(type) {
 		case string:
-			path := st.normalizePath(typedValue)
-
-			ok, err := st.fileExists(path)
+			paths, skip, err := st.storage().resolveFile(missingFileHandler, "values", typedValue)
 			if err != nil {
 				return nil, err
 			}
-			if !ok {
-				if missingFileHandler == nil || *missingFileHandler == "Error" {
-					return nil, fmt.Errorf("file does not exist: %s", path)
-				} else if *missingFileHandler == "Warn" {
-					st.logger.Warnf("skipping missing values file \"%s\"", path)
-					continue
-				} else if *missingFileHandler == "Info" {
-					st.logger.Infof("skipping missing values file \"%s\"", path)
-					continue
-				} else {
-					st.logger.Debugf("skipping missing values file \"%s\"", path)
-					continue
-				}
+			if skip {
+				continue
 			}
 
+			if len(paths) > 1 {
+				return nil, fmt.Errorf("glob patterns in release values and secrets is not supported yet. please submit a feature request if necessary")
+			}
+			path := paths[0]
+
 			yamlBytes, err := st.RenderValuesFileToBytes(path)
 			if err != nil {
 				return nil, fmt.Errorf("failed to render values files \"%s\": %v", typedValue, err)
@@ -1241,7 +1369,7 @@ func (st *HelmState) generateTemporaryValuesFiles(values []interface{}, missingF
 			}
 			generatedFiles = append(generatedFiles, valfile.Name())
 		default:
-			return nil, fmt.Errorf("unexpected type of values entry: %T", typedValue)
+			return nil, fmt.Errorf("unexpected type of value: value=%v, type=%T", typedValue, typedValue)
 		}
 	}
 	return generatedFiles, nil
@@ -1257,7 +1385,7 @@ func (st *HelmState) namespaceAndValuesFlags(helm helmexec.Interface, release *R
 	for _, v := range release.Values {
 		switch typedValue := v.(type) {
 		case string:
-			path := st.normalizePath(release.ValuesPathPrefix + typedValue)
+			path := st.storage().normalizePath(release.ValuesPathPrefix + typedValue)
 			values = append(values, path)
 		default:
 			values = append(values, v)
@@ -1276,26 +1404,19 @@ func (st *HelmState) namespaceAndValuesFlags(helm helmexec.Interface, release *R
 	release.generatedValues = append(release.generatedValues, generatedFiles...)
 
 	for _, value := range release.Secrets {
-		path := st.normalizePath(release.ValuesPathPrefix + value)
-		ok, err := st.fileExists(path)
+		paths, skip, err := st.storage().resolveFile(release.MissingFileHandler, "secrets", release.ValuesPathPrefix+value)
 		if err != nil {
 			return nil, err
 		}
-		if !ok {
-			if release.MissingFileHandler == nil || *release.MissingFileHandler == "Error" {
-				return nil, err
-			} else if *release.MissingFileHandler == "Warn" {
-				st.logger.Warnf("skipping missing secrets file \"%s\"", path)
-				continue
-			} else if *release.MissingFileHandler == "Info" {
-				st.logger.Infof("skipping missing secrets file \"%s\"", path)
-				continue
-			} else {
-				st.logger.Debugf("skipping missing secrets file \"%s\"", path)
-				continue
-			}
+		if skip {
+			continue
 		}
 
+		if len(paths) > 1 {
+			return nil, fmt.Errorf("glob patterns in release secret file is not supported yet. please submit a feature request if necessary")
+		}
+		path := paths[0]
+
 		decryptFlags := st.appendTillerFlags([]string{}, release)
 		valfile, err := helm.DecryptSecret(st.createHelmContext(release, workerIndex), path, decryptFlags...)
 		if err != nil {
@@ -1310,7 +1431,7 @@ func (st *HelmState) namespaceAndValuesFlags(helm helmexec.Interface, release *R
 			if set.Value != "" {
 				flags = append(flags, "--set", fmt.Sprintf("%s=%s", escape(set.Name), escape(set.Value)))
 			} else if set.File != "" {
-				flags = append(flags, "--set-file", fmt.Sprintf("%s=%s", escape(set.Name), st.normalizePath(set.File)))
+				flags = append(flags, "--set-file", fmt.Sprintf("%s=%s", escape(set.Name), st.storage().normalizePath(set.File)))
 			} else if len(set.Values) > 0 {
 				items := make([]string, len(set.Values))
 				for i, raw := range set.Values {
@@ -1387,3 +1508,43 @@ func escape(value string) string {
 	intermediate = strings.Replace(intermediate, "}", "\\}", -1)
 	return strings.Replace(intermediate, ",", "\\,", -1)
 }
+
+//UnmarshalYAML will unmarshal the helmfile yaml section and fill the SubHelmfileSpec structure
+//this is required to keep allowing string scalar for defining helmfile
+func (hf *SubHelmfileSpec) UnmarshalYAML(unmarshal func(interface{}) error) error {
+
+	var tmp interface{}
+	if err := unmarshal(&tmp); err != nil {
+		return err
+	}
+
+	switch i := tmp.(type) {
+	case string: // single path definition without sub items, legacy sub helmfile definition
+		hf.Path = i
+	case map[interface{}]interface{}: // helmfile path with sub section
+		var subHelmfileSpecTmp struct {
+			Path               string   `yaml:"path"`
+			Selectors          []string `yaml:"selectors"`
+			SelectorsInherited bool     `yaml:"selectorsInherited"`
+
+			Environment SubhelmfileEnvironmentSpec `yaml:",inline"`
+		}
+		if err := unmarshal(&subHelmfileSpecTmp); err != nil {
+			return err
+		}
+		hf.Path = subHelmfileSpecTmp.Path
+		hf.Selectors = subHelmfileSpecTmp.Selectors
+		hf.SelectorsInherited = subHelmfileSpecTmp.SelectorsInherited
+		hf.Environment = subHelmfileSpecTmp.Environment
+	}
+	//since we cannot make sur the "console" string can be red after the "path" we must check we don't have
+	//a SubHelmfileSpec with only selector and no path
+	if hf.Selectors != nil && hf.Path == "" {
+		return fmt.Errorf("found 'selectors' definition without path: %v", hf.Selectors)
+	}
+	//also exclude SelectorsInherited to true and explicit selectors
+	if hf.SelectorsInherited && len(hf.Selectors) > 0 {
+		return fmt.Errorf("You cannot use 'SelectorsInherited: true' along with and explicit selector for path: %v", hf.Path)
+	}
+	return nil
+}

pkg/state/state_exec_tmpl.go

@@ -0,0 +1,67 @@
+package state
+
+import (
+	"fmt"
+	"github.com/imdario/mergo"
+	"github.com/roboll/helmfile/pkg/maputil"
+	"github.com/roboll/helmfile/pkg/tmpl"
+)
+
+func (st *HelmState) Values() (map[string]interface{}, error) {
+	vals := map[string]interface{}{}
+
+	if err := mergo.Merge(&vals, st.Env.Defaults, mergo.WithOverride); err != nil {
+		return nil, err
+	}
+	if err := mergo.Merge(&vals, st.Env.Values, mergo.WithOverride); err != nil {
+		return nil, err
+	}
+
+	vals, err := maputil.CastKeysToStrings(vals)
+	if err != nil {
+		return nil, err
+	}
+
+	return vals, nil
+}
+
+func (st *HelmState) mustLoadVals() map[string]interface{} {
+	vals, err := st.Values()
+	if err != nil {
+		panic(err)
+	}
+	return vals
+}
+
+func (st *HelmState) valuesFileTemplateData() EnvironmentTemplateData {
+	return EnvironmentTemplateData{
+		Environment: st.Env,
+		Namespace:   st.Namespace,
+		Values:      st.mustLoadVals(),
+	}
+}
+
+func (st *HelmState) ExecuteTemplates() (*HelmState, error) {
+	r := *st
+
+	vals, err := st.Values()
+	if err != nil {
+		return nil, err
+	}
+
+	for i, rt := range st.Releases {
+		tmplData := releaseTemplateData{
+			Environment: st.Env,
+			Release:     rt,
+			Values:      vals,
+		}
+		renderer := tmpl.NewFileRenderer(st.readFile, st.basePath, tmplData)
+		r, err := rt.ExecuteTemplateExpressions(renderer)
+		if err != nil {
+			return nil, fmt.Errorf("failed executing templates in release \"%s\".\"%s\": %v", st.FilePath, rt.Name, err)
+		}
+		st.Releases[i] = *r
+	}
+
+	return &r, nil
+}

pkg/state/state_exec_tmpl_test.go

@@ -1,10 +1,9 @@
 package state
 
 import (
+	"github.com/roboll/helmfile/pkg/environment"
 	"reflect"
 	"testing"
-
-	"github.com/roboll/helmfile/environment"
 )
 
 func TestHelmState_executeTemplates(t *testing.T) {
@@ -17,7 +16,7 @@ func TestHelmState_executeTemplates(t *testing.T) {
 			name: "Has template expressions in chart, values, and secrets",
 			input: ReleaseSpec{
 				Chart:     "test-charts/{{ .Release.Name }}",
-				Version:   "0.1",
+				Version:   "{{ .Release.Name }}-0.1",
 				Verify:    nil,
 				Name:      "test-app",
 				Namespace: "test-namespace-{{ .Release.Name }}",
@@ -26,7 +25,7 @@ func TestHelmState_executeTemplates(t *testing.T) {
 			},
 			want: ReleaseSpec{
 				Chart:     "test-charts/test-app",
-				Version:   "0.1",
+				Version:   "test-app-0.1",
 				Verify:    nil,
 				Name:      "test-app",
 				Namespace: "test-namespace-test-app",
@@ -36,7 +35,8 @@ func TestHelmState_executeTemplates(t *testing.T) {
 		},
 	}
 
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			state := &HelmState{
 				basePath: ".",
@@ -71,6 +71,9 @@ func TestHelmState_executeTemplates(t *testing.T) {
 			if !reflect.DeepEqual(actual.Secrets, tt.want.Secrets) {
 				t.Errorf("expected %+v, got %+v", tt.want.Secrets, actual.Secrets)
 			}
+			if !reflect.DeepEqual(actual.Version, tt.want.Version) {
+				t.Errorf("expected %+v, got %+v", tt.want.Version, actual.Version)
+			}
 		})
 	}
 }

pkg/state/state_run.go

@@ -2,7 +2,7 @@ package state
 
 import (
 	"fmt"
-	"github.com/roboll/helmfile/helmexec"
+	"github.com/roboll/helmfile/pkg/helmexec"
 	"sync"
 )
 

pkg/state/state_test.go

@@ -1,19 +1,30 @@
 package state
 
 import (
+	"io/ioutil"
 	"os"
+	"path/filepath"
 	"reflect"
 	"testing"
 
+	"github.com/roboll/helmfile/pkg/helmexec"
+	"github.com/roboll/helmfile/pkg/testhelper"
+
 	"errors"
 	"strings"
 
 	"fmt"
-	"github.com/roboll/helmfile/helmexec"
 )
 
 var logger = helmexec.NewLogger(os.Stdout, "warn")
 
+func injectFs(st *HelmState, fs *testhelper.TestFs) *HelmState {
+	st.glob = fs.Glob
+	st.readFile = fs.ReadFile
+	st.fileExists = fs.FileExists
+	return st
+}
+
 func TestLabelParsing(t *testing.T) {
 	cases := []struct {
 		labelString    string
@@ -113,7 +124,8 @@ func TestHelmState_applyDefaultsTo(t *testing.T) {
 			want: specWithNamespaceFromFields,
 		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			state := &HelmState{
 				basePath:           tt.fields.BaseChartPath,
@@ -503,7 +515,8 @@ func TestHelmState_flagsForUpgrade(t *testing.T) {
 			},
 		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			state := &HelmState{
 				basePath:          "./",
@@ -551,7 +564,7 @@ func Test_isLocalChart(t *testing.T) {
 			args: args{
 				chart: "",
 			},
-			want: false,
+			want: true,
 		},
 		{
 			name: "parent local path",
@@ -567,11 +580,26 @@ func Test_isLocalChart(t *testing.T) {
 			},
 			want: true,
 		},
+		{
+			name: "absolute path",
+			args: args{
+				chart: "/foo/bar/baz",
+			},
+			want: true,
+		},
+		{
+			name: "local chart in 3-level deep dir",
+			args: args{
+				chart: "foo/bar/baz",
+			},
+			want: true,
+		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			if got := isLocalChart(tt.args.chart); got != tt.want {
-				t.Errorf("pathExists() = %v, want %v", got, tt.want)
+				t.Errorf("%s(\"%s\") isLocalChart(): got %v, want %v", tt.name, tt.args.chart, got, tt.want)
 			}
 		})
 	}
@@ -620,7 +648,8 @@ func Test_normalizeChart(t *testing.T) {
 			want: "/app",
 		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			if got := normalizeChart(tt.args.basePath, tt.args.chart); got != tt.want {
 				t.Errorf("normalizeChart() = %v, want %v", got, tt.want)
@@ -643,6 +672,8 @@ type mockHelmExec struct {
 	deleted  []mockRelease
 	lists    map[listKey]string
 	diffed   []mockRelease
+
+	updateDepsCallbacks map[string]func(string) error
 }
 
 type mockRelease struct {
@@ -658,9 +689,18 @@ type mockAffected struct {
 
 func (helm *mockHelmExec) UpdateDeps(chart string) error {
 	if strings.Contains(chart, "error") {
-		return errors.New("error")
+		return fmt.Errorf("simulated UpdateDeps failure for chart: %s", chart)
 	}
 	helm.charts = append(helm.charts, chart)
+
+	if helm.updateDepsCallbacks != nil {
+		callback, exists := helm.updateDepsCallbacks[chart]
+		if exists {
+			if err := callback(chart); err != nil {
+				return err
+			}
+		}
+	}
 	return nil
 }
 
@@ -787,7 +827,8 @@ func TestHelmState_SyncRepos(t *testing.T) {
 			want: []string{"name", "http://example.com/", "", "", "example_user", "example_password"},
 		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			for k, v := range tt.envs {
 				err := os.Setenv(k, v)
@@ -902,7 +943,8 @@ func TestHelmState_SyncReleases(t *testing.T) {
 			wantReleases: []mockRelease{{"releaseName", []string{"--set", "foo.bar[0]={A,B}"}}},
 		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			state := &HelmState{
 				Releases: tt.releases,
@@ -915,6 +957,118 @@ func TestHelmState_SyncReleases(t *testing.T) {
 	}
 }
 
+func TestHelmState_SyncReleases_MissingValuesFileForUndesiredRelease(t *testing.T) {
+	no := false
+	tests := []struct {
+		name          string
+		release       ReleaseSpec
+		listResult    string
+		expectedError string
+	}{
+		{
+			name: "should install",
+			release: ReleaseSpec{
+				Name:  "foo",
+				Chart: "../../foo-bar",
+			},
+			listResult:    ``,
+			expectedError: ``,
+		},
+		{
+			name: "should upgrade",
+			release: ReleaseSpec{
+				Name:  "foo",
+				Chart: "../../foo-bar",
+			},
+			listResult: `NAME 	REVISION	UPDATED                 	STATUS  	CHART                      	APP VERSION	NAMESPACE
+										foo	1       	Wed Apr 17 17:39:04 2019	DEPLOYED	foo-bar-2.0.4	0.1.0      	default`,
+			expectedError: ``,
+		},
+		{
+			name: "should uninstall",
+			release: ReleaseSpec{
+				Name:      "foo",
+				Chart:     "../../foo-bar",
+				Installed: &no,
+			},
+			listResult: `NAME 	REVISION	UPDATED                 	STATUS  	CHART                      	APP VERSION	NAMESPACE
+										foo	1       	Wed Apr 17 17:39:04 2019	DEPLOYED	foo-bar-2.0.4	0.1.0      	default`,
+			expectedError: ``,
+		},
+		{
+			name: "should fail installing due to missing values file",
+			release: ReleaseSpec{
+				Name:   "foo",
+				Chart:  "../../foo-bar",
+				Values: []interface{}{"noexistent.values.yaml"},
+			},
+			listResult:    ``,
+			expectedError: `failed processing release foo: values file matching "noexistent.values.yaml" does not exist in "."`,
+		},
+		{
+			name: "should fail upgrading due to missing values file",
+			release: ReleaseSpec{
+				Name:   "foo",
+				Chart:  "../../foo-bar",
+				Values: []interface{}{"noexistent.values.yaml"},
+			},
+			listResult: `NAME 	REVISION	UPDATED                 	STATUS  	CHART                      	APP VERSION	NAMESPACE
+										foo	1       	Wed Apr 17 17:39:04 2019	DEPLOYED	foo-bar-2.0.4	0.1.0      	default`,
+			expectedError: `failed processing release foo: values file matching "noexistent.values.yaml" does not exist in "."`,
+		},
+		{
+			name: "should uninstall even when there is a missing values file",
+			release: ReleaseSpec{
+				Name:      "foo",
+				Chart:     "../../foo-bar",
+				Values:    []interface{}{"noexistent.values.yaml"},
+				Installed: &no,
+			},
+			listResult: `NAME 	REVISION	UPDATED                 	STATUS  	CHART                      	APP VERSION	NAMESPACE
+										foo	1       	Wed Apr 17 17:39:04 2019	DEPLOYED	foo-bar-2.0.4	0.1.0      	default`,
+			expectedError: ``,
+		},
+	}
+	for i := range tests {
+		tt := tests[i]
+		t.Run(tt.name, func(t *testing.T) {
+			state := &HelmState{
+				basePath: ".",
+				Releases: []ReleaseSpec{tt.release},
+				logger:   logger,
+			}
+			fs := testhelper.NewTestFs(map[string]string{})
+			state = injectFs(state, fs)
+			helm := &mockHelmExec{
+				lists: map[listKey]string{},
+			}
+			//simulate the helm.list call result
+			helm.lists[listKey{filter: "^" + tt.release.Name + "$"}] = tt.listResult
+
+			affectedReleases := AffectedReleases{}
+			errs := state.SyncReleases(&affectedReleases, helm, []string{}, 1)
+
+			if tt.expectedError != "" {
+				if len(errs) == 0 {
+					t.Fatalf("expected error not occurred: expected=%s, got none", tt.expectedError)
+				}
+				if len(errs) != 1 {
+					t.Fatalf("too many errors: expected %d, got %d: %v", 1, len(errs), errs)
+				}
+				err := errs[0]
+				if err.Error() != tt.expectedError {
+					t.Fatalf("unexpected error: expected=%s, got=%v", tt.expectedError, err)
+				}
+			} else {
+				if len(errs) > 0 {
+					t.Fatalf("unexpected error(s): expected=0, got=%d: %v", len(errs), errs)
+				}
+			}
+
+		})
+	}
+}
+
 func TestHelmState_SyncReleasesAffectedRealeases(t *testing.T) {
 	no := false
 	tests := []struct {
@@ -989,7 +1143,8 @@ func TestHelmState_SyncReleasesAffectedRealeases(t *testing.T) {
 			wantAffected: mockAffected{[]*mockRelease{{"releaseNameFoo", []string{}}}, []*mockRelease{{"releaseNameBar", []string{}}}, []*mockRelease{{"releaseNameFoo-error", []string{}}}},
 		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			state := &HelmState{
 				Releases: tt.releases,
@@ -1078,8 +1233,19 @@ func TestGetDeployedVersion(t *testing.T) {
 										foo	1       	Wed Apr 17 17:39:04 2019	DEPLOYED	foo-bar-1.0.0-alpha+001	0.1.0      	default`,
 			installedVersion: "1.0.0-alpha+001",
 		},
+		{
+			name: "chart version with dash and release with dash",
+			release: ReleaseSpec{
+				Name:  "foo-bar",
+				Chart: "registry/foo-bar",
+			},
+			listResult: `NAME 	REVISION	UPDATED                 	STATUS  	CHART                      	APP VERSION	NAMESPACE
+										foo-bar-release	1       	Wed Apr 17 17:39:04 2019	DEPLOYED	foo-bar-1.0.0-alpha+001	0.1.0      	default`,
+			installedVersion: "1.0.0-alpha+001",
+		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			state := &HelmState{
 				Releases: []ReleaseSpec{tt.release},
@@ -1198,7 +1364,8 @@ func TestHelmState_DiffReleases(t *testing.T) {
 			wantReleases: []mockRelease{{"releaseName", []string{"--set", "foo.bar[0]={A,B}"}}},
 		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			state := &HelmState{
 				Releases: tt.releases,
@@ -1267,28 +1434,22 @@ func TestHelmState_SyncReleasesCleanup(t *testing.T) {
 			expectedNumRemovedFiles: 2,
 		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			numRemovedFiles := 0
 			state := &HelmState{
 				Releases: tt.releases,
 				logger:   logger,
-				readFile: func(f string) ([]byte, error) {
-					if f != "someFile" {
-						return nil, fmt.Errorf("unexpected file to read: %s", f)
-					}
-					someFileContent := []byte(`foo: bar
-`)
-					return someFileContent, nil
-				},
 				removeFile: func(f string) error {
 					numRemovedFiles += 1
 					return nil
 				},
-				fileExists: func(f string) (bool, error) {
-					return true, nil
-				},
 			}
+			testfs := testhelper.NewTestFs(map[string]string{
+				"/path/to/someFile": `foo: FOO`,
+			})
+			state = injectFs(state, testfs)
 			if errs := state.SyncReleases(&AffectedReleases{}, tt.helm, []string{}, 1); errs != nil && len(errs) > 0 {
 				t.Errorf("unexpected errors: %v", errs)
 			}
@@ -1356,28 +1517,23 @@ func TestHelmState_DiffReleasesCleanup(t *testing.T) {
 			expectedNumRemovedFiles: 2,
 		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			numRemovedFiles := 0
 			state := &HelmState{
 				Releases: tt.releases,
 				logger:   logger,
-				readFile: func(f string) ([]byte, error) {
-					if f != "someFile" {
-						return nil, fmt.Errorf("unexpected file to read: %s", f)
-					}
-					someFileContent := []byte(`foo: bar
-`)
-					return someFileContent, nil
-				},
 				removeFile: func(f string) error {
 					numRemovedFiles += 1
 					return nil
 				},
-				fileExists: func(f string) (bool, error) {
-					return true, nil
-				},
 			}
+			testfs := testhelper.NewTestFs(map[string]string{
+				"/path/to/someFile": `foo: bar
+`,
+			})
+			state = injectFs(state, testfs)
 			if _, errs := state.DiffReleases(tt.helm, []string{}, 1, false, false, false); errs != nil && len(errs) > 0 {
 				t.Errorf("unexpected errors: %v", errs)
 			}
@@ -1394,8 +1550,39 @@ func TestHelmState_DiffReleasesCleanup(t *testing.T) {
 }
 
 func TestHelmState_UpdateDeps(t *testing.T) {
+	helm := &mockHelmExec{
+		updateDepsCallbacks: map[string]func(string) error{},
+	}
+
+	var generatedDir string
+	tempDir := func(dir, prefix string) (string, error) {
+		var err error
+		generatedDir, err = ioutil.TempDir(dir, prefix)
+		if err != nil {
+			return "", err
+		}
+		helm.updateDepsCallbacks[generatedDir] = func(chart string) error {
+			content := []byte(`dependencies:
+- name: envoy
+  repository: https://kubernetes-charts.storage.googleapis.com
+  version: 1.5.0
+- name: envoy
+  repository: https://kubernetes-charts.storage.googleapis.com
+  version: 1.4.0
+digest: sha256:8194b597c85bb3d1fee8476d4a486e952681d5c65f185ad5809f2118bc4079b5
+generated: 2019-05-16T15:42:45.50486+09:00
+`)
+			filename := filepath.Join(generatedDir, "requirements.lock")
+			logger.Debugf("test: writing %s: %s", filename, content)
+			return ioutil.WriteFile(filename, content, 0644)
+		}
+		return generatedDir, nil
+	}
+
+	logger := helmexec.NewLogger(os.Stderr, "debug")
 	state := &HelmState{
 		basePath: "/src",
+		FilePath: "/src/helmfile.yaml",
 		Releases: []ReleaseSpec{
 			{
 				Chart: "./..",
@@ -1413,19 +1600,89 @@ func TestHelmState_UpdateDeps(t *testing.T) {
 				Chart: "published/deeper",
 			},
 			{
-				Chart: ".error",
+				Chart:   "stable/envoy",
+				Version: "1.5.0",
+			},
+			{
+				Chart:   "stable/envoy",
+				Version: "1.4.0",
 			},
 		},
+		Repositories: []RepositorySpec{
+			{
+				Name: "stable",
+				URL:  "https://kubernetes-charts.storage.googleapis.com",
+			},
+		},
+		tempDir: tempDir,
+		logger:  logger,
 	}
 
-	want := []string{"/", "/examples", "/helmfile"}
-	helm := &mockHelmExec{}
 	errs := state.UpdateDeps(helm)
+	want := []string{"/", "/examples", "/helmfile", "/src/published", generatedDir}
 	if !reflect.DeepEqual(helm.charts, want) {
 		t.Errorf("HelmState.UpdateDeps() = %v, want %v", helm.charts, want)
 	}
 	if len(errs) != 0 {
-		t.Errorf("HelmState.UpdateDeps() - no errors, but got: %v", len(errs))
+		t.Errorf("HelmState.UpdateDeps() - no errors, but got %d: %v", len(errs), errs)
+	}
+
+	resolved, err := state.ResolveDeps()
+	if err != nil {
+		t.Errorf("unexpected error: %v", err)
+	}
+
+	if resolved.Releases[5].Version != "1.5.0" {
+		t.Errorf("unexpected version number: expected=1.5.0, got=%s", resolved.Releases[5].Version)
+	}
+	if resolved.Releases[6].Version != "1.4.0" {
+		t.Errorf("unexpected version number: expected=1.4.0, got=%s", resolved.Releases[6].Version)
+	}
+}
+
+func TestHelmState_ResolveDeps_NoLockFile(t *testing.T) {
+	logger := helmexec.NewLogger(os.Stderr, "debug")
+	state := &HelmState{
+		basePath: "/src",
+		FilePath: "/src/helmfile.yaml",
+		Releases: []ReleaseSpec{
+			{
+				Chart: "./..",
+			},
+			{
+				Chart: "../examples",
+			},
+			{
+				Chart: "../../helmfile",
+			},
+			{
+				Chart: "published",
+			},
+			{
+				Chart: "published/deeper",
+			},
+			{
+				Chart: "stable/envoy",
+			},
+		},
+		Repositories: []RepositorySpec{
+			{
+				Name: "stable",
+				URL:  "https://kubernetes-charts.storage.googleapis.com",
+			},
+		},
+		logger: logger,
+		readFile: func(f string) ([]byte, error) {
+			if f != "helmfile.lock" {
+				return nil, fmt.Errorf("stub: unexpected file: %s", f)
+			}
+			return nil, os.ErrNotExist
+		},
+	}
+
+	_, err := state.ResolveDeps()
+	if err != nil {
+		t.Errorf("unexpected error: %v", err)
 	}
 }
 
@@ -1496,8 +1753,9 @@ func TestHelmState_ReleaseStatuses(t *testing.T) {
 			want: []mockRelease{{"releaseA", []string{"--tiller-namespace", "tillerns"}}},
 		},
 	}
-	for _, tt := range tests {
-		i := func(t *testing.T) {
+	for i := range tests {
+		tt := tests[i]
+		f := func(t *testing.T) {
 			state := &HelmState{
 				Releases: tt.releases,
 				logger:   logger,
@@ -1523,7 +1781,7 @@ func TestHelmState_ReleaseStatuses(t *testing.T) {
 				t.Errorf("HelmState.ReleaseStatuses() for [%s] = %v, want %v", tt.name, tt.helm.releases, tt.want)
 			}
 		}
-		t.Run(tt.name, i)
+		t.Run(tt.name, f)
 	}
 }
 
@@ -1580,8 +1838,9 @@ func TestHelmState_TestReleasesNoCleanUp(t *testing.T) {
 			want: []mockRelease{{"releaseA", []string{"--timeout", "1", "--tiller-namespace", "tillerns"}}},
 		},
 	}
-	for _, tt := range tests {
-		i := func(t *testing.T) {
+	for i := range tests {
+		tt := tests[i]
+		f := func(t *testing.T) {
 			state := &HelmState{
 				Releases: tt.releases,
 				logger:   logger,
@@ -1595,7 +1854,7 @@ func TestHelmState_TestReleasesNoCleanUp(t *testing.T) {
 				t.Errorf("HelmState.TestReleases() for [%s] = %v, want %v", tt.name, tt.helm.releases, tt.want)
 			}
 		}
-		t.Run(tt.name, i)
+		t.Run(tt.name, f)
 	}
 }
 
@@ -1630,19 +1889,21 @@ func TestHelmState_NoReleaseMatched(t *testing.T) {
 			wantErr: false,
 		},
 	}
-	for _, tt := range tests {
-		i := func(t *testing.T) {
+	for i := range tests {
+		tt := tests[i]
+		f := func(t *testing.T) {
 			state := &HelmState{
 				Releases: releases,
 				logger:   logger,
 			}
-			errs := state.FilterReleases([]string{tt.labels})
+			state.Selectors = []string{tt.labels}
+			errs := state.FilterReleases()
 			if (errs != nil) != tt.wantErr {
 				t.Errorf("ReleaseStatuses() for %s error = %v, wantErr %v", tt.name, errs, tt.wantErr)
 				return
 			}
 		}
-		t.Run(tt.name, i)
+		t.Run(tt.name, f)
 	}
 }
 
@@ -1748,8 +2009,9 @@ func TestHelmState_Delete(t *testing.T) {
 			deleted:         []mockRelease{{"releaseA", []string{"--purge", "--tiller-namespace", "tillerns"}}},
 		},
 	}
-	for _, tt := range tests {
-		i := func(t *testing.T) {
+	for i := range tests {
+		tt := tests[i]
+		f := func(t *testing.T) {
 			name := "releaseA"
 			if tt.wantErr {
 				name = "releaseA-error"
@@ -1784,6 +2046,6 @@ func TestHelmState_Delete(t *testing.T) {
 				t.Errorf("unexpected deletions happened: expected %v, got %v", &affectedReleases.Deleted, tt.deleted)
 			}
 		}
-		t.Run(tt.name, i)
+		t.Run(tt.name, f)
 	}
 }

pkg/state/storage.go

@@ -0,0 +1,99 @@
+package state
+
+import (
+	"fmt"
+	"go.uber.org/zap"
+	"net/url"
+	"path/filepath"
+	"sort"
+)
+
+type Storage struct {
+	logger *zap.SugaredLogger
+
+	FilePath string
+
+	basePath string
+	glob     func(string) ([]string, error)
+}
+
+func NewStorage(forFile string, logger *zap.SugaredLogger, glob func(string) ([]string, error)) *Storage {
+	return &Storage{
+		FilePath: forFile,
+		basePath: filepath.Dir(forFile),
+		logger:   logger,
+		glob:     glob,
+	}
+}
+
+func (st *Storage) resolveFile(missingFileHandler *string, tpe, path string) ([]string, bool, error) {
+	title := fmt.Sprintf("%s file", tpe)
+
+	files, err := st.ExpandPaths(path)
+	if err != nil {
+		return nil, false, err
+	}
+
+	var handlerId string
+
+	if missingFileHandler != nil {
+		handlerId = *missingFileHandler
+	} else {
+		handlerId = MissingFileHandlerError
+	}
+
+	if len(files) == 0 {
+		switch handlerId {
+		case MissingFileHandlerError:
+			return nil, false, fmt.Errorf("%s matching \"%s\" does not exist in \"%s\"", title, path, st.basePath)
+		case MissingFileHandlerWarn:
+			st.logger.Warnf("skipping missing %s matching \"%s\"", title, path)
+			return nil, true, nil
+		case MissingFileHandlerInfo:
+			st.logger.Infof("skipping missing %s matching \"%s\"", title, path)
+			return nil, true, nil
+		case MissingFileHandlerDebug:
+			st.logger.Debugf("skipping missing %s matching \"%s\"", title, path)
+			return nil, true, nil
+		default:
+			available := []string{
+				MissingFileHandlerError,
+				MissingFileHandlerWarn,
+				MissingFileHandlerInfo,
+				MissingFileHandlerDebug,
+			}
+			return nil, false, fmt.Errorf("invalid missing file handler \"%s\" while processing \"%s\" in \"%s\": it must be one of %s", handlerId, path, st.FilePath, available)
+		}
+	}
+
+	return files, false, nil
+}
+
+func (st *Storage) ExpandPaths(globPattern string) ([]string, error) {
+	result := []string{}
+	absPathPattern := st.normalizePath(globPattern)
+	matches, err := st.glob(absPathPattern)
+	if err != nil {
+		return nil, fmt.Errorf("failed processing %s: %v", globPattern, err)
+	}
+
+	sort.Strings(matches)
+
+	result = append(result, matches...)
+	return result, nil
+}
+
+// normalizes relative path to absolute one
+func (st *Storage) normalizePath(path string) string {
+	u, _ := url.Parse(path)
+	if u.Scheme != "" || filepath.IsAbs(path) {
+		return path
+	} else {
+		return st.JoinBase(path)
+	}
+}
+
+// JoinBase returns an absolute path in the form basePath/relative
+func (st *Storage) JoinBase(relPath string) string {
+	return filepath.Join(st.basePath, relPath)
+}

pkg/state/types.go

@@ -1,6 +1,8 @@
 package state
 
-import "github.com/roboll/helmfile/environment"
+import (
+	"github.com/roboll/helmfile/pkg/environment"
+)
 
 // TemplateSpec defines the structure of a reusable and composable template for helm releases.
 type TemplateSpec struct {
@@ -13,12 +15,16 @@ type EnvironmentTemplateData struct {
 	Environment environment.Environment
 	// Namespace is accessible as `.Namespace` from any non-values template executed by the renderer
 	Namespace string
+	// Values is accessible as `.Values` and it contains default state values overrode by environment values and override values.
+	Values map[string]interface{}
 }
 
-// ReleaseTemplateData provides variables accessible while executing golang text/template expressions in releases of a helmfile YAML file
-type ReleaseTemplateData struct {
+// releaseTemplateData provides variables accessible while executing golang text/template expressions in releases of a helmfile YAML file
+type releaseTemplateData struct {
 	// Environment is accessible as `.Environment` from any template expression executed by the renderer
 	Environment environment.Environment
 	// Release is accessible as `.Release` from any template expression executed by the renderer
 	Release ReleaseSpec
+	// Values is accessible as `.Values` and it contains default state values overrode by environment values and override values.
+	Values map[string]interface{}
 }

pkg/testhelper/testfs.go

@@ -0,0 +1,137 @@
+package testhelper
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+type TestFs struct {
+	Cwd   string
+	dirs  map[string]bool
+	files map[string]string
+
+	GlobFixtures map[string][]string
+
+	fileReaderCalls int
+	successfulReads []string
+}
+
+func NewTestFs(files map[string]string) *TestFs {
+	dirs := map[string]bool{}
+	for abs, _ := range files {
+		for d := filepath.Dir(abs); !dirs[d]; d = filepath.Dir(d) {
+			dirs[d] = true
+			fmt.Fprintf(os.Stderr, "testfs: recognized dir: %s\n", d)
+		}
+	}
+	return &TestFs{
+		Cwd:   "/path/to",
+		dirs:  dirs,
+		files: files,
+
+		successfulReads: []string{},
+
+		GlobFixtures: map[string][]string{},
+	}
+}
+
+func (f *TestFs) FileExistsAt(path string) bool {
+	var ok bool
+	if strings.Contains(path, "/") {
+		_, ok = f.files[path]
+	} else {
+		_, ok = f.files[filepath.Join(f.Cwd, path)]
+	}
+	return ok
+}
+
+func (f *TestFs) FileExists(path string) (bool, error) {
+	return f.FileExistsAt(path), nil
+}
+
+func (f *TestFs) DirectoryExistsAt(path string) bool {
+	var ok bool
+	if strings.Contains(path, "/") {
+		_, ok = f.dirs[path]
+	} else {
+		_, ok = f.dirs[filepath.Join(f.Cwd, path)]
+	}
+	return ok
+}
+
+func (f *TestFs) ReadFile(filename string) ([]byte, error) {
+	var str string
+	var ok bool
+	if filename[0] == '/' {
+		str, ok = f.files[filename]
+	} else {
+		str, ok = f.files[filepath.Join(f.Cwd, filename)]
+	}
+	if !ok {
+		return []byte(nil), fmt.Errorf("no registered file found: %s", filename)
+	}
+
+	f.fileReaderCalls += 1
+
+	f.successfulReads = append(f.successfulReads, filename)
+
+	return []byte(str), nil
+}
+
+func (f *TestFs) SuccessfulReads() []string {
+	return f.successfulReads
+}
+
+func (f *TestFs) FileReaderCalls() int {
+	return f.fileReaderCalls
+}
+
+func (f *TestFs) Glob(relPattern string) ([]string, error) {
+	var pattern string
+	if relPattern[0] == '/' {
+		pattern = relPattern
+	} else {
+		pattern = filepath.Join(f.Cwd, relPattern)
+	}
+
+	fixtures, ok := f.GlobFixtures[pattern]
+	if ok {
+		return fixtures, nil
+	}
+
+	matches := []string{}
+	for name, _ := range f.files {
+		matched, err := filepath.Match(pattern, name)
+		if err != nil {
+			return nil, err
+		}
+		if matched {
+			matches = append(matches, name)
+		}
+	}
+	return matches, nil
+}
+
+func (f *TestFs) Abs(path string) (string, error) {
+	var p string
+	if path[0] == '/' {
+		p = path
+	} else {
+		p = filepath.Join(f.Cwd, path)
+	}
+	return filepath.Clean(p), nil
+}
+
+func (f *TestFs) Getwd() (string, error) {
+	return f.Cwd, nil
+}
+
+func (f *TestFs) Chdir(dir string) error {
+	if _, ok := f.dirs[dir]; ok {
+		f.Cwd = dir
+		return nil
+	}
+	return fmt.Errorf("unexpected chdir \"%s\"", dir)
+}

pkg/tmpl/context_tmpl.go

@@ -13,9 +13,9 @@ func (c *Context) stringTemplate() *template.Template {
 	}
 	tmpl := template.New("stringTemplate").Funcs(funcMap)
 	if c.preRender {
-		tmpl.Option("missingkey=zero")
+		tmpl = tmpl.Option("missingkey=zero")
 	} else {
-		tmpl.Option("missingkey=error")
+		tmpl = tmpl.Option("missingkey=error")
 	}
 	return tmpl
 }

pkg/tmpl/context_tmpl_test.go

@@ -230,7 +230,8 @@ func Test_renderTemplateToString(t *testing.T) {
 			wantErr: true,
 		},
 	}
-	for _, tt := range tests {
+	for i := range tests {
+		tt := tests[i]
 		t.Run(tt.name, func(t *testing.T) {
 			for k, v := range tt.args.envs {
 				err := os.Setenv(k, v)

pkg/tmpl/file_renderer.go

@@ -46,6 +46,8 @@ func (r *FileRenderer) RenderTemplateFileToBuffer(file string) (*bytes.Buffer, e
 	return r.RenderTemplateContentToBuffer(content)
 }
 
+// RenderToBytes loads the content of the file.
+// If its extension is `gotmpl` it treats the content as a go template and renders it.
 func (r *FileRenderer) RenderToBytes(path string) ([]byte, error) {
 	var yamlBytes []byte
 	splits := strings.Split(path, ".")

pkg/tmpl/file_renderer_test.go

@@ -2,7 +2,7 @@ package tmpl
 
 import (
 	"fmt"
-	"github.com/roboll/helmfile/environment"
+	"github.com/roboll/helmfile/pkg/environment"
 	"reflect"
 	"testing"
 )

state/create.go

@@ -1,194 +0,0 @@
-package state
-
-import (
-	"bytes"
-	"fmt"
-	"io"
-	"io/ioutil"
-	"os"
-	"path/filepath"
-
-	"github.com/imdario/mergo"
-	"github.com/roboll/helmfile/environment"
-	"github.com/roboll/helmfile/helmexec"
-	"github.com/roboll/helmfile/tmpl"
-	"go.uber.org/zap"
-	"gopkg.in/yaml.v2"
-)
-
-type StateLoadError struct {
-	msg   string
-	Cause error
-}
-
-func (e *StateLoadError) Error() string {
-	return fmt.Sprintf("%s: %v", e.msg, e.Cause)
-}
-
-type UndefinedEnvError struct {
-	msg string
-}
-
-func (e *UndefinedEnvError) Error() string {
-	return e.msg
-}
-
-func createFromYaml(content []byte, file string, env string, logger *zap.SugaredLogger) (*HelmState, error) {
-	c := &creator{
-		logger,
-		ioutil.ReadFile,
-		filepath.Abs,
-		true,
-	}
-	return c.CreateFromYaml(content, file, env)
-}
-
-type creator struct {
-	logger   *zap.SugaredLogger
-	readFile func(string) ([]byte, error)
-	abs      func(string) (string, error)
-
-	Strict bool
-}
-
-func NewCreator(logger *zap.SugaredLogger, readFile func(string) ([]byte, error), abs func(string) (string, error)) *creator {
-	return &creator{
-		logger:   logger,
-		readFile: readFile,
-		abs:      abs,
-		Strict:   true,
-	}
-}
-
-func (c *creator) CreateFromYaml(content []byte, file string, env string) (*HelmState, error) {
-	var state HelmState
-
-	basePath, err := c.abs(filepath.Dir(file))
-	if err != nil {
-		return nil, &StateLoadError{fmt.Sprintf("failed to read %s", file), err}
-	}
-	state.FilePath = file
-	state.basePath = basePath
-
-	decoder := yaml.NewDecoder(bytes.NewReader(content))
-	if !c.Strict {
-		decoder.SetStrict(false)
-	} else {
-		decoder.SetStrict(true)
-	}
-	i := 0
-	for {
-		i++
-
-		var intermediate HelmState
-
-		err := decoder.Decode(&intermediate)
-		if err == io.EOF {
-			break
-		} else if err != nil {
-			return nil, &StateLoadError{fmt.Sprintf("failed to read %s: reading document at index %d", file, i), err}
-		}
-
-		if err := mergo.Merge(&state, &intermediate, mergo.WithAppendSlice); err != nil {
-			return nil, &StateLoadError{fmt.Sprintf("failed to read %s: merging document at index %d", file, i), err}
-		}
-	}
-
-	if len(state.DeprecatedReleases) > 0 {
-		if len(state.Releases) > 0 {
-			return nil, fmt.Errorf("failed to parse %s: you can't specify both `charts` and `releases` sections", file)
-		}
-		state.Releases = state.DeprecatedReleases
-		state.DeprecatedReleases = []ReleaseSpec{}
-	}
-
-	if state.DeprecatedContext != "" && state.HelmDefaults.KubeContext == "" {
-		state.HelmDefaults.KubeContext = state.DeprecatedContext
-	}
-
-	state.logger = c.logger
-
-	e, err := state.loadEnv(env, c.readFile)
-	if err != nil {
-		return nil, &StateLoadError{fmt.Sprintf("failed to read %s", file), err}
-	}
-	state.Env = *e
-
-	state.readFile = c.readFile
-	state.removeFile = os.Remove
-	state.fileExists = func(path string) (bool, error) {
-		_, err := os.Stat(path)
-
-		if err != nil {
-			if os.IsNotExist(err) {
-				return false, nil
-			}
-			return false, err
-		}
-		return true, nil
-	}
-
-	return &state, nil
-}
-
-func (st *HelmState) loadEnv(name string, readFile func(string) ([]byte, error)) (*environment.Environment, error) {
-	envVals := map[string]interface{}{}
-	envSpec, ok := st.Environments[name]
-	if ok {
-		for _, envvalFile := range envSpec.Values {
-			envvalFullPath := filepath.Join(st.basePath, envvalFile)
-			tmplData := EnvironmentTemplateData{Environment: environment.EmptyEnvironment, Namespace: ""}
-			r := tmpl.NewFileRenderer(readFile, filepath.Dir(envvalFullPath), tmplData)
-			bytes, err := r.RenderToBytes(envvalFullPath)
-			if err != nil {
-				return nil, fmt.Errorf("failed to load environment values file \"%s\": %v", envvalFile, err)
-			}
-			m := map[string]interface{}{}
-			if err := yaml.Unmarshal(bytes, &m); err != nil {
-				return nil, fmt.Errorf("failed to load environment values file \"%s\": %v", envvalFile, err)
-			}
-			if err := mergo.Merge(&envVals, &m, mergo.WithOverride); err != nil {
-				return nil, fmt.Errorf("failed to load \"%s\": %v", envvalFile, err)
-			}
-		}
-
-		if len(envSpec.Secrets) > 0 {
-			helm := helmexec.New(st.logger, "")
-			for _, secFile := range envSpec.Secrets {
-				path := filepath.Join(st.basePath, secFile)
-				if _, err := os.Stat(path); os.IsNotExist(err) {
-					return nil, err
-				}
-				// Work-around to allow decrypting environment secrets
-				//
-				// We don't have releases loaded yet and therefore unable to decide whether
-				// helmfile should use helm-tiller to call helm-secrets or not.
-				//
-				// This means that, when you use environment secrets + tillerless setup, you still need a tiller
-				// installed on the cluster, just for decrypting secrets!
-				// Related: https://github.com/futuresimple/helm-secrets/issues/83
-				release := &ReleaseSpec{}
-				flags := st.appendTillerFlags([]string{}, release)
-				decFile, err := helm.DecryptSecret(st.createHelmContext(release, 0), path, flags...)
-				if err != nil {
-					return nil, err
-				}
-				bytes, err := readFile(decFile)
-				if err != nil {
-					return nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", secFile, err)
-				}
-				m := map[string]interface{}{}
-				if err := yaml.Unmarshal(bytes, &m); err != nil {
-					return nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", secFile, err)
-				}
-				if err := mergo.Merge(&envVals, &m, mergo.WithOverride); err != nil {
-					return nil, fmt.Errorf("failed to load \"%s\": %v", secFile, err)
-				}
-			}
-		}
-	} else if name != DefaultEnv {
-		return nil, &UndefinedEnvError{msg: fmt.Sprintf("environment \"%s\" is not defined", name)}
-	}
-
-	return &environment.Environment{Name: name, Values: envVals}, nil
-}

state/environment.go

@@ -1,6 +0,0 @@
-package state
-
-type EnvironmentSpec struct {
-	Values  []string `yaml:"values"`
-	Secrets []string `yaml:"secrets"`
-}

state/state_exec_tmpl.go

@@ -1,32 +0,0 @@
-package state
-
-import (
-	"fmt"
-	"github.com/roboll/helmfile/tmpl"
-)
-
-func (st *HelmState) envTemplateData() EnvironmentTemplateData {
-	return EnvironmentTemplateData{
-		st.Env,
-		st.Namespace,
-	}
-}
-
-func (st *HelmState) ExecuteTemplates() (*HelmState, error) {
-	r := *st
-
-	for i, rt := range st.Releases {
-		tmplData := ReleaseTemplateData{
-			Environment: st.Env,
-			Release:     rt,
-		}
-		renderer := tmpl.NewFileRenderer(st.readFile, st.basePath, tmplData)
-		r, err := rt.ExecuteTemplateExpressions(renderer)
-		if err != nil {
-			return nil, fmt.Errorf("failed executing templates in release \"%s\".\"%s\": %v", st.FilePath, rt.Name, err)
-		}
-		st.Releases[i] = *r
-	}
-
-	return &r, nil
-}

test/integration/environment.values.yaml

@@ -0,0 +1 @@
+mysecret: MYSECRET

test/integration/happypath.yaml

@@ -1,6 +1,13 @@
+environments:
+  default:
+    values:
+      - environment.values.yaml
+---
 repositories:
   - name: stable
     url: https://kubernetes-charts.storage.googleapis.com/
+  - name: incubator
+    url: https://kubernetes-charts-incubator.storage.googleapis.com
 
 helmDefaults:
   kubeContext: minikube
@@ -12,3 +19,9 @@ releases:
     set:
       - name: ingress.enabled
         value: false
+
+  - name: raw
+    chart: incubator/raw
+    values:
+      - mysecret: {{ .Environment.Values.mysecret }}
+      - values.yaml

test/integration/run.sh

@@ -76,6 +76,16 @@ ${helmfile} -f ${dir}/happypath.yaml apply
 code=$?
 [ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile apply: ${code}"
 
+info "Locking dependencies"
+${helmfile} -f ${dir}/happypath.yaml deps
+code=$?
+[ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile deps: ${code}"
+
+info "Applying ${dir}/happypath.yaml with locked dependencies"
+${helmfile} -f ${dir}/happypath.yaml apply
+code=$?
+[ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile apply: ${code}"
+
 info "Deleting release"
 ${helmfile} -f ${dir}/happypath.yaml delete
 ${helm} status --namespace=${test_ns} httpbin &> /dev/null && fail "release should not exist anymore after a delete"

test/integration/values.yaml

@@ -0,0 +1,9 @@
+resources: []
+templates:
+  - |
+    apiVersion: v1
+    kind: Secret
+    metadata:
+      name: common-secret
+    stringData:
+      mykey: {{ .Values.mysecret }}