Commit Graph

69 Commits

Author SHA1 Message Date
Fedor Korotkov dc3eeef5b1
Support URLs in hostDir policies (#146)
* Support URLs in hostDir policies

We can't just blindly allow remote URLs since they might contain symlinks leading to outside the archive. Instead, let's support specifying URLs where the remote archive can come from.

Fixes #145

* Ignore Lint issue

* Reverted old validation logic
2023-10-24 21:01:34 +04:00
dependabot[bot] 7c2c466d65
Bump golang.org/x/net from 0.7.0 to 0.17.0 (#143)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.7.0 to 0.17.0.
- [Commits](https://github.com/golang/net/compare/v0.7.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-16 11:55:24 +04:00
Nikolay Edigaryev fe8cc21cba
HostDir: support URLs in path (#142) 2023-10-11 18:37:24 +04:00
Nikolay Edigaryev 13b4e192f0
Introduce "orchard {port-forward, vnc} worker WORKER_NAME" (#140)
* Fix potential NPE in Client.wsRequest()

* Introduce "orchard {port-forward, vnc} worker WORKER_NAME"

* portspec.go: simplify logic and respect [LOCAL_PORT]:REMOTE_PORT format
2023-10-09 18:51:34 +04:00
Nikolay Edigaryev 063405672f
Worker: better cleanup mechanisms (#139)
* Always Close() the Worker instance

* orchard list vms: show assigned worker for each of the VMs

* Stop the failed VMs before we schedule new VMs

To avoid violating resource constraints.

* syncOnDiskVMs: don't ignore running VMs

* Worker: show correct remote and local VM counts
2023-09-28 19:09:43 +00:00
Nikolay Edigaryev 40f58e4aee
More RPC-related logs (#136)
* More RPC-related logs

* Notifier should be set before we use it in the scheduler
2023-09-27 20:16:00 +04:00
Nikolay Edigaryev 64987f6d59
Send http.Server errors to zap at debug level and provide --debug flag (#135) 2023-09-25 18:17:23 +04:00
Nikolay Edigaryev 8c62df0eba
Only allow simple names when creating workers, VMs, etc. and escape paths in API client (#129)
* Controller: only allow simple names when creating workers, VMs, etc.

* Client: escape paths

* simplename: allow ':' character
2023-09-22 14:51:43 -04:00
Nikolay Edigaryev 2adb17c3c4
Introduce "orchard get worker" command (#127) 2023-09-13 23:50:58 +04:00
Nikolay Edigaryev 3d0e375ede
Don't stop and delete VMs that failed to clone (#125)
* NewVM() never returns an error

* Don't stop and delete VMs that failed to clone
2023-09-13 19:39:10 +04:00
Nikolay Edigaryev bb3d6edcd5
Fix Tart VM IP detection in bridged mode (#124) 2023-09-12 08:52:21 +00:00
Nikolay Edigaryev 036eb954be
Retry DB transactions on badger.ErrConflict (#114)
* Log HTTP 500 errors in more detail

* Log errors in storeView and storeUpdate

* Retry on badger.ErrConflict
2023-08-15 15:18:47 +04:00
Tim Peeters 9f4cd5bae7
Fix typo in documentation of the new image-pull-policy option. (#113) 2023-08-07 12:48:57 -04:00
Nikolay Edigaryev 6759618f28
orchard create vm: support --image-pull-policy=Always (#110) 2023-07-26 17:43:14 +04:00
Nikolay Edigaryev fd88ce5890
Introduce ORCHARD_LICENSE_TIER environment variable (#111)
* Introduce ORCHARD_LICENSE_TIER environment variable

* Only parse ORCHARD_LICENSE_TIER if it was provided
2023-07-26 17:28:38 +04:00
Nikolay Edigaryev a52c205c34
API(port forward endpoint): handle normal WebSocket closure gracefully (#108) 2023-07-20 20:55:42 +00:00
Nikolay Edigaryev c5c729c96f
Document --no-pki (#107)
* Document --no-pki

* and re-verify → and will re-verify

* against → only against

* Less "we"'s
2023-07-10 19:13:10 +04:00
Nikolay Edigaryev 0ba7db1a70
orchard list vms: add "Created" row (#106) 2023-07-04 18:11:26 +04:00
Nikolay Edigaryev d57d18d380
Support for sharing files with the host system (#103)
* Support for sharing files with the host system

* Integration tests

* Added back TestVMGarbageCollection comment
2023-07-04 18:10:53 +04:00
Nikolay Edigaryev 6a325daf74
Switch from golang.org/x/net/websocket to nhooyr.io/websocket and handle NotFound errors (#105)
* Switch from golang.org/x/net/websocket to nhooyr.io/websocket

* Do not attach errors that we can handle to the Gin's context

* Add missing newline to "no credentials specified or found, ..." message

* Fix potential NPE in ChooseUsernameAndPassword()

* Fix type in PortForward() error message in "orchard ssh vm"

* Fix potential NPE in Connections()

* Use header.Set() for consistency's sake for Authorization header
2023-07-04 18:10:41 +04:00
Nikolay Edigaryev 3c3b8e8180
Do not treat controller registration error as fatal (#100) 2023-06-29 19:29:32 +04:00
Nikolay Edigaryev 9b0cce182f
Use os.MkdirAll instead of os.Mkdir to create Orchard's home directory (#96)
* Use os.MkdirAll instead of os.Mkdir to create Orchard's home directory

* README.md: document ORCHARD_HOME

* README.md: sort environment variables
2023-06-26 19:35:50 +04:00
Nikolay Edigaryev c4c1851aff
Certificate-less bootstrap tokens (#93) 2023-06-23 00:53:03 +04:00
Nikolay Edigaryev dcc954631b
TLS improvements (#90)
* Fix typo when passing arguments to tls.LoadX509KeyPair()

* Support TLS 1.2 too

* Do not require a controller to only present a single certificate

* No need to set ServerName since we use InsecureSkipVerify

* Use host's root CA set by default and support normal SNI scenarios
2023-06-07 15:31:55 +04:00
Fedor Korotkov f6b48b7c42
Change event prefix to preserve order under load (#89)
* Change event prefix to preserve order under load

When there are a lot of events streamed from a worker, it's possible to have two batches coming for the same timestamp (which is a timestamp of the event on the worker). This way the existing logic would mess up the order because `index` and the random number doesn't guarantee the order.

To fix this I've changed the format of the prefix for the event to include tro things:

1. Timestamp in nanoseconds of the injection time on the controller so two sequential batches will have guaranteed order unless they are processed within a nanosecond.
2. Made the `index` being fixed length with trailing zeros, so they are properly lexicographically sorted (`000001`, `000002`, ...).

* No need to disable linting
2023-06-05 17:01:12 +00:00
Grigory Entin c12f93a90a
Fixed `logs` gluing the lines together. (#87)
* Fixed logs gluing the lines.

* .golangci.yml: disable "depguard" linter

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2023-06-04 09:42:02 +00:00
dependabot[bot] ece7c3dcc8
Bump github.com/gin-gonic/gin from 1.8.2 to 1.9.0 (#84) 2023-05-05 07:19:12 -04:00
Nikolay Edigaryev 60e564da88
Implement restart policy for VMs (#83)
* Implement restart policy for VMs

* Do not update VM.Resource, we only use it as a read-only specification

* Err()/setErr(): use atomic.Pointer instead of sync.Mutex
2023-04-24 19:30:08 +04:00
Fedor Korotkov 010df300a3
Add basic Prometheus metrics (#82)
Fixes #71
2023-04-21 10:05:01 +04:00
Fedor Korotkov dd5e588eb0
Support Bridged Network (#78)
* Support Bridged Network

Inspired by https://github.com/cirruslabs/tart/issues/473

* Fixed tests
2023-04-20 15:04:07 +04:00
Fedor Korotkov a8771a046e
Orchard Social Card (#79) 2023-04-20 02:05:48 +04:00
Nikolay Edigaryev 06de1094ba
Remove worker role (#77) 2023-04-12 12:03:24 +04:00
Nikolay Edigaryev 77656517fd
Controller info endpoint and API integration examples (#75)
* Controller API: introduce controller's information endpoint

* Prevent generation of empty events after channel closure

* Allow events to be buffered in the events channel

* Controller API: introduce controller's information endpoint[1]

* IntegrationGuide.md: a couple of Python and Golang examples

* Rephrase a sentence

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

---------

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2023-04-11 07:28:46 +00:00
Nikolay Edigaryev 84633d0e45
Introduce "orchard pause" and "orchard resume" commands (#73) 2023-04-07 22:59:41 +04:00
Nikolay Edigaryev fd1be695d4
Environment variable overrides (#74)
* Environment variable overrides

* Remove ORCHARD_CONTEXT
2023-04-07 17:49:13 +00:00
Nikolay Edigaryev 316f785a0c
orchard ssh vm: introduce exec mode (#72)
* orchard ssh vm: introduce exec mode

* Document "tart vm ssh" exec mode
2023-04-05 11:05:27 -04:00
Nikolay Edigaryev 4eafec99a5
Fail VMs if the worker had crashed/is unhealthy (#70)
* Fail VMs if the worker had crashed/is unhealthy

* OnDiskName: properly handle cases when VM's name contains hyphens

* Worker: introduce Offline() method and check it before scheduling

* tart.List(): use Tart's JSON output

* OnDiskName: remove empty parts check

* Scheduler: move health-checking logic to a separate function

* Only fail "running" VMs

* Only fail orphaned VMs if they're in terminal state

* Integration tests

* Run healthCheckingLoopIteration() before schedulingLoopIteration()

* Worker: sync on-disk VMs only once at start
2023-04-03 16:47:49 +04:00
Fedor Korotkov ea1e5c8578
Documented how to access VMs (#69) 2023-03-29 16:09:57 +04:00
Fedor Korotkov f152043f19
Reactive Scheduling (#67)
Before we had two main loops: controller loop to assign VMs and worker loop to start VMs. Each of the loops was performed upon an interval every N seconds.

This change introduces a mechanism for reactively requesting loop execution:

 1. Controller loop will be executed upon VM creation to try to immediately schedule.
 2. A worker will be notified upon a VM assigment and worker loop will be requested to sync immediately.

 Fixes #31
2023-03-28 20:51:41 +04:00
Fedor Korotkov 5eaf6b24d4
Make port-forward endpoint to wait for the VM (#65)
* Make port-forward endpoint to wait for the VM

Fixes #62

* Fixes after rebase
2023-03-27 23:52:21 +04:00
Nikolay Edigaryev 357a042937
REST API: provide error messages in error responses (#66)
* REST API: provide error messages in error responses

* Fix role checking logic and add tests

* Ignore testpackage linter error

* Rename NewError() to NewErrorResponse()
2023-03-27 14:12:03 -04:00
Nikolay Edigaryev cb39836ee0
Resources support (#63)
* Resources support

* Ability to provide VM and worker resources via the CLI

* orchard dev: always listen on :6120

* orchard dev: support --resources

* REST API: provide resource defaults when creating VM

* OpenAPI: document "resources" field

* orchard dev: serve Swagger API documentation on /v1/

* Integration guide
2023-03-27 17:30:54 +04:00
Fedor Korotkov f42026824c
Configure workers to log to a file (#59)
* Configure workers to log to a file

As a best practice. Inspired by 845ca51d5b/FAQ.md (does-zap-support-log-rotation)

* Removed redundant error check

* Make log file optional
2023-03-24 18:03:27 +00:00
dependabot[bot] 67702edd59
Bump golang.org/x/net from 0.5.0 to 0.7.0 (#50)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-24 13:31:11 -04:00
dependabot[bot] 1ebec7a268
Bump golang.org/x/crypto from 0.0.0-20211215153901-e495a2d5b3d3 to 0.1.0 (#49)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.0.0-20211215153901-e495a2d5b3d3 to 0.1.0.
- [Release notes](https://github.com/golang/crypto/releases)
- [Commits](https://github.com/golang/crypto/commits/v0.1.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-24 13:23:56 -04:00
Nikolay Edigaryev 7647ccdc10
Remove Generation field (#57) 2023-03-24 17:23:07 +00:00
Nikolay Edigaryev 49753ebf4c
Tests: use separate controller listening ports to prevent conflicts (#58) 2023-03-24 17:22:58 +00:00
Fedor Korotkov 63ba8b5532
Separate context for `orchard dev` (#56)
Fixes #51
2023-03-24 13:10:35 -04:00
Fedor Korotkov b1813a2a92
Deployment guide to GCP (#44)
* Deployment guide to GCP

* Update DeploymentGuide.md

* Refer to Ansible Playbook

* read permissions as well
2023-03-24 09:49:06 -04:00
Fedor Korotkov 362ea85b4f
Always require a client for running a worker (#52)
* Always require a client for running a worker

* Actually validate roles

* Delete worker

Fixes #46

* Update internal/worker/worker.go

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2023-03-24 17:44:20 +04:00