Commit Graph
63 Commits
Author SHA1 Message Date
Nikolay Edigaryev 72c6bbffa0 Move documentation to Tart's website (#199) 2024-09-03 21:08:11 +04:00
Nikolay Edigaryev 8aaf05c4f7 controller run: make bootstrap process more user-friendly (#201)
* controller run: make bootstrap process more user-friendly

* Badger: log to zap instead of standard error
2024-09-03 18:54:28 +04:00
Nikolay Edigaryev b8bc62cf28 .goreleaser.yml: publish .deb and .rpm packages to Fury.io (#200) 2024-08-27 17:32:58 +04:00
Nikolay Edigaryev cd9794197b API: update service account fields on PUT (#198)
* API: update service account fields on PUT

* Disable G115 integer overflow linter of gosec
2024-08-21 20:03:52 +04:00
Nikolay Edigaryev eecf61f714 .cirrus.yml: specify resources required on the "dev-mini" (#194) 2024-07-16 16:18:49 +04:00
Nikolay Edigaryev 0fd17db586 Document standard options of ingesting metrics into GCP (#193)
* Document standard options of ingesting metrics into GCP

* Remove "the"
2024-07-11 16:51:25 +04:00
Nikolay Edigaryev 4df43e6432 Default ?wait= to 0 seconds (#190) 2024-07-03 23:07:14 +04:00
Nikolay Edigaryev 76f192bdb0 API endpoint and associated RPC changes to resolve VMs IP's (#188)
* API endpoint and associated RPC changes to resolve VMs IP's

* Fix "Missing expected argument '<name>'" error when doing "tart set"

* Implement TestIPEndpoint() and IP() method in controller HTTP client
2024-07-03 22:56:43 +04:00
Nikolay Edigaryev 8119b22817 orchard controller run: introduce --insecure-ssh-no-client-auth (#187) 2024-06-28 23:55:18 +04:00
Nikolay EdigaryevandFedor Korotkov ff0497b1d8 Produce OpenTelemetry metrics (#185)
* .golangci.yml: remove mentions of deprecated linters

* Fix "staticcheck" linter error by using grpc.NewClient

* Configure OpenTelemetry

Metrics only for now.

* Produce OpenTelemetry metrics

* Update DeploymentGuide.md

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

* Update DeploymentGuide.md

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

* Introduce "org.cirruslabs.orchard.controller.worker_status"

---------

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2024-06-24 18:19:51 +04:00
Nikolay Edigaryev 9442b6c5c3 Randomize the VM's MAC-address when using bridged networking (#182)
* Randomize the VM's MAC-address when using bridged networking

* CI: bypass test caching for "go test"
2024-06-18 11:19:56 +04:00
Nikolay Edigaryev 772654a408 .goreleaser.yml: upgrade to GoReleaser v2 (#180) 2024-06-11 22:16:54 +04:00
Nikolay Edigaryev d59bc7f8a7 Orchard Controller: implement an SSH server that acts as a jump host (#179)
* proxy.Connections(): require io.ReadWriteCloser instead of net.Conn

* Orchard Controller: implement an SSH server that acts as a jump host

* Issue a warning if the name used will be invalid in the future

* Further restrict uppercase characters in names in the future

The rationale is similar to https://github.com/kubernetes/kubernetes/issues/71140.

We won't want to munge the user's input and introduce subtle bugs doing
lowercase comparisons.
2024-06-11 19:32:45 +04:00
Nikolay Edigaryev c845f3b2fd API: do not return null when methods returning a list have no items (#170)
* API: do not return null when methods returning a list have no items

* Use "omitempty" in all API structs
2024-04-29 15:49:09 -04:00
Nikolay Edigaryev 8790136f6a Use macOS Sonoma images by default, instead of macOS Ventura (#166) 2024-04-17 17:56:08 +04:00
Nikolay Edigaryev 7fb0a85834 API(VM): new image FQN (fully-qualified name) field (#165) 2024-04-15 20:14:44 +04:00
Nikolay Edigaryev 510a25956f Asynchronous VM creation (#163) 2024-04-12 15:56:00 +04:00
Nikolay Edigaryev a5c653d0a6 Introduce --bootstrap-token-stdin and --{username,password} (#160)
* orchard worker run: introduce --bootstrap-token-stdin flag

* orchard create vm: introduce --username and --password
2024-04-04 15:55:07 +04:00
Nikolay Edigaryev 2c5b038df2 Set HTTP client timeout to 30 seconds (#153)
* Set HTTP client timeout to 30 seconds

* Add an explanation for why we set http.Server's Timeout
2024-02-05 18:32:31 +04:00
Nikolay Edigaryev 969f0bd729 Association flow improvements for "orchard context create" and better URL crafting (#149)
* Client: prevent double slashes at the end of URLs

* orchard context create: let the user know which association flow is used

* Client: rename parsePath() to formatPath()

* Client: grab the ServerName from the trusted certificate
2023-11-21 18:43:42 +04:00
Nikolay Edigaryev fe8cc21cba HostDir: support URLs in path (#142) 2023-10-11 18:37:24 +04:00
Nikolay Edigaryev 13b4e192f0 Introduce "orchard {port-forward, vnc} worker WORKER_NAME" (#140)
* Fix potential NPE in Client.wsRequest()

* Introduce "orchard {port-forward, vnc} worker WORKER_NAME"

* portspec.go: simplify logic and respect [LOCAL_PORT]:REMOTE_PORT format
2023-10-09 18:51:34 +04:00
Nikolay Edigaryev 063405672f Worker: better cleanup mechanisms (#139)
* Always Close() the Worker instance

* orchard list vms: show assigned worker for each of the VMs

* Stop the failed VMs before we schedule new VMs

To avoid violating resource constraints.

* syncOnDiskVMs: don't ignore running VMs

* Worker: show correct remote and local VM counts
2023-09-28 19:09:43 +00:00
Nikolay Edigaryev 40f58e4aee More RPC-related logs (#136)
* More RPC-related logs

* Notifier should be set before we use it in the scheduler
2023-09-27 20:16:00 +04:00
Nikolay Edigaryev 64987f6d59 Send http.Server errors to zap at debug level and provide --debug flag (#135) 2023-09-25 18:17:23 +04:00
Nikolay Edigaryev 8c62df0eba Only allow simple names when creating workers, VMs, etc. and escape paths in API client (#129)
* Controller: only allow simple names when creating workers, VMs, etc.

* Client: escape paths

* simplename: allow ':' character
2023-09-22 14:51:43 -04:00
Nikolay Edigaryev 2adb17c3c4 Introduce "orchard get worker" command (#127) 2023-09-13 23:50:58 +04:00
Nikolay Edigaryev 3d0e375ede Don't stop and delete VMs that failed to clone (#125)
* NewVM() never returns an error

* Don't stop and delete VMs that failed to clone
2023-09-13 19:39:10 +04:00
Nikolay Edigaryev bb3d6edcd5 Fix Tart VM IP detection in bridged mode (#124) 2023-09-12 08:52:21 +00:00
Nikolay Edigaryev 036eb954be Retry DB transactions on badger.ErrConflict (#114)
* Log HTTP 500 errors in more detail

* Log errors in storeView and storeUpdate

* Retry on badger.ErrConflict
2023-08-15 15:18:47 +04:00
Nikolay Edigaryev 6759618f28 orchard create vm: support --image-pull-policy=Always (#110) 2023-07-26 17:43:14 +04:00
Nikolay Edigaryev fd88ce5890 Introduce ORCHARD_LICENSE_TIER environment variable (#111)
* Introduce ORCHARD_LICENSE_TIER environment variable

* Only parse ORCHARD_LICENSE_TIER if it was provided
2023-07-26 17:28:38 +04:00
Nikolay Edigaryev a52c205c34 API(port forward endpoint): handle normal WebSocket closure gracefully (#108) 2023-07-20 20:55:42 +00:00
Nikolay Edigaryev c5c729c96f Document --no-pki (#107)
* Document --no-pki

* and re-verify → and will re-verify

* against → only against

* Less "we"'s
2023-07-10 19:13:10 +04:00
Nikolay Edigaryev 0ba7db1a70 orchard list vms: add "Created" row (#106) 2023-07-04 18:11:26 +04:00
Nikolay Edigaryev d57d18d380 Support for sharing files with the host system (#103)
* Support for sharing files with the host system

* Integration tests

* Added back TestVMGarbageCollection comment
2023-07-04 18:10:53 +04:00
Nikolay Edigaryev 6a325daf74 Switch from golang.org/x/net/websocket to nhooyr.io/websocket and handle NotFound errors (#105)
* Switch from golang.org/x/net/websocket to nhooyr.io/websocket

* Do not attach errors that we can handle to the Gin's context

* Add missing newline to "no credentials specified or found, ..." message

* Fix potential NPE in ChooseUsernameAndPassword()

* Fix type in PortForward() error message in "orchard ssh vm"

* Fix potential NPE in Connections()

* Use header.Set() for consistency's sake for Authorization header
2023-07-04 18:10:41 +04:00
Nikolay Edigaryev 3c3b8e8180 Do not treat controller registration error as fatal (#100) 2023-06-29 19:29:32 +04:00
Nikolay Edigaryev 9b0cce182f Use os.MkdirAll instead of os.Mkdir to create Orchard's home directory (#96)
* Use os.MkdirAll instead of os.Mkdir to create Orchard's home directory

* README.md: document ORCHARD_HOME

* README.md: sort environment variables
2023-06-26 19:35:50 +04:00
Nikolay Edigaryev c4c1851aff Certificate-less bootstrap tokens (#93) 2023-06-23 00:53:03 +04:00
Nikolay Edigaryev dcc954631b TLS improvements (#90)
* Fix typo when passing arguments to tls.LoadX509KeyPair()

* Support TLS 1.2 too

* Do not require a controller to only present a single certificate

* No need to set ServerName since we use InsecureSkipVerify

* Use host's root CA set by default and support normal SNI scenarios
2023-06-07 15:31:55 +04:00
Nikolay Edigaryev 60e564da88 Implement restart policy for VMs (#83)
* Implement restart policy for VMs

* Do not update VM.Resource, we only use it as a read-only specification

* Err()/setErr(): use atomic.Pointer instead of sync.Mutex
2023-04-24 19:30:08 +04:00
Nikolay Edigaryev 06de1094ba Remove worker role (#77) 2023-04-12 12:03:24 +04:00
Nikolay EdigaryevandFedor Korotkov 77656517fd Controller info endpoint and API integration examples (#75)
* Controller API: introduce controller's information endpoint

* Prevent generation of empty events after channel closure

* Allow events to be buffered in the events channel

* Controller API: introduce controller's information endpoint[1]

* IntegrationGuide.md: a couple of Python and Golang examples

* Rephrase a sentence

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

---------

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2023-04-11 07:28:46 +00:00
Nikolay Edigaryev 84633d0e45 Introduce "orchard pause" and "orchard resume" commands (#73) 2023-04-07 22:59:41 +04:00
Nikolay Edigaryev fd1be695d4 Environment variable overrides (#74)
* Environment variable overrides

* Remove ORCHARD_CONTEXT
2023-04-07 17:49:13 +00:00
Nikolay Edigaryev 316f785a0c orchard ssh vm: introduce exec mode (#72)
* orchard ssh vm: introduce exec mode

* Document "tart vm ssh" exec mode
2023-04-05 11:05:27 -04:00
Nikolay Edigaryev 4eafec99a5 Fail VMs if the worker had crashed/is unhealthy (#70)
* Fail VMs if the worker had crashed/is unhealthy

* OnDiskName: properly handle cases when VM's name contains hyphens

* Worker: introduce Offline() method and check it before scheduling

* tart.List(): use Tart's JSON output

* OnDiskName: remove empty parts check

* Scheduler: move health-checking logic to a separate function

* Only fail "running" VMs

* Only fail orphaned VMs if they're in terminal state

* Integration tests

* Run healthCheckingLoopIteration() before schedulingLoopIteration()

* Worker: sync on-disk VMs only once at start
2023-04-03 16:47:49 +04:00
Nikolay Edigaryev 357a042937 REST API: provide error messages in error responses (#66)
* REST API: provide error messages in error responses

* Fix role checking logic and add tests

* Ignore testpackage linter error

* Rename NewError() to NewErrorResponse()
2023-03-27 14:12:03 -04:00
Nikolay Edigaryev cb39836ee0 Resources support (#63)
* Resources support

* Ability to provide VM and worker resources via the CLI

* orchard dev: always listen on :6120

* orchard dev: support --resources

* REST API: provide resource defaults when creating VM

* OpenAPI: document "resources" field

* orchard dev: serve Swagger API documentation on /v1/

* Integration guide
2023-03-27 17:30:54 +04:00
Nikolay Edigaryev 7647ccdc10 Remove Generation field (#57) 2023-03-24 17:23:07 +00:00
Nikolay Edigaryev 49753ebf4c Tests: use separate controller listening ports to prevent conflicts (#58) 2023-03-24 17:22:58 +00:00
Nikolay Edigaryev af074f499d Remove UID for now and use machine ID to differentiate workers (#48)
* Remove UID for now and use machine ID to differentiate workers

* Rename MetadataWorkerKey back to MetadataWorkerNameKey
2023-03-23 23:38:54 +04:00
Nikolay Edigaryev 5c3b47a247 --help fixes and new "orchard logs vm" command (#41)
* Make sure we list names/VM names as required argument in the --help

* Introduce "orchard logs vm" command

* Make sure each command has a Short field

* Future-proof port-forward, ssh and vnc commands

To support not only the VM resource.
2023-03-23 15:59:57 +04:00
Nikolay Edigaryev 10f56bb5e3 Introduce "orchard ssh" and "orchard vnc" commands (#36)
* proxy.Connections(): handle "use of closed network connection" error

* Controller: less strict timeouts that work nicely for WebSockets

* Worker: only attempt connect to the gRPC once our UID is known

* Introduce "orchard ssh" and "orchard vnc" commands

* Worker: prevent context leak by moving logic into a separate function

* Fix linter errors

* Port forwarding integration test

* Check for "uname -mo" output
2023-03-21 14:58:24 -04:00
Nikolay Edigaryev 47fef47d1c Port forwarding support (#30)
* Port forwarding support

* .golangci.yml: remove and replace deprecated and archived linters

* Client: pass credentials when calling WebSocket API methods

* API: require ServiceAccountRoleComputeWrite role for port forwarding

* Use Buf

* Rename Poll() RPC method to Watch()

* Split Rendezvous into two parts: Watcher and Proxy (#32)

* Split Rendezvous into two parts: Watcher and Proxy

* Implement Proxy cancellation

* Use Protocol Buffers structure directly in Watcher

* Fix TestWatcher after switching to Protocol Buffers structure

* portForwardVM(): ensure we also check for gin's context
2023-03-14 11:31:13 -04:00
Nikolay Edigaryev 8df31f7c2d Introduce service accounts and bootstrap tokens (#22) 2023-02-21 11:34:12 -05:00
Nikolay Edigaryev 0b9b96b8c9 Introduce "orchard context" (#18) 2023-02-07 19:48:31 +04:00
Nikolay Edigaryev a7264370f5 Introduce "controller init" and generate self-signed X.509 certificate (#17) 2023-02-04 11:40:07 +04:00
Nikolay Edigaryev ab3ffe82fa Remove janitor (#16)
* Remove controller's janitor

* Properly handle err == nil in mapErr() function
2023-02-01 19:02:56 +04:00
Nikolay Edigaryev 6bcc02d815 Use golangci-lint (#15) 2023-01-31 22:22:28 +04:00
Nikolay EdigaryevandFedor Korotkov 92e8732d46 Initial version of the Orchard orchestration system (#3)
* Initial version of the Orchard orchestration system

* Update README.md

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2023-01-26 23:46:23 +04:00
Nikolay Edigaryev dcbbb8b8de Add README.md 2023-01-24 21:23:11 +04:00