Compare commits

...
83 Commits
Author SHA1 Message Date
Nikolay Edigaryev b7b3b702ac Sentry: upgrade and attach command-line arguments (#774)
* Sentry: upgrade and attach command-line arguments

* Sentry's setContext(): explicitly pass a String
2024-04-02 18:31:16 +04:00
Tor Arne Vestbø 560dba79e4 Report operating system in tart get (#772)
Can be useful to know from outside the VM.
2024-03-31 14:18:26 -04:00
Tor Arne Vestbø 2b33b8f9e6 Report progress when downloading IPSW files (#768)
The URLSession async/await functions do not report progress through
the normal URLSessionTaskDelegate callbacks, as reported in:

 https://developer.apple.com/forums/thread/723015

We don't want to use URLSession.bytes, as that results in a much
slower download speed compared to URLSession.download, but we can
work around the lack of progress callbacks by observing the
progress on the URLSessionTask itself.

Fixes #767
2024-03-28 19:15:06 +04:00
Tor Arne Vestbø d8b010c79c Support cancellation of installation process (#770)
We wrap the installation with a withTaskCancellationHandler, which
ensures that the SIGINT signal handling code in main() will trigger
a cancellation of the installer.

As the VZMacOSInstaller must be both created and interacted with
on the VM's queue, which in our case is the main queue, we need
to move the logic to a separate function tagged with @MainActor.
This makes sense either way, as it cleans up the code a bit.
2024-03-28 00:14:33 +04:00
Nikolay Edigaryev 5cd83c38cd Introduce Golang-based benchmarking utility (#769)
* Introduce Golang-based benchmarking utility

* benchmark fio: properly configure logger level

* benchmark: properly terminate on Ctrl+C when initializing/running Tart

* benchmark(fio): increase runtime to 30 seconds

* benchmark(fio): IOPS are already per second

* benchmark(fio): --numjobs 1 --iodepth 1 --end_fsync 1

* benchmark(README.md): add results
2024-03-27 18:45:01 +04:00
Bartek Pacia 1a3b862631 goreleaser: set up automatic installation of shell completion files (#766) 2024-03-26 12:08:09 +04:00
Fedor Korotkov ae2d59e5c2 Revert "Do not magically set --no-graphics when --vnc is passed (#763)
* Revert "Do not magically set `--no-graphics` when `--vnc` is passed (#732)"

This reverts commit a48f4d4ec9.

* Mark `--graphics` as private
2024-03-19 08:28:27 +00:00
Evgeniy Baranov 7eac45702b Fix the --insecure flag issue by disabling ATS in Info.plist (#760) 2024-03-19 03:42:14 -04:00
Nikolay Edigaryev e06d89f95d integration-tests: test_run() with --no-graphics (#757) 2024-03-12 15:12:02 +04:00
Fedor Korotkov 0602d6e0e1 Pack additional resources into brew releases (#756)
To fix missing icon since #746
2024-03-12 10:46:56 +00:00
Fedor Korotkov ac5d0baa0c Fixed --no-graphics mode (#755)
Regression introduced in #746
2024-03-12 10:42:35 +00:00
Nikolay Edigaryev ee27cc57bb tart run: introduce --net-softnet-allow command-line argument (#753) 2024-03-11 22:17:34 +04:00
Fedor Korotkov a9e2a19015 Lowercase tart.app (#751)
So signing and `embedded.provisionprofile` work as expected.

Related to #746
2024-03-11 12:23:12 -04:00
Fedor Korotkov 89ff5f6b65 Respect name in case of a single directory mount (#750)
Fixes #748
2024-03-11 15:01:06 +00:00
Nikolay Edigaryev 6bf39e73d0 Prefer USB keyboard and screen coordinate pointing devices (#747) 2024-03-11 13:16:40 +00:00
Tor Arne VestbøandNikolay Edigaryev 0b693f6bc9 Improve macOS app integration (#746)
* Improve macOS app integration

Tart is now a proper application bundle, with the name and icon
declared in the Info.plist, which we were missing.

This also allows us to declare the app as LSBackgroundOnly
as a default, which means that 'tart create' and similar
background commands will not show the application icon in
the dock, while 'tart run' will, thanks to it overriding
the activation policy of the app.

For now the logic of creating the Tart.app bundle is duplicated
between the CI packaging scripts and the run-signed.sh script.
Now that these scripts are growing, it makes sense to look
at whether we can share the logic somehow, e.g. by building
the application bundle directly during build, and packaging
that, instead of creating it as a post install step.

* Integration tests: fix DockerContainer import

To work around the breaking change in 4.0.0,
see https://github.com/testcontainers/testcontainers-python/commit/383b12e9d63b4b105c65e587e87c380355871a36.

* .cirrus.yml(Release (Dry Run)): no need to install Sentry CLI

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2024-03-11 09:05:56 -04:00
Fedor Korotkov 99bbd838a1 Introduce --net-host flag to enforce host-only network (#743)
* Experimental `--net-host-only` option

* Use Softnet's host networking
2024-03-01 13:57:39 +04:00
Fedor Korotkov 5c7743b7cd Show both size and actual size of files (#742)
Right now we show only actual size of files on disk which excludes empty blocks of the recently introduced sparced format in #671. This makes impossible to get info about disk size that we just set via `tart set`.

Here is an example of `tart list` output before the change:

```
Source Name                                                                                                            Size State
local  sonoma-base                                                                                                     22   stopped
local  sonoma-vanilla                                                                                                  18   stopped
local  sonoma-xcode                                                                                                    67   stopped
local  ubuntu                                                                                                          1    stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base:latest                                                                     22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base@sha256:16c1593bbaf787b20b3c0bc094c5b6baf71c937d22c2e4596da85ac55c92e6cc    22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla:14.3                                                                    17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla@sha256:23c4e853d48d00a4333346d66a32b2b5aad900cc0dc10e7ecb9dbe67b6f587f4 17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode:latest                                                                    67   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode@sha256:d0cb8d01424a68b89e0f16f5371bf2152b2c115bd886341a6ba8da42121d1f41   67   stopped
oci    ghcr.io/cirruslabs/ubuntu:22.04                                                                                 1    stopped
oci    ghcr.io/cirruslabs/ubuntu@sha256:037763feb7a15d6077edeb7a097738c34313637d16036764b4c196d28d8b429c               1    stopped
```

And here is the output after the change:

```
Source Name                                                                                                            Disk Size State
local  sonoma-base                                                                                                     50   22   stopped
local  sonoma-vanilla                                                                                                  50   18   stopped
local  sonoma-xcode                                                                                                    90   67   stopped
local  ubuntu                                                                                                          20   1    stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base:latest                                                                     50   22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base@sha256:16c1593bbaf787b20b3c0bc094c5b6baf71c937d22c2e4596da85ac55c92e6cc    50   22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla:14.3                                                                    50   17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla@sha256:23c4e853d48d00a4333346d66a32b2b5aad900cc0dc10e7ecb9dbe67b6f587f4 50   17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode:latest                                                                    90   67   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode@sha256:d0cb8d01424a68b89e0f16f5371bf2152b2c115bd886341a6ba8da42121d1f41   90   67   stopped
oci    ghcr.io/cirruslabs/ubuntu:22.04                                                                                 20   1    stopped
oci    ghcr.io/cirruslabs/ubuntu@sha256:037763feb7a15d6077edeb7a097738c34313637d16036764b4c196d28d8b429c               20   1    stopped
```

Additionally, `tart get` will print actual size with a 3 decimal point precision which will help to track growth in disk images for our templates.

`tart get` before:

```
CPU Memory Disk Display  State
4   8192   67   1024x768 stopped
```

`tart get` after:

```
CPU Memory Disk Size   Display  State
4   8192   90   67.333 1024x768 stopped
```
2024-02-27 20:13:38 +04:00
Fedor Korotkov a40e104c03 Update Platinum Tier (#741)
5x jump in price from Gold to Platinum is a bit too high. Most of known large deployments target 200-300 hosts. Let's accommodate such users by lowering Platinum tier.
2024-02-23 15:27:30 +00:00
Nikolay Edigaryev e2d6c13ed0 DHCP MAC-address resolver: handle duplicate leases (#740) 2024-02-23 16:49:06 +04:00
Nikolay Edigaryev 3f17884ac2 Introduce "tart fqn" command (#735)
* tart pull: experimental --json-digest option

* Introduce "tart fqn" command

* Revert "tart pull: experimental --json-digest option"

This reverts commit 842052f5bd.
2024-02-19 21:13:45 +00:00
Fedor Korotkov 7dcebf9c04 Allow to override VirtioFS tag for shared directories (#733) 2024-02-19 20:51:25 +00:00
Tor Arne Vestbø f6c56ed8eb Add option to disable audio pass-though (#728) 2024-02-19 12:26:31 -05:00
Fedor Korotkov a48f4d4ec9 Do not magically set --no-graphics when --vnc is passed (#732)
From a discussion in #728 it appeared that having both `--graphics` and `--no-graphics` is a bit confusing.

`--graphics` was introduced in #248 to support having both VNC and UI for debugging Packer plugin in cirruslabs/packer-plugin-tart#21. This is because `--vnc` flag has a side effect of hiding UI which I think was wrong in retrospective. One can run `tart run --vnc --no-graphics`. In most of the cases this is automated via Alfred or something like that.

Now we have so many arguments that IMO it's worth to remove `--graphics` for overall consistency in arguments: everything is enabled by default and can be disabled via `--no-*` flags.
2024-02-19 08:47:27 -05:00
Noah Martin 9bb71a4051 Improve rename error message (#723) 2024-01-30 10:01:42 -05:00
Fedor Korotkov 18d462dd3d Build x86 binary (#716)
* Build x86 binary

To support Linux VMs on Intel aka x86_64

* Fixed paths and formatting

* Unique IDs

* Fixed Goreleaser

* Skip creation integration test for now

* import

* Reenable create test

* Revert "Reenable create test"

This reverts commit 4c947c1f0e.

* Reenable create test
2024-01-26 13:09:05 +00:00
Fedor Korotkov cd6a97f842 Some SEO optimizations (#720)
Removed Anka mentions and Codemagic/Testingbot since we don't know if they still use Tart.
2024-01-24 12:15:06 -05:00
Fedor Korotkov 3e4bc73ff0 [docs] Deprioritize Anna (#714) 2024-01-20 15:58:52 +04:00
Fedor KorotkovandNikolay Edigaryev 89fff42d0a [docs] Resize linux images after cloning (#712)
* [docs] Resize linux images after cloning

Fixes #711

* Update quick-start.md

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Update quick-start.md

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2024-01-20 05:35:41 -05:00
Evan Martin 7bb50b5e4b fix typo in FAQ (#713) 2024-01-20 10:51:05 +04:00
Nikolay Edigaryev b5cbf67ee6 Document default admin/admin credentials (#709) 2024-01-20 02:34:21 +04:00
Fedor Korotkov 090a8232fc [docs] Improved table of content for licensing.md (#705) 2024-01-17 19:00:15 +04:00
Fedor Korotkov 306ace792c Added a testimonial from Mitchell Hashimoto (#704) 2024-01-15 12:55:27 +04:00
Nikolay Edigaryev 96f6f94fa7 tart set: make --disk-size change idempotent (#698)
...when the size is not changed.
2024-01-08 14:34:13 +00:00
Nikolay Edigaryev fbc481250d tart run: disable dynamic display reconfiguration for Linux (#697) 2024-01-08 14:25:56 +00:00
Nikolay Edigaryev 35538c2c5d tart set: fix typo in --disk-size help (#695) 2024-01-04 01:09:25 +00:00
Nikolay Edigaryev 4c33064916 tart set: bring back the --disk-size command-line argument (#694)
* tart set: bring back the --disk-size command-line argument

* Add a --disk-size explainer
2024-01-04 00:34:55 +04:00
Nikolay Edigaryev 1a267d4a39 tart create --linux: allow scaling VM down to 1 CPU and 256 MiB (#693)
* tart create --linux: allow scaling VM down to 1 CPU and 256 MiB

* Revert "tart create --linux: allow scaling VM down to 1 CPU and 256 MiB"

This reverts commit 7a31443eea.

* Check minimum CPU and memory sizes in "tart set"
2024-01-03 19:48:13 +04:00
Nikolay Edigaryev 36c54d95cb Document how to unlock the Keychain over SSH (#691)
* Document how to unlock the Keychain over SSH

* Fix MD028 markdown linter error

* Add link to Keychain page in Wiki
2023-12-19 15:36:56 +01:00
Nikolay Edigaryev 1d8bfafde5 tart create --from-ipsw: expand tilde (~) in path (#688) 2023-12-18 13:05:48 -05:00
Nikolay Edigaryev 537f0ae5db OCI storage: unconditionally remove the old link when link()'ing (#686) 2023-12-08 17:20:49 +04:00
Fedor Korotkov 02f1ff5238 Fixed image url 2023-12-08 03:23:54 -05:00
Fedor Korotkov 9c9bcd586e Highlight AWS Marketplace availability (#683)
* Highlight AWS Marketplace availability

* Updated image

* Changed height
2023-12-08 10:44:41 +04:00
Fedor Korotkov 60f0eac7a8 Cache only non-empty archives (#685)
Fixes #684. But I'm not sure how it got into this state in the first place. `URLSession.shared.data` should've throw.
2023-12-08 10:43:43 +04:00
Andrew Malchuk 35377a3475 Fix the filesystem corruption on Linux VMs (#675)
* Use NVMe drive, cached mode and full synchronization mode on Linux

* Inline getting storage device attachment
2023-12-01 15:56:58 +00:00
Nikolay Edigaryev dda4e91a91 Document Buildkite Tart Plugin (#677)
* Document Buildkite Tart Plugin

* Fix cropped screenshot
2023-12-01 15:35:45 +00:00
Nikolay Edigaryev ac5f794e6d tart delete: prevent the deletion of running VMs (#676)
And introduce a VMDirectory.lock() method to avoid duplication of
the PIDLock(lockURL: vmDir.configURL) snippet.
2023-12-01 09:33:01 -05:00
Nikolay Edigaryev 5bcbc77249 Document available VM images on the website (#674)
* Document available VM images on the website

* Fix indents
2023-11-28 16:31:39 +00:00
Fedor Korotkov bf03873c8d Validate that a disk is not amd64 (#673)
To improve UX for cases like #672
2023-11-28 14:55:34 +00:00
Nikolay Edigaryev bad37b129c DiskV2: write layers sparsely to avoid unnecessary disk usage (#671) 2023-11-27 23:27:07 +04:00
Nikolay Edigaryev 0f47cca746 MAC address resolver: skip expired leases (#669) 2023-11-27 10:12:36 -05:00
Fedor Korotkov d70eca4484 Document Cirrus Runners Discounts (#663) 2023-11-22 20:00:45 +04:00
Fedor Korotkov 25887b075f Use ssh from our tap (#662)
Fixes #661
2023-11-20 20:01:57 +00:00
Simon B. Støvring 8c011623be Adds Shape logo to README (#658) 2023-11-15 14:51:10 +00:00
Fedor Korotkov 2dccdfb306 Document ECR Public Mirror (#656)
Fixes https://github.com/cirruslabs/tart/discussions/652
2023-11-13 18:18:51 +00:00
Fedor KorotkovandNikolay Edigaryev aca768a838 Print put errors from Docker Helpers (#654)
* Print put errors from Docker Helpers

* Update Sources/tart/Credentials/DockerConfigCredentialsProvider.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Check output data is not empty

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2023-11-10 22:44:51 +04:00
Tor Arne Vestbø 68b3557747 Hide dock icon in no graphics mode (#653)
* Package tart binary into app bundle when running via run-signed.sh

This is what happens when installing the tart application package
as built by CI. We should stay as close as possible to the install
situation during development, so that we get bug/behavior parity.

For example, an app bundle behaves differently than a standalone
executable when it comes to bringing up a Dock icon for the app.

* Set activation policy to prohibited when starting in no graphics mode

This ensures that the Dock icon is hidden.
2023-11-10 09:05:20 -05:00
Fedor Korotkov b2c923f2fe Properly enter main even loop in headless mode (#651)
Fixes #638
2023-11-09 00:05:00 +04:00
Fedor Korotkov c75009e46f Introduce --capture-system-keys flag (#650)
To allow guest to capture things like Cmd+Tab.

Fixes #636
2023-11-08 20:21:55 +04:00
Riain Condon 43e74ab769 fix docs to specify inside VM for gitlab runner (#649)
just adds specifically VM in the gitlab runner docs to avoid confusion of where the build and cache dirs are
2023-11-08 09:05:19 -05:00
Fedor Korotkov 1338864ed6 Don't install Sentry CLI via brew (#648)
Seems it installas 1.x version instead of 2.x. Sentry's documentation [recommends to use their script](https://docs.sentry.io/product/cli/installation/?original_referrer=https%3A%2F%2Fwww.google.com%2F#automatic-installation).
2023-11-07 16:23:20 +00:00
Nikolay Edigaryev 70040b633c Introduce AuthenticationKeeper actor to serialize authn modification (#647) 2023-11-06 14:58:23 -05:00
Nikolay Edigaryev f4bc02d175 DiskV2.push(): map disk into memory to avoid large allocations (#645) 2023-11-03 17:13:10 +04:00
Fedor Korotkov 6c24aa639a [blog] New dashboard with insights into performance of Cirrus Runners (#644) 2023-11-03 12:17:40 +04:00
Nikolay Edigaryev d8b69de52d Fetcher.fetchViaFile(): use an mmap(2)-ed file, similarly to DiskV1 (#641)
* Fetcher.fetchViaFile(): use an mmap(2)-ed file, similarly to DiskV1

* No need to convert Data to Data
2023-11-01 15:00:48 +04:00
Nikolay Edigaryev c4c2bfeded tart-dev.entitlements: add "com.apple.security.get-task-allow" (#642) 2023-11-01 14:35:55 +04:00
Fedor Korotkov b95585b56b Don't forget to finalize output stream (#640)
There is a suspicion that this might leak memory
2023-11-01 12:19:11 +04:00
Fedor Korotkov 8d5574ed3f Removed usage of deprecated APIs (#628)
See https://developer.apple.com/documentation/virtualization/vzmacauxiliarystorage/3816043-init
2023-10-11 17:05:49 -04:00
Fedor Korotkov 457c2bc7db Adjusted live installation counter (#627) 2023-10-11 20:55:51 +00:00
Fedor Korotkov 4bf9bdd531 Document Tart on AWS (#625)
Fixes #581
2023-10-07 12:06:06 +04:00
Fedor Korotkov 8e9d61d5f5 Validate that a suspendable VM doesn't have shared directories (#623) 2023-10-04 20:21:45 +04:00
Fedor KorotkovandNikolay Edigaryev 71d03226fe Support mounting remote archives (#620)
* Support mounting remote archives

Allow to pass an HTTPS link instead of a local path to `tart run --dir` argument. HTTPS link should point to a gzipped Tar archive aka `*.tar.gz` file.

In this situation Tart will download an archive by the link if necessary, will cache it and will unarchive it into a temporary folder inside `$TART_HOME` to be mounted to the VM.

This use case is useful for mounting something external that updates faster than the VM itself. For example, GitHub Actions Runner installation.

* Don't use async/await APIs to prevent from deadlocks because of the MainActor thing

* Prefer cached data

* Moved comment

* Fix URLCache caching files in memory instead of on-disk (#622)

* Fix URLCache caching files in memory instead of on-disk

* Fix disk capacity typo

* Moved log

* Moved fetching logic to `DirectoryShare#createConfiguration` method

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2023-10-03 23:01:29 +04:00
Nikolay Edigaryev 36dab9878d tart: bump max password characters from 256 to 1024 (#618) 2023-10-02 13:58:55 +00:00
Nikolay Edigaryev f634002813 tart run: disable console device when --suspendable is requested (#615) 2023-09-30 21:13:44 +04:00
Fedor Korotkov 8e79669afb Configure Markdown Linter (#614) 2023-09-29 03:35:17 -04:00
Fedor KorotkovandNikolay Edigaryev 2da8bc0fb5 Document XL Cirrus Runners (#613)
* Document XL Cirrus Runners

Also tried to put everything about Cirrus Runners in one place rather than having the information spreaded between https://tart.run and https://github.com/apps/cirrus-runners.

Plus updated docs to use Sonoma.

* Apply suggestions from code review

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2023-09-28 15:53:13 -04:00
Nikolay Edigaryev 2d984ba194 .cirrus.yml: add an execution_lock: for integration tests (#610)
* .cirrus.yml: add an execution_lock: for integration tests

* Use Persistent Worker's resources instead of grabbing an execution lock
2023-09-22 22:52:52 +04:00
Fedor Korotkov 50ce44c3eb Support block devices on Sonoma (#611)
* Support block devices on Sonoma

* Updated docs

* Removed unused error
2023-09-22 22:30:18 +04:00
Fedor Korotkov c9e49ceb39 Missing user
So it's an even number of them
2023-09-22 12:10:13 -04:00
Fedor Korotkov 6df50e55d8 --suspendable devices fallback on Ventura host (#605)
Fixes #604
2023-09-22 07:38:13 -04:00
Nikolay Edigaryev 1fd710d00d web: fix GitLab Runner integration link (#608)
Resolves https://github.com/cirruslabs/tart/issues/607.
2023-09-22 07:38:02 -04:00
Rui Marinho 4f6c7e79e1 Add Uphold as a Tart user (#606) 2023-09-22 07:37:02 -04:00
fedor 1afb43e85b Updated announcement link 2023-09-20 10:58:17 -04:00
89 changed files with 2254 additions and 630 deletions
+2
View File
@@ -7,6 +7,8 @@ export VERSION="${CIRRUS_TAG:-0}"
mkdir -p .ci/pkg/
cp .build/arm64-apple-macosx/release/tart .ci/pkg/tart
cp Resources/embedded.provisionprofile .ci/pkg/embedded.provisionprofile
cp Resources/AppIcon.png .ci/pkg/AppIcon.png
cp Resources/Info.plist .ci/pkg/Info.plist
pkgbuild --root .ci/pkg/ --identifier com.github.cirruslabs.tart --version $VERSION \
--scripts .ci/pkg/scripts \
--install-location "/Library/Application Support/Tart" \
+3 -1
View File
@@ -3,9 +3,11 @@
set -e
# fix structure
mkdir -p "$2/tart.app/Contents/MacOS"
mkdir -p "$2/tart.app/Contents/MacOS" "$2/tart.app/Resources"
mv "$2/tart" "$2/tart.app/Contents/MacOS/tart"
mv "$2/embedded.provisionprofile" "$2/tart.app/Contents/embedded.provisionprofile"
mv "$2/AppIcon.png" "$2/tart.app/Resources/AppIcon.png"
mv "$2/Info.plist" "$2/tart.app/Contents/Info.plist"
echo "#!/bin/sh" > /usr/local/bin/tart
echo "exec '$2/tart.app/Contents/MacOS/tart' \"\$@\"" >> /usr/local/bin/tart
+25 -17
View File
@@ -1,15 +1,16 @@
use_compute_credits: true
env:
XCODE_TAG: 15
XCODE_TAG: 15.2
task:
name: Test on Ventura
name: Test on Sonoma
alias: test
use_compute_credits: $CIRRUS_USER_COLLABORATOR == 'true'
persistent_worker:
labels:
name: dev-mini
resources:
tart-vms: 1
test_script:
- swift test
integration_test_script:
@@ -28,12 +29,19 @@ task:
path: "integration-tests/pytest-junit.xml"
format: junit
task:
name: Markdown Lint
only_if: $CIRRUS_BRANCH != 'gh-pages' && changesInclude('**.md')
container:
image: node:latest
install_script: npm install -g markdownlint-cli
lint_script: markdownlint --config=docs/.markdownlint.yml docs/
task:
name: Lint
alias: lint
use_compute_credits: $CIRRUS_USER_COLLABORATOR == 'true'
macos_instance:
image: ghcr.io/cirruslabs/macos-ventura-xcode:$XCODE_TAG
image: ghcr.io/cirruslabs/macos-sonoma-xcode:$XCODE_TAG
lint_script:
- swift package plugin --allow-writing-to-package-directory swiftformat --cache ignore --lint --report swiftformat.json .
always:
@@ -43,15 +51,18 @@ task:
task:
only_if: $CIRRUS_TAG == ''
name: Build
env:
matrix:
BUILD_ARCH: arm64
BUILD_ARCH: x86_64
name: Build ($BUILD_ARCH)
alias: build
use_compute_credits: $CIRRUS_USER_COLLABORATOR == 'true'
macos_instance:
image: ghcr.io/cirruslabs/macos-ventura-xcode:$XCODE_TAG
build_script: swift build --product tart
sign_script: codesign --sign - --entitlements Resources/tart-dev.entitlements --force .build/debug/tart
image: ghcr.io/cirruslabs/macos-sonoma-xcode:$XCODE_TAG
build_script: swift build --arch $BUILD_ARCH --product tart
sign_script: codesign --sign - --entitlements Resources/tart-dev.entitlements --force .build/$BUILD_ARCH-apple-macosx/debug/tart
binary_artifacts:
path: .build/debug/tart
path: .build/$BUILD_ARCH-apple-macosx/debug/tart
task:
only_if: $CIRRUS_TAG == '' && ($CIRRUS_USER_PERMISSION == 'write' || $CIRRUS_USER_PERMISSION == 'admin')
@@ -59,9 +70,8 @@ task:
depends_on:
- lint
- build
use_compute_credits: $CIRRUS_USER_COLLABORATOR == 'true'
macos_instance:
image: ghcr.io/cirruslabs/macos-ventura-xcode:$XCODE_TAG
image: ghcr.io/cirruslabs/macos-sonoma-xcode:$XCODE_TAG
env:
MACOS_CERTIFICATE: ENCRYPTED[552b9d275d1c2bdbc1bff778b104a8f9a53cbd0d59344d4b7f6d0ca3c811a5cefb97bef9ba0ef31c219cb07bdacdd2c2]
AC_PASSWORD: ENCRYPTED[4a761023e7e06fe2eb350c8b6e8e7ca961af193cb9ba47605f25f1d353abc3142606f412e405be48fd897a78787ea8c2]
@@ -77,7 +87,7 @@ task:
- security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k password101 build.keychain
- xcrun notarytool store-credentials "notarytool" --apple-id "hello@cirruslabs.org" --team-id "9M2P8L4D89" --password $AC_PASSWORD
install_script:
- brew install go goreleaser/tap/goreleaser-pro getsentry/tools/sentry-cli
- brew install go goreleaser/tap/goreleaser-pro
- brew install mitchellh/gon/gon
info_script:
- security find-identity -v
@@ -95,9 +105,8 @@ task:
- lint
- test
- build
use_compute_credits: $CIRRUS_USER_COLLABORATOR == 'true'
macos_instance:
image: ghcr.io/cirruslabs/macos-ventura-xcode:$XCODE_TAG
image: ghcr.io/cirruslabs/macos-sonoma-xcode:$XCODE_TAG
env:
MACOS_CERTIFICATE: ENCRYPTED[552b9d275d1c2bdbc1bff778b104a8f9a53cbd0d59344d4b7f6d0ca3c811a5cefb97bef9ba0ef31c219cb07bdacdd2c2]
AC_PASSWORD: ENCRYPTED[4a761023e7e06fe2eb350c8b6e8e7ca961af193cb9ba47605f25f1d353abc3142606f412e405be48fd897a78787ea8c2]
@@ -142,7 +151,6 @@ task:
task:
name: Deploy Documentation
only_if: $CIRRUS_BRANCH == 'main'
use_compute_credits: $CIRRUS_USER_COLLABORATOR == 'true'
container:
image: ghcr.io/cirruslabs/mkdocs-material-insiders:latest
registry_config: ENCRYPTED[!cf1a0f25325aa75bad3ce6ebc890bc53eb0044c02efa70d8cefb83ba9766275a994b4831706c52630a0692b2fa9cfb9e!]
+17 -14
View File
@@ -3,27 +3,35 @@ project_name: tart
before:
hooks:
- .ci/set-version.sh
- swift build -c release --product tart
- swift build --arch x86_64 -c release --product tart
- swift build --arch arm64 -c release --product tart
- gon gon.hcl
- mkdir -p tart.app/Contents/MacOS
- cp .build/arm64-apple-macosx/release/tart tart.app/Contents/MacOS/
builds:
- builder: prebuilt
- id: tart
builder: prebuilt
goamd64: [v1]
goos:
- darwin
goarch:
- arm64
- amd64
binary: tart.app/Contents/MacOS/tart
prebuilt:
path: tart.app/Contents/MacOS/tart
path: '.build/{{- if eq .Arch "arm64" }}arm64{{- else }}x86_64{{ end }}-apple-macosx/release/tart'
archives:
- name_template: "{{ .ProjectName }}"
- name_template: "{{ .ProjectName }}-{{ .Arch }}"
files:
- src: Resources/embedded.provisionprofile
dst: tart.app/Contents
strip_parent: true
- src: Resources/Info.plist
dst: tart.app/Contents
strip_parent: true
- src: Resources/AppIcon.png
dst: tart.app/Contents/Resources
strip_parent: true
- LICENSE
release:
@@ -31,24 +39,19 @@ release:
brews:
- name: tart
tap:
repository:
owner: cirruslabs
name: homebrew-cli
caveats: See the GitHub repository for more information
homepage: https://github.com/cirruslabs/tart
license: "Fair Source"
description: Run macOS VMs on Apple Silicon
description: Run macOS and Linux VMs on Apple Hardware
skip_upload: auto
dependencies:
- "cirruslabs/cli/softnet"
install: |
libexec.install Dir["*"]
bin.write_exec_script "#{libexec}/tart.app/Contents/MacOS/tart"
generate_completions_from_executable(bin/"tart", "--generate-completion-script")
custom_block: |
depends_on :macos => :ventura
on_macos do
unless Hardware::CPU.arm?
odie "Tart only works on Apple Silicon!"
end
end
+11 -2
View File
@@ -23,8 +23,8 @@
"kind" : "remoteSourceControl",
"location" : "https://github.com/getsentry/sentry-cocoa",
"state" : {
"revision" : "d277532e1c8af813981ba01f591b15bbdd735615",
"version" : "8.8.0"
"revision" : "bf7bdd75e25556d0f97ad54fb804b4287863e106",
"version" : "8.22.4"
}
},
{
@@ -108,6 +108,15 @@
"version" : "0.50.6"
}
},
{
"identity" : "swiftradix",
"kind" : "remoteSourceControl",
"location" : "https://github.com/orchetect/SwiftRadix",
"state" : {
"revision" : "a52c37a4c213403f7377ae77b4c68451bcab8330",
"version" : "1.3.1"
}
},
{
"identity" : "texttable",
"kind" : "remoteSourceControl",
+3 -1
View File
@@ -18,9 +18,10 @@ let package = Package(
.package(url: "https://github.com/antlr/antlr4", branch: "dev"),
.package(url: "https://github.com/apple/swift-atomics.git", .upToNextMajor(from: "1.0.0")),
.package(url: "https://github.com/nicklockwood/SwiftFormat", from: "0.50.6"),
.package(url: "https://github.com/getsentry/sentry-cocoa", from: "8.8.0"),
.package(url: "https://github.com/getsentry/sentry-cocoa", from: "8.22.4"),
.package(url: "https://github.com/cfilipov/TextTable", branch: "master"),
.package(url: "https://github.com/sersoft-gmbh/swift-sysctl.git", from: "1.0.0"),
.package(url: "https://github.com/orchetect/SwiftRadix", from: "1.3.0")
],
targets: [
.executableTarget(name: "tart", dependencies: [
@@ -34,6 +35,7 @@ let package = Package(
.product(name: "Sentry", package: "sentry-cocoa"),
.product(name: "TextTable", package: "TextTable"),
.product(name: "Sysctl", package: "swift-sysctl"),
.product(name: "SwiftRadix", package: "SwiftRadix"),
], exclude: [
"OCI/Reference/Makefile",
"OCI/Reference/Reference.g4",
+16 -18
View File
@@ -8,7 +8,7 @@ Built by CI engineers for your automation needs. Here are some highlights of Tar
* Use Tart Packer Plugin to automate VM creation.
* Easily integrates with any CI system.
Tart powers [Cirrus Runners](https://tart.run/integrations/github-actions/?utm_source=github&utm_medium=referral)
Tart powers [Cirrus Runners](https://cirrus-runners.app/)
service — a drop-in replacement for the standard GitHub-hosted runners, offering 2-3 times better performance for a fraction of the price.
<p align="center">
@@ -17,21 +17,7 @@ service — a drop-in replacement for the standard GitHub-hosted runners, offeri
</a>
</p>
Tart is also adopted by several other automation services:
<p align="center">
<a href="https://cirrus-ci.org/guide/macOS/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/CirrusCI.png" height="65"/>
</a>
<a href="https://codemagic.io/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/Codemagic.png" height="65"/>
</a>
<a href="https://testingbot.com/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/TestingBot.png" height="65"/>
</a>
</p>
Many more companies are using Tart in their internal setups. Here are a few of them:
Many companies are using Tart in their internal setups. Here are a few of them:
<p align="center">
<a href="https://ahrefs.com/" target=_blank>
@@ -43,6 +29,9 @@ Many more companies are using Tart in their internal setups. Here are a few of t
<a href="https://mullvad.net/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/Mullvad.png" height="65"/>
</a>
<a href="https://shape.dk/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/shape.png" height="65"/>
</a>
<a href="https://suran.com/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/Suran.png" height="65"/>
</a>
@@ -52,6 +41,9 @@ Many more companies are using Tart in their internal setups. Here are a few of t
<a href="https://transloadit.com/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/Transloadit.png" height="65"/>
</a>
<a href="https://uphold.com/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/Uphold.png" height="65"/>
</a>
<a href="https://www.pitsdatarecovery.net/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/PITSGlobalDataRecoveryServices.png" height="65"/>
</a>
@@ -59,14 +51,20 @@ Many more companies are using Tart in their internal setups. Here are a few of t
**Note:** If your company or project is using Tart please consider [adding yourself to the list above](/Resources/Users/HowToAddYourself.md).
<p align="center">
<a href="https://aws.amazon.com/marketplace/pp/prodview-qczco34wlkdws?utm_source=github&utm_medium=referral" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/AWSMarkeplaceLogo.png" height="90"/>
</a>
</p>
## Usage
Try running a Tart VM on your Apple Silicon device running macOS 13.0 (Ventura) or later (will download a 25 GB image):
```bash
brew install cirruslabs/cli/tart
tart clone ghcr.io/cirruslabs/macos-ventura-base:latest ventura-base
tart run ventura-base
tart clone ghcr.io/cirruslabs/macos-sonoma-base:latest sonoma-base
tart run sonoma-base
```
Please check the [official documentation](https://tart.run) for more information and/or feel free to use [discussions](https://github.com/cirruslabs/tart/discussions)
Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

+23
View File
@@ -0,0 +1,23 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleName</key>
<string>tart</string>
<key>CFBundleIdentifier</key>
<string>org.cirruslabs.tart</string>
<key>CFBundleExecutable</key>
<string>tart</string>
<key>LSBackgroundOnly</key>
<string>1</string>
<key>CFBundleIconFiles</key>
<array>
<string>AppIcon.png</string>
</array>
<key>NSAppTransportSecurity</key>
<dict>
<key>NSAllowsArbitraryLoads</key>
<true/>
</dict>
</dict>
</plist>
Binary file not shown.

After

Width:  |  Height:  |  Size: 8.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.0 KiB

+2
View File
@@ -4,5 +4,7 @@
<dict>
<key>com.apple.security.virtualization</key>
<true/>
<key>com.apple.security.get-task-allow</key>
<true/>
</dict>
</plist>
+1 -1
View File
@@ -73,7 +73,7 @@ struct Clone: AsyncParsableCommand {
// APFS is doing copy-on-write so the above cloning operation (just copying files on disk)
// is not actually claiming new space until the VM is started and it writes something to disk.
// So once we clone the VM let's try to claim a little bit of space for the VM to run.
try Prune.reclaimIfNeeded(UInt64(sourceVM.sizeBytes()), sourceVM)
try Prune.reclaimIfNeeded(UInt64(sourceVM.allocatedSizeBytes()), sourceVM)
}, onCancel: {
try? FileManager.default.removeItem(at: tmpVMDir.baseURL)
})
+28 -12
View File
@@ -1,7 +1,8 @@
import ArgumentParser
import Dispatch
import SwiftUI
import Foundation
import SwiftUI
import Virtualization
struct Create: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Create a VM")
@@ -22,6 +23,11 @@ struct Create: AsyncParsableCommand {
if fromIPSW == nil && !linux {
throw ValidationError("Please specify either a --from-ipsw or --linux option!")
}
#if arch(x86_64)
if fromIPSW != nil {
throw ValidationError("Only Linux VMs are supported on Intel!")
}
#endif
}
func run() async throws {
@@ -32,19 +38,29 @@ struct Create: AsyncParsableCommand {
try tmpVMDirLock.lock()
try await withTaskCancellationHandler(operation: {
if let fromIPSW = fromIPSW {
let ipswURL: URL
#if arch(arm64)
if let fromIPSW = fromIPSW {
let ipswURL: URL
if fromIPSW == "latest" {
ipswURL = try await VM.latestIPSWURL()
} else if fromIPSW.starts(with: "http://") || fromIPSW.starts(with: "https://") {
ipswURL = URL(string: fromIPSW)!
} else {
ipswURL = URL(fileURLWithPath: fromIPSW)
if fromIPSW == "latest" {
defaultLogger.appendNewLine("Looking up the latest supported IPSW...")
let image = try await withCheckedThrowingContinuation { continuation in
VZMacOSRestoreImage.fetchLatestSupported() { result in
continuation.resume(with: result)
}
}
ipswURL = image.url
} else if fromIPSW.starts(with: "http://") || fromIPSW.starts(with: "https://") {
ipswURL = URL(string: fromIPSW)!
} else {
ipswURL = URL(fileURLWithPath: NSString(string: fromIPSW).expandingTildeInPath)
}
_ = try await VM(vmDir: tmpVMDir, ipswURL: ipswURL, diskSizeGB: diskSize)
}
_ = try await VM(vmDir: tmpVMDir, ipswURL: ipswURL, diskSizeGB: diskSize)
}
#endif
if linux {
_ = try await VM.linux(vmDir: tmpVMDir, diskSizeGB: diskSize)
+22
View File
@@ -0,0 +1,22 @@
import ArgumentParser
import Foundation
import SystemConfiguration
struct FQN: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Get a fully-qualified VM name", shouldDisplay: false)
@Argument(help: "VM name")
var name: String
func run() async throws {
if var remoteName = try? RemoteName(name) {
let digest = try VMStorageOCI().digest(remoteName)
remoteName.reference = Reference(digest: digest)
print(remoteName)
} else {
print(name)
}
}
}
+3 -2
View File
@@ -2,9 +2,11 @@ import ArgumentParser
import Foundation
fileprivate struct VMInfo: Encodable {
let OS: OS
let CPU: Int
let Memory: UInt64
let Disk: Int
let Size: String
let Display: String
let Running: Bool
let State: String
@@ -22,10 +24,9 @@ struct Get: AsyncParsableCommand {
func run() async throws {
let vmDir = try VMStorageLocal().open(name)
let vmConfig = try VMConfig(fromURL: vmDir.configURL)
let diskSizeInGb = try vmDir.sizeGB()
let memorySizeInMb = vmConfig.memorySize / 1024 / 1024
let info = VMInfo(CPU: vmConfig.cpuCount, Memory: memorySizeInMb, Disk: diskSizeInGb,
let info = VMInfo(OS: vmConfig.os, CPU: vmConfig.cpuCount, Memory: memorySizeInMb, Disk: try vmDir.sizeGB(), Size: String(format: "%.3f", Float(try vmDir.allocatedSizeBytes()) / 1000 / 1000 / 1000),
Display: vmConfig.display.description, Running: try vmDir.running(), State: try vmDir.state())
print(format.renderSingle(info))
}
+3 -2
View File
@@ -5,6 +5,7 @@ import SwiftUI
fileprivate struct VMInfo: Encodable {
let Source: String
let Name: String
let Disk: Int
let Size: Int
let Running: Bool
let State: String
@@ -37,13 +38,13 @@ struct List: AsyncParsableCommand {
if source == nil || source == "local" {
infos += sortedInfos(try VMStorageLocal().list().map { (name, vmDir) in
try VMInfo(Source: "local", Name: name, Size: vmDir.sizeGB(), Running: vmDir.running(), State: vmDir.state())
try VMInfo(Source: "local", Name: name, Disk: vmDir.sizeGB(), Size: vmDir.allocatedSizeGB(), Running: vmDir.running(), State: vmDir.state())
})
}
if source == nil || source == "oci" {
infos += sortedInfos(try VMStorageOCI().list().map { (name, vmDir, _) in
try VMInfo(Source: "oci", Name: name, Size: vmDir.sizeGB(), Running: vmDir.running(), State: vmDir.state())
try VMInfo(Source: "oci", Name: name, Disk: vmDir.sizeGB(), Size: vmDir.allocatedSizeGB(), Running: vmDir.running(), State: vmDir.state())
})
}
+4 -4
View File
@@ -89,7 +89,7 @@ struct Prune: AsyncParsableCommand {
var prunablesToDelete: [Prunable] = []
for prunable in prunables {
let prunableSizeBytes = UInt64(try prunable.sizeBytes())
let prunableSizeBytes = UInt64(try prunable.allocatedSizeBytes())
if prunableSizeBytes <= spaceBudgetBytes {
// Don't mark for deletion as
@@ -158,7 +158,7 @@ struct Prune: AsyncParsableCommand {
.sorted { try $0.accessDate() < $1.accessDate() }
// Does it even make sense to start?
let cacheUsedBytes = try prunables.map { try $0.sizeBytes() }.reduce(0, +)
let cacheUsedBytes = try prunables.map { try $0.allocatedSizeBytes() }.reduce(0, +)
if cacheUsedBytes < reclaimBytes {
return
}
@@ -177,9 +177,9 @@ struct Prune: AsyncParsableCommand {
continue
}
try SentrySDK.span?.setData(value: prunable.sizeBytes(), key: prunable.url.path)
try SentrySDK.span?.setData(value: prunable.allocatedSizeBytes(), key: prunable.url.path)
cacheReclaimedBytes += try prunable.sizeBytes()
cacheReclaimedBytes += try prunable.allocatedSizeBytes()
try prunable.delete()
}
+1 -1
View File
@@ -24,7 +24,7 @@ struct Rename: AsyncParsableCommand {
}
if localStorage.exists(newName) {
throw ValidationError("failed to rename VM \(name), target VM \(name) already exists, delete it first!")
throw ValidationError("failed to rename VM \(name), target VM \(newName) already exists, delete it first!")
}
try localStorage.rename(name, newName)
+359 -162
View File
@@ -1,5 +1,6 @@
import ArgumentParser
import Cocoa
import Darwin
import Dispatch
import SwiftUI
import Virtualization
@@ -32,58 +33,77 @@ struct Run: AsyncParsableCommand {
))
var serialPath: String?
@Flag(help: "Force open a UI window, even when VNC is enabled.")
@Flag(help: ArgumentHelp("Force open a UI window, even when VNC is enabled.", visibility: .private))
var graphics: Bool = false
@Flag(help: "Boot into recovery mode")
@Flag(help: "Disable audio pass-through to host.")
var noAudio: Bool = false
#if arch(arm64)
@Flag(help: "Boot into recovery mode")
#endif
var recovery: Bool = false
@Flag(help: ArgumentHelp(
"Use screen sharing instead of the built-in UI.",
discussion: "Useful since Screen Sharing supports copy/paste, drag and drop, etc.\n"
+ "Note that Remote Login option should be enabled inside the VM."))
#if arch(arm64)
@Flag(help: ArgumentHelp(
"Use screen sharing instead of the built-in UI.",
discussion: "Useful since Screen Sharing supports copy/paste, drag and drop, etc.\n"
+ "Note that Remote Login option should be enabled inside the VM."))
#endif
var vnc: Bool = false
@Flag(help: ArgumentHelp(
"Use Virtualization.Framework's VNC server instead of the build-in UI.",
discussion: "Useful since this type of VNC is available in recovery mode and in macOS installation.\n"
+ "Note that this feature is experimental and there may be bugs present when using VNC."))
#if arch(arm64)
@Flag(help: ArgumentHelp(
"Use Virtualization.Framework's VNC server instead of the build-in UI.",
discussion: "Useful since this type of VNC is available in recovery mode and in macOS installation.\n"
+ "Note that this feature is experimental and there may be bugs present when using VNC."))
#endif
var vncExperimental: Bool = false
@Option(help: ArgumentHelp("""
Additional disk attachments with an optional read-only specifier\n(e.g. --disk=\"disk.bin\" --disk=\"ubuntu.iso:ro\")
Additional disk attachments with an optional read-only specifier\n(e.g. --disk=\"disk.bin\" --disk=\"ubuntu.iso:ro\" --disk=\"/dev/disk0\")
""", discussion: """
Can be either a disk image file or a block device like a local SSD on AWS EC2 Mac instances.
Learn how to create a disk image using Disk Utility here:
https://support.apple.com/en-gb/guide/disk-utility/dskutl11888/mac
To work with block devices 'tart' binary must be executed as root which affects locating Tart VMs.
To workaround this issue pass TART_HOME explicitly:
sudo TART_HOME="$HOME/.tart" tart run sonoma --disk=/dev/disk0
""", valueName: "path[:ro]"))
var disk: [String] = []
@Option(name: [.customLong("rosetta")], help: ArgumentHelp(
"Attaches a Rosetta share to the guest Linux VM with a specific tag (e.g. --rosetta=\"rosetta\")",
discussion: """
Requires host to be macOS 13.0 (Ventura) with Rosetta installed. The latter can be done
by running "softwareupdate --install-rosetta" (without quotes) in the Terminal.app.
#if arch(arm64)
@Option(name: [.customLong("rosetta")], help: ArgumentHelp(
"Attaches a Rosetta share to the guest Linux VM with a specific tag (e.g. --rosetta=\"rosetta\")",
discussion: """
Requires host to be macOS 13.0 (Ventura) with Rosetta installed. The latter can be done
by running "softwareupdate --install-rosetta" (without quotes) in the Terminal.app.
Note that you also have to configure Rosetta in the guest Linux VM by following the
steps from "Mount the Shared Directory and Register Rosetta" section here:
https://developer.apple.com/documentation/virtualization/running_intel_binaries_in_linux_vms_with_rosetta#3978496
""",
valueName: "tag"
))
Note that you also have to configure Rosetta in the guest Linux VM by following the
steps from "Mount the Shared Directory and Register Rosetta" section here:
https://developer.apple.com/documentation/virtualization/running_intel_binaries_in_linux_vms_with_rosetta#3978496
""",
valueName: "tag"
))
#endif
var rosettaTag: String?
@Option(help: ArgumentHelp("""
Additional directory shares with an optional read-only specifier\n(e.g. --dir=\"~/src/build\" or --dir=\"~/src/sources:ro\")
""", discussion: """
Requires host to be macOS 13.0 (Ventura) or newer.
A shared directory is automatically mounted to "/Volumes/My Shared Files" directory on macOS,
while on Linux you have to do it manually: "mount -t virtiofs com.apple.virtio-fs.automount /mount/point".
For macOS guests, they must be running macOS 13.0 (Ventura) or newer.
@Option(help: ArgumentHelp("Additional directory shares with an optional read-only and mount tag options (e.g. --dir=\"~/src/build\" or --dir=\"~/src/sources:ro\")", discussion: """
Requires host to be macOS 13.0 (Ventura) or newer. macOS guests must be running macOS 13.0 (Ventura) or newer too.
In case of passing multiple directories it is required to prefix them with names e.g. --dir=\"build:~/src/build\" --dir=\"sources:~/src/sources:ro\"
These names will be used as directory names under the mounting point inside guests. For the example above it will be
"/Volumes/My Shared Files/build" and "/Volumes/My Shared Files/sources" respectively.
""", valueName: "[name:]path[:ro]"))
Options are comma-separated and are as follows:
* ro — mount this directory share in read-only mode instead of the default read-write (e.g. --dir=\"~/src/sources:ro\")
* tag=<TAG> — by default, the \"com.apple.virtio-fs.automount\" mount tag is used for all directory shares. On macOS, this causes the directories to be automatically mounted to "/Volumes/My Shared Files" directory. On Linux, you have to do it manually: "mount -t virtiofs com.apple.virtio-fs.automount /mount/point".
Mount tag can be overridden by appending tag property to the directory share (e.g. --dir=\"~/src/build:tag=build\" or --dir=\"~/src/build:ro,tag=build\"). Then it can be mounted via "mount_virtiofs build ~/build" inside guest macOS and "mount -t virtiofs build ~/build" inside guest Linux.
In case of passing multiple directories per mount tag it is required to prefix them with names e.g. --dir=\"build:~/src/build\" --dir=\"sources:~/src/sources:ro\". These names will be used as directory names under the mounting point inside guests. For the example above it will be "/Volumes/My Shared Files/build" and "/Volumes/My Shared Files/sources" respectively.
""", valueName: "[name:]path[:options]"))
var dir: [String] = []
@Option(help: ArgumentHelp("""
@@ -97,27 +117,63 @@ struct Run: AsyncParsableCommand {
discussion: "Learn how to configure Softnet for use with Tart here: https://github.com/cirruslabs/softnet"))
var netSoftnet: Bool = false
@Flag(help: ArgumentHelp("Disables audio and entropy devices and switches to only Mac-specific input devices.", discussion: "Useful for running a VM that can be suspended via \"tart suspend\"."))
@Option(help: ArgumentHelp("Comma-separated list of CIDRs to allow the traffic to when using Softnet isolation\n(e.g. --net-softnet-allow=192.168.0.0/24)", valueName: "comma-separated CIDRs"))
var netSoftnetAllow: String?
@Flag(help: ArgumentHelp("Restrict network access to the host-only network"))
var netHost: Bool = false
#if arch(arm64)
@Flag(help: ArgumentHelp("Disables audio and entropy devices and switches to only Mac-specific input devices.", discussion: "Useful for running a VM that can be suspended via \"tart suspend\"."))
#endif
var suspendable: Bool = false
#if arch(arm64)
@Flag(help: ArgumentHelp("Whether system hot keys should be sent to the guest instead of the host",
discussion: "If enabled then system hot keys like Cmd+Tab will be sent to the guest instead of the host."))
#endif
var captureSystemKeys: Bool = false
mutating func validate() throws {
if vnc && vncExperimental {
throw ValidationError("--vnc and --vnc-experimental are mutually exclusive")
}
if netBridged.count > 0 && netSoftnet {
throw ValidationError("--net-bridged and --net-softnet are mutually exclusive")
// check that not more than one network option is specified
var netFlags = 0
if netBridged.count > 0 { netFlags += 1 }
if netSoftnet { netFlags += 1 }
if netHost { netFlags += 1 }
if netFlags > 1 {
throw ValidationError("--net-bridged, --net-softnet and --net-host are mutually exclusive")
}
if graphics && noGraphics {
throw ValidationError("--graphics and --no-graphics are mutually exclusive")
}
if (noGraphics || vnc || vncExperimental) && captureSystemKeys {
throw ValidationError("--captures-system-keys can only be used with the default VM view")
}
let localStorage = VMStorageLocal()
let vmDir = try localStorage.open(name)
if try vmDir.state() == "suspended" {
suspendable = true
}
if suspendable {
if dir.count > 0 {
throw ValidationError("Suspending VMs with shared directories is not supported")
}
}
for disk in disk {
if disk.hasSuffix("-amd64.iso") {
throw ValidationError("Seems you have a disk targeting x86 architecture (hence amd64 in the name). Please use an 'arm64' version of the disk.")
}
}
}
@MainActor
@@ -140,26 +196,12 @@ struct Run: AsyncParsableCommand {
}
}
if netSoftnet && isInteractiveSession() {
if (netSoftnet || netHost) && isInteractiveSession() {
try Softnet.configureSUIDBitIfNeeded()
}
let additionalDiskAttachments = try additionalDiskAttachments()
// Error out if the disk is locked by the host (e.g. it was mounted in Finder),
// see https://github.com/cirruslabs/tart/issues/323 for more details.
for additionalDiskAttachment in additionalDiskAttachments {
// Read-only attachments do not seem to acquire the lock
if additionalDiskAttachment.isReadOnly {
continue
}
if try !FileLock(lockURL: additionalDiskAttachment.url).trylock() {
throw RuntimeError.DiskAlreadyInUse("disk \(additionalDiskAttachment.url.path) seems to be already in use, "
+ "unmount it first in Finder")
}
}
var serialPorts: [VZSerialPortConfiguration] = []
if serial {
let tty_fd = createPTY()
@@ -181,10 +223,11 @@ struct Run: AsyncParsableCommand {
vm = try VM(
vmDir: vmDir,
network: userSpecifiedNetwork(vmDir: vmDir) ?? NetworkShared(),
additionalDiskAttachments: additionalDiskAttachments,
additionalStorageDevices: additionalDiskAttachments,
directorySharingDevices: directoryShares() + rosettaDirectoryShare(),
serialPorts: serialPorts,
suspendable: suspendable
suspendable: suspendable,
audio: !noAudio
)
let vncImpl: VNC? = try {
@@ -210,7 +253,7 @@ struct Run: AsyncParsableCommand {
// configuration file, otherwise we will loose the lock.
//
// [1]: https://man.openbsd.org/fcntl
let lock = try PIDLock(lockURL: vmDir.configURL)
let lock = try vmDir.lock()
if try !lock.trylock() {
throw RuntimeError.VMAlreadyRunning("VM \"\(name)\" is already running!")
}
@@ -222,15 +265,17 @@ struct Run: AsyncParsableCommand {
do {
var resume = false
if #available(macOS 14, *) {
if FileManager.default.fileExists(atPath: vmDir.stateURL.path) {
print("restoring VM state from a snapshot...")
try await vm!.virtualMachine.restoreMachineStateFrom(url: vmDir.stateURL)
try FileManager.default.removeItem(at: vmDir.stateURL)
resume = true
print("resuming VM...")
#if arch(arm64)
if #available(macOS 14, *) {
if FileManager.default.fileExists(atPath: vmDir.stateURL.path) {
print("restoring VM state from a snapshot...")
try await vm!.virtualMachine.restoreMachineStateFrom(url: vmDir.stateURL)
try FileManager.default.removeItem(at: vmDir.stateURL)
resume = true
print("resuming VM...")
}
}
}
#endif
try await vm!.start(recovery: recovery, resume: resume)
@@ -276,23 +321,25 @@ struct Run: AsyncParsableCommand {
sigusr1Src.setEventHandler {
Task {
do {
if #available(macOS 14, *) {
try vm!.configuration.validateSaveRestoreSupport()
#if arch(arm64)
if #available(macOS 14, *) {
try vm!.configuration.validateSaveRestoreSupport()
print("pausing VM to take a snapshot...")
try await vm!.virtualMachine.pause()
print("pausing VM to take a snapshot...")
try await vm!.virtualMachine.pause()
print("creating a snapshot...")
try await vm!.virtualMachine.saveMachineStateTo(url: vmDir.stateURL)
print("creating a snapshot...")
try await vm!.virtualMachine.saveMachineStateTo(url: vmDir.stateURL)
print("snapshot created successfully! shutting down the VM...")
print("snapshot created successfully! shutting down the VM...")
task.cancel()
} else {
print(RuntimeError.SuspendFailed("this functionality is only supported on macOS 14 (Sonoma) or newer"))
task.cancel()
} else {
print(RuntimeError.SuspendFailed("this functionality is only supported on macOS 14 (Sonoma) or newer"))
Foundation.exit(1)
}
Foundation.exit(1)
}
#endif
} catch (let e) {
print(RuntimeError.SuspendFailed(e.localizedDescription))
@@ -304,9 +351,11 @@ struct Run: AsyncParsableCommand {
let useVNCWithoutGraphics = (vnc || vncExperimental) && !graphics
if noGraphics || useVNCWithoutGraphics {
dispatchMain()
// enter the main even loop, without bringing up any UI,
// and just wait for the VM to exit.
NSApplication.shared.run()
} else {
runUI(suspendable)
runUI(suspendable, captureSystemKeys)
}
}
@@ -328,10 +377,21 @@ struct Run: AsyncParsableCommand {
if netSoftnet {
let config = try VMConfig.init(fromURL: vmDir.configURL)
return try Softnet(vmMACAddress: config.macAddress.string)
var extraArguments: [String] = []
if let netSoftnetAllow = netSoftnetAllow {
extraArguments += ["--allow", netSoftnetAllow]
}
return try Softnet(vmMACAddress: config.macAddress.string, extraArguments: extraArguments)
}
if netBridged.count > 0 {
if netHost {
let config = try VMConfig.init(fromURL: vmDir.configURL)
return try Softnet(vmMACAddress: config.macAddress.string, extraArguments: ["--vm-net-type", "host"])
}
if netBridged.count > 0 {
func findBridgedInterface(_ name: String) throws -> VZBridgedNetworkInterface {
let interface = VZBridgedNetworkInterface.networkInterfaces.first { interface in
interface.identifier == name || interface.localizedDisplayName == name
@@ -361,22 +421,43 @@ struct Run: AsyncParsableCommand {
}
}
func additionalDiskAttachments() throws -> [VZDiskImageStorageDeviceAttachment] {
var result: [VZDiskImageStorageDeviceAttachment] = []
func additionalDiskAttachments() throws -> [VZStorageDeviceConfiguration] {
var result: [VZStorageDeviceConfiguration] = []
let readOnlySuffix = ":ro"
let expandedDiskPaths = disk.map { NSString(string:$0).expandingTildeInPath }
for rawDisk in expandedDiskPaths {
if rawDisk.hasSuffix(readOnlySuffix) {
result.append(try VZDiskImageStorageDeviceAttachment(
url: URL(fileURLWithPath: String(rawDisk.prefix(rawDisk.count - readOnlySuffix.count))),
readOnly: true
))
let diskReadOnly = rawDisk.hasSuffix(readOnlySuffix)
let diskPath = diskReadOnly ? String(rawDisk.prefix(rawDisk.count - readOnlySuffix.count)) : rawDisk
let diskURL = URL(fileURLWithPath: diskPath)
// check if `diskPath` is a block device or a directory
if pathHasMode(diskPath, mode: S_IFBLK) || pathHasMode(diskPath, mode: S_IFDIR) {
print("Using block device\n")
guard #available(macOS 14, *) else {
throw UnsupportedOSError("attaching block devices", "are")
}
let fileHandle = FileHandle(forUpdatingAtPath: diskPath)
guard fileHandle != nil else {
if ProcessInfo.processInfo.userName != "root" {
throw RuntimeError.VMConfigurationError("need to run as root to work with block devices")
}
throw RuntimeError.VMConfigurationError("block device \(diskURL.url.path) seems to be already in use, unmount it first via 'diskutil unmount'")
}
let attachment = try VZDiskBlockDeviceStorageDeviceAttachment(fileHandle: fileHandle!, readOnly: diskReadOnly, synchronizationMode: .full)
result.append(VZVirtioBlockDeviceConfiguration(attachment: attachment))
} else {
result.append(try VZDiskImageStorageDeviceAttachment(
url: URL(fileURLWithPath: rawDisk),
readOnly: false
))
// Error out if the disk is locked by the host (e.g. it was mounted in Finder),
// see https://github.com/cirruslabs/tart/issues/323 for more details.
if try !diskReadOnly && !FileLock(lockURL: diskURL).trylock() {
throw RuntimeError.DiskAlreadyInUse("disk \(diskURL.url.path) seems to be already in use, unmount it first in Finder")
}
let diskImageAttachment = try VZDiskImageStorageDeviceAttachment(
url: diskURL,
readOnly: diskReadOnly
)
result.append(VZVirtioBlockDeviceConfiguration(attachment: diskImageAttachment))
}
}
@@ -392,76 +473,81 @@ struct Run: AsyncParsableCommand {
throw UnsupportedOSError("directory sharing", "is")
}
var directoryShares: [DirectoryShare] = []
var allDirectoryShares: [DirectoryShare] = []
var allNamedShares = true
for rawDir in dir {
let directoryShare = try DirectoryShare(parseFrom: rawDir)
if (directoryShare.name == nil) {
allNamedShares = false
allDirectoryShares.append(try DirectoryShare(parseFrom: rawDir))
}
return try Dictionary(grouping: allDirectoryShares, by: {$0.mountTag}).map { mountTag, directoryShares in
let sharingDevice = VZVirtioFileSystemDeviceConfiguration(tag: mountTag)
var allNamedShares = true
for directoryShare in directoryShares {
if directoryShare.name == nil {
allNamedShares = false
}
}
directoryShares.append(directoryShare)
if directoryShares.count == 1 && directoryShares.first!.name == nil {
let directoryShare = directoryShares.first!
let singleDirectoryShare = VZSingleDirectoryShare(directory: try directoryShare.createConfiguration())
sharingDevice.share = singleDirectoryShare
} else if !allNamedShares {
throw ValidationError("invalid --dir syntax: for multiple directory shares each one of them should be named")
} else {
var directories: [String : VZSharedDirectory] = Dictionary()
try directoryShares.forEach { directories[$0.name!] = try $0.createConfiguration() }
sharingDevice.share = VZMultipleDirectoryShare(directories: directories)
}
return sharingDevice
}
let automountTag = VZVirtioFileSystemDeviceConfiguration.macOSGuestAutomountTag
let sharingDevice = VZVirtioFileSystemDeviceConfiguration(tag: automountTag)
if allNamedShares {
var directories: [String : VZSharedDirectory] = Dictionary()
directoryShares.forEach { directories[$0.name!] = VZSharedDirectory(url: $0.path, readOnly: $0.readOnly) }
sharingDevice.share = VZMultipleDirectoryShare(directories: directories)
} else if dir.count > 1 {
throw ValidationError("invalid --dir syntax: for multiple directory shares each one of them should be named")
} else if dir.count == 1 {
let directoryShare = directoryShares.first!
let singleDirectoryShare = VZSingleDirectoryShare(directory: VZSharedDirectory(url: directoryShare.path, readOnly: directoryShare.readOnly))
sharingDevice.share = singleDirectoryShare
}
return [sharingDevice]
}
private func rosettaDirectoryShare() throws -> [VZDirectorySharingDeviceConfiguration] {
guard let rosettaTag = rosettaTag else {
return []
}
#if arch(arm64)
guard #available(macOS 13, *) else {
throw UnsupportedOSError("Rosetta directory share", "is")
}
guard #available(macOS 13, *) else {
throw UnsupportedOSError("Rosetta directory share", "is")
}
switch VZLinuxRosettaDirectoryShare.availability {
case .notInstalled:
throw UnsupportedOSError("Rosetta directory share", "is", "that have Rosetta installed")
case .notSupported:
throw UnsupportedOSError("Rosetta directory share", "is", "running Apple silicon")
default:
break
}
switch VZLinuxRosettaDirectoryShare.availability {
case .notInstalled:
throw UnsupportedOSError("Rosetta directory share", "is", "that have Rosetta installed")
case .notSupported:
throw UnsupportedOSError("Rosetta directory share", "is", "running Apple silicon")
default:
break
}
try VZVirtioFileSystemDeviceConfiguration.validateTag(rosettaTag)
let device = VZVirtioFileSystemDeviceConfiguration(tag: rosettaTag)
device.share = try VZLinuxRosettaDirectoryShare()
try VZVirtioFileSystemDeviceConfiguration.validateTag(rosettaTag)
let device = VZVirtioFileSystemDeviceConfiguration(tag: rosettaTag)
device.share = try VZLinuxRosettaDirectoryShare()
return [device]
return [device]
#elseif arch(x86_64)
// there is no Rosetta on Intel
return []
#endif
}
private func runUI(_ suspendable: Bool) {
private func runUI(_ suspendable: Bool, _ captureSystemKeys: Bool) {
let nsApp = NSApplication.shared
nsApp.setActivationPolicy(.regular)
nsApp.activate(ignoringOtherApps: true)
nsApp.applicationIconImage = NSImage(data: AppIconData)
struct MainApp: App {
static var disappearSignal: Int32 = SIGINT
static var capturesSystemKeys: Bool = false
@NSApplicationDelegateAdaptor private var appDelegate: MinimalMenuAppDelegate
var body: some Scene {
WindowGroup(vm!.name) {
Group {
VMView(vm: vm!).onAppear {
VMView(vm: vm!, capturesSystemKeys: MainApp.capturesSystemKeys).onAppear {
NSWindow.allowsAutomaticWindowTabbing = false
}.onDisappear {
let ret = kill(getpid(), MainApp.disappearSignal)
@@ -511,6 +597,7 @@ struct Run: AsyncParsableCommand {
}
MainApp.disappearSignal = suspendable ? SIGUSR1 : SIGINT
MainApp.capturesSystemKeys = captureSystemKeys
MainApp.main()
}
}
@@ -560,18 +647,19 @@ struct VMView: NSViewRepresentable {
typealias NSViewType = VZVirtualMachineView
@ObservedObject var vm: VM
var capturesSystemKeys: Bool
func makeNSView(context: Context) -> NSViewType {
let machineView = VZVirtualMachineView()
// Do not capture system keys so that shortcuts like
// Shift-Command-4 + Space (capture a screenshot of window)
// work on the host instead of the guest
machineView.capturesSystemKeys = false
machineView.capturesSystemKeys = capturesSystemKeys
// Enable automatic display reconfiguration
// for guests that support it
if #available(macOS 14.0, *) {
//
// This is disabled for Linux because of poor HiDPI
// support, which manifests in fonts being too small
if #available(macOS 14.0, *), vm.config.os != .linux {
machineView.automaticallyReconfiguresDisplay = true
}
@@ -587,38 +675,147 @@ struct DirectoryShare {
let name: String?
let path: URL
let readOnly: Bool
let mountTag: String
init(parseFrom: String) throws {
let splits = parseFrom.split(maxSplits: 2) { $0 == ":" }
var parseFrom = parseFrom
if splits.count == 3 {
if splits[2] == "ro" {
readOnly = true
} else {
throw ValidationError("invalid --dir syntax: optional read-only specifier can only be \"ro\"")
}
name = String(splits[0])
path = String(splits[1]).toFilePathURL()
} else if splits.count == 2 {
if splits[1] == "ro" {
name = nil
path = String(splits[0]).toFilePathURL()
readOnly = true
} else {
name = String(splits[0])
path = String(splits[1]).toFilePathURL()
readOnly = false
}
} else {
name = nil
path = String(splits[0]).toFilePathURL()
readOnly = false
// Consume options
(self.readOnly, self.mountTag, parseFrom) = Self.parseOptions(parseFrom)
// Special case for URLs
if parseFrom.hasPrefix("http:") || parseFrom.hasPrefix("https:") {
self.name = nil
self.path = URL(string: parseFrom)!
return
}
let arguments = parseFrom.split(separator: ":", maxSplits: 1)
if arguments.count == 2 {
self.name = String(arguments[0])
self.path = String(arguments[1]).toRemoteOrLocalURL()
} else {
self.name = nil
self.path = String(arguments[0]).toRemoteOrLocalURL()
}
}
static func parseOptions(_ parseFrom: String) -> (Bool, String, String) {
var arguments = parseFrom.split(separator: ":")
let options = arguments.last!.split(separator: ",")
var readOnly: Bool = false
var mountTag: String = VZVirtioFileSystemDeviceConfiguration.macOSGuestAutomountTag
var found: Bool = false
for option in options {
switch true {
case option == "ro":
readOnly = true
found = true
case option.hasPrefix("tag="):
mountTag = String(option.dropFirst(4))
found = true
default:
continue
}
}
if found {
arguments.removeLast()
}
return (readOnly, mountTag, arguments.joined(separator: ":"))
}
func createConfiguration() throws -> VZSharedDirectory {
if (path.isFileURL) {
return VZSharedDirectory(url: path, readOnly: readOnly)
}
let urlCache = URLCache(memoryCapacity: 0, diskCapacity: 1 * 1024 * 1024 * 1024)
let archiveRequest = URLRequest(url: path, cachePolicy: .returnCacheDataElseLoad)
var response: CachedURLResponse? = urlCache.cachedResponse(for: archiveRequest)
if (response == nil || response?.data.isEmpty == true) {
print("Downloading \(path)...")
// download and unarchive remote directories if needed here
// use old school API to prevent deadlocks since we are running via MainActor
let downloadSemaphore = DispatchSemaphore(value: 0)
Task {
do {
let (archiveData, archiveResponse) = try await URLSession.shared.data(for: archiveRequest)
if archiveData.isEmpty {
print("Remote archive is empty!")
} else {
urlCache.storeCachedResponse(CachedURLResponse(response: archiveResponse, data: archiveData, storagePolicy: .allowed), for: archiveRequest)
print("Cached for future invocations!")
}
} catch {
print("Download failed: \(error)")
}
downloadSemaphore.signal()
}
downloadSemaphore.wait()
response = urlCache.cachedResponse(for: archiveRequest)
} else {
print("Using cached archive for \(path)...")
}
if (response == nil) {
throw ValidationError("Failed to fetch a remote archive!")
}
let temporaryLocation = try Config().tartTmpDir.appendingPathComponent(UUID().uuidString + ".volume")
try FileManager.default.createDirectory(atPath: temporaryLocation.path, withIntermediateDirectories: true)
let lock = try FileLock(lockURL: temporaryLocation)
try lock.lock()
guard let executableURL = resolveBinaryPath("tar") else {
throw ValidationError("tar not found in PATH")
}
let process = Process.init()
process.executableURL = executableURL
process.currentDirectoryURL = temporaryLocation
process.arguments = ["-xz"]
let inPipe = Pipe()
process.standardInput = inPipe
process.launch()
inPipe.fileHandleForWriting.write(response!.data)
try inPipe.fileHandleForWriting.close()
process.waitUntilExit()
if !(process.terminationReason == .exit && process.terminationStatus == 0) {
throw ValidationError("Unarchiving failed!")
}
print("Unarchived into a temporary directory!")
return VZSharedDirectory(url: temporaryLocation, readOnly: readOnly)
}
}
extension String {
func toFilePathURL() -> URL {
URL(fileURLWithPath: NSString(string: self).expandingTildeInPath)
func toRemoteOrLocalURL() -> URL {
if (starts(with: "https://") || starts(with: "https://")) {
URL(string: self)!
} else {
URL(fileURLWithPath: NSString(string: self).expandingTildeInPath)
}
}
}
func pathHasMode(_ path: String, mode: mode_t) -> Bool {
var st = stat()
let statRes = stat(path, &st)
guard statRes != -1 else {
return false
}
return (Int32(st.st_mode) & Int32(mode)) == Int32(mode)
}
+12 -1
View File
@@ -16,7 +16,18 @@ struct Set: AsyncParsableCommand {
@Option(help: "VM display resolution in a format of <width>x<height>. For example, 1200x800")
var display: VMDisplayConfig?
@Option(help: .hidden)
@Option(help: ArgumentHelp("Resize the VMs disk to the specified size in GB (note that the disk size can only be increased to avoid losing data)",
discussion: """
Disk resizing works on most cloud-ready Linux distributions out-of-the box (e.g. Ubuntu Cloud Images
have the \"cloud-initramfs-growroot\" package installed that runs on boot) and on the rest of the
distributions by running the \"growpart\" or \"resize2fs\" commands.
For macOS, however, things are a bit more complicated: you need to remove the recovery partition
first and then run various \"diskutil\" commands, see Tart's packer plugin source code for more
details[1].
[1]: https://github.com/cirruslabs/packer-plugin-tart/blob/main/builder/tart/step_disk_resize.go
"""))
var diskSize: UInt16?
func run() async throws {
+2 -2
View File
@@ -29,12 +29,12 @@ struct Stop: AsyncParsableCommand {
}
func stopRunning(_ vmDir: VMDirectory) async throws {
let lock = try PIDLock(lockURL: vmDir.configURL)
let lock = try vmDir.lock()
// Find the VM's PID
var pid = try lock.pid()
if pid == 0 {
throw RuntimeError.VMNotRunning("VM \"\(name)\" is not running")
throw RuntimeError.VMNotRunning(name)
}
// Try to gracefully terminate the VM
+1 -1
View File
@@ -11,7 +11,7 @@ struct Suspend: AsyncParsableCommand {
func run() async throws {
let vmDir = try VMStorageLocal().open(name)
let lock = try PIDLock(lockURL: vmDir.configURL)
let lock = try vmDir.lock()
// Find the VM's PID
var pid = try lock.pid()
@@ -41,13 +41,18 @@ class DockerConfigCredentialsProvider: CredentialsProvider {
process.waitUntilExit()
let outputData = try outPipe.fileHandleForReading.readToEnd()
if !(process.terminationReason == .exit && process.terminationStatus == 0) {
if let outputData = outputData {
print(String(decoding: outputData, as: UTF8.self))
}
throw CredentialsProviderError.Failed(message: "Docker helper failed!")
}
if outputData == nil || outputData?.count == 0 {
throw CredentialsProviderError.Failed(message: "Docker helper output is empty!")
}
let getOutput = try JSONDecoder().decode(
DockerGetOutput.self, from: outPipe.fileHandleForReading.readDataToEndOfFile()
)
let getOutput = try JSONDecoder().decode(DockerGetOutput.self, from: outputData!)
return (getOutput.Username, getOutput.Secret)
}
@@ -13,7 +13,7 @@ class StdinCredentials {
return (user, password)
}
private static func readStdinCredential(name: String, prompt: String, maxCharacters: Int = 255, isSensitive: Bool) throws -> String {
private static func readStdinCredential(name: String, prompt: String, maxCharacters: Int = 1024, isSensitive: Bool) throws -> String {
var buf = [CChar](repeating: 0, count: maxCharacters + 1 /* sentinel */ + 1 /* NUL */)
guard let rawCredential = readpassphrase(prompt, &buf, buf.count, isSensitive ? RPP_ECHO_OFF : RPP_ECHO_ON) else {
throw StdinCredentialsError.CredentialRequired(which: name)
File diff suppressed because one or more lines are too long
+27 -14
View File
@@ -3,19 +3,32 @@ import AsyncAlgorithms
fileprivate let urlSession = createURLSession()
class Fetcher {
static func fetch(_ request: URLRequest, viaFile: Bool = false) async throws -> (AsyncThrowingChannel<Data, Error>, HTTPURLResponse) {
if viaFile {
return try await fetchViaFile(request)
}
return try await fetchViaMemory(request)
class DownloadDelegate: NSObject, URLSessionTaskDelegate {
let progress: Progress
init(_ progress: Progress) throws {
self.progress = progress
}
private static func fetchViaMemory(_ request: URLRequest) async throws -> (AsyncThrowingChannel<Data, Error>, HTTPURLResponse) {
func urlSession(_ session: URLSession, didCreateTask task: URLSessionTask) {
self.progress.addChild(task.progress, withPendingUnitCount: self.progress.totalUnitCount)
}
}
class Fetcher {
static func fetch(_ request: URLRequest, viaFile: Bool = false, progress: Progress? = nil) async throws -> (AsyncThrowingChannel<Data, Error>, HTTPURLResponse) {
let delegate = progress != nil ? try DownloadDelegate(progress!) : nil
if viaFile {
return try await fetchViaFile(request, delegate: delegate)
}
return try await fetchViaMemory(request, delegate: delegate)
}
private static func fetchViaMemory(_ request: URLRequest, delegate: URLSessionTaskDelegate? = nil) async throws -> (AsyncThrowingChannel<Data, Error>, HTTPURLResponse) {
let dataCh = AsyncThrowingChannel<Data, Error>()
let (data, response) = try await urlSession.data(for: request)
let (data, response) = try await urlSession.data(for: request, delegate: delegate)
Task {
await dataCh.send(data)
@@ -26,21 +39,21 @@ class Fetcher {
return (dataCh, response as! HTTPURLResponse)
}
private static func fetchViaFile(_ request: URLRequest) async throws -> (AsyncThrowingChannel<Data, Error>, HTTPURLResponse) {
private static func fetchViaFile(_ request: URLRequest, delegate: URLSessionTaskDelegate? = nil) async throws -> (AsyncThrowingChannel<Data, Error>, HTTPURLResponse) {
let dataCh = AsyncThrowingChannel<Data, Error>()
let (fileURL, response) = try await urlSession.download(for: request)
let (fileURL, response) = try await urlSession.download(for: request, delegate: delegate)
// Acquire a handle to the downloaded file and then remove it.
//
// This keeps a working reference to that file, yet we don't
// have to deal with the cleanup any more.
let fh = try FileHandle(forReadingFrom: fileURL)
let mappedFile = try Data(contentsOf: fileURL, options: [.alwaysMapped])
try FileManager.default.removeItem(at: fileURL)
Task {
while let data = try fh.read(upToCount: 64 * 1024 * 1024) {
await dataCh.send(data)
for chunk in (0 ..< mappedFile.count).chunks(ofCount: 64 * 1024 * 1024) {
await dataCh.send(mappedFile.subdata(in: chunk))
}
dataCh.finish()
@@ -1,13 +1,17 @@
import Foundation
import Network
import SwiftRadix
struct Lease {
var mac: MACAddress
var ip: IPv4Address
var expiresAt: Date
init?(fromRawLease: [String : String]) {
// Retrieve the required fields
guard let hwAddress = fromRawLease["hw_address"] else { return nil }
guard let ipAddress = fromRawLease["ip_address"] else { return nil }
guard let lease = fromRawLease["lease"] else { return nil }
// Parse MAC address
let hwAddressSplits = hwAddress.split(separator: ",")
@@ -26,7 +30,13 @@ struct Lease {
return nil
}
// Parse expiration timestamp
guard let leaseTimestamp = lease.hex?.value else {
return nil
}
self.ip = ip
self.mac = mac
self.expiresAt = Date(timeIntervalSince1970: TimeInterval(leaseTimestamp))
}
}
+11 -6
View File
@@ -37,13 +37,18 @@ class Leases {
}
init(_ fromString: String) throws {
var leases: [MACAddress : Lease] = Dictionary()
let leases = try Self.retrieveRawLeases(fromString).compactMap({ rawLease in
Lease(fromRawLease: rawLease)
}).filter({ lease in
lease.expiresAt.isInFuture
}).map({ lease in
(lease.mac, lease)
})
for lease in try Self.retrieveRawLeases(fromString).compactMap({ Lease(fromRawLease: $0) }) {
leases[lease.mac] = lease
self.leases = Dictionary(leases) { (left, right) in
// When duplicate lease is found, prefer a newer lease over the older one
(left.expiresAt > right.expiresAt) ? left : right
}
self.leases = leases
}
/// Parse leases from the host cache similarly to the PLCache_read() function found in Apple's Open Source releases.
@@ -107,7 +112,7 @@ class Leases {
return rawLeases
}
func ResolveMACAddress(macAddress: MACAddress) throws -> IPv4Address? {
func ResolveMACAddress(macAddress: MACAddress) -> IPv4Address? {
leases[macAddress]?.ip
}
}
+2 -2
View File
@@ -15,7 +15,7 @@ class Softnet: Network {
let vmFD: Int32
init(vmMACAddress: String) throws {
init(vmMACAddress: String, extraArguments: [String] = []) throws {
let fds = UnsafeMutablePointer<Int32>.allocate(capacity: MemoryLayout<Int>.stride * 2)
let ret = socketpair(AF_UNIX, SOCK_DGRAM, 0, fds)
@@ -30,7 +30,7 @@ class Softnet: Network {
try setSocketBuffers(softnetFD, 1 * 1024 * 1024);
process.executableURL = try Self.softnetExecutableURL()
process.arguments = ["--vm-fd", String(STDIN_FILENO), "--vm-mac-address", vmMACAddress]
process.arguments = ["--vm-fd", String(STDIN_FILENO), "--vm-mac-address", vmMACAddress] + extraArguments
process.standardInput = FileHandle(fileDescriptor: softnetFD, closeOnDealloc: false)
}
@@ -0,0 +1,25 @@
import Foundation
actor AuthenticationKeeper {
var authentication: Authentication? = nil
func set(_ authentication: Authentication) {
self.authentication = authentication
}
func header() -> (String, String)? {
if let authentication = authentication {
// Do not suggest any headers if the
// authentication token has expired
if !authentication.isValid() {
return nil
}
return authentication.header()
}
// Do not suggest any headers if the
// authentication token is not set
return nil
}
}
+44 -8
View File
@@ -4,17 +4,22 @@ import Compression
class DiskV2: Disk {
private static let bufferSizeBytes = 4 * 1024 * 1024
private static let layerLimitBytes = 500 * 1000 * 1000
private static let holeGranularityBytes = 64 * 1024
static func push(diskURL: URL, registry: Registry, chunkSizeMb: Int, progress: Progress) async throws -> [OCIManifestLayer] {
var pushedLayers: [OCIManifestLayer] = []
// Open the disk file
let disk = try FileHandle(forReadingFrom: diskURL)
var mappedDisk = try Data(contentsOf: diskURL, options: [.alwaysMapped])
// Compress the disk file as multiple individually decompressible streams,
// each equal ``Self.layerLimitBytes`` bytes or slightly larger due to the
// internal compressor's buffer
while let (compressedData, uncompressedSize, uncompressedDigest) = try compressNextLayerOfLimitBytesOrMore(disk: disk) {
var offset: UInt64 = 0
while let (compressedData, uncompressedSize, uncompressedDigest) = try compressNextLayerOfLimitBytesOrMore(mappedDisk: mappedDisk, offset: offset) {
offset += uncompressedSize
let layerDigest = try await registry.pushBlob(fromData: compressedData, chunkSizeMb: chunkSizeMb)
pushedLayers.append(OCIManifestLayer(
@@ -88,14 +93,38 @@ class DiskV2: Disk {
return
}
// Open the disk file at the specific offset
// Open the disk file
let disk = try FileHandle(forWritingTo: diskURL)
try disk.seek(toOffset: diskWritingOffset)
// A zero chunk for faster than byte-by-byte comparisons
//
// Assumes that the other Data(...) is equal in size, but it's fine to get a false-negative
// on the last block since it costs only 64 KiB of excess data per 500 MB layer.
//
// Some simple benchmarks ("sync && sudo purge" command was used to negate the disk caching effects):
// +--------------------------------------+---------------------------------------------------+
// | Operation | time(1) result |
// +--------------------------------------+---------------------------------------------------+
// | Data(...) == zeroChunk | 2.16s user 11.71s system 73% cpu 18.928 total |
// | Data(...).contains(where: {$0 != 0}) | 603.68s user 12.97s system 99% cpu 10:22.85 total |
// +--------------------------------------+---------------------------------------------------+
let zeroChunk = Data(count: holeGranularityBytes)
var diskWritingOffset = diskWritingOffset
// Pull and decompress a single layer into the specific offset on disk
let filter = try OutputFilter(.decompress, using: .lz4, bufferCapacity: Self.bufferSizeBytes) { data in
if let data = data {
disk.write(data)
guard let data = data else {
return
}
for chunk in data.chunks(ofCount: holeGranularityBytes) {
// Only write chunks that are not zero
if chunk != zeroChunk {
try disk.seek(toOffset: diskWritingOffset)
disk.write(chunk)
}
diskWritingOffset += UInt64(chunk.count)
}
}
@@ -106,6 +135,8 @@ class DiskV2: Disk {
progress.completedUnitCount += Int64(data.count)
}
try filter.finalize()
try disk.close()
}
@@ -114,7 +145,7 @@ class DiskV2: Disk {
}
}
private static func compressNextLayerOfLimitBytesOrMore(disk: FileHandle) throws -> (Data, UInt64, String)? {
private static func compressNextLayerOfLimitBytesOrMore(mappedDisk: Data, offset: UInt64) throws -> (Data, UInt64, String)? {
var compressedData = Data()
var bytesRead: UInt64 = 0
let digest = Digest()
@@ -126,10 +157,15 @@ class DiskV2: Disk {
return nil
}
guard let uncompressedChunk = try disk.read(upToCount: bufferSizeBytes) else {
let readFromByte = Int(offset + bytesRead)
let numBytesToRead = min(mappedDisk.count - readFromByte, bufferSizeBytes)
if numBytesToRead == 0 {
return nil
}
let uncompressedChunk = mappedDisk.subdata(in: readFromByte ..< (readFromByte + numBytesToRead))
bytesRead += UInt64(uncompressedChunk.count)
digest.update(uncompressedChunk)
+5 -12
View File
@@ -102,8 +102,7 @@ class Registry {
private let baseURL: URL
let namespace: String
let credentialsProviders: [CredentialsProvider]
var currentAuthToken: Authentication? = nil
let authenticationKeeper = AuthenticationKeeper()
var host: String? {
guard let host = baseURL.host else { return nil }
@@ -253,7 +252,7 @@ class Registry {
for try await part in channel {
try Task.checkCancellation()
try await handler(Data(part))
try await handler(part)
}
}
@@ -305,11 +304,6 @@ class Registry {
request.httpBody = body
}
// Invalidate token if it has expired
if currentAuthToken?.isValid() == false {
currentAuthToken = nil
}
var (channel, response) = try await authAwareRequest(request: request, viaFile: viaFile)
if doAuth && response.statusCode == HTTPCode.Unauthorized.rawValue {
@@ -331,7 +325,7 @@ class Registry {
if wwwAuthenticate.scheme.lowercased() == "basic" {
if let (user, password) = try lookupCredentials() {
currentAuthToken = BasicAuthentication(user: user, password: password)
await authenticationKeeper.set(BasicAuthentication(user: user, password: password))
}
return
@@ -378,7 +372,7 @@ class Registry {
+ "while retrieving an authentication token", details: data.asText())
}
currentAuthToken = try TokenResponse.parse(fromData: data)
await authenticationKeeper.set(try TokenResponse.parse(fromData: data))
}
private func lookupCredentials() throws -> (String, String)? {
@@ -399,8 +393,7 @@ class Registry {
private func authAwareRequest(request: URLRequest, viaFile: Bool = false) async throws -> (AsyncThrowingChannel<Data, Error>, HTTPURLResponse) {
var request = request
if let token = currentAuthToken {
let (name, value) = token.header()
if let (name, value) = await authenticationKeeper.header() {
request.addValue(value, forHTTPHeaderField: name)
}
+110 -100
View File
@@ -6,121 +6,131 @@ struct UnsupportedHostOSError: Error, CustomStringConvertible {
}
}
struct Darwin: PlatformSuspendable {
var ecid: VZMacMachineIdentifier
var hardwareModel: VZMacHardwareModel
#if arch(arm64)
init(ecid: VZMacMachineIdentifier, hardwareModel: VZMacHardwareModel) {
self.ecid = ecid
self.hardwareModel = hardwareModel
}
struct Darwin: PlatformSuspendable {
var ecid: VZMacMachineIdentifier
var hardwareModel: VZMacHardwareModel
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
let encodedECID = try container.decode(String.self, forKey: .ecid)
guard let data = Data.init(base64Encoded: encodedECID) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize Data using the provided value")
}
guard let ecid = VZMacMachineIdentifier.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize VZMacMachineIdentifier using the provided value")
}
self.ecid = ecid
let encodedHardwareModel = try container.decode(String.self, forKey: .hardwareModel)
guard let data = Data.init(base64Encoded: encodedHardwareModel) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
guard let hardwareModel = VZMacHardwareModel.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
self.hardwareModel = hardwareModel
}
func encode(to encoder: Encoder) throws {
var container = encoder.container(keyedBy: CodingKeys.self)
try container.encode(ecid.dataRepresentation.base64EncodedString(), forKey: .ecid)
try container.encode(hardwareModel.dataRepresentation.base64EncodedString(), forKey: .hardwareModel)
}
func os() -> OS {
.darwin
}
func bootLoader(nvramURL: URL) throws -> VZBootLoader {
VZMacOSBootLoader()
}
func platform(nvramURL: URL) throws -> VZPlatformConfiguration {
let result = VZMacPlatformConfiguration()
result.machineIdentifier = ecid
result.auxiliaryStorage = VZMacAuxiliaryStorage(contentsOf: nvramURL)
if !hardwareModel.isSupported {
// At the moment support of M1 chip is not yet dropped in any macOS version
// This mean that host software is not supporting this hardware model and should be updated
throw UnsupportedHostOSError()
init(ecid: VZMacMachineIdentifier, hardwareModel: VZMacHardwareModel) {
self.ecid = ecid
self.hardwareModel = hardwareModel
}
result.hardwareModel = hardwareModel
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
return result
}
let encodedECID = try container.decode(String.self, forKey: .ecid)
guard let data = Data.init(base64Encoded: encodedECID) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize Data using the provided value")
}
guard let ecid = VZMacMachineIdentifier.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize VZMacMachineIdentifier using the provided value")
}
self.ecid = ecid
func graphicsDevice(vmConfig: VMConfig) -> VZGraphicsDeviceConfiguration {
let result = VZMacGraphicsDeviceConfiguration()
let encodedHardwareModel = try container.decode(String.self, forKey: .hardwareModel)
guard let data = Data.init(base64Encoded: encodedHardwareModel) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
guard let hardwareModel = VZMacHardwareModel.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
self.hardwareModel = hardwareModel
}
func encode(to encoder: Encoder) throws {
var container = encoder.container(keyedBy: CodingKeys.self)
try container.encode(ecid.dataRepresentation.base64EncodedString(), forKey: .ecid)
try container.encode(hardwareModel.dataRepresentation.base64EncodedString(), forKey: .hardwareModel)
}
func os() -> OS {
.darwin
}
func bootLoader(nvramURL: URL) throws -> VZBootLoader {
VZMacOSBootLoader()
}
func platform(nvramURL: URL) throws -> VZPlatformConfiguration {
let result = VZMacPlatformConfiguration()
result.machineIdentifier = ecid
result.auxiliaryStorage = VZMacAuxiliaryStorage(url: nvramURL)
if !hardwareModel.isSupported {
// At the moment support of M1 chip is not yet dropped in any macOS version
// This mean that host software is not supporting this hardware model and should be updated
throw UnsupportedHostOSError()
}
result.hardwareModel = hardwareModel
return result
}
func graphicsDevice(vmConfig: VMConfig) -> VZGraphicsDeviceConfiguration {
let result = VZMacGraphicsDeviceConfiguration()
if let hostMainScreen = NSScreen.main {
let vmScreenSize = NSSize(width: vmConfig.display.width, height: vmConfig.display.height)
result.displays = [
VZMacGraphicsDisplayConfiguration(for: hostMainScreen, sizeInPoints: vmScreenSize)
]
return result
}
if let hostMainScreen = NSScreen.main {
let vmScreenSize = NSSize(width: vmConfig.display.width, height: vmConfig.display.height)
result.displays = [
VZMacGraphicsDisplayConfiguration(for: hostMainScreen, sizeInPoints: vmScreenSize)
VZMacGraphicsDisplayConfiguration(
widthInPixels: vmConfig.display.width,
heightInPixels: vmConfig.display.height,
// A reasonable guess according to Apple's documentation[1]
// [1]: https://developer.apple.com/documentation/coregraphics/1456599-cgdisplayscreensize
pixelsPerInch: 72
)
]
return result
}
result.displays = [
VZMacGraphicsDisplayConfiguration(
widthInPixels: vmConfig.display.width,
heightInPixels: vmConfig.display.height,
// A reasonable guess according to Apple's documentation[1]
// [1]: https://developer.apple.com/documentation/coregraphics/1456599-cgdisplayscreensize
pixelsPerInch: 72
)
]
func keyboards() -> [VZKeyboardConfiguration] {
if #available(macOS 14, *) {
// Mac keyboard is only supported by guests starting with macOS Ventura
return [VZUSBKeyboardConfiguration(), VZMacKeyboardConfiguration()]
} else {
return [VZUSBKeyboardConfiguration()]
}
}
return result
}
func keyboardsSuspendable() -> [VZKeyboardConfiguration] {
if #available(macOS 14, *) {
return [VZMacKeyboardConfiguration()]
} else {
// fallback to the regular configuration
return keyboards()
}
}
func keyboards() -> [VZKeyboardConfiguration] {
if #available(macOS 14, *) {
// Mac keyboard is only supported by guests starting with macOS Ventura
return [VZMacKeyboardConfiguration(), VZUSBKeyboardConfiguration()]
} else {
return [VZUSBKeyboardConfiguration()]
func pointingDevices() -> [VZPointingDeviceConfiguration] {
// Trackpad is only supported by guests starting with macOS Ventura
[VZUSBScreenCoordinatePointingDeviceConfiguration(), VZMacTrackpadConfiguration()]
}
func pointingDevicesSuspendable() -> [VZPointingDeviceConfiguration] {
if #available(macOS 14, *) {
return [VZMacTrackpadConfiguration()]
} else {
// fallback to the regular configuration
return pointingDevices()
}
}
}
func keyboardsSuspendable() -> [VZKeyboardConfiguration] {
if #available(macOS 14, *) {
return [VZMacKeyboardConfiguration()]
} else {
return []
}
}
func pointingDevices() -> [VZPointingDeviceConfiguration] {
// Trackpad is only supported by guests starting with macOS Ventura
[VZMacTrackpadConfiguration(), VZUSBScreenCoordinatePointingDeviceConfiguration()]
}
func pointingDevicesSuspendable() -> [VZPointingDeviceConfiguration] {
[VZMacTrackpadConfiguration()]
}
}
#endif
+3
View File
@@ -8,5 +8,8 @@ protocol Prunable {
var url: URL { get }
func delete() throws
func accessDate() throws -> Date
// size on disk as seen in Finder including empty blocks
func sizeBytes() throws -> Int
// actual size on disk without empty blocks
func allocatedSizeBytes() throws -> Int
}
+5
View File
@@ -26,6 +26,7 @@ struct Root: AsyncParsableCommand {
Rename.self,
Stop.self,
Delete.self,
FQN.self,
])
public static func main() async throws {
@@ -49,6 +50,10 @@ struct Root: AsyncParsableCommand {
}
defer { SentrySDK.flush(timeout: 2.seconds.timeInterval) }
SentrySDK.configureScope { scope in
scope.setExtra(value: ProcessInfo.processInfo.arguments, key: "Command-line arguments")
}
// Enrich future events with Cirrus CI-specific tags
if let tags = ProcessInfo.processInfo.environment["CIRRUS_SENTRY_TAGS"] {
SentrySDK.configureScope { scope in
+5 -1
View File
@@ -9,7 +9,11 @@ extension URL: Prunable {
try FileManager.default.removeItem(at: self)
}
func sizeBytes() throws -> Int {
func allocatedSizeBytes() throws -> Int {
try resourceValues(forKeys: [.totalFileAllocatedSizeKey]).totalFileAllocatedSize!
}
func sizeBytes() throws -> Int {
try resourceValues(forKeys: [.totalFileSizeKey]).totalFileSize!
}
}
+122 -103
View File
@@ -42,10 +42,11 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
init(vmDir: VMDirectory,
network: Network = NetworkShared(),
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment] = [],
additionalStorageDevices: [VZStorageDeviceConfiguration] = [],
directorySharingDevices: [VZDirectorySharingDeviceConfiguration] = [],
serialPorts: [VZSerialPortConfiguration] = [],
suspendable: Bool = false
suspendable: Bool = false,
audio: Bool = true
) throws {
name = vmDir.name
config = try VMConfig.init(fromURL: vmDir.configURL)
@@ -58,10 +59,11 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
self.network = network
configuration = try Self.craftConfiguration(diskURL: vmDir.diskURL,
nvramURL: vmDir.nvramURL, vmConfig: config,
network: network, additionalDiskAttachments: additionalDiskAttachments,
network: network, additionalStorageDevices: additionalStorageDevices,
directorySharingDevices: directorySharingDevices,
serialPorts: serialPorts,
suspendable: suspendable
suspendable: suspendable,
audio: audio
)
virtualMachine = VZVirtualMachine(configuration: configuration)
@@ -89,12 +91,17 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// Download the IPSW
defaultLogger.appendNewLine("Fetching \(remoteURL.lastPathComponent)...")
let (channel, response) = try await Fetcher.fetch(URLRequest(url: remoteURL), viaFile: true)
let downloadProgress = Progress(totalUnitCount: 100)
ProgressObserver(downloadProgress).log(defaultLogger)
let progress = Progress(totalUnitCount: response.expectedContentLength)
ProgressObserver(progress).log(defaultLogger)
let request = URLRequest(url: remoteURL)
let (channel, response) = try await Fetcher.fetch(request, viaFile: true, progress: downloadProgress)
let temporaryLocation = try Config().tartTmpDir.appendingPathComponent(UUID().uuidString + ".ipsw")
defaultLogger.appendNewLine("Computing digest for \(temporaryLocation.path)...")
let digestProgress = Progress(totalUnitCount: response.expectedContentLength)
ProgressObserver(digestProgress).log(defaultLogger)
FileManager.default.createFile(atPath: temporaryLocation.path, contents: nil)
let lock = try FileLock(lockURL: temporaryLocation)
try lock.lock()
@@ -106,7 +113,7 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
let chunkAsData = Data(chunk)
fileHandle.write(chunkAsData)
digest.update(chunkAsData)
progress.completedUnitCount += Int64(chunk.count)
digestProgress.completedUnitCount += Int64(chunk.count)
}
try fileHandle.close()
@@ -116,18 +123,6 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
return try FileManager.default.replaceItemAt(finalLocation, withItemAt: temporaryLocation)!
}
static func latestIPSWURL() async throws -> URL {
defaultLogger.appendNewLine("Looking up the latest supported IPSW...")
let image = try await withCheckedThrowingContinuation { continuation in
VZMacOSRestoreImage.fetchLatestSupported() { result in
continuation.resume(with: result)
}
}
return image.url
}
var inFinalState: Bool {
get {
virtualMachine.state == VZVirtualMachine.State.stopped ||
@@ -137,82 +132,90 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
}
init(
vmDir: VMDirectory,
ipswURL: URL,
diskSizeGB: UInt16,
network: Network = NetworkShared(),
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment] = [],
directorySharingDevices: [VZDirectorySharingDeviceConfiguration] = [],
serialPorts: [VZSerialPortConfiguration] = []
) async throws {
var ipswURL = ipswURL
#if arch(arm64)
init(
vmDir: VMDirectory,
ipswURL: URL,
diskSizeGB: UInt16,
network: Network = NetworkShared(),
additionalStorageDevices: [VZStorageDeviceConfiguration] = [],
directorySharingDevices: [VZDirectorySharingDeviceConfiguration] = [],
serialPorts: [VZSerialPortConfiguration] = []
) async throws {
var ipswURL = ipswURL
if !ipswURL.isFileURL {
ipswURL = try await VM.retrieveIPSW(remoteURL: ipswURL)
}
// We create a temporary TART_HOME directory in tests, which has its "cache" folder symlinked
// to the users Tart cache directory (~/.tart/cache). However, the Virtualization.Framework
// cannot deal with paths that contain symlinks, so expand them here first.
ipswURL.resolveSymlinksInPath()
// Load the restore image and try to get the requirements
// that match both the image and our platform
let image = try await withCheckedThrowingContinuation { continuation in
VZMacOSRestoreImage.load(from: ipswURL) { result in
continuation.resume(with: result)
if !ipswURL.isFileURL {
ipswURL = try await VM.retrieveIPSW(remoteURL: ipswURL)
}
}
guard let requirements = image.mostFeaturefulSupportedConfiguration else {
throw UnsupportedRestoreImageError()
}
// We create a temporary TART_HOME directory in tests, which has its "cache" folder symlinked
// to the users Tart cache directory (~/.tart/cache). However, the Virtualization.Framework
// cannot deal with paths that contain symlinks, so expand them here first.
ipswURL.resolveSymlinksInPath()
// Create NVRAM
_ = try VZMacAuxiliaryStorage(creatingStorageAt: vmDir.nvramURL, hardwareModel: requirements.hardwareModel)
// Create disk
try vmDir.resizeDisk(diskSizeGB)
name = vmDir.name
// Create config
config = VMConfig(
platform: Darwin(ecid: VZMacMachineIdentifier(), hardwareModel: requirements.hardwareModel),
cpuCountMin: requirements.minimumSupportedCPUCount,
memorySizeMin: requirements.minimumSupportedMemorySize
)
// allocate at least 4 CPUs because otherwise VMs are frequently freezing
try config.setCPU(cpuCount: max(4, requirements.minimumSupportedCPUCount))
try config.save(toURL: vmDir.configURL)
// Initialize the virtual machine and its configuration
self.network = network
configuration = try Self.craftConfiguration(diskURL: vmDir.diskURL, nvramURL: vmDir.nvramURL,
vmConfig: config, network: network,
additionalDiskAttachments: additionalDiskAttachments,
directorySharingDevices: directorySharingDevices,
serialPorts: serialPorts
)
virtualMachine = VZVirtualMachine(configuration: configuration)
super.init()
virtualMachine.delegate = self
// Run automated installation
try await withCheckedThrowingContinuation { (continuation: CheckedContinuation<Void, Error>) in
DispatchQueue.main.async { [ipswURL] in
let installer = VZMacOSInstaller(virtualMachine: self.virtualMachine, restoringFromImageAt: ipswURL)
defaultLogger.appendNewLine("Installing OS...")
ProgressObserver(installer.progress).log(defaultLogger)
installer.install { result in
// Load the restore image and try to get the requirements
// that match both the image and our platform
let image = try await withCheckedThrowingContinuation { continuation in
VZMacOSRestoreImage.load(from: ipswURL) { result in
continuation.resume(with: result)
}
}
guard let requirements = image.mostFeaturefulSupportedConfiguration else {
throw UnsupportedRestoreImageError()
}
// Create NVRAM
_ = try VZMacAuxiliaryStorage(creatingStorageAt: vmDir.nvramURL, hardwareModel: requirements.hardwareModel)
// Create disk
try vmDir.resizeDisk(diskSizeGB)
name = vmDir.name
// Create config
config = VMConfig(
platform: Darwin(ecid: VZMacMachineIdentifier(), hardwareModel: requirements.hardwareModel),
cpuCountMin: requirements.minimumSupportedCPUCount,
memorySizeMin: requirements.minimumSupportedMemorySize
)
// allocate at least 4 CPUs because otherwise VMs are frequently freezing
try config.setCPU(cpuCount: max(4, requirements.minimumSupportedCPUCount))
try config.save(toURL: vmDir.configURL)
// Initialize the virtual machine and its configuration
self.network = network
configuration = try Self.craftConfiguration(diskURL: vmDir.diskURL, nvramURL: vmDir.nvramURL,
vmConfig: config, network: network,
additionalStorageDevices: additionalStorageDevices,
directorySharingDevices: directorySharingDevices,
serialPorts: serialPorts
)
virtualMachine = VZVirtualMachine(configuration: configuration)
super.init()
virtualMachine.delegate = self
// Run automated installation
try await install(ipswURL)
}
}
@MainActor
private func install(_ url: URL) async throws {
let installer = VZMacOSInstaller(virtualMachine: self.virtualMachine, restoringFromImageAt: url)
defaultLogger.appendNewLine("Installing OS...")
ProgressObserver(installer.progress).log(defaultLogger)
try await withTaskCancellationHandler(operation: {
try await withCheckedThrowingContinuation { continuation in
installer.install { result in
continuation.resume(with: result)
}
}
}, onCancel: {
installer.progress.cancel()
})
}
#endif
@available(macOS 13, *)
static func linux(vmDir: VMDirectory, diskSizeGB: UInt16) async throws -> VM {
@@ -257,9 +260,13 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
@MainActor
private func start(_ recovery: Bool) async throws {
let startOptions = VZMacOSVirtualMachineStartOptions()
startOptions.startUpFromMacOSRecovery = recovery
try await virtualMachine.start(options: startOptions)
#if arch(arm64)
let startOptions = VZMacOSVirtualMachineStartOptions()
startOptions.startUpFromMacOSRecovery = recovery
try await virtualMachine.start(options: startOptions)
#else
try await virtualMachine.start()
#endif
}
@MainActor
@@ -277,10 +284,11 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
nvramURL: URL,
vmConfig: VMConfig,
network: Network = NetworkShared(),
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment],
additionalStorageDevices: [VZStorageDeviceConfiguration],
directorySharingDevices: [VZDirectorySharingDeviceConfiguration],
serialPorts: [VZSerialPortConfiguration],
suspendable: Bool = false
suspendable: Bool = false,
audio: Bool = true
) throws -> VZVirtualMachineConfiguration {
let configuration = VZVirtualMachineConfiguration()
@@ -298,7 +306,7 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
configuration.graphicsDevices = [vmConfig.platform.graphicsDevice(vmConfig: vmConfig)]
// Audio
if !suspendable {
if audio && !suspendable {
let soundDeviceConfiguration = VZVirtioSoundDeviceConfiguration()
let inputAudioStreamConfiguration = VZVirtioSoundDeviceInputStreamConfiguration()
inputAudioStreamConfiguration.source = VZHostAudioInputStreamSource()
@@ -326,11 +334,20 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
// Storage
var attachments = [try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false)]
attachments.append(contentsOf: additionalDiskAttachments)
configuration.storageDevices = attachments.map {
VZVirtioBlockDeviceConfiguration(attachment: $0)
let attachment: VZDiskImageStorageDeviceAttachment = vmConfig.os == .linux ?
// Use "cached" caching mode for virtio drive to prevent fs corruption on linux
try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false, cachingMode: .cached, synchronizationMode: .full) :
try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false)
var device: VZStorageDeviceConfiguration
if #available(macOS 14, *), vmConfig.os == .linux {
device = VZNVMExpressControllerDeviceConfiguration(attachment: attachment)
} else {
device = VZVirtioBlockDeviceConfiguration(attachment: attachment)
}
var devices: [VZStorageDeviceConfiguration] = [device]
devices.append(contentsOf: additionalStorageDevices)
configuration.storageDevices = devices
// Entropy
if !suspendable {
@@ -347,13 +364,15 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
//
// A dummy console device useful for implementing
// host feature checks in the guest agent software.
let consolePort = VZVirtioConsolePortConfiguration()
consolePort.name = "tart-version-\(CI.version)"
if !suspendable {
let consolePort = VZVirtioConsolePortConfiguration()
consolePort.name = "tart-version-\(CI.version)"
let consoleDevice = VZVirtioConsoleDeviceConfiguration()
consoleDevice.ports[0] = consolePort
let consoleDevice = VZVirtioConsoleDeviceConfiguration()
consoleDevice.ports[0] = consolePort
configuration.consoleDevices.append(consoleDevice)
configuration.consoleDevices.append(consoleDevice)
}
try configuration.validate()
+20 -3
View File
@@ -95,7 +95,14 @@ struct VMConfig: Codable {
arch = try container.decodeIfPresent(Architecture.self, forKey: .arch) ?? .arm64
switch os {
case .darwin:
platform = try Darwin(from: decoder)
#if arch(arm64)
platform = try Darwin(from: decoder)
#else
throw DecodingError.dataCorruptedError(
forKey: .os,
in: container,
debugDescription: "Darwin VMs are only supported on Apple Silicon hosts")
#endif
case .linux:
platform = try Linux(from: decoder)
}
@@ -132,20 +139,30 @@ struct VMConfig: Codable {
}
mutating func setCPU(cpuCount: Int) throws {
if cpuCount < cpuCountMin {
if os == .darwin && cpuCount < cpuCountMin {
throw LessThanMinimalResourcesError("VM should have \(cpuCountMin) CPU cores"
+ " at minimum (requested \(cpuCount))")
}
if cpuCount < VZVirtualMachineConfiguration.minimumAllowedCPUCount {
throw LessThanMinimalResourcesError("VM should have \(VZVirtualMachineConfiguration.minimumAllowedCPUCount) CPU cores"
+ " at minimum (requested \(cpuCount))")
}
self.cpuCount = cpuCount
}
mutating func setMemory(memorySize: UInt64) throws {
if memorySize < memorySizeMin {
if os == .darwin && memorySize < memorySizeMin {
throw LessThanMinimalResourcesError("VM should have \(memorySizeMin) bytes"
+ " of memory at minimum (requested \(memorySize))")
}
if memorySize < VZVirtualMachineConfiguration.minimumAllowedMemorySize {
throw LessThanMinimalResourcesError("VM should have \(VZVirtualMachineConfiguration.minimumAllowedMemorySize) bytes"
+ " of memory at minimum (requested \(memorySize))")
}
self.memorySize = memorySize
}
}
+32 -3
View File
@@ -30,13 +30,17 @@ struct VMDirectory: Prunable {
baseURL
}
func lock() throws -> PIDLock {
try PIDLock(lockURL: configURL)
}
func running() throws -> Bool {
// The most common reason why PIDLock() instantiation fails is a race with "tart delete" (ENOENT),
// which is fine to report as "not running".
//
// The other reasons are unlikely and the cost of getting a false positive is way less than
// the cost of crashing with an exception when calling "tart list" on a busy machine, for example.
guard let lock = try? PIDLock(lockURL: configURL) else {
guard let lock = try? lock() else {
return false
}
@@ -130,20 +134,45 @@ struct VMDirectory: Prunable {
if !FileManager.default.fileExists(atPath: diskURL.path) {
FileManager.default.createFile(atPath: diskURL.path, contents: nil, attributes: nil)
}
let diskFileHandle = try FileHandle.init(forWritingTo: diskURL)
// macOS considers kilo being 1000 and not 1024
try diskFileHandle.truncate(atOffset: UInt64(sizeGB) * 1000 * 1000 * 1000)
let currentDiskFileLength = try diskFileHandle.seekToEnd()
let desiredDiskFileLength = UInt64(sizeGB) * 1000 * 1000 * 1000
if desiredDiskFileLength < currentDiskFileLength {
let currentLengthHuman = ByteCountFormatter().string(fromByteCount: Int64(currentDiskFileLength))
let desiredLengthHuman = ByteCountFormatter().string(fromByteCount: Int64(desiredDiskFileLength))
throw RuntimeError.InvalidDiskSize("new disk size of \(desiredLengthHuman) should be larger " +
"than the current disk size of \(currentLengthHuman)")
} else if desiredDiskFileLength > currentDiskFileLength {
try diskFileHandle.truncate(atOffset: desiredDiskFileLength)
}
try diskFileHandle.close()
}
func delete() throws {
let lock = try lock()
if try !lock.trylock() {
throw RuntimeError.VMIsRunning(name)
}
try FileManager.default.removeItem(at: baseURL)
try lock.unlock()
}
func accessDate() throws -> Date {
try baseURL.accessDate()
}
func allocatedSizeBytes() throws -> Int {
try configURL.allocatedSizeBytes() + diskURL.allocatedSizeBytes() + nvramURL.allocatedSizeBytes()
}
func allocatedSizeGB() throws -> Int {
try allocatedSizeBytes() / 1000 / 1000 / 1000
}
func sizeBytes() throws -> Int {
try configURL.sizeBytes() + diskURL.sizeBytes() + nvramURL.sizeBytes()
}
+9 -3
View File
@@ -50,10 +50,12 @@ enum RuntimeError : Error {
case VMConfigurationError(_ message: String)
case VMDoesNotExist(name: String)
case VMMissingFiles(_ message: String)
case VMNotRunning(_ message: String)
case VMIsRunning(_ name: String)
case VMNotRunning(_ name: String)
case VMAlreadyRunning(_ message: String)
case NoIPAddressFound(_ message: String)
case DiskAlreadyInUse(_ message: String)
case InvalidDiskSize(_ message: String)
case FailedToUpdateAccessDate(_ message: String)
case PIDLockFailed(_ message: String)
case FailedToParseRemoteName(_ message: String)
@@ -81,14 +83,18 @@ extension RuntimeError : CustomStringConvertible {
return "the specified VM \"\(name)\" does not exist"
case .VMMissingFiles(let message):
return message
case .VMNotRunning(let message):
return message
case .VMIsRunning(let name):
return "VM \"\(name)\" is running"
case .VMNotRunning(let name):
return "VM \"\(name)\" is not running"
case .VMAlreadyRunning(let message):
return message
case .NoIPAddressFound(let message):
return message
case .DiskAlreadyInUse(let message):
return message
case .InvalidDiskSize(let message):
return message
case .FailedToUpdateAccessDate(let message):
return message
case .PIDLockFailed(let message):
+1 -1
View File
@@ -39,7 +39,7 @@ class VMStorageLocal: PrunableStorage {
}
func delete(_ name: String) throws {
try FileManager.default.removeItem(at: vmURL(name))
try VMDirectory(baseURL: vmURL(name)).delete()
}
func list() throws -> [(String, VMDirectory)] {
+2 -4
View File
@@ -134,7 +134,7 @@ class VMStorageOCI: PrunableStorage {
func pull(_ name: RemoteName, registry: Registry, concurrency: UInt) async throws {
SentrySDK.configureScope { scope in
scope.setContext(value: ["imageName": name], key: "OCI")
scope.setContext(value: ["imageName": name.description], key: "OCI")
}
defaultLogger.appendNewLine("pulling manifest...")
@@ -219,9 +219,7 @@ class VMStorageOCI: PrunableStorage {
}
func link(from: RemoteName, to: RemoteName) throws {
if FileManager.default.fileExists(atPath: vmURL(from).path) {
try FileManager.default.removeItem(at: vmURL(from))
}
try? FileManager.default.removeItem(at: vmURL(from))
try FileManager.default.createSymbolicLink(at: vmURL(from), withDestinationURL: vmURL(to))
+45 -3
View File
@@ -1,6 +1,8 @@
import XCTest
@testable import tart
import Virtualization
final class DirectoryShareTests: XCTestCase {
func testNamedParsing() throws {
let share = try DirectoryShare(parseFrom: "build:/Users/admin/build")
@@ -8,25 +10,65 @@ final class DirectoryShareTests: XCTestCase {
XCTAssertEqual(share.path, URL(filePath: "/Users/admin/build"))
XCTAssertFalse(share.readOnly)
}
func testNamedReadOnlyParsing() throws {
let share = try DirectoryShare(parseFrom: "build:/Users/admin/build:ro")
XCTAssertEqual(share.name, "build")
XCTAssertEqual(share.path, URL(filePath: "/Users/admin/build"))
XCTAssertTrue(share.readOnly)
}
func testOptionalNameParsing() throws {
let share = try DirectoryShare(parseFrom: "/Users/admin/build")
XCTAssertNil(share.name)
XCTAssertEqual(share.path, URL(filePath: "/Users/admin/build"))
XCTAssertFalse(share.readOnly)
}
func testOptionalNameReadOnlyParsing() throws {
let share = try DirectoryShare(parseFrom: "/Users/admin/build:ro")
XCTAssertNil(share.name)
XCTAssertEqual(share.path, URL(filePath: "/Users/admin/build"))
XCTAssertTrue(share.readOnly)
}
func testMountTagParsing() throws {
let share = try DirectoryShare(parseFrom: "/Users/admin/build:tag=foo-bar")
XCTAssertNil(share.name)
XCTAssertEqual(share.path, URL(filePath: "/Users/admin/build"))
XCTAssertFalse(share.readOnly)
XCTAssertEqual(share.mountTag, "foo-bar")
let roShare = try DirectoryShare(parseFrom: "/Users/admin/build:ro,tag=foo-bar")
XCTAssertNil(roShare.name)
XCTAssertEqual(roShare.path, URL(filePath: "/Users/admin/build"))
XCTAssertTrue(roShare.readOnly)
XCTAssertEqual(roShare.mountTag, "foo-bar")
let inverseRoShare = try DirectoryShare(parseFrom: "/Users/admin/build:tag=foo-bar,ro")
XCTAssertNil(inverseRoShare.name)
XCTAssertEqual(inverseRoShare.path, URL(filePath: "/Users/admin/build"))
XCTAssertTrue(inverseRoShare.readOnly)
XCTAssertEqual(inverseRoShare.mountTag, "foo-bar")
}
func testURL() throws {
let archiveWithoutNameOrOptions = try DirectoryShare(parseFrom: "https://example.com/archive.tar.gz")
XCTAssertNil(archiveWithoutNameOrOptions.name)
XCTAssertEqual(archiveWithoutNameOrOptions.path, URL(string: "https://example.com/archive.tar.gz")!)
XCTAssertFalse(archiveWithoutNameOrOptions.readOnly)
XCTAssertEqual(archiveWithoutNameOrOptions.mountTag, VZVirtioFileSystemDeviceConfiguration.macOSGuestAutomountTag)
let archiveWithOptions = try DirectoryShare(parseFrom: "https://example.com/archive.tar.gz:ro,tag=sometag")
XCTAssertNil(archiveWithOptions.name)
XCTAssertEqual(archiveWithOptions.path, URL(string: "https://example.com/archive.tar.gz")!)
XCTAssertTrue(archiveWithOptions.readOnly)
XCTAssertEqual(archiveWithOptions.mountTag, "sometag")
let archiveWithNameAndOptions = try DirectoryShare(parseFrom: "somename:https://example.com/archive.tar.gz:ro,tag=sometag")
XCTAssertEqual(archiveWithNameAndOptions.name, "somename")
XCTAssertEqual(archiveWithNameAndOptions.path, URL(string: "https://example.com/archive.tar.gz")!)
XCTAssertTrue(archiveWithNameAndOptions.readOnly)
XCTAssertEqual(archiveWithNameAndOptions.mountTag, "sometag")
}
}
+18
View File
@@ -0,0 +1,18 @@
import XCTest
@testable import tart
import Network
import SwiftRadix
final class LeaseTests: XCTestCase {
func testCorrectTimezone() throws {
let lease = Lease(fromRawLease: [
"hw_address": "1,11:22:33:44:55:66",
"ip_address": "1.2.3.4",
"lease": "0x6565da9e",
])
XCTAssertNotNil(lease)
XCTAssertEqual(lease!.expiresAt.toISO(), "2023-11-28T12:18:38Z")
}
}
+61
View File
@@ -0,0 +1,61 @@
import XCTest
@testable import tart
import Network
import SwiftDate
final class LeasesTests: XCTestCase {
func testNoExpired() throws {
let macAddress = MACAddress(fromString: "11:22:33:44:55:66")!
let leases = try Leases("""
{
name=whatever
ip_address=66.66.66.66
hw_address=1,\(macAddress)
identifier=1,\(macAddress)
lease=\(Int((Date() - 1.seconds).timeIntervalSince1970).hex)
}
{
name=whatever
ip_address=1.2.3.4
hw_address=1,\(macAddress)
identifier=1,\(macAddress)
lease=\(Int((Date() + 10.minutes).timeIntervalSince1970).hex)
}
{
name=whatever
ip_address=66.66.66.66
hw_address=1,\(macAddress)
identifier=1,\(macAddress)
lease=\(Int((Date() - 1.seconds).timeIntervalSince1970).hex)
}
""")
XCTAssertEqual(IPv4Address("1.2.3.4"), leases.ResolveMACAddress(macAddress: macAddress))
}
func testDuplicateYetNotExpiredLeases() throws {
let macAddress = MACAddress(fromString: "11:22:33:44:55:66")!
let leases = try Leases("""
{
name=debian
ip_address=192.168.64.1
hw_address=1,\(macAddress)
identifier=1,\(macAddress)
lease=\(Int((Date() + 10.minutes).timeIntervalSince1970).hex)
}
{
name=debian
ip_address=192.168.64.2
hw_address=1,\(macAddress)
identifier=1,\(macAddress)
lease=\(Int((Date() + 5.minutes).timeIntervalSince1970).hex)
}
""")
XCTAssertEqual(IPv4Address("192.168.64.1"), leases.ResolveMACAddress(macAddress: macAddress))
}
}
@@ -8,11 +8,12 @@ final class MACAddressResolverTests: XCTestCase {
{
ip_address=1.2.3.4
hw_address=1,00:11:22:33:44:55
lease=0x7fffffff
}
""")
XCTAssertEqual(IPv4Address("1.2.3.4"),
try leases.ResolveMACAddress(macAddress: MACAddress(fromString: "00:11:22:33:44:55")!))
leases.ResolveMACAddress(macAddress: MACAddress(fromString: "00:11:22:33:44:55")!))
}
func testMultipleEntries() throws {
@@ -20,16 +21,18 @@ final class MACAddressResolverTests: XCTestCase {
{
ip_address=1.2.3.4
hw_address=1,00:11:22:33:44:55
lease=0x7fffffff
}
{
ip_address=5.6.7.8
hw_address=1,AA:BB:CC:DD:EE:FF
lease=0x7fffffff
}
""")
XCTAssertEqual(IPv4Address("1.2.3.4"),
try leases.ResolveMACAddress(macAddress: MACAddress(fromString: "00:11:22:33:44:55")!))
leases.ResolveMACAddress(macAddress: MACAddress(fromString: "00:11:22:33:44:55")!))
XCTAssertEqual(IPv4Address("5.6.7.8"),
try leases.ResolveMACAddress(macAddress: MACAddress(fromString: "AA:BB:CC:DD:EE:FF")!))
leases.ResolveMACAddress(macAddress: MACAddress(fromString: "AA:BB:CC:DD:EE:FF")!))
}
}
+1
View File
@@ -0,0 +1 @@
root = true
+92
View File
@@ -0,0 +1,92 @@
run:
timeout: 5m
linters:
enable-all: true
disable:
# Messages like "struct of size 104 bytes could be of size 96 bytes" from a package
# that was last updated 2 years ago[1] are barely helpful.
#
# After all, we're writing the code for other people, so let's trust the compiler here (that's
# constantly evolving compared to this linter) and revisit this if memory usage becomes a problem.
#
# [1]: https://github.com/mdempsky/maligned/commit/6e39bd26a8c8b58c5a22129593044655a9e25959
- maligned
# We don't have high-performance requirements at this moment, so sacrificing
# the code readability for marginal performance gains is not worth it.
- prealloc
# New linters that require a lot of codebase churn and noise, but perhaps we can enable them in the future.
- nlreturn
- wrapcheck
- errorlint
# Unfortunately, we use globals due to how spf13/cobra works.
- gochecknoglobals
# That's fine that some Proto objects don't have all fields initialized
- exhaustivestruct
# Style linters that are total nuts.
- wsl
- gofumpt
- goimports
- funlen
# This conflicts with the Protocol Buffers Version 3 design,
# which is largely based on default values for struct fields.
- exhaustivestruct
# Enough parallelism for now.
- paralleltest
# Ill-based assumptions about identifiers like fmt.Println without taking context into account.
- forbidigo
# Advantages of using t.Helper() are too small to waste developer's cognitive stamina on it.
- thelper
# Too restrictive defaults, plus there's already a gocyclo linter in place.
- cyclop
# Gives false positives for textbook examples[1][2]
# [1]: https://github.com/charithe/durationcheck/issues/7
# [2]: https://golang.org/pkg/time/ (see "To convert an integer number of units to a Duration, multiply:")
- durationcheck
# No way to disable the "exported" check for the whole project[1]
# [1]: https://github.com/mgechev/revive/issues/244#issuecomment-560512162
- revive
# Unfortunately too much false-positives, e.g. for a 0700 umask or number 10 when using strconv.FormatInt()
- gomnd
# Needs package whitelists
- depguard
# Generates absolutely useless errors, e.g.
# "string `.yml` has 3 occurrences, make it a constant"
- goconst
# It's OK to not sort imports
- gci
# It's OK to not initialize some struct fields
- exhaustruct
# This is not a library, so it's OK to use dynamic errors
- goerr113
# fmt.Sprintf() looks a bit nicer than string addition
- perfsprint
# We can control this ourselves
- varnamelen
- contextcheck
issues:
# Don't hide multiple issues that belong to one class since GitHub annotations can handle them all nicely.
max-issues-per-linter: 0
max-same-issues: 0
+39
View File
@@ -0,0 +1,39 @@
# Benchmark
Tart comes with a Golang-based benchmarking utility that allows one to easily compare host and guest performance.
Currently, only Flexible I/O tester workloads are supported. To run them, [install Golang](https://go.dev/) and run the following command from this (`benchmark/`) directory:
```shell
go run cmd/main.go fio
```
You can also enable the debugging output to diagnose issues:
```shell
go run cmd/main.go fio --debug
```
## Results
Host:
* Hardware: Mac mini (Apple M2 Pro, 8 performance and 4 efficiency cores, 32 GB RAM, `Mac14,12`)
* OS: macOS Sonoma 14.4.1
Guest:
* Hardware: [Virtualization.Framework](https://developer.apple.com/documentation/virtualization)
* OS: macOS Sonoma 14.4.1
```
Name Executor Bandwidth I/O operations
Random writing of 1MB local 2.6 GB/s 649.35 kIOPS
Random writing of 1MB Tart 2.5 GB/s 620.22 kIOPS
Random writing of 10MB local 2.6 GB/s 651.74 kIOPS
Random writing of 10MB Tart 2.5 GB/s 615.52 kIOPS
Random writing of 100MB local 1.9 GB/s 481.51 kIOPS
Random writing of 100MB Tart 2.0 GB/s 493.31 kIOPS
Random writing of 1000MB local 1.7 GB/s 414.89 kIOPS
Random writing of 1000MB Tart 1.1 GB/s 287.4 kIOPS
```
+23
View File
@@ -0,0 +1,23 @@
package main
import (
"context"
"github.com/cirruslabs/tart/benchmark/internal/command"
"log"
"os"
"os/signal"
)
func main() {
// Set up a signal-interruptible context
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
// Run the root command
if err := command.NewCommand().ExecuteContext(ctx); err != nil {
cancel()
log.Fatal(err)
}
cancel()
}
+29
View File
@@ -0,0 +1,29 @@
module github.com/cirruslabs/tart/benchmark
go 1.22.1
require (
github.com/avast/retry-go/v4 v4.5.1
github.com/dustin/go-humanize v1.0.1
github.com/google/uuid v1.6.0
github.com/gosuri/uitable v0.0.4
github.com/spf13/cobra v1.8.0
github.com/stretchr/testify v1.9.0
go.uber.org/zap v1.27.0
golang.org/x/crypto v0.21.0
)
require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/fatih/color v1.16.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.15 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/spf13/pflag v1.0.5 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/sys v0.18.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
+52
View File
@@ -0,0 +1,52 @@
github.com/avast/retry-go/v4 v4.5.1 h1:AxIx0HGi4VZ3I02jr78j5lZ3M6x1E0Ivxa6b0pUUh7o=
github.com/avast/retry-go/v4 v4.5.1/go.mod h1:/sipNsvNB3RRuT5iNcb6h73nw3IBmXJ/H3XrCQYSOpc=
github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM=
github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gosuri/uitable v0.0.4 h1:IG2xLKRvErL3uhY6e1BylFzG+aJiwQviDDTfOKeKTpY=
github.com/gosuri/uitable v0.0.4/go.mod h1:tKR86bXuXPZazfOTG1FIzvjIdXzd0mo4Vtn16vt0PJo=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZgg3U=
github.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
golang.org/x/crypto v0.21.0 h1:X31++rzVUdKhX5sWmSOFZxx8UW/ldWx55cbf08iNAMA=
golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.18.0 h1:FcHjZXDMxI8mM3nwhX9HlKop4C0YQvCVCdwYl2wOtE8=
golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
@@ -0,0 +1,29 @@
package fio
type Benchmark struct {
Name string
Command string
}
var benchmarks = []Benchmark{
{
Name: "Random writing of 1MB",
Command: "fio --rw randwrite --runtime 30 --time_based --unlink 1 --output-format json " +
"--size 1MB --name unnamed --numjobs 1 --iodepth 1 --end_fsync 1",
},
{
Name: "Random writing of 10MB",
Command: "fio --rw randwrite --runtime 30 --time_based --unlink 1 --output-format json " +
"--size 10MB --name unnamed --numjobs 1 --iodepth 1 --end_fsync 1",
},
{
Name: "Random writing of 100MB",
Command: "fio --rw randwrite --runtime 30 --time_based --unlink 1 --output-format json " +
"--size 100MB --name unnamed --numjobs 1 --iodepth 1 --end_fsync 1",
},
{
Name: "Random writing of 1000MB",
Command: "fio --rw randwrite --runtime 30 --time_based --unlink 1 --output-format json " +
"--size 1000MB --name unnamed --numjobs 1 --iodepth 1 --end_fsync 1",
},
}
+131
View File
@@ -0,0 +1,131 @@
package fio
import (
"context"
"encoding/json"
"errors"
"fmt"
"github.com/cirruslabs/tart/benchmark/internal/executor"
"github.com/cirruslabs/tart/benchmark/internal/executor/local"
"github.com/cirruslabs/tart/benchmark/internal/executor/tart"
"github.com/dustin/go-humanize"
"github.com/gosuri/uitable"
"github.com/spf13/cobra"
"go.uber.org/zap"
)
var debug bool
func NewCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "fio",
Short: "run Flexible I/O tester (fio) benchmarks",
RunE: run,
}
cmd.Flags().BoolVar(&debug, "debug", false, "enable debug logging")
return cmd
}
func run(cmd *cobra.Command, args []string) error {
config := zap.NewProductionConfig()
if debug {
config.Level = zap.NewAtomicLevelAt(zap.DebugLevel)
}
logger, err := config.Build()
if err != nil {
return err
}
defer func() {
_ = logger.Sync()
}()
executors, err := initializeExecutors(cmd.Context(), logger)
if err != nil {
return err
}
defer func() {
errs := []error{err}
for _, executor := range executors {
if err := executor.Close(); err != nil {
errs = append(errs, fmt.Errorf("failed to close executor %s: %w", executor.Name(), err))
}
}
err = errors.Join(errs...)
}()
table := uitable.New()
table.AddRow("Name", "Executor", "Bandwidth", "I/O operations")
for _, benchmark := range benchmarks {
for _, executor := range executors {
logger.Sugar().Infof("running benchmark %q on %s executor", benchmark.Name, executor.Name())
stdout, err := executor.Run(cmd.Context(), benchmark.Command)
if err != nil {
return err
}
var fioResult Result
if err := json.Unmarshal(stdout, &fioResult); err != nil {
return err
}
if len(fioResult.Jobs) != 1 {
return fmt.Errorf("expected exactly 1 job from fio's JSON output, got %d",
len(fioResult.Jobs))
}
job := fioResult.Jobs[0]
writeBandwidth := humanize.Bytes(uint64(job.Write.BW)*humanize.KByte) + "/s"
writeIOPS := humanize.SIWithDigits(job.Write.IOPS, 2, "IOPS")
logger.Sugar().Infof("write bandwidth: %s, write IOPS: %s\n", writeBandwidth, writeIOPS)
table.AddRow(benchmark.Name, executor.Name(), writeBandwidth, writeIOPS)
}
}
fmt.Println(table.String())
return nil
}
func initializeExecutors(ctx context.Context, logger *zap.Logger) ([]executor.Executor, error) {
var result []executor.Executor
logger.Info("initializing local executor")
local, err := local.New(logger)
if err != nil {
return nil, err
}
result = append(result, local)
logger.Info("local executor initialized")
logger.Info("initializing Tart executor")
tart, err := tart.New(ctx, logger)
if err != nil {
return nil, err
}
result = append(result, tart)
logger.Info("Tart executor initialized")
for _, executor := range result {
logger.Sugar().Infof("installing Flexible I/O tester (fio) on %s executor", executor.Name())
if _, err := executor.Run(ctx, "brew install fio"); err != nil {
return nil, err
}
}
return result, nil
}
+15
View File
@@ -0,0 +1,15 @@
package fio
type Result struct {
Jobs []Job `json:"jobs"`
}
type Job struct {
Name string `json:"jobname"`
Write Write `json:"write"`
}
type Write struct {
BW float64 `json:"bw"`
IOPS float64 `json:"iops"`
}
+20
View File
@@ -0,0 +1,20 @@
package command
import (
"github.com/cirruslabs/tart/benchmark/internal/command/fio"
"github.com/spf13/cobra"
)
func NewCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "benchmark",
SilenceUsage: true,
SilenceErrors: true,
}
cmd.AddCommand(
fio.NewCommand(),
)
return cmd
}
+11
View File
@@ -0,0 +1,11 @@
package executor
import (
"context"
)
type Executor interface {
Name() string
Run(ctx context.Context, command string) ([]byte, error)
Close() error
}
@@ -0,0 +1,42 @@
package local
import (
"bytes"
"context"
"go.uber.org/zap"
"go.uber.org/zap/zapio"
"io"
"os/exec"
)
type Local struct {
logger *zap.Logger
}
func New(logger *zap.Logger) (*Local, error) {
return &Local{
logger: logger,
}, nil
}
func (local *Local) Name() string {
return "local"
}
func (local *Local) Run(ctx context.Context, command string) ([]byte, error) {
cmd := exec.CommandContext(ctx, "zsh", "-c", command)
loggerWriter := &zapio.Writer{Log: local.logger, Level: zap.DebugLevel}
stdoutBuf := &bytes.Buffer{}
cmd.Stdout = io.MultiWriter(stdoutBuf, loggerWriter)
cmd.Stderr = loggerWriter
err := cmd.Run()
return stdoutBuf.Bytes(), err
}
func (local *Local) Close() error {
return nil
}
@@ -0,0 +1,20 @@
package local_test
import (
"context"
"github.com/cirruslabs/tart/benchmark/internal/executor/local"
"github.com/stretchr/testify/require"
"go.uber.org/zap"
"testing"
)
func TestLocal(t *testing.T) {
local, err := local.New(zap.NewNop())
require.NoError(t, err)
output, err := local.Run(context.Background(), "echo \"this is a test\"")
require.NoError(t, err)
require.Equal(t, "this is a test\n", string(output))
require.NoError(t, local.Close())
}
+35
View File
@@ -0,0 +1,35 @@
package tart
import (
"bytes"
"context"
"go.uber.org/zap"
"go.uber.org/zap/zapio"
"io"
"os/exec"
"strings"
)
const tartBinaryName = "tart"
func Cmd(ctx context.Context, logger *zap.Logger, args ...string) error {
_, err := CmdWithOutput(ctx, logger, args...)
return err
}
func CmdWithOutput(ctx context.Context, logger *zap.Logger, args ...string) (string, error) {
logger.Sugar().Debugf("running %s %s", tartBinaryName, strings.Join(args, " "))
cmd := exec.CommandContext(ctx, tartBinaryName, args...)
loggerWriter := &zapio.Writer{Log: logger, Level: zap.DebugLevel}
stdoutBuf := &bytes.Buffer{}
cmd.Stdout = io.MultiWriter(stdoutBuf, loggerWriter)
cmd.Stderr = loggerWriter
err := cmd.Run()
return stdoutBuf.String(), err
}
+133
View File
@@ -0,0 +1,133 @@
package tart
import (
"bytes"
"context"
"errors"
"fmt"
"github.com/avast/retry-go/v4"
"github.com/google/uuid"
"go.uber.org/zap"
"golang.org/x/crypto/ssh"
"net"
"strings"
"time"
)
const baseImage = "ghcr.io/cirruslabs/macos-sonoma-base:latest"
type Tart struct {
vmRunCancel context.CancelFunc
vmName string
sshClient *ssh.Client
logger *zap.Logger
}
func New(ctx context.Context, logger *zap.Logger) (*Tart, error) {
tart := &Tart{
vmName: fmt.Sprintf("tart-benchmark-%s", uuid.NewString()),
logger: logger,
}
if err := Cmd(ctx, tart.logger, "pull", baseImage); err != nil {
return nil, err
}
if err := Cmd(ctx, tart.logger, "clone", baseImage, tart.vmName); err != nil {
return nil, err
}
vmRunCtx, vmRunCancel := context.WithCancel(ctx)
tart.vmRunCancel = vmRunCancel
go func() {
_ = Cmd(vmRunCtx, tart.logger, "run", "--no-graphics", tart.vmName)
}()
ip, err := CmdWithOutput(ctx, tart.logger, "ip", "--wait", "60", tart.vmName)
if err != nil {
return nil, tart.Close()
}
err = retry.Do(func() error {
dialer := net.Dialer{
Timeout: 1 * time.Second,
}
addr := fmt.Sprintf("%s:22", strings.TrimSpace(ip))
netConn, err := dialer.DialContext(ctx, "tcp", addr)
if err != nil {
return err
}
sshConn, chans, reqs, err := ssh.NewClientConn(netConn, addr, &ssh.ClientConfig{
User: "admin",
Auth: []ssh.AuthMethod{
ssh.Password("admin"),
},
HostKeyCallback: func(_ string, _ net.Addr, _ ssh.PublicKey) error {
return nil
},
})
if err != nil {
return err
}
tart.sshClient = ssh.NewClient(sshConn, chans, reqs)
return nil
}, retry.RetryIf(func(err error) bool {
return !errors.Is(err, context.Canceled)
}))
if err != nil {
return nil, tart.Close()
}
return tart, nil
}
func (tart *Tart) Name() string {
return "Tart"
}
func (tart *Tart) Run(ctx context.Context, command string) ([]byte, error) {
sshSession, err := tart.sshClient.NewSession()
if err != nil {
return nil, err
}
// Work around x/crypto/ssh not being context.Context-friendly (e.g. https://github.com/golang/go/issues/20288)
monitorCtx, monitorCancel := context.WithCancel(ctx)
go func() {
<-monitorCtx.Done()
_ = sshSession.Close()
}()
defer monitorCancel()
stdoutBuf := &bytes.Buffer{}
sshSession.Stdin = bytes.NewBufferString(command)
sshSession.Stdout = stdoutBuf
if err := sshSession.Shell(); err != nil {
return nil, err
}
if err := sshSession.Wait(); err != nil {
return nil, err
}
return stdoutBuf.Bytes(), nil
}
func (tart *Tart) Close() error {
if tart.sshClient != nil {
_ = tart.sshClient.Close()
}
tart.vmRunCancel()
_ = Cmd(context.Background(), tart.logger, "delete", tart.vmName)
return nil
}
@@ -0,0 +1,22 @@
package tart_test
import (
"context"
"github.com/cirruslabs/tart/benchmark/internal/executor/tart"
"github.com/stretchr/testify/require"
"go.uber.org/zap"
"testing"
)
func TestTart(t *testing.T) {
ctx := context.Background()
tart, err := tart.New(ctx, zap.NewNop())
require.NoError(t, err)
output, err := tart.Run(ctx, "echo \"this is a test\"")
require.NoError(t, err)
require.Equal(t, "this is a test\n", string(output))
require.NoError(t, tart.Close())
}
+12
View File
@@ -0,0 +1,12 @@
"default": true
"MD002": false # First heading should be a top level heading
"MD007": # Unordered list indentation
indent: 4
"MD009": false # Trailing spaces
"MD013": false # Line length
"MD025": false # Multiple top level headings in the same document
"MD026": false # Trailing punctuation in heading
"MD033": false # Inline HTML
"MD041": false # First line in file should be a top level heading
"MD045": false # OK not to have a description for an image
"MD046": false # Code block style [Expected: fenced; Actual: indented]
Binary file not shown.

After

Width:  |  Height:  |  Size: 232 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 602 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.8 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 602 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 538 KiB

@@ -62,7 +62,7 @@ and before reaching the 100 CPU cores limit is royalty-free and does not have th
When an organization surpasses the 100 CPU cores limit, they will be required to obtain a [Gold Tier License](/licensing#license-tiers),
which costs \$1000 per month. Upon reaching a limit of 500 CPU cores, a [Platinum Tier License](/licensing#license-tiers)
(\$5000 per month) will be required, and for organizations that exceed 5000 CPU cores, a custom [Diamond Tier License](/licensing#license-tiers)
(\$3000 per month) will be required, and for organizations that exceed 3000 CPU cores, a custom [Diamond Tier License](/licensing#license-tiers)
(\$1 per core per month) will be necessary. **All paid license tiers will include priority feature development and SLAs on support with urgent issues.**
## Have we considered alternatives?
@@ -56,7 +56,7 @@ On bootstrap, each Orchard worker establishes a `Watch()` RPC stream and waits f
Once `PortForward` instruction is received, the worker connects to the specified VM and port locally and opens a new `PortForward()` RPC stream with the controller, carrying the unique `session` identifier in the gRPC metadata to help distinguish several port forwarding requests.
We’re using a pretty ingenious Golang package that turns any gRPC stream into a `net.Conn`: https://github.com/mitchellh/go-grpc-net-conn. This allows us to abstract from the gRPC details and simply proxy two `net.Conns`, thus providing the port forwarding functionality.
We’re using a pretty ingenious [Golang package that turns any gRPC stream into a `net.Conn`](https://github.com/mitchellh/go-grpc-net-conn). This allows us to abstract from the gRPC details and simply proxy two `net.Conns`, thus providing the port forwarding functionality.
We’ve also initially considered using [Yamux](https://github.com/hashicorp/yamux) to only keep a single connection with each worker, however, that involves the burden of dealing with flow control and potential implementation bugs associated with it, so we’ve decided to simply open an additional connection for each port forwarding session and let the OS deal with it.
@@ -74,25 +74,25 @@ Secondly, we’ve exposed three commands in the Orchard CLI that all use this en
Opens a TCP port locally and forwards everything sent to it to the specified VM (and vice versa).
For example, `orchard port-forward vm ventura-builder 2222:22` will forward traffic from the local TCP port `2222` to the `ventura-builder` VM’s TCP port `22`.
For example, `orchard port-forward vm sonoma-builder 2222:22` will forward traffic from the local TCP port `2222` to the `ventura-builder` VM’s TCP port `22`.
### `orchard ssh`
Connects to the specified VM on the default SSH port `22`, optionally only launching a command (if specified), similarly to what the official OpenSSH client does.
For example, `orchard ssh vm ventura-builder` will open an interactive session with the `ventura-builder` VM.
For example, `orchard ssh vm sonoma-builder` will open an interactive session with the `ventura-builder` VM.
You can also send local scripts for execution by utilizing redirection:
```shell
orchard ssh vm ventura-builder 'sh -s' < script.sh
orchard ssh vm sonoma-builder 'sh -s' < script.sh
```
### `orchard vnc`
Establishes a port forwarding to the specified VM’s default VNC port `5900` and opens the default macOS Screen Sharing app.
For example, `orchard vnc vm ventura-builder` will establish a port-forwarding to the `ventura-builder` VM's port `5900` under the hood and launch macOS Screen Sharing app.
For example, `orchard vnc vm sonoma-builder` will establish a port-forwarding to the `ventura-builder` VM's port `5900` under the hood and launch macOS Screen Sharing app.
Note that the SSH and VNC commands expect the VM resource to specify credentials in it’s definition (can be done via `orchard create vm`), and will otherwise fall back to the credentials specified by `--username` and `--password`, or if none specified — to de-facto standard of `admin:admin` credentials.
+6 -4
View File
@@ -29,6 +29,8 @@ dozens of companies that we know of are using Tart in their daily workflows. If
- ![](https://github.com/cirruslabs/tart/raw/main/Resources/Users/Suran.png){ height="65" }
- ![](https://github.com/cirruslabs/tart/raw/main/Resources/Users/Symflower.png){ height="65" }
- ![](https://github.com/cirruslabs/tart/raw/main/Resources/Users/Transloadit.png){ height="65" }
- ![](https://github.com/cirruslabs/tart/raw/main/Resources/Users/PITSGlobalDataRecoveryServices.png){ height="65" }
- ![](https://github.com/cirruslabs/tart/raw/main/Resources/Users/Uphold.png){ height="65" }
</div>
@@ -41,10 +43,10 @@ allocate time to continue improving Tart which brings us to the section below.
In the last 7 months we've had 12 feature releases that brought a lot of features requested by the community. Here are just
a few of them to highlight:
* [Custom GitLab Runner Executor](/integrations/gitlab-runner.md).
* [Cluster Management via Orchard](2023-04-25-orchard-ga.md).
* Numerous compatibility improvements for all kinds of OCI-registries.
* Sonoma Support (see details [below](#macos-sonoma-updates)).
-[Custom GitLab Runner Executor](/integrations/gitlab-runner/).
-[Cluster Management via Orchard](2023-04-25-orchard-ga.md).
-Numerous compatibility improvements for all kinds of OCI-registries.
-Sonoma Support (see details [below](#macos-sonoma-updates)).
But one of the most requested features/complaints was around pulling huge Tart images from remote OCI-compatible registries.
With an ideal network conditions `tart pull` worked pretty good but in case of any network issues it was required to
+71
View File
@@ -0,0 +1,71 @@
---
draft: false
date: 2023-10-06
search:
exclude: true
authors:
- fkorotkov
categories:
- announcement
---
# Tart is now available on AWS Marketplace
Announcing [official AMIs for EC2 Mac Instances](https://aws.amazon.com/marketplace/pp/prodview-qczco34wlkdws)
with preconfigured Tart installation that is optimized to work within AWS infrastructure.
EC2 Mac Instances is a gem of engineering powered by AWS Nitro devices. Just imagine there is a physical Mac Mini with
a plugged in Nitro device that can push the physical power button!
![EC2 M2 Pro](/blog/images/ec2-mac2-m2pro.png)
This clever synergy between Apple Hardware and Nitro System allows seamless integration with VPC networking and booting macOS from an EBS volume.
In this blog post we’ll see how a virtualization solution like Tart can compliment and elevate experience with EC2 Mac Instances.
<!-- more -->
Let’s start from the basics, what EC2 Mac Instances allow to do compared to physical Mac Minis seating in offices of
many companies around the world?
First and foremost, EC2 Mac Instances sit inside AWS data centers and can leverage all the goodies of VPC networking
within your company's existing infrastructure. No need to connect your Macs in the office through a VPN and deal
with networking and security.
Additionally, EC2 Mac Instances are booting from EBS volumes which means it is possible to always have reproducible instances
and apply all the best practices of Infrastructure-as-Code. Managing a fleet of physical Macs is a pain and it's very hard
to make them configured in a reproducible and stable way. With booting from identical EBS volumes your team is always sure
about the identical initial state of the fleet.
## Compromises of EC2 Mac Instances
The flexibility of EBS volumes for macOS comes with some compromises that virtualization solutions like Tart can help with.
The initial boot from an EBS volume takes some time and not instant. macOS itself is pretty heavy and a Nitro device needs
to download tens of gigabytes that macOS requires in order to boot. This means that **resetting a EC2 Mac Instance to a clean state
is not instant and usually takes a couple of minutes** when you can’t utilize the precious resources for your workloads.
It is much easier to tailor such EBS volumes with tools like Packer but there is still a **friction to test newly created EBS volumes**
since one needs to start and run a EC2 Mac Instance and it’s not possible to test things locally. Similarly it is even harder
to test beta versions of macOS that require manual interaction with a running instance.
## Solution
Tart can help with all the compromises! Tart virtual machines (VMs) have nearly native performance thanks to utilizing
native `Virtualization.Framework` that was developed along the first Apple Silicon chip. **Tart VMs can be copied/disposed
instantly and booting a fresh Tart VM takes only several seconds**. It is also possible to run two different Tart VMs in parallel
that can have completely different versions of macOS and packages. For example, it is possible to have the latest stable macOS
with the release version of Xcode along with the next version of macOS with the latest beta of Xcode.
Creation of Tart VMs can be automated with [a Packer plugin](https://github.com/cirruslabs/packer-plugin-tart) the same way as
creation of EC2 AMIs with one caveat that **Tart Packer Plugin works locally so you can test the same virtual machine locally
as you would run it in the cloud**.
Lightweight nature of Tart VMs with a focus on an easy-to-integrate Tart CLI compliments any macOS automation and helps to reduce
the feedback cycle and improves reproducibility of macOS environments even further.
## Conclusion
We are excited to bring [official AMIs that include Tart installation optimized to work within AWS](https://aws.amazon.com/marketplace/pp/prodview-qczco34wlkdws).
In the coming weeks when macOS Sonoma will become available on AWS we’ll release another update specifically targeting EC2 Mac Instances.
This update will simplify access to local SSDs of Mac Instances that are slightly faster than EBS volumes. Stay tuned and don’t hesitate
to ask any [questions](https://tart.run/licensing/).
@@ -0,0 +1,59 @@
---
draft: false
date: 2023-11-03
search:
exclude: true
authors:
- fkorotkov
categories:
- announcement
---
# New dashboard with insights into performance of Cirrus Runners
This month we are celebrating one year since launching Cirrus Runners — managed Apple Silicon infrastructure for your
GitHub Actions. During the last 12 months we ran millions of workflows for our customers and now ready to share some insights
into price performance of them for our customers.
One of the key difference with Cirrus Runners is how they are getting billed for. Customers purchase Cirrus Runners via monthly subscription
that costs $150 per each Cirrus Runner. Each runner can be used 24 hours a day 7 days a week to run GitHub Actions workflows
for an organization. If there are more outstanding jobs than available runners then they are queued and executed as soon as
there is a free runner. This is different from how GitHub-managed GitHub Actions are billed for — you pay for each minute of execution time.
The benefit of a fixed price is that you can run as many jobs as you want without worrying about the cost. The downside is that
you need to make sure that you are using your runners efficiently. This is where the new dashboard comes in handy.
<!-- more -->
But first, **let's see theoretically the lowest price per minute** of a Cirrus Runners. If you run 24 hours a day 7 days a week
then you will get 43,200 minutes of execution time per month. This means that the price per minute is $0.0035 if your runners
utilization is 100%. But even if your engineering teams is located in a single time zone and works 8 hours a day 5 days a week
then you will get 9,600 minutes of execution time per month which comes down to $0.015 per-minute. This is still more than 10 times cheaper
than recently announced Apple Silicon GitHub-manged runners that cost $0.16 per minute.
Now lets take a look at the new Cirrus Runners dashboard of a real customers that run their workflows on Cirrus Runners
and **practically pushing the price performance pretty close to the theoretical minimum**.
![Cirrus Runners Dashboard](/blog/images/runners-price-performance-2.png)
As you can see above Cirrus Runners Dashboard focuses on 4 core metrics:
1. **Minutes Used** — overall amount of minutes that Cirrus Runners were executing jobs.
2. **Workflow Runs** — absolute number of workflow runs that were executed on Cirrus Runners.
3. **Queue Size** — number of jobs that were queued and waiting for a free Cirrus Runner.
4. **Queue Time** — average time that jobs were waiting in the queue.
In this particular example price performance of Cirrus Runners is $0.006 per minute which is 2 times more than the theoretical minimum
and **26 times better than GitHub-managed Apple Silicon runners**. But this is a extreme example, looking at queue time and queue size
we can see that the downside of such great price performance is that jobs are waiting in the queue on average around 5 minutes.
Here is another example of Cirrus Runners Dashboard for a different customer that has a slightly higher price performance of $0.017 per minute
but at the same time doesn't experience queue time at all. **Note that $0.017 is still 10 times cheaper than GitHub-managed Apple Silicon runners**.
![Cirrus Runners Dashboard](/blog/images/runners-price-performance-3.png)
## Conclusion
Having a fixed price for Cirrus Runners is a great way to save money on your CI/CD infrastructure and just in general have predictable budged.
But it requires keeping the balance between price per minute and queue time. Cirrus Runners Dashboard helps you to keep an eye on this balance
and make sure that you are getting the most out of your Cirrus Runners.
+33 -11
View File
@@ -3,19 +3,9 @@ hide:
- navigation
---
## How Tart is different from Anka?
Under the hood Tart is using the same technology as Anka 3.0 so there should be no real difference in performance
or features supported. If there is some feature missing please don't hesitate to [create a feature request](https://github.com/cirruslabs/tart/issues).
Instead of Anka Registry, Tart can work with any OCI-compatible container registry. This provides a much more consistent
and scalable experience for distributing virtual machines.
Tart doesn't yet have an analogue of Anka Controller for managing long living VMs but [soon will be](https://github.com/cirruslabs/tart/issues/372).
## VM location on disk
Tart stores all it's files in `~/.tart/` directory. Local images that you can run are stored in `~/.tart/vms/`.
Tart stores all its files in `~/.tart/` directory. Local images that you can run are stored in `~/.tart/vms/`.
Remote images are pulled into `~/.tart/cache/OCIs/`.
## Nested virtualization support?
@@ -66,3 +56,35 @@ sudo defaults write /Library/Preferences/SystemConfiguration/com.apple.InternetS
```
Note that this tweak persists across reboots, so normally you'll only need to do it once per new host.
## Running login/clone/pull/push commands over SSH
When invoking the Tart in an SSH session, you might get error like this:
>Keychain returned unsuccessful status -25308
...or this:
>Keychain failed to update item: User interaction is not allowed.
This is because Tart uses [Keychain](https://en.wikipedia.org/wiki/Keychain_(software)) to store and retrieve OCI registry credentials by default, but Keychain is only automatically/semi-automatically unlocked in GUI sessions.
To unlock the Keychain in an SSH session, run the following command, which will ask for your user's password:
```shell
security unlock-keychain
```
This command also supports the `-p` command-line argument that allows you to supply the password and unlock non-interactively, which is great for scripts.
If that doesn't work for you for some reason, you can pass the credentials via the environment variables, see [Registry Authorization](integrations/vm-management.md#registry-authorization) for more details on how to do that.
## How Tart is different from Anka?
Under the hood Tart is using the same technology as Anka 3.0 so there should be no real difference in performance
or features supported. If there is some feature missing please don't hesitate to [create a feature request](https://github.com/cirruslabs/tart/issues).
Instead of Anka Registry, Tart can work with any OCI-compatible container registry. This provides a much more consistent
and scalable experience for distributing virtual machines.
Tart doesn't yet have an analogue of Anka Controller for managing long living VMs but [soon will be](https://github.com/cirruslabs/tart/issues/372).
+21
View File
@@ -0,0 +1,21 @@
# Buildkite
It is possible to run [Buildkite](https://buildkite.com/) pipeline steps in isolated ephemeral Tart Virtual Machines with the help of [Tart Buildkite Plugin](https://github.com/cirruslabs/tart-buildkite-plugin):
![](/assets/images/BuildkiteTartPlugin.png)
## Configuration
The most basic configuration looks like this:
```yaml
steps:
- command: uname -a
plugins:
- cirruslabs/tart#main:
image: ghcr.io/cirruslabs/macos-sonoma-base:latest
```
This will run `uname -r` in a macOS Tart VM cloned from `ghcr.io/cirruslabs/macos-sonoma-base:latest`.
See plugin's [Configuration section](https://github.com/cirruslabs/tart-buildkite-plugin#configuration) for the full list of available options.
+2 -2
View File
@@ -13,7 +13,7 @@ task:
name: hello
macos_instance:
# can be a remote or a local virtual machine
image: ghcr.io/cirruslabs/macos-ventura-base:latest
image: ghcr.io/cirruslabs/macos-sonoma-base:latest
hello_script:
- echo "Hello from within a Tart VM!"
- echo "Here is my CPU info:"
@@ -45,7 +45,7 @@ exposes it via [`artifacts` instruction](https://cirrus-ci.org/guide/writing-tas
task:
name: Build
macos_instance:
image: ghcr.io/cirruslabs/macos-ventura-xcode:latest
image: ghcr.io/cirruslabs/macos-sonoma-xcode:latest
build_script: swift build --product tart
binary_artifacts:
path: .build/debug/tart
+111 -16
View File
@@ -1,33 +1,128 @@
# GitHub Actions
# Cirrus Runners for GitHub Actions
Tart already powers several CI services mentioned above including our own [Cirrus CI](https://cirrus-ci.org/guide/macOS/) which offers unlimited concurrency with per-second billing.
For services that haven't leveraged Tart yet, we offer fully managed runners via a monthly subscription.
*Cirrus Runners* is the fastest way to get your current CI workflows to benefit from Apple Silicon hardware. No need to manage infrastructure or migrate to another CI provider.
*Cirrus Runners* is the fastest and most cost-efficient way to get your current CI workflows to benefit from Apple Silicon hardware. No need to manage infrastructure or migrate to another CI provider.
Your actions will be executed in clean macOS virtual machines with 4 Apple M2 cores.
## Testimonials from customers
Sebastian Jachec, Mobile Engineer at [Daybridge](https://www.daybridge.com/).
Mitchell Hashimoto, HashiCorp co-founder:
> It’s been plain-sailing with the Cirrus Runners — they’ve been great! They’re consistently 60+% faster on workflows that we previously used Github Actions’ macOS runners for.
> I've been using "Cirrus Runners" since [that tweet](https://twitter.com/mitchellh/status/1731071326201561194) and it has been fantastic. Huge speed increase, huge cost decrease, zero maintenance, exactly what I wanted.
Max Lapides, Senior Mobile Engineer at [Tonal](https://www.tonal.com/).
Max Lapides, Senior Mobile Engineer at [Tonal](https://www.tonal.com/):
> Previously, we were using the GitHub‑hosted macOS runners and our iOS build took ~30 minutes. Now with Cirrus Runners, the iOS build only takes ~12 minutes. That’s a huge boost to our productivity, and for only $150/month per runner it is much less expensive too.
John A., Software Engineer at [GitKraken](https://www.gitkraken.com/):
> GitHub Actions MacOS-x86 runners have become increasingly unreliable, so we're moving our Mac builds over to arm64 because Cirrus Labs' M1 runners are not only ~3 times faster, they've also been far more stable.
Sebastian Jachec, Mobile Engineer at [Daybridge](https://www.daybridge.com/):
> It’s been plain-sailing with the Cirrus Runners — they’ve been great! They’re consistently 60+% faster on workflows that we previously used Github Actions’ macOS runners for.
## Pricing
Each Cirrus Runner costs $150 a month and there is no limit on the amount of minutes for your actions.
We recommend to purchase several Cirrus Runners depending on your team size, so you can run actions in
parallel. Note that you can change your subscription at any time via [this page](https://billing.stripe.com/p/login/3cs7vNbzo92p7fy3cc)
or by emailing [support@cirruslabs.org](mailto:support@cirruslabs.org).
### Discounts
We offer two mutually exclusive discounts:
- 10% "Volume Discount" for subscriptions of 10 or more Cirrus Runners.
- 15% "Annual Discount" for 12 months subscription commitment of any amount of Cirrus Runners.
Please contact [support@cirruslabs.org](mailto:support@cirruslabs.org) after activating the subscription in order to get the discount applied.
### Priority Support
Subscriptions of 20 or more Cirrus Runners include access to [Priority Support](../licensing.md#priority-support).
Please contact [sales@cirruslabs.org](mailto:sales@cirruslabs.org) in order to get all the details.
### CPU and Memory resources of Cirrus Runners
By default, a single Cirrus Runner is allocated with 4 M2 cores and 12 GB of unified memory which is enough for most of the workloads.
For workloads that require more resources it is possible to use XL Cirrus Runners which have twice the resources: a full M2 chip with 8 cores
and 24 GB of unified memory. Note that a single XL Cirrus Runner also uses twice the concurrency.
In order to use an XL Cirrus Runner for a job please append `-xl` suffix to your `runs-on` property. More on that down below.
## Installation
Once you configure [Cirrus Runners App](https://github.com/apps/cirrus-runners) for your organization, you'll be redirected
to a checkout page powered by Stripe. During the checkout process you'll be able to configure a subscription for
a desired amount of parallel Cirrus Runners and try it for free for 10 days.
Once configured, please follow instruction below. If you have any questions please contact [support@cirruslabs.org](mailto:support@cirruslabs.org).
Subscriptions with more than 10 runners also include Priority Support
## Configuring Cirrus Runners
Configuring Cirrus Runners for GitHub Actions is as simple as installing [Cirrus Runners App](https://github.com/apps/cirrus-runners).
After successful installation and subscription configuration, use any of [Ventura images managed by us](https://github.com/cirruslabs/macos-image-templates) in `runs-on`:
In order for Cirrus Runners to be used by your GitHub Actions workflow jobs, specify a desired image in the `runs-on` property.
```yaml
name: Test Suite
jobs:
test:
runs-on: ghcr.io/cirruslabs/macos-ventura-xcode:latest
```
=== "Default Cirrus Runner"
```yaml
name: Tests
jobs:
test:
runs-on: ghcr.io/cirruslabs/macos-sonoma-xcode:latest
```
=== "XL Cirrus Runner"
```yaml
name: Integration Tests
jobs:
test:
runs-on: ghcr.io/cirruslabs/macos-sonoma-xcode:latest-xl
```
List of all available images can be found in [this repository](https://github.com/cirruslabs/macos-image-templates).
Note that Tart VM images don't have the same set of pre-installed packages as the official Intel GitHub runners.
If something is missing please [create an issue within this repository](https://github.com/cirruslabs/macos-image-templates/issues/new).
When workflows are executing you'll see Cirrus on-demand runners on your organization's settings page at `https://github.com/organizations/<ORGANIZATION>/settings/actions/runners`.
Note that Cirrus Runners will get added to the default runner group. By default, only private repositories can access runners in a default runner group, but you can override this in your organization's settings.
Note that Cirrus Runners will get added to the default runner group.
!!! tip "Using Cirrus Runners with public repositories"
By default, only private repositories can access runners in a default runner group, but you can override this in your organization's settings:
```https://github.com/organizations/<YOUR ORGANIZATION NAME>/settings/actions/runner-groups/1```
![](/assets/images/TartGHARunners.png)
### Dashboard
You can also see the status of your runners on the [Cirrus Runners Dashboard](https://cirrus-runners.app/). This dashboard
also provides insights into price performance of your Cirrus Runners. Please check out [this blog post](/blog/2023/11/03/new-dashboard-with-insights-into-performance-of-cirrus-runners/)
to learn more about what this dashboard can do for you.
![](/assets/images/RunnersDashboard.png)
## Data handling flow
By design Cirrus Runners service never sees any of your secrets or source code and acts as compute platform with the lastest
Apple Silicon hardware that can quickly allocate CPU/Memory resources for your jobs.
Here is a high-level overview of how Cirrus Runners service manages runners for your organization:
- Cirrus Runner GitHub App is subscribed to [`workflow_job`](https://docs.github.com/en/webhooks/webhook-events-and-payloads#workflow_job).
- Upon receiving a new event targeting Cirrus Runners via `runs-on` property the following steps take place:
- Non-personal information about your job is saved to perform health checking of Cirrus Runners execution.
- Cirrus Runners GitHub App has only one permission that allows generating temporary registration tokens for
self-hosted GitHub Actions Runners. Note that Cirrus Runners GitHub App itself doesn't have access to contents of
repositories in your organization.
- Cirrus Runners Service creates a new single use Tart VM, generates a temporary registration tokens for self-hosted runners
and passes it without storing inside the VM for the GitHub Actions Runner service to [start a ephemeral runner](https://github.blog/changelog/2021-09-20-github-actions-ephemeral-self-hosted-runners-new-webhooks-for-auto-scaling/).
- Cirrus Runners service continuously monitors health of the Tart VM executing your job to make sure it runs to completion.
- After the job finishes the ephemeral Tart VM is getting destroyed with all the information of the job run.
If you have any questions or concerns please feel free to reach out to [support@cirruslabs.org](mailto:support@cirruslabs.org).
+2 -2
View File
@@ -19,7 +19,7 @@ concurrent = 2
[[runners]]
# ...
executor = "custom"
builds_dir = "/Users/admin/builds" # directory inside the
builds_dir = "/Users/admin/builds" # directory inside the VM
cache_dir = "/Users/admin/cache"
[runners.feature_flags]
FF_RESOLVE_FULL_TLS_CHAIN = false
@@ -37,7 +37,7 @@ Now you can use Tart Images in your `.gitlab-ci.yml`:
```yaml
# You can use any remote Tart Image.
# Tart Executor will pull it from the registry and use it for creating ephemeral VMs.
image: ghcr.io/cirruslabs/macos-ventura-base:latest
image: ghcr.io/cirruslabs/macos-sonoma-base:latest
test:
tags:
+6 -4
View File
@@ -11,8 +11,8 @@ Tart can create VMs from `*.ipsw` files. You can download a specific `*.ipsw` fi
use `latest` instead of a path to `*.ipsw` to download the latest available version:
```bash
tart create --from-ipsw=latest ventura-vanilla
tart run ventura-vanilla
tart create --from-ipsw=latest sonoma-vanilla
tart run sonoma-vanilla
```
After the initial booting of the VM you'll need to manually go through the macOS installation process. As a convention we recommend creating an `admin` user with an `admin` password. After the regular installation please do some additional modifications in the VM:
@@ -67,8 +67,8 @@ packer {
}
source "tart-cli" "tart" {
vm_base_name = "ghcr.io/cirruslabs/macos-ventura-base:latest"
vm_name = "my-custom-ventura"
vm_base_name = "ghcr.io/cirruslabs/macos-sonoma-base:latest"
vm_name = "my-custom-sonoma"
cpu_count = 4
memory_gb = 8
disk_size_gb = 70
@@ -92,7 +92,9 @@ Here is a [repository with Packer templates](https://github.com/cirruslabs/macos
## Working with a Remote OCI Container Registry
<!-- markdownlint-disable MD034 -->
For example, let's say you want to push/pull images to a registry hosted at https://acme.io/.
<!-- markdownlint-enable MD034 -->
### Registry Authorization
+17 -12
View File
@@ -12,11 +12,8 @@ will be required to obtain a paid license.
The virtual CPU cores of Tart VMs are not tied to specific physical cores of the host CPU. Instead, for optimal performance
Tart VMs will automatically try to balance compute between all available cores of the host CPU. As a result,
all performance and energy-efficient cores of the host CPU are always counted towards the license usage.
# License Tiers
## Free Tier
## License Tiers
By default, when no [license is purchased](#get-the-license), it is assumed that an organization is using a Free Tier license.
You can find the Free Tier license text in [Tart](https://github.com/cirruslabs/tart/blob/main/LICENSE) and [Orchard](https://github.com/cirruslabs/orchard/blob/main/LICENSE) repositories.
@@ -35,29 +32,37 @@ Free Tier license has a 100 CPU core limit for Tart and 4 Orchard Workers limit
- Using Tart on 13 Mac Minis with 8 CPUs each.
- Creating an Orchard cluster of 5 Mac Minis workers with 8 CPUs each.
## Gold Tier
### Gold Tier
If an organization wishes to exceed the limits of the Free Tier license, a purchase of the [Gold Tier License](#get-the-license) is required, which costs \$1000 per month.
Gold Tier license has a 500 CPU core limit for Tart and 20 Orchard Workers limit for Orchard.
## Platinum Tier
### Platinum Tier
If an organization wishes to exceed the limits of the Gold Tier license, a purchase of the [Platinum Tier License](#get-the-license) is required, which costs \$5000 per month.
If an organization wishes to exceed the limits of the Gold Tier license, a purchase of the [Platinum Tier License](#get-the-license) is required, which costs \$3000 per month.
Platinum Tier license has a 5,000 CPU core limit for Tart and 200 Orchard Workers limit for Orchard.
Platinum Tier license has a 3,000 CPU core limit for Tart and 200 Orchard Workers limit for Orchard.
## Diamond Tier
### Diamond Tier
For organizations that wish to exceed the limits of the Platinum Tier license, a purchase of a [custom Diamond Tier License](#get-the-license) is required, which costs \$1 per CPU core per month and gives the ability to run unlimited Orchard Workers.
# Get the license
## Get the license
If your organization is interested in purchasing one of the license tiers, please email [licensing@cirruslabs.org](mailto:licensing@cirruslabs.org).
You can see a template of a license subscription agreement [here](assets/TartLicenseSubscription.pdf).
# General Support
!!! info "Running on AWS?"
There are [official AMIs for EC2 Mac Instances](https://aws.amazon.com/marketplace/pp/prodview-qczco34wlkdws)
with preconfigured Tart installation that is optimized to work within AWS infrastructure.
Additionally, there is a [ECR Pulic Gallery mirror](https://gallery.ecr.aws/cirruslabs/macos) of all the
[Tart VM images managed by us](https://github.com/cirruslabs/macos-image-templates).
## General Support
The best way to ask general questions about particular use cases is to email our support team at [support@cirruslabs.org](mailto:support@cirruslabs.org).
Our support team is trying our best to respond ASAP, but there is no guarantee on a response time unless your organization
@@ -66,7 +71,7 @@ has a paid license subscription which includes [Priority Support](#priority-supp
If you have a feature request or noticed lack of some documentation please feel free to [create a GitHub issue](https://github.com/cirruslabs/tart/issues/new).
Our support team will answer it by replying to the issue or by updating the documentation.
# Priority Support
## Priority Support
In addition to the general support we provide a *Priority Support* with guaranteed response times included in all the paid license tiers.
+55 -9
View File
@@ -7,8 +7,8 @@ Try running a Tart VM on your Apple Silicon device running macOS 13.0 (Ventura)
```bash
brew install cirruslabs/cli/tart
tart clone ghcr.io/cirruslabs/macos-ventura-base:latest ventura-base
tart run ventura-base
tart clone ghcr.io/cirruslabs/macos-sonoma-base:latest sonoma-base
tart run sonoma-base
```
??? info "Manual installation from a release archive"
@@ -17,8 +17,8 @@ tart run ventura-base
```bash
curl -LO https://github.com/cirruslabs/tart/releases/latest/download/tart.tar.gz
tar -xzvf tart.tar.gz
./tart.app/Contents/MacOS/tart clone ghcr.io/cirruslabs/macos-ventura-base:latest ventura-base
./tart.app/Contents/MacOS/tart run ventura-base
./tart.app/Contents/MacOS/tart clone ghcr.io/cirruslabs/macos-sonoma-base:latest sonoma-base
./tart.app/Contents/MacOS/tart run sonoma-base
```
Please note that `./tart.app/Contents/MacOS/tart` binary is required to be used in order to trick macOS
@@ -28,12 +28,59 @@ tart run ventura-base
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/TartScreenshot.png"/>
</p>
## VM images
The following macOS images are currently available:
* macOS 14 (Sonoma)
* `ghcr.io/cirruslabs/macos-sonoma-vanilla:latest`
* `ghcr.io/cirruslabs/macos-sonoma-base:latest`
* `ghcr.io/cirruslabs/macos-sonoma-xcode:latest`
* macOS 13 (Ventura)
* `ghcr.io/cirruslabs/macos-ventura-vanilla:latest`
* `ghcr.io/cirruslabs/macos-ventura-base:latest`
* `ghcr.io/cirruslabs/macos-ventura-xcode:latest`
* macOS 12 (Monterey)
* `ghcr.io/cirruslabs/macos-monterey-vanilla:latest`
* `ghcr.io/cirruslabs/macos-monterey-base:latest`
* `ghcr.io/cirruslabs/macos-monterey-xcode:latest`
There's also a [full list of images](https://github.com/orgs/cirruslabs/packages?tab=packages&q=macos-) in which you can discovery specific tags (e.g. `ghcr.io/cirruslabs/macos-monterey-xcode:15`) and [macOS-specific Packer templates](https://github.com/cirruslabs/macos-image-templates) that were used to generate these images.
For, Linux the options are as follows:
* Ubuntu
* `ghcr.io/cirruslabs/ubuntu:latest`
* Debian
* `ghcr.io/cirruslabs/debian:latest`
* Fedora
* `ghcr.io/cirruslabs/fedora:latest`
Note that these Linux images have a minimal disk size of 20 GB, and you might want to resize them right after cloning:
```bash
tart clone ghcr.io/cirruslabs/ubuntu:latest ubuntu
tart set ubuntu --disk-size 50
tart run ubuntu
```
These Linux images can be ran natively on [Vetu](https://github.com/cirruslabs/vetu), our virtualization solution for Linux, assuming that Vetu itself is running on an `arm64` machine.
Similarly to macOS, there's also a [full list of images](https://github.com/orgs/cirruslabs/packages?repo_name=linux-image-templates) in which you can discovery specific tags (e.g. `ghcr.io/cirruslabs/ubuntu:22.04`) and [Linux-specific Packer templates](https://github.com/cirruslabs/linux-image-templates) that were used to generate these images.
All images above use the following credentials:
* Username: `admin`
* Password: `admin`
These credentials work both for logging in via GUI, console (Linux) and SSH.
## SSH access
If the guest VM is running and configured to accept incoming SSH connections you can conveniently connect to it like so:
```bash
ssh admin@$(tart ip ventura-base)
ssh admin@$(tart ip sonoma-base)
```
!!! tip "Running scripts inside Tart virtual machines"
@@ -41,9 +88,9 @@ ssh admin@$(tart ip ventura-base)
from within Tart virtual machines. Alternatively, you can use plain ssh connection and `tart ip` command:
```bash
brew install sshpass
sshpass -p admin ssh -o "StrictHostKeyChecking no" admin@$(tart ip ventura-base) "uname -a"
sshpass -p admin ssh -o "StrictHostKeyChecking no" admin@$(tart ip ventura-base) < script.sh
brew install cirruslabs/cli/sshpass
sshpass -p admin ssh -o "StrictHostKeyChecking no" admin@$(tart ip sonoma-base) "uname -a"
sshpass -p admin ssh -o "StrictHostKeyChecking no" admin@$(tart ip sonoma-base) < script.sh
```
## Mounting directories
@@ -100,4 +147,3 @@ mount -t virtiofs com.apple.virtio-fs.automount /mnt/shared
```
The directory we've mounted above will be accessible from the `/mnt/shared/project` path inside a guest VM.
+29 -41
View File
@@ -1,11 +1,5 @@
{% extends "base.html" %}
{% block announce %}
<a href="/blog/2023/04/25/announcing-orchard-orchestration-for-managing-macos-virtual-machines-at-scale/">
🚀🚀🚀&nbsp&nbspAnnouncing <strong>Orchard</strong> orchestration for managing macOS virtual machines at scale&nbsp;&nbsp;🚀🚀🚀
</a>
{% endblock %}
<!-- Render landing page under tabs -->
{% block tabs %} {{ super() }}
@@ -177,15 +171,10 @@
<figcaption class="md-typeset">
<h2>Seamless integration with your existing automations</h2>
<p>
Tart powers several continuous integration systems including
<a href="/integrations/github-actions"
>on&#8209;demand GitHub Actions Runners</a
>
and
<a href="https://cirrus-ci.org/guide/macOS/" target="_blank"
>Cirrus&nbsp;CI</a
>. Double the performance of&nbsp;your macOS actions with
a&nbsp;couple lines of&nbsp;code.
Tart integrates with many continuous integration systems, including a dedicated
service of on-demand GitHub Actions Runners. With a single line change, you can cut your
CI/CD costs by up to <b>30 times</b> by using <a href="https://cirrus-runners.app/">Cirrus Runners</a>
to run your workflows.
</p>
</figcaption>
</figure>
@@ -242,12 +231,15 @@
}
let counterElement = document.getElementById('installation-counter');
if (counterElement) {
counterElement.textContent = Math.round(allDownloads / 1000) + ",000"
// Live installation count is available starting version 1.0.0
// Prior Tart was installed a little over 14,000 times, let's count them too
let installationPriorV1 = 14
counterElement.textContent = (installationPriorV1 + Math.round(allDownloads / 1000)) + ",000"
}
})
</script>
<h2>
With more than <strong id="installation-counter">10,000</strong> installations to date, Tart has been adopted for various scenarios.
With more than <strong id="installation-counter">25,000</strong> installations to date, Tart has been adopted for various scenarios.
Its applications range from powering CI/CD pipelines and reproducible local development environments,
to helping in the testing of device management systems without actual physical devices.
</h2>
@@ -272,6 +264,26 @@
</h1>
</header>
<div class="mdx-users">
<figure class="mdx-users__testimonial">
<img
src="assets/images/users/mitchell-hashimoto.webp"
alt="Mitchell Hashimoto"
loading="lazy"
width="200"
height="200"
/>
<figcaption class="md-typeset">
<h2>Mitchell Hashimoto</h2>
<h3>
<a href="https://www.hashicorp.com/" target="_blank">HashiCorp</a> co-founder
</h3>
<hr/>
<cite>
I've been using "Cirrus Runners" since <a href="https://x.com/mitchellh/status/1731071326201561194" target="_blank">that tweet</a> and
it has been fantastic. Huge speed increase, huge cost decrease, zero maintenance, exactly what I wanted.
</cite>
</figcaption>
</figure>
<figure class="mdx-users__testimonial">
<img
src="assets/images/users/seb-jachec.webp"
@@ -296,30 +308,6 @@
</cite>
</figcaption>
</figure>
<figure class="mdx-users__testimonial">
<img
src="assets/images/users/mikhail-tokarev.webp"
alt="Mikhail Tokarev"
loading="lazy"
width="200"
height="200"
/>
<figcaption class="md-typeset">
<h2>Mikhail Tokarev</h2>
<h3>
CTO at
<a href="https://codemagic.io/start/" target="_blank"
>Codemagic</a
>
</h3>
<hr/>
<cite>
Thanks to the minimal overhead of using the Apple Virtualization
API, we’ve seen some performance improvements in booting new
virtual machines compared with Anka.
</cite>
</figcaption>
</figure>
<figure class="mdx-users__testimonial">
<img
src="assets/images/users/max-lapides.webp"
+4 -1
View File
@@ -1,4 +1,7 @@
source = [".build/arm64-apple-macosx/release/tart"]
source = [
".build/x86_64-apple-macosx/release/tart",
".build/arm64-apple-macosx/release/tart"
]
bundle_id = "com.github.cirruslabs.tart"
apple_id {
+1 -1
View File
@@ -1,7 +1,7 @@
import requests
from testcontainers.core.waiting_utils import wait_container_is_ready
from testcontainers.general import DockerContainer
from testcontainers.core.container import DockerContainer
class DockerRegistry(DockerContainer):
+5 -3
View File
@@ -1,16 +1,18 @@
import uuid
import pytest
from paramiko.client import SSHClient, AutoAddPolicy
def test_run(tart):
@pytest.mark.parametrize("run_opts", [[], ["--no-graphics"]])
def test_run(tart, run_opts):
vm_name = f"integration-test-run-{uuid.uuid4()}"
# Instantiate a VM with admin:admin SSH access
tart.run(["clone", "ghcr.io/cirruslabs/macos-ventura-base:latest", vm_name])
tart.run(["clone", "ghcr.io/cirruslabs/macos-sonoma-base:latest", vm_name])
# Run the VM asynchronously
tart_run_process = tart.run_async(["run", vm_name])
tart_run_process = tart.run_async(["run", vm_name] + run_opts)
# Obtain the VM's IP
stdout, _ = tart.run(["ip", vm_name, "--wait", "120"])
+1
View File
@@ -91,6 +91,7 @@ nav:
- "Integrations":
- "GitHub Actions": integrations/github-actions.md
- "GitLab Runner": integrations/gitlab-runner.md
- "Buildkite": integrations/buildkite.md
- "Self-hosted CI": integrations/cirrus-cli.md
- "Managing VMs": integrations/vm-management.md
- "Support & Licensing": licensing.md
+9 -2
View File
@@ -1,11 +1,18 @@
#!/bin/sh
# helper script to build and run a signed tart binary
# usage: ./scripts/run-signed.sh run ventura-base
# usage: ./scripts/run-signed.sh run sonoma-base
set -e
swift build --product tart
codesign --sign - --entitlements Resources/tart-dev.entitlements --force .build/debug/tart
.build/debug/tart "$@"
rm -Rf .build/tart.app/
mkdir -p .build/tart.app/Contents/MacOS .build/tart.app/Contents/Resources
cp -c .build/debug/tart .build/tart.app/Contents/MacOS/tart
cp -c Resources/embedded.provisionprofile .build/tart.app/Contents/embedded.provisionprofile
cp -c Resources/Info.plist .build/tart.app/Contents/Info.plist
cp -c Resources/AppIcon.png .build/tart.app/Contents/Resources
.build/tart.app/Contents/MacOS/tart "$@"