Compare commits

...
Author SHA1 Message Date
Fedor Korotkov a65ab5a712 Claude Code Review workflow 2025-07-10 20:23:26 +04:00
Fedor Korotkov 3f7b6d2a61 Claude PR Assistant workflow 2025-07-10 20:23:25 +04:00
Nikolay Edigaryev 0debec1266 docs: include full article content in RSS (#1104) 2025-07-08 21:06:37 +04:00
Nikolay Edigaryev 294c5fc5e5 Upgrade Swift Argument Parser to 1.6.1 (#1103)
* Upgrade Swift Argument Parser to 1.6.1

* Remove ArgumentParser workaround
2025-07-08 00:19:50 +04:00
Fedor Korotkov 99777b6740 Update README example to use macOS Tahoe (#1101) 2025-07-07 13:50:16 +04:00
Fedor KorotkovandClaude a2972aa4d9 feat: prioritize pruning of old SHA when pulling updated tags (#1102)
* feat: prioritize pruning of old SHA when pulling updated tags

When pulling a new version of a tagged image (e.g., ghcr.io/cirruslabs/macos-runner:sonoma),
set the access date of the previous SHA to epoch time (1970-01-01). This ensures that the
old SHA will be prioritized for pruning, even if it was accessed more recently than other
cached images.

This helps manage disk space more efficiently by automatically cleaning up superseded
versions of frequently-updated tagged images.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* format

* Review comments

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-07-07 08:52:05 +00:00
Fedor KorotkovandNikolay Edigaryev 3a6c5fb81d feat: Add disk image format selection with ASIF support (#1094)
* feat: Add disk image format selection with ASIF support

* fixed goreleaser-pro

* Fix ASIF disk format compatibility issues

- Use .uncached caching mode for ASIF disks to avoid Virtualization framework compatibility issues
- Improve caching mode selection logic for better maintainability
- Fix compiler warning by changing var to let for attachment variable

This resolves VM startup failures when using ASIF disk format by ensuring proper disk attachment configuration.

* Update goreleaser installation to use tap-specific formula

Change from 'brew install --cask goreleaser-pro' to 'brew install --cask goreleaser/tap/goreleaser-pro' for proper installation from the official goreleaser tap.

* Remove VS Code configuration and add to gitignore

- Remove .vscode/launch.json from repository
- Add .vscode/ to .gitignore to prevent VS Code settings from being tracked

* Implement ASIF disk resize using diskutil

- Add support for resizing ASIF disk images using diskutil image resize
- Detect disk format from VM config and route to appropriate resize method
- Use diskutil image info to get current ASIF disk size and validate resize
- Remove restriction that prevented ASIF disk resizing in Set command
- Add FailedToResizeDisk error case for proper error handling
- Maintain backward compatibility with raw disk resizing
- Add comprehensive size validation to prevent data loss

* Update Sources/tart/Commands/Create.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Update Sources/tart/DiskImageFormat.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Update Sources/tart/DiskImageFormat.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Fix test naming and remove redundant test cases

- Rename testFormatArgument to testCaseInsensitivity for clarity
- Remove redundant 'raw' and 'invalid' test cases already covered in testFormatFromString
- Remove testFormatDescriptions test as it's not very useful

Addresses review comment: https://github.com/cirruslabs/tart/pull/1094#discussion_r2152093510

* Remove canCreate property and simplify DiskImageFormat

- Remove canCreate property since it's the same as isSupported
- Remove description property entirely as it's not used
- Fix displayName for RAW format (remove UDIF reference)
- Remove checkDiskutilASIFSupport helper function

Addresses review comments:
- https://github.com/cirruslabs/tart/pull/1094#discussion_r2152109450
- https://github.com/cirruslabs/tart/pull/1094#discussion_r2152115610
- https://github.com/cirruslabs/tart/pull/1094#discussion_r2152124330

* Update Create command validation and help text

- Simplify ArgumentParser help text to let it show possible values automatically
- Remove canCreate validation since property was removed
- Simplify error message for unsupported disk formats

Addresses review comment: https://github.com/cirruslabs/tart/pull/1094#discussion_r2152113480

* Add disk format validation to Run command

- Add validation to ensure ASIF disk format is supported on current system
- Check disk format compatibility before attempting to run VM

Addresses review comment: https://github.com/cirruslabs/tart/pull/1094#discussion_r2152109450

* Use proper namespaced constant for OCI label

- Add diskFormatLabelAnnotation constant in Manifest.swift
- Use org.cirruslabs.tart.disk.format namespace for consistency
- Use variable shadowing instead of new variable name for labels

Addresses review comment: https://github.com/cirruslabs/tart/pull/1094#discussion_r2152163515

* Remove special ASIF caching mode

- Remove .uncached caching mode for ASIF disks
- Use default caching logic for all disk formats
- Testing shows .cached mode works fine on macOS 26.0

Addresses review comment: https://github.com/cirruslabs/tart/pull/1094#discussion_r2152133589

* Improve code structure in VMDirectory

- Use guard let instead of nested if let for better readability
- Reduce nesting in resizeASIFDisk function
- Improve error handling flow

Addresses review comment: https://github.com/cirruslabs/tart/pull/1094#discussion_r2152141916

* diskFormatLabel

* reverted caching mode

* Use PropertyListDecoder

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2025-06-19 18:27:30 +04:00
Nikolay Edigaryev 5793935317 tart ip: implement --resolver=agent (#1095)
* tart ip: implement --resolver=agent

* CI: fix GoReleaser installation
2025-06-19 13:07:06 +04:00
Nikolay Edigaryev 8dc8b644b2 tart exec: do not limit RPC call duration to 1 second 2025-06-11 20:15:18 +02:00
Nikolay Edigaryev b625c04131 tart exec: make sure <name> goes after flags like -i and -t in --help (#1090) 2025-06-11 21:58:30 +04:00
Nikolay Edigaryev a0c03dcce6 docs: new "Bridging the gaps with the Tart Guest Agent" blog post (#1080) 2025-06-01 19:54:45 -04:00
Nikolay Edigaryev 8539b8faae Delay Sentry initialization until after we parse the CLI arguments (#1085) 2025-05-30 17:24:19 +04:00
Nikolay Edigaryev 71159373e5 tart run: allow "--dir" with "--suspendable" (#1082) 2025-05-30 17:24:09 +04:00
Fedor Korotkov 8248f19943 Update sentry (#1079) 2025-05-28 23:06:10 +00:00
fedor 1cbc1e2cda Suspendable VMs now support consoles 2025-05-28 17:09:54 -04:00
Nikolay Edigaryev 0187834c34 tart exec: explain that Tart Guest Agent is required (#1078)
* tart exec: explain that Tart Guest Agent is required

Also handle decrease the connection timeout to 1 second
and provide a hint to the user.

* execute() can be made private

* Include error.localizedDescription
2025-05-27 12:57:56 +04:00
Nikolay Edigaryev dfbdb5559c Introduce "tart exec" command as an alternative to SSH (#1074)
* Introduce "tart exec" command as an alternative to SSH

* Simplify control socket machinery by using NIO async/await primitives

* No reason to print the "vm" object directly, just refer to it as "VM"

* Log to Apple’s Unified Logging System
2025-05-22 17:28:14 +04:00
Nikolay Edigaryev 40ab5c3af4 Fix unescaped commas in generated ArgumentParser completions (#1066)
* Fix unescaped commas in generated ArgumentParser completions

* Improve completion hints
2025-05-06 14:24:43 +04:00
fedor 280a31f707 Update docs, examples and CI to Sequoia 2025-05-04 20:49:23 -04:00
Nikolay EdigaryevandFedor Korotkov 8d49404337 Enable clipboard sharing on macOS too (#1046)
* Enable clipboard sharing on macOS too

And document which packages need to be installed on these operating
systems.

* We now use Tart Guest Agent

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

---------

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2025-04-29 21:15:03 +04:00
Nikolay EdigaryevandCopilot 64a3999a58 Improve Orchard docs (#1064)
* Iterate over Orchard Architecture description

* Document Orchard Controller customization (e.g. --listen-ssh)

* New section: "Using Orchard CLI"

* Fix Markdown unordered list indentation

* Fix "fenced code blocks should have a language specified"

* the context → a context

* Clarify different port

* Simplify labels explanation

* Studios → Studio

* Better explain resources

* crate → create

* only to place → only place

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Surround "Using resources when creating VMs" header by blank lines

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-04-29 18:10:14 +04:00
Fedor Korotkov 5c1f5a61c1 Add --no-trackpad option to disable trackpad on macOS VMs (#1060)
* Add --no-trackpad option to disable trackpad on macOS VMs

* Cleanup after AI
2025-04-22 10:17:19 -04:00
Fedor Korotkov 1310220f05 Add NSLocalNetworkUsageDescription (#1058) 2025-04-18 18:27:32 +04:00
dependabot[bot] 1fe2f1ff88 Bump golang.org/x/crypto from 0.21.0 to 0.35.0 in /benchmark (#1057)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.21.0 to 0.35.0.
- [Commits](https://github.com/golang/crypto/compare/v0.21.0...v0.35.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.35.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-18 18:01:55 +04:00
df3de33f1a Posibility to add Labels when pushing OCI Image (#1052)
* Posibility to add Labels when pushing OCI Image

Example running:
tart push $image ${registry}/org/${image}-testing --labels com.org.revision=testing --labels com.org.repo.buildid=123456

* Fix Linting

Run swift package plugin --allow-writing-to-package-directory swiftformat --cache ignore

* Update Sources/tart/Commands/Push.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Update Sources/tart/Commands/Push.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Update Sources/tart/Commands/Push.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Update Sources/tart/OCI/Manifest.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Update Sources/tart/Commands/Push.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Update Sources/tart/Commands/Push.swift

* Do not use a variable to store parseLabels() results

* Trim spaces before splitting labels and support empty values

---------

Co-authored-by: Victor Serbu <victors@4psa.com>
Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2025-04-14 16:10:12 +00:00
Fedor Korotkov 1560e4d312 Update Orchard Worker Instructions (#1055)
To include workaround for https://github.com/cirruslabs/orchard/pull/302
2025-04-14 14:21:28 +00:00
Samuel 318202fa81 fix: correct typo in validation error message for nested virtualization support (#1050) 2025-04-04 11:20:43 -04:00
Fedor Korotkov cb92a3fa67 Use full host resources for Xcode benchmarks (#1045) 2025-03-23 16:55:32 -04:00
Nikolay Edigaryev 9c30638079 docs(FAQ): document disk resizing procedure (#1042) 2025-03-18 21:36:30 +04:00
Nikolay Edigaryev a4edc6af50 Make tart set --random-serial no-nop for Linux VMs (#1027) 2025-02-11 19:16:03 +04:00
Gavinkaa 2890dda847 fixing typo (#1026) 2025-02-11 14:25:04 +00:00
Nikolay Edigaryev 2d55f3b9fa docs(FAQ): document unsupported DHCP client identifiers (#1009)
* docs(FAQ): document unsupported DHCP client identifiers

* New section "Resolving the VMs IP with bridged networking"

And a more clearer explanation of what "tart ip" does.

* Remove extraneous space in ` --resolver=arp`

* Better section name

* Add a note about Linux talkativeness

* Explain "talkativeness" a bit better
2025-01-20 19:26:08 +04:00
Fedor Korotkov d3104c71b9 [docs] update manual installation script (#1008) 2025-01-20 12:56:50 +04:00
Andrew Malchuk 3ddad55372 Fix #1004: Wrong binary path in distro (#1005) 2025-01-18 13:32:54 +04:00
Nikolay Edigaryev 72a81ca84a .goreleaser.yml: caveats stanza with DHCP fix information (#1002) 2025-01-17 17:06:02 +04:00
Nikolay Edigaryev d8945503d6 Benchmark: run XcodeBenchmark with different disk settings (#1000)
* Benchmark: run XcodeBenchmark with different disk settings

* Add Xcode benchmark results
2025-01-16 23:58:26 +04:00
Nikolay Edigaryev a0fd5435de tart run: automatically enable --net-softnet when its related opts used (#994) 2025-01-16 17:47:39 +04:00
Andrew Malchuk 4cf68fc061 Build universal binary instead of architecture dependent (#995)
* Build universal binary instead of architecture dependent

* Added universal_binaries stage to goreleaser

* Fixed paths to compiled binary in .cirrus.yml

* Revert changes in .cirrus.yml, use builtin venv module instead of virtualenv only
2025-01-16 08:22:05 -05:00
Nikolay Edigaryev b626ed415b Always use write(contentsOf:) instead of write(_:) (#997) 2025-01-15 00:26:30 +04:00
Nikolay Edigaryev dd7bace92d tart run: clarify --net-softnet-expose limitations w.r.t. PF rdr rules (#996) 2025-01-14 21:52:36 +04:00
Nikolay Edigaryev 94376ca355 tart run: introduce --net-softnet-expose (#990)
* tart run: introduce --net-softnet-expose

* --net-softnet-expose: add discussion

* --net-softnet-expose: add a note about Softnet restrictions

...and how to disable them.

* LAN → local network

* Better clarify what --net-softnet does

And how --net-softnet-allow can change that behavior.
2025-01-10 05:36:51 +04:00
Fedor Korotkov 60a481857f FAQ to help with troubleshooting (#988) 2025-01-03 14:05:50 -05:00
Nikolay Edigaryev 876271dceb docs: a firewall rule needs to be created when deploying Orchard to GCE (#983) 2024-12-24 16:39:26 +04:00
Frederic BOLTZandNikolay Edigaryev 6dd43abf03 Update FAQ.md (#979)
* Update FAQ.md

* Update docs/faq.md

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2024-12-21 18:44:36 -10:00
Nikolay Edigaryev 04c6df2efb Registry: limit the text output on unexpected status code (#981)
* Registry: limit the text output on unexpected status code

* pullBlob(): limit channel read-out on error to 4 KiB

* No need to always read channel until end

This was introduced in https://github.com/cirruslabs/tart/pull/284
because we were blocking in "urlSession(_ session: URLSession, dataTask:
URLSessionDataTask, didReceive data: Data)", which we don't do anymore.

* Fetcher.fetch(): remove "progress" argument as we don't need it anymore
2024-12-20 11:55:08 +04:00
Nikolay Edigaryev 5a8b48a392 Assorted documentation improvements (#982) 2024-12-19 19:10:24 -10:00
Nikolay Edigaryev b96ea087f5 tart pull: re-try disk layer downloads by specifying "Range" header (#980) 2024-12-19 21:21:33 +04:00
74 changed files with 2065 additions and 278 deletions
+12 -11
View File
@@ -1,7 +1,7 @@
use_compute_credits: true
task:
name: Test on Sonoma
name: Test on Sequoia
alias: test
persistent_worker:
labels:
@@ -17,8 +17,7 @@ task:
- export PATH=$(pwd)/.build/arm64-apple-macosx/debug:$PATH
# Run integration tests
- cd integration-tests
- HOMEBREW_NO_AUTO_UPDATE=1 brew install virtualenv
- virtualenv venv
- python3 -m venv --symlinks venv
- source venv/bin/activate
- pip install -r requirements.txt
- pytest --verbose --junit-xml=pytest-junit.xml
@@ -38,7 +37,7 @@ task:
name: Lint
alias: lint
macos_instance:
image: ghcr.io/cirruslabs/macos-runner:sonoma
image: ghcr.io/cirruslabs/macos-runner:sequoia
lint_script:
- swift package plugin --allow-writing-to-package-directory swiftformat --cache ignore --lint --report swiftformat.json .
always:
@@ -55,7 +54,7 @@ task:
name: Build ($BUILD_ARCH)
alias: build
macos_instance:
image: ghcr.io/cirruslabs/macos-runner:sonoma
image: ghcr.io/cirruslabs/macos-runner:sequoia
build_script: swift build --arch $BUILD_ARCH --product tart
sign_script: codesign --sign - --entitlements Resources/tart-dev.entitlements --force .build/$BUILD_ARCH-apple-macosx/debug/tart
binary_artifacts:
@@ -68,7 +67,7 @@ task:
- lint
- build
macos_instance:
image: ghcr.io/cirruslabs/macos-runner:sonoma
image: ghcr.io/cirruslabs/macos-runner:sequoia
env:
MACOS_CERTIFICATE: ENCRYPTED[552b9d275d1c2bdbc1bff778b104a8f9a53cbd0d59344d4b7f6d0ca3c811a5cefb97bef9ba0ef31c219cb07bdacdd2c2]
AC_PASSWORD: ENCRYPTED[4a761023e7e06fe2eb350c8b6e8e7ca961af193cb9ba47605f25f1d353abc3142606f412e405be48fd897a78787ea8c2]
@@ -84,8 +83,9 @@ task:
- security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k password101 build.keychain
- xcrun notarytool store-credentials "notarytool" --apple-id "hello@cirruslabs.org" --team-id "9M2P8L4D89" --password $AC_PASSWORD
install_script:
- brew install go goreleaser/tap/goreleaser-pro
- brew install go
- brew install mitchellh/gon/gon
- brew install --cask goreleaser/tap/goreleaser-pro
info_script:
- security find-identity -v
- xcodebuild -version
@@ -103,7 +103,7 @@ task:
- test
- build
macos_instance:
image: ghcr.io/cirruslabs/macos-runner:sonoma
image: ghcr.io/cirruslabs/macos-runner:sequoia
env:
MACOS_CERTIFICATE: ENCRYPTED[552b9d275d1c2bdbc1bff778b104a8f9a53cbd0d59344d4b7f6d0ca3c811a5cefb97bef9ba0ef31c219cb07bdacdd2c2]
AC_PASSWORD: ENCRYPTED[4a761023e7e06fe2eb350c8b6e8e7ca961af193cb9ba47605f25f1d353abc3142606f412e405be48fd897a78787ea8c2]
@@ -121,9 +121,10 @@ task:
- security import certificate.p12 -k build.keychain -P password101 -T /usr/bin/codesign -T /usr/bin/pkgbuild
- security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k password101 build.keychain
- xcrun notarytool store-credentials "notarytool" --apple-id "hello@cirruslabs.org" --team-id "9M2P8L4D89" --password $AC_PASSWORD
install_script:
- brew install go goreleaser/tap/goreleaser-pro getsentry/tools/sentry-cli
install_script:
- brew install go getsentry/tools/sentry-cli
- brew install mitchellh/gon/gon
- brew install --cask goreleaser/tap/goreleaser-pro
info_script:
- security find-identity -v
- xcodebuild -version
@@ -153,7 +154,7 @@ task:
registry_config: ENCRYPTED[!cf1a0f25325aa75bad3ce6ebc890bc53eb0044c02efa70d8cefb83ba9766275a994b4831706c52630a0692b2fa9cfb9e!]
env:
DEPLOY_TOKEN: ENCRYPTED[!45ed45666558902ed1c2400add734ec063103bec31841847e8c8764802fca229bfa6d85c690e16ad159e047574b48793!]
deploy_script:
deploy_script:
- git config --global user.name "Cirrus CI"
- git config --global user.name "hello@cirruslabs.org"
- git remote set-url origin https://$DEPLOY_TOKEN@github.com/cirruslabs/tart/
+78
View File
@@ -0,0 +1,78 @@
name: Claude Code Review
on:
pull_request:
types: [opened, synchronize]
# Optional: Only run on specific file changes
# paths:
# - "src/**/*.ts"
# - "src/**/*.tsx"
# - "src/**/*.js"
# - "src/**/*.jsx"
jobs:
claude-review:
# Optional: Filter by PR author
# if: |
# github.event.pull_request.user.login == 'external-contributor' ||
# github.event.pull_request.user.login == 'new-developer' ||
# github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code Review
id: claude-review
uses: anthropics/claude-code-action@beta
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# Optional: Specify model (defaults to Claude Sonnet 4, uncomment for Claude Opus 4)
# model: "claude-opus-4-20250514"
# Direct prompt for automated review (no @claude mention needed)
direct_prompt: |
Please review this pull request and provide feedback on:
- Code quality and best practices
- Potential bugs or issues
- Performance considerations
- Security concerns
- Test coverage
Be constructive and helpful in your feedback.
# Optional: Use sticky comments to make Claude reuse the same comment on subsequent pushes to the same PR
# use_sticky_comment: true
# Optional: Customize review based on file types
# direct_prompt: |
# Review this PR focusing on:
# - For TypeScript files: Type safety and proper interface usage
# - For API endpoints: Security, input validation, and error handling
# - For React components: Performance, accessibility, and best practices
# - For tests: Coverage, edge cases, and test quality
# Optional: Different prompts for different authors
# direct_prompt: |
# ${{ github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR' &&
# 'Welcome! Please review this PR from a first-time contributor. Be encouraging and provide detailed explanations for any suggestions.' ||
# 'Please provide a thorough code review focusing on our coding standards and best practices.' }}
# Optional: Add specific tools for running tests or linting
# allowed_tools: "Bash(npm run test),Bash(npm run lint),Bash(npm run typecheck)"
# Optional: Skip review for certain conditions
# if: |
# !contains(github.event.pull_request.title, '[skip-review]') &&
# !contains(github.event.pull_request.title, '[WIP]')
+64
View File
@@ -0,0 +1,64 @@
name: Claude Code
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
jobs:
claude:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@beta
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: |
actions: read
# Optional: Specify model (defaults to Claude Sonnet 4, uncomment for Claude Opus 4)
# model: "claude-opus-4-20250514"
# Optional: Customize the trigger phrase (default: @claude)
# trigger_phrase: "/claude"
# Optional: Trigger when specific user is assigned to an issue
# assignee_trigger: "claude-bot"
# Optional: Allow Claude to run specific commands
# allowed_tools: "Bash(npm install),Bash(npm run build),Bash(npm run test:*),Bash(npm run lint:*)"
# Optional: Add custom instructions for Claude to customize its behavior for your project
# custom_instructions: |
# Follow our coding standards
# Ensure all new code has tests
# Use TypeScript for new files
# Optional: Custom environment variables for Claude
# claude_env: |
# NODE_ENV: test
+3
View File
@@ -8,6 +8,9 @@ tart.xcodeproj/
# AppCode
.idea/
# VS Code
.vscode/
# Swift
.build/
+18 -5
View File
@@ -1,11 +1,12 @@
version: 2
project_name: tart
before:
hooks:
- .ci/set-version.sh
- swift build --arch x86_64 -c release --product tart
- swift build --arch arm64 -c release --product tart
- gon gon.hcl
- swift build --arch arm64 --configuration release --product tart
- swift build --arch x86_64 --configuration release --product tart
builds:
- id: tart
@@ -20,8 +21,14 @@ builds:
prebuilt:
path: '.build/{{- if eq .Arch "arm64" }}arm64{{- else }}x86_64{{ end }}-apple-macosx/release/tart'
universal_binaries:
- name_template: tart.app/Contents/MacOS/tart
replace: true
hooks:
post: gon gon.hcl
archives:
- name_template: "{{ .ProjectName }}-{{ .Arch }}"
- name_template: "{{ .ProjectName }}"
files:
- src: Resources/embedded.provisionprofile
dst: tart.app/Contents
@@ -42,7 +49,13 @@ brews:
repository:
owner: cirruslabs
name: homebrew-cli
caveats: See the GitHub repository for more information
caveats: |
Tart has been installed. You might want to reduce the default DHCP lease time
from 86,400 to 600 seconds to avoid DHCP shortage when running lots of VMs daily:
sudo defaults write /Library/Preferences/SystemConfiguration/com.apple.InternetSharing.default.plist bootpd -dict DHCPLeaseTimeSecs -int 600
See https://tart.run/faq/#changing-the-default-dhcp-lease-time for more details.
homepage: https://github.com/cirruslabs/tart
license: "Fair Source"
description: Run macOS and Linux VMs on Apple Hardware
+1
View File
@@ -29,6 +29,7 @@ Table of Contents
1. Code should follow camel case
2. Code should follow [SwiftFormat](https://github.com/nicklockwood/SwiftFormat#swift-package-manager-plugin) guidelines. You can auto-format the code by running the following command:
```bash
swift package plugin --allow-writing-to-package-directory swiftformat --cache ignore .
```
+122 -5
View File
@@ -1,5 +1,5 @@
{
"originHash" : "22b3726bc4e4c6e9c04ac97cb08a82967feb39960a93d2909768a16e11576748",
"originHash" : "668bad809d4882f75f097e66a12a6dbc8e61ec998f1800a7e09439c854fadda1",
"pins" : [
{
"identity" : "antlr4",
@@ -10,6 +10,24 @@
"version" : "4.13.2"
}
},
{
"identity" : "cirruslabs_tart-guest-agent_apple_swift",
"kind" : "remoteSourceControl",
"location" : "https://buf.build/gen/swift/git/1.28.2-00000000000000-17d7dedafb88.1/cirruslabs_tart-guest-agent_apple_swift.git",
"state" : {
"revision" : "ccfae5de1917cdb0d7c5000008fa5ed0bad032bf",
"version" : "1.28.2-00000000000000-17d7dedafb88.1"
}
},
{
"identity" : "cirruslabs_tart-guest-agent_grpc_swift",
"kind" : "remoteSourceControl",
"location" : "https://buf.build/gen/swift/git/1.24.2-00000000000000-17d7dedafb88.1/cirruslabs_tart-guest-agent_grpc_swift.git",
"state" : {
"branch" : "1.24.2-00000000000000-17d7dedafb88.1",
"revision" : "b8421f137325fe8de737ff5b61238f6f2131b2a8"
}
},
{
"identity" : "dynamic",
"kind" : "remoteSourceControl",
@@ -19,6 +37,15 @@
"revision" : "772883073d044bc754d401cabb6574624eb3778f"
}
},
{
"identity" : "grpc-swift",
"kind" : "remoteSourceControl",
"location" : "https://github.com/grpc/grpc-swift.git",
"state" : {
"revision" : "8c5e99d0255c373e0330730d191a3423c57373fb",
"version" : "1.24.2"
}
},
{
"identity" : "semaphore",
"kind" : "remoteSourceControl",
@@ -33,8 +60,8 @@
"kind" : "remoteSourceControl",
"location" : "https://github.com/getsentry/sentry-cocoa",
"state" : {
"revision" : "5575af93efb776414f243e93d6af9f6258dc539a",
"version" : "8.36.0"
"revision" : "65b3d2a7608685e8d4a37c68fa2c64f28d0b537e",
"version" : "8.51.1"
}
},
{
@@ -51,8 +78,8 @@
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-argument-parser",
"state" : {
"revision" : "41982a3656a71c768319979febd796c6fd111d5c",
"version" : "1.5.0"
"revision" : "309a47b2b1d9b5e991f36961c983ecec72275be3",
"version" : "1.6.1"
}
},
{
@@ -64,6 +91,33 @@
"version" : "1.2.0"
}
},
{
"identity" : "swift-collections",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-collections.git",
"state" : {
"revision" : "671108c96644956dddcd89dd59c203dcdb36cec7",
"version" : "1.1.4"
}
},
{
"identity" : "swift-http-structured-headers",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-http-structured-headers.git",
"state" : {
"revision" : "db6eea3692638a65e2124990155cd220c2915903",
"version" : "1.3.0"
}
},
{
"identity" : "swift-http-types",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-http-types.git",
"state" : {
"revision" : "a0a57e949a8903563aba4615869310c0ebf14c03",
"version" : "1.4.0"
}
},
{
"identity" : "swift-log",
"kind" : "remoteSourceControl",
@@ -73,6 +127,51 @@
"version" : "1.6.1"
}
},
{
"identity" : "swift-nio",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio.git",
"state" : {
"revision" : "34d486b01cd891297ac615e40d5999536a1e138d",
"version" : "2.83.0"
}
},
{
"identity" : "swift-nio-extras",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-extras.git",
"state" : {
"revision" : "f1f6f772198bee35d99dd145f1513d8581a54f2c",
"version" : "1.26.0"
}
},
{
"identity" : "swift-nio-http2",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-http2.git",
"state" : {
"revision" : "4281466512f63d1bd530e33f4aa6993ee7864be0",
"version" : "1.36.0"
}
},
{
"identity" : "swift-nio-ssl",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-ssl.git",
"state" : {
"revision" : "4b38f35946d00d8f6176fe58f96d83aba64b36c7",
"version" : "2.31.0"
}
},
{
"identity" : "swift-nio-transport-services",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-transport-services.git",
"state" : {
"revision" : "cd1e89816d345d2523b11c55654570acd5cd4c56",
"version" : "1.24.0"
}
},
{
"identity" : "swift-numerics",
"kind" : "remoteSourceControl",
@@ -82,6 +181,15 @@
"version" : "1.0.2"
}
},
{
"identity" : "swift-protobuf",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-protobuf.git",
"state" : {
"revision" : "ebc7251dd5b37f627c93698e4374084d98409633",
"version" : "1.28.2"
}
},
{
"identity" : "swift-retry",
"kind" : "remoteSourceControl",
@@ -100,6 +208,15 @@
"version" : "1.8.0"
}
},
{
"identity" : "swift-system",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-system.git",
"state" : {
"revision" : "a34201439c74b53f0fd71ef11741af7e7caf01e1",
"version" : "1.4.2"
}
},
{
"identity" : "swift-xattr",
"kind" : "remoteSourceControl",
+6 -2
View File
@@ -10,20 +10,22 @@ let package = Package(
.executable(name: "tart", targets: ["tart"])
],
dependencies: [
.package(url: "https://github.com/apple/swift-argument-parser", from: "1.3.1"),
.package(url: "https://github.com/apple/swift-argument-parser", from: "1.6.1"),
.package(url: "https://github.com/mhdhejazi/Dynamic", branch: "master"),
.package(url: "https://github.com/apple/swift-algorithms", from: "1.2.0"),
.package(url: "https://github.com/malcommac/SwiftDate", from: "7.0.0"),
.package(url: "https://github.com/antlr/antlr4", exact: "4.13.2"),
.package(url: "https://github.com/apple/swift-atomics.git", .upToNextMajor(from: "1.2.0")),
.package(url: "https://github.com/nicklockwood/SwiftFormat", from: "0.53.6"),
.package(url: "https://github.com/getsentry/sentry-cocoa", from: "8.36.0"),
.package(url: "https://github.com/getsentry/sentry-cocoa", from: "8.51.1"),
.package(url: "https://github.com/cfilipov/TextTable", branch: "master"),
.package(url: "https://github.com/sersoft-gmbh/swift-sysctl.git", from: "1.8.0"),
.package(url: "https://github.com/orchetect/SwiftRadix", from: "1.3.1"),
.package(url: "https://github.com/groue/Semaphore", from: "0.0.8"),
.package(url: "https://github.com/fumoboy007/swift-retry", from: "0.2.3"),
.package(url: "https://github.com/jozefizso/swift-xattr", from: "3.0.0"),
.package(url: "https://github.com/grpc/grpc-swift.git", .upToNextMajor(from: "1.24.2")),
.package(url: "https://buf.build/gen/swift/git/1.24.2-00000000000000-17d7dedafb88.1/cirruslabs_tart-guest-agent_grpc_swift.git", revision: "1.24.2-00000000000000-17d7dedafb88.1"),
],
targets: [
.executableTarget(name: "tart", dependencies: [
@@ -40,6 +42,8 @@ let package = Package(
.product(name: "Semaphore", package: "Semaphore"),
.product(name: "DMRetry", package: "swift-retry"),
.product(name: "XAttr", package: "swift-xattr"),
.product(name: "GRPC", package: "grpc-swift"),
.product(name: "Cirruslabs_TartGuestAgent_Grpc_Swift", package: "cirruslabs_tart-guest-agent_grpc_swift"),
], exclude: [
"OCI/Reference/Makefile",
"OCI/Reference/Reference.g4",
+2 -2
View File
@@ -66,8 +66,8 @@ Try running a Tart VM on your Apple Silicon device running macOS 13.0 (Ventura)
```bash
brew install cirruslabs/cli/tart
tart clone ghcr.io/cirruslabs/macos-sonoma-base:latest sonoma-base
tart run sonoma-base
tart clone ghcr.io/cirruslabs/macos-tahoe-base:latest tahoe-base
tart run tahoe-base
```
Please check the [official documentation](https://tart.run) for more information and/or feel free to use [discussions](https://github.com/cirruslabs/tart/discussions)
+2
View File
@@ -21,5 +21,7 @@
<key>NSAllowsArbitraryLoads</key>
<true/>
</dict>
<key>NSLocalNetworkUsageDescription</key>
<string>Access to OCI registries on the local network</string>
</dict>
</plist>
+11 -3
View File
@@ -10,7 +10,7 @@ struct Create: AsyncParsableCommand {
@Argument(help: "VM name")
var name: String
@Option(help: ArgumentHelp("create a macOS VM using path to the IPSW file or URL (or \"latest\", to fetch the latest supported IPSW automatically)", valueName: "path"))
@Option(help: ArgumentHelp("create a macOS VM using path to the IPSW file or URL (or \"latest\", to fetch the latest supported IPSW automatically)", valueName: "path"), completion: .file())
var fromIPSW: String?
@Flag(help: "create a Linux VM")
@@ -19,6 +19,9 @@ struct Create: AsyncParsableCommand {
@Option(help: ArgumentHelp("Disk size in GB"))
var diskSize: UInt16 = 50
@Option(help: ArgumentHelp("Disk image format", discussion: "ASIF format provides better performance but requires macOS 26 Tahoe or later"))
var diskFormat: DiskImageFormat = .raw
func validate() throws {
if fromIPSW == nil && !linux {
throw ValidationError("Please specify either a --from-ipsw or --linux option!")
@@ -28,6 +31,11 @@ struct Create: AsyncParsableCommand {
throw ValidationError("Only Linux VMs are supported on Intel!")
}
#endif
// Validate disk format support
if !diskFormat.isSupported {
throw ValidationError("Disk format '\(diskFormat.rawValue)' is not supported on this system.")
}
}
func run() async throws {
@@ -58,12 +66,12 @@ struct Create: AsyncParsableCommand {
ipswURL = URL(fileURLWithPath: NSString(string: fromIPSW).expandingTildeInPath)
}
_ = try await VM(vmDir: tmpVMDir, ipswURL: ipswURL, diskSizeGB: diskSize)
_ = try await VM(vmDir: tmpVMDir, ipswURL: ipswURL, diskSizeGB: diskSize, diskFormat: diskFormat)
}
#endif
if linux {
_ = try await VM.linux(vmDir: tmpVMDir, diskSizeGB: diskSize)
_ = try await VM.linux(vmDir: tmpVMDir, diskSizeGB: diskSize, diskFormat: diskFormat)
}
try VMStorageLocal().move(name, from: tmpVMDir)
+180
View File
@@ -0,0 +1,180 @@
import ArgumentParser
import Foundation
import NIOPosix
import GRPC
import Cirruslabs_TartGuestAgent_Grpc_Swift
struct ExecCustomExitCodeError: Error {
let exitCode: Int32
}
struct Exec: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Execute a command in a running VM", discussion: """
Requires Tart Guest Agent running in a guest VM.
Note that all non-vanilla Cirrus Labs VM images already have the Tart Guest Agent installed.
""")
@Flag(name: [.customShort("i")], help: "Attach host's standard input to a remote command")
var interactive: Bool = false
@Flag(name: [.customShort("t")], help: "Allocate a remote pseudo-terminal (PTY)")
var tty: Bool = false
@Argument(help: "VM name", completion: .custom(completeLocalMachines))
var name: String
@Argument(parsing: .captureForPassthrough, help: "Command to execute")
var command: [String]
func run() async throws {
// We only have withThrowingDiscardingTaskGroup available starting from macOS 14
if #unavailable(macOS 14) {
throw RuntimeError.Generic("\"tart exec\" is only available on macOS 14 (Sonoma) or newer")
}
// Open VM's directory
let vmDir = try VMStorageLocal().open(name)
// Ensure that the VM is running
if try !vmDir.running() {
throw RuntimeError.VMNotRunning(name)
}
// Create a gRPC channel connected to the VM's control socket
let group = MultiThreadedEventLoopGroup(numberOfThreads: 1)
defer {
try! group.syncShutdownGracefully()
}
let channel = try GRPCChannelPool.with(
target: .unixDomainSocket(vmDir.controlSocketURL.path()),
transportSecurity: .plaintext,
eventLoopGroup: group,
)
defer {
try! channel.close().wait()
}
// Switch controlling terminal into raw mode when remote pseudo-terminal is requested
var state: State? = nil
if tty && Term.IsTerminal() {
state = try Term.MakeRaw()
}
defer {
// Restore terminal to its initial state
if let state {
try! Term.Restore(state)
}
}
// Execute a command in a running VM
do {
try await execute(channel)
} catch let error as GRPCConnectionPoolError {
throw RuntimeError.Generic("Failed to connect to the VM using its control socket: \(error.localizedDescription), is the Tart Guest Agent running?")
}
}
private func execute(_ channel: GRPCChannel) async throws {
let agentAsyncClient = AgentAsyncClient(channel: channel)
let execCall = agentAsyncClient.makeExecCall()
try await execCall.requestStream.send(.with {
$0.type = .command(.with {
$0.name = command[0]
$0.args = Array(command.dropFirst(1))
$0.interactive = interactive
$0.tty = tty
$0.terminalSize = .with {
let (width, height) = try! Term.GetSize()
$0.cols = UInt32(width)
$0.rows = UInt32(height)
}
})
})
// Process command events and optionally send our standard input and/or terminal dimensions
try await withThrowingTaskGroup { group in
// Stream host's standard input if interactive mode is enabled
if interactive {
let stdinStream = AsyncStream<Data> { continuation in
let handle = FileHandle.standardInput
handle.readabilityHandler = { handle in
let data = handle.availableData
continuation.yield(data)
if data.isEmpty {
continuation.finish()
}
}
}
group.addTask {
for await stdinData in stdinStream {
try await execCall.requestStream.send(.with {
$0.type = .standardInput(.with {
$0.data = stdinData
})
})
}
}
}
// Stream host's terminal dimensions if pseudo-terminal is requested
signal(SIGWINCH, SIG_IGN)
let sigwinchSrc = DispatchSource.makeSignalSource(signal: SIGWINCH)
sigwinchSrc.activate()
if tty {
let terminalDimensionsStream = AsyncStream { continuation in
sigwinchSrc.setEventHandler {
continuation.yield(try! Term.GetSize())
}
}
group.addTask {
for await (width, height) in terminalDimensionsStream {
try await execCall.requestStream.send(.with {
$0.type = .terminalResize(.with {
$0.cols = UInt32(width)
$0.rows = UInt32(height)
})
})
}
}
}
// Process command events
group.addTask {
for try await response in execCall.responseStream {
switch response.type {
case .standardOutput(let ioChunk):
try FileHandle.standardOutput.write(contentsOf: ioChunk.data)
case .standardError(let ioChunk):
try FileHandle.standardError.write(contentsOf: ioChunk.data)
case .exit(let exit):
throw ExecCustomExitCodeError(exitCode: exit.code)
default:
// Unknown event, do nothing
continue
}
}
}
while !group.isEmpty {
do {
try await group.next()
} catch {
group.cancelAll()
throw error
}
}
}
}
}
+1 -1
View File
@@ -7,7 +7,7 @@ struct Export: AsyncParsableCommand {
@Argument(help: "Source VM name.", completion: .custom(completeMachines))
var name: String
@Argument(help: "Path to the destination file.")
@Argument(help: "Path to the destination file.", completion: .file())
var path: String?
func run() async throws {
+2 -1
View File
@@ -6,6 +6,7 @@ fileprivate struct VMInfo: Encodable {
let CPU: Int
let Memory: UInt64
let Disk: Int
let DiskFormat: String
let Size: String
let Display: String
let Running: Bool
@@ -26,7 +27,7 @@ struct Get: AsyncParsableCommand {
let vmConfig = try VMConfig(fromURL: vmDir.configURL)
let memorySizeInMb = vmConfig.memorySize / 1024 / 1024
let info = VMInfo(OS: vmConfig.os, CPU: vmConfig.cpuCount, Memory: memorySizeInMb, Disk: try vmDir.sizeGB(), Size: String(format: "%.3f", Float(try vmDir.allocatedSizeBytes()) / 1000 / 1000 / 1000), Display: vmConfig.display.description, Running: try vmDir.running(), State: try vmDir.state().rawValue)
let info = VMInfo(OS: vmConfig.os, CPU: vmConfig.cpuCount, Memory: memorySizeInMb, Disk: try vmDir.sizeGB(), DiskFormat: vmConfig.diskFormat.rawValue, Size: String(format: "%.3f", Float(try vmDir.allocatedSizeBytes()) / 1000 / 1000 / 1000), Display: vmConfig.display.description, Running: try vmDir.running(), State: try vmDir.state().rawValue)
print(format.renderSingle(info))
}
}
+19 -11
View File
@@ -5,9 +5,9 @@ import SystemConfiguration
import Sentry
enum IPResolutionStrategy: String, ExpressibleByArgument, CaseIterable {
case dhcp, arp
case dhcp, arp, agent
private(set) static var allValueStrings: [String] = Format.allCases.map { "\($0)"}
private(set) static var allValueStrings: [String] = Self.allCases.map { "\($0)"}
}
struct IP: AsyncParsableCommand {
@@ -19,13 +19,11 @@ struct IP: AsyncParsableCommand {
@Option(help: "Number of seconds to wait for a potential VM booting")
var wait: UInt16 = 0
@Option(help: ArgumentHelp("Strategy for resolving IP address: dhcp or arp",
@Option(help: ArgumentHelp("Strategy for resolving IP address",
discussion: """
By default, Tart is looking up and parsing DHCP lease file to determine the IP of the VM.\n
This method is fast and the most reliable but only returns local IP adresses.\n
Alternatively, Tart can call external `arp` executable and parse it's output.\n
In case of enabled Bridged Networking this method will return VM's IP address on the network interface used for Bridged Networking.\n
Note that `arp` strategy won't work for VMs using `--net-softnet`.
By default, Tart is using a "dhcp" resolver which parses the DHCP lease file on host and tries to find an entry containing the VM's MAC address. This method is fast and the most reliable, but only works for VMs are not using the bridged networking.\n
Alternatively, Tart has an "arp" resolver which calls an external "arp" executable and parses it's output. This works for VMs using bridged networking and returns their IP, but when they generate enough network activity to populate the host's ARP table. Note that "arp" strategy won't work for VMs using the Softnet networking.\n
A third strategy, "agent" works in all cases reliably, but requires Guest agent for Tart VMs (https://github.com/cirruslabs/tart-guest-agent) to be installed inside of a VM.
"""))
var resolver: IPResolutionStrategy = .dhcp
@@ -34,14 +32,16 @@ struct IP: AsyncParsableCommand {
let vmConfig = try VMConfig.init(fromURL: vmDir.configURL)
let vmMACAddress = MACAddress(fromString: vmConfig.macAddress.string)!
guard let ip = try await IP.resolveIP(vmMACAddress, resolutionStrategy: resolver, secondsToWait: wait) else {
guard let ip = try await IP.resolveIP(vmMACAddress, resolutionStrategy: resolver, secondsToWait: wait, controlSocketURL: vmDir.controlSocketURL) else {
var message = "no IP address found"
if try !vmDir.running() {
message += ", is your VM running?"
}
if (vmConfig.os == .linux && resolver == .arp) {
if (resolver == .agent) {
message += " (also make sure that Guest agent for Tart is running inside of a VM)"
} else if (vmConfig.os == .linux && resolver == .arp) {
message += " (not all Linux distributions are compatible with the ARP resolver)"
}
@@ -51,7 +51,7 @@ struct IP: AsyncParsableCommand {
print(ip)
}
static public func resolveIP(_ vmMACAddress: MACAddress, resolutionStrategy: IPResolutionStrategy = .dhcp, secondsToWait: UInt16 = 0) async throws -> IPv4Address? {
static public func resolveIP(_ vmMACAddress: MACAddress, resolutionStrategy: IPResolutionStrategy = .dhcp, secondsToWait: UInt16 = 0, controlSocketURL: URL? = nil) async throws -> IPv4Address? {
let waitUntil = Calendar.current.date(byAdding: .second, value: Int(secondsToWait), to: Date.now)!
repeat {
@@ -64,6 +64,14 @@ struct IP: AsyncParsableCommand {
if let leases = try Leases(), let ip = leases.ResolveMACAddress(macAddress: vmMACAddress) {
return ip
}
case .agent:
guard let controlSocketURL = controlSocketURL else {
throw RuntimeError.Generic("Cannot perform IP resolution via Tart Guest Agent when control socket URL is not set")
}
if let ip = try await AgentResolver.ResolveIP(controlSocketURL) {
return ip
}
}
// wait a second
+2 -2
View File
@@ -4,10 +4,10 @@ import Foundation
struct Import: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Import VM from a compressed .tvm file")
@Argument(help: "Path to a file created with \"tart export\".")
@Argument(help: "Path to a file created with \"tart export\".", completion: .file())
var path: String
@Argument(help: "Destination VM name.")
@Argument(help: "Destination VM name.", completion: .custom(completeLocalMachines))
var name: String
func validate() throws {
+1 -1
View File
@@ -18,7 +18,7 @@ struct List: AsyncParsableCommand {
@Option(help: ArgumentHelp("Only display VMs from the specified source (e.g. --source local, --source oci)."))
var source: String?
@Option(help: "Output format: text or json")
@Option(help: "Output format: text or json", completion: .list(["text", "json"]))
var format: Format = .text
@Flag(name: [.short, .long], help: ArgumentHelp("Only display VM names."))
+1 -1
View File
@@ -7,7 +7,7 @@ import SwiftDate
struct Prune: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Prune OCI and IPSW caches or local VMs")
@Option(help: ArgumentHelp("Entries to remove: \"caches\" targets OCI and IPSW caches and \"vms\" targets local VMs."))
@Option(help: ArgumentHelp("Entries to remove: \"caches\" targets OCI and IPSW caches and \"vms\" targets local VMs."), completion: .list(["caches", "vms"]))
var entries: String = "caches"
@Option(help: ArgumentHelp("Remove entries that were last accessed more than n days ago",
+29 -1
View File
@@ -26,6 +26,11 @@ struct Push: AsyncParsableCommand {
"""))
var chunkSize: Int = 0
@Option(name: [.customLong("label")], help: ArgumentHelp("additional metadata to attach to the OCI image configuration in key=value format",
discussion: "Can be specified multiple times to attach multiple labels."))
var labels: [String] = []
@Option(help: .hidden)
var diskFormat: String = "v2"
@@ -81,7 +86,8 @@ struct Push: AsyncParsableCommand {
references: references,
chunkSizeMb: chunkSize,
diskFormat: diskFormat,
concurrency: concurrency
concurrency: concurrency,
labels: parseLabels()
)
// Populate the local cache (if requested)
if populateCache {
@@ -115,6 +121,28 @@ struct Push: AsyncParsableCommand {
return RemoteName(host: registry.host!, namespace: registry.namespace,
reference: Reference(digest: digest))
}
// Helper method to convert labels array to dictionary
func parseLabels() -> [String: String] {
var result = [String: String]()
for label in labels {
let parts = label.trimmingCharacters(in: .whitespaces).split(separator: "=", maxSplits: 1, omittingEmptySubsequences: false)
let key = parts.count > 0 ? String(parts[0]) : ""
let value = parts.count > 1 ? String(parts[1]) : ""
// It sometimes makes sense to provide an empty value,
// but not an empty key
if key.isEmpty {
continue
}
result[key] = value
}
return result
}
}
extension Collection where Element == RemoteName {
+87 -16
View File
@@ -81,7 +81,7 @@ struct Run: AsyncParsableCommand {
@Option(help: ArgumentHelp(
"Attach an externally created serial console",
discussion: "Alternative to `--serial` flag for programmatic integrations."
))
), completion: .file())
var serialPath: String?
@Flag(help: ArgumentHelp("Force open a UI window, even when VNC is enabled.", visibility: .private))
@@ -92,7 +92,7 @@ struct Run: AsyncParsableCommand {
@Flag(help: ArgumentHelp(
"Disable clipboard sharing between host and guest.",
discussion: "Only works with Linux-based guest operating systems."))
discussion: "Clipboard sharing requires spice-vdagent package on Linux and https://github.com/cirruslabs/tart-guest-agent on macOS."))
var noClipboard: Bool = false
#if arch(arm64)
@@ -117,7 +117,7 @@ struct Run: AsyncParsableCommand {
var vncExperimental: Bool = false
@Option(help: ArgumentHelp("""
Additional disk attachments with an optional read-only and synchronization options (e.g. --disk="disk.bin", --disk="ubuntu.iso:ro", --disk="/dev/disk0", --disk "ghcr.io/cirruslabs/xcode:16.0:ro" or --disk="nbd://localhost:10809/myDisk:sync=none")
Additional disk attachments with an optional read-only and synchronization options in the form of path[:options] (e.g. --disk="disk.bin", --disk="ubuntu.iso:ro", --disk="/dev/disk0", --disk "ghcr.io/cirruslabs/xcode:16.0:ro" or --disk="nbd://localhost:10809/myDisk:sync=none")
""", discussion: """
The disk attachment can be a:
@@ -138,8 +138,8 @@ struct Run: AsyncParsableCommand {
To work around this pass TART_HOME explicitly:
sudo TART_HOME="$HOME/.tart" tart run sonoma --disk=/dev/disk0
""", valueName: "path[:options]"))
sudo TART_HOME="$HOME/.tart" tart run sequoia --disk=/dev/disk0
""", valueName: "path[:options]"), completion: .file())
var disk: [String] = []
#if arch(arm64)
@@ -158,7 +158,7 @@ struct Run: AsyncParsableCommand {
#endif
var rosettaTag: String?
@Option(help: ArgumentHelp("Additional directory shares with an optional read-only and mount tag options (e.g. --dir=\"~/src/build\" or --dir=\"~/src/sources:ro\")", discussion: """
@Option(help: ArgumentHelp("Additional directory shares with an optional read-only and mount tag options in the form of [name:]path[:options] (e.g. --dir=\"~/src/build\" or --dir=\"~/src/sources:ro\")", discussion: """
Requires host to be macOS 13.0 (Ventura) or newer. macOS guests must be running macOS 13.0 (Ventura) or newer too.
Options are comma-separated and are as follows:
@@ -170,7 +170,7 @@ struct Run: AsyncParsableCommand {
Mount tag can be overridden by appending tag property to the directory share (e.g. --dir=\"~/src/build:tag=build\" or --dir=\"~/src/build:ro,tag=build\"). Then it can be mounted via "mount_virtiofs build ~/build" inside guest macOS and "mount -t virtiofs build ~/build" inside guest Linux.
In case of passing multiple directories per mount tag it is required to prefix them with names e.g. --dir=\"build:~/src/build\" --dir=\"sources:~/src/sources:ro\". These names will be used as directory names under the mounting point inside guests. For the example above it will be "/Volumes/My Shared Files/build" and "/Volumes/My Shared Files/sources" respectively.
""", valueName: "[name:]path[:options]"))
""", valueName: "[name:]path[:options]"), completion: .directory)
var dir: [String] = []
@Flag(help: ArgumentHelp("Enable nested virtualization if possible"))
@@ -183,13 +183,49 @@ struct Run: AsyncParsableCommand {
""", valueName: "interface name"))
var netBridged: [String] = []
@Flag(help: ArgumentHelp("Use software networking instead of the default shared (NAT) networking",
discussion: "Learn how to configure Softnet for use with Tart here: https://github.com/cirruslabs/softnet"))
@Flag(help: ArgumentHelp("Use software networking provided by Softnet instead of the default shared (NAT) networking",
discussion: """
Softnet provides better network isolation and alleviates DHCP shortage on production systems. Tart invokes Softnet when this option is specified as a sub-process and communicates with it over socketpair(2).
It is essentially a userspace packet filter which restricts the VM networking and prevents a class of security issues, such as ARP spoofing. By default, the VM will only be able to:
* send traffic from its own MAC-address
* send traffic from the IP-address assigned to it by the DHCP
* send traffic to globally routable IPv4 addresses
* send traffic to gateway IP of the vmnet bridge (this would normally be \"bridge100\" interface)
* receive any incoming traffic
In addition, Softnet tunes macOS built-in DHCP server to decrease its lease time from the default 86,400 seconds (one day) to 600 seconds (10 minutes). This is especially important when you use Tart to clone and run a lot of ephemeral VMs over a period of one day.
More on Softnet here: https://github.com/cirruslabs/softnet
"""))
var netSoftnet: Bool = false
@Option(help: ArgumentHelp("Comma-separated list of CIDRs to allow the traffic to when using Softnet isolation\n(e.g. --net-softnet-allow=192.168.0.0/24)", valueName: "comma-separated CIDRs"))
@Option(help: ArgumentHelp("Comma-separated list of CIDRs to allow the traffic to when using Softnet isolation\n(e.g. --net-softnet-allow=192.168.0.0/24)", discussion: """
This option allows you bypass the private IPv4 address space restrictions imposed by --net-softnet.
For example, you can allow the VM to communicate with the local network with e.g. --net-softnet-allow=10.0.0.0/16 or to completely disable the destination based restrictions with --net-softnet-allow=0.0.0.0/0.
Implies --net-softnet.
""", valueName: "comma-separated CIDRs"))
var netSoftnetAllow: String?
@Option(help: ArgumentHelp("Comma-separated list of TCP ports to expose (e.g. --net-softnet-expose 2222:22,8080:80)", discussion: """
Options are comma-separated and are as follows:
* EXTERNAL_PORT:INTERNAL_PORT — forward TCP traffic from the EXTERNAL_PORT on a host's egress interface (automatically detected and could be Wi-Fi, Ethernet and a VPN interface) to the INTERNAL_PORT on guest's IP (as reported by "tart ip")
Note that for the port forwarding to work correctly:
* the software in guest listening on INTERNAL_PORT should either listen on 0.0.0.0 or on an IP address assigned to that guest
* connection to the EXTERNAL_PORT should be performed from the local network that the host is attached to or from the internet, it's not possible to connect to that forwarded port from the host itself
Another thing to keep in mind is that regular Softnet restrictions will still apply even to port forwarding. So if you're planning to access your VM from local network, and your local network is 192.168.0.0/24, for example, then add --net-softnet-allow=192.168.0.0/24. If you only need port forwarding, to completely disable Softnet restrictions you can use --net-softnet-allow=0.0.0.0/0.
Implies --net-softnet.
""", valueName: "comma-separated port specifications"))
var netSoftnetExpose: String?
@Flag(help: ArgumentHelp("Restrict network access to the host-only network"))
var netHost: Bool = false
@@ -224,12 +260,22 @@ struct Run: AsyncParsableCommand {
#endif
var captureSystemKeys: Bool = false
#if arch(arm64)
@Flag(help: ArgumentHelp("Don't add trackpad as a pointing device on macOS VMs"))
#endif
var noTrackpad: Bool = false
mutating func validate() throws {
if vnc && vncExperimental {
throw ValidationError("--vnc and --vnc-experimental are mutually exclusive")
}
// check that not more than one network option is specified
// Automatically enable --net-softnet when any of its related options are specified
if netSoftnetAllow != nil || netSoftnetExpose != nil {
netSoftnet = true
}
// Check that no more than one network option is specified
var netFlags = 0
if netBridged.count > 0 { netFlags += 1 }
if netSoftnet { netFlags += 1 }
@@ -249,7 +295,7 @@ struct Run: AsyncParsableCommand {
if nested {
if #unavailable(macOS 15) {
throw ValidationError("Nested virtualization is supported on hosts starting with macOS 15 (Sequia), and later.")
throw ValidationError("Nested virtualization is supported on hosts starting with macOS 15 (Sequoia), and later.")
} else if !VZGenericPlatformConfiguration.isNestedVirtualizationSupported {
throw ValidationError("Nested virtualization is available for Mac with the M3 chip, and later.")
}
@@ -266,8 +312,16 @@ struct Run: AsyncParsableCommand {
if !(config.platform is PlatformSuspendable) {
throw ValidationError("You can only suspend macOS VMs")
}
if dir.count > 0 {
throw ValidationError("Suspending VMs with shared directories is not supported")
if noTrackpad {
throw ValidationError("--no-trackpad cannot be used with --suspendable")
}
}
if noTrackpad {
let config = try VMConfig.init(fromURL: vmDir.configURL)
if config.os != .darwin {
throw ValidationError("--no-trackpad can only be used with macOS VMs")
}
}
@@ -283,6 +337,12 @@ struct Run: AsyncParsableCommand {
let localStorage = VMStorageLocal()
let vmDir = try localStorage.open(name)
// Validate disk format support
let vmConfig = try VMConfig(fromURL: vmDir.configURL)
if !vmConfig.diskFormat.isSupported {
throw ValidationError("Disk format '\(vmConfig.diskFormat.rawValue)' is not supported on this system.")
}
let storageLock = try FileLock(lockURL: Config().tartHomeDir)
try storageLock.lock()
// check if there is a running VM with the same MAC address
@@ -332,7 +392,8 @@ struct Run: AsyncParsableCommand {
audio: !noAudio,
clipboard: !noClipboard,
sync: VZDiskImageSynchronizationMode(diskOptions.syncModeRaw),
caching: VZDiskImageCachingMode(diskOptions.cachingModeRaw)
caching: VZDiskImageCachingMode(diskOptions.cachingModeRaw),
noTrackpad: noTrackpad
)
let vncImpl: VNC? = try {
@@ -423,6 +484,12 @@ struct Run: AsyncParsableCommand {
}
}
if #available(macOS 14, *) {
Task {
try await ControlSocket(vmDir.controlSocketURL).run()
}
}
try await vm!.run()
if let vncImpl = vncImpl {
@@ -527,6 +594,10 @@ struct Run: AsyncParsableCommand {
softnetExtraArguments += ["--allow", netSoftnetAllow]
}
if let netSoftnetExpose = netSoftnetExpose {
softnetExtraArguments += ["--expose", netSoftnetExpose]
}
if netSoftnet {
let config = try VMConfig.init(fromURL: vmDir.configURL)
@@ -1029,7 +1100,7 @@ struct DirectoryShare {
process.standardInput = inPipe
process.launch()
inPipe.fileHandleForWriting.write(response!.data)
try inPipe.fileHandleForWriting.write(contentsOf: response!.data)
try inPipe.fileHandleForWriting.close()
process.waitUntilExit()
+2 -11
View File
@@ -33,15 +33,7 @@ struct Set: AsyncParsableCommand {
@Option(help: ArgumentHelp("Resize the VMs disk to the specified size in GB (note that the disk size can only be increased to avoid losing data)",
discussion: """
Disk resizing works on most cloud-ready Linux distributions out-of-the box (e.g. Ubuntu Cloud Images
have the \"cloud-initramfs-growroot\" package installed that runs on boot) and on the rest of the
distributions by running the \"growpart\" or \"resize2fs\" commands.
For macOS, however, things are a bit more complicated: you need to remove the recovery partition
first and then run various \"diskutil\" commands, see Tart's packer plugin source code for more
details[1].
[1]: https://github.com/cirruslabs/packer-plugin-tart/blob/main/builder/tart/step_disk_resize.go
See https://tart.run/faq/#disk-resizing for more details.
"""))
var diskSize: UInt16?
@@ -73,8 +65,7 @@ struct Set: AsyncParsableCommand {
}
#if arch(arm64)
if randomSerial {
let oldPlatform = vmConfig.platform as! Darwin
if randomSerial, let oldPlatform = vmConfig.platform as? Darwin {
vmConfig.platform = Darwin(ecid: VZMacMachineIdentifier(), hardwareModel: oldPlatform.hardwareModel)
}
#endif
+89
View File
@@ -0,0 +1,89 @@
import Foundation
import Network
import os.log
import NIO
import NIOPosix
@available(macOS 14, *)
class ControlSocket {
let controlSocketURL: URL
let vmPort: UInt32
let eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: 1)
let logger: os.Logger = os.Logger(subsystem: "org.cirruslabs.tart.control-socket", category: "network")
init(_ controlSocketURL: URL, vmPort: UInt32 = 8080) {
self.controlSocketURL = controlSocketURL
self.vmPort = vmPort
}
func run() async throws {
// Remove control socket file from previous "tart run" invocations,
// if any, otherwise we may get the "address already in use" error
try? FileManager.default.removeItem(atPath: controlSocketURL.path())
let serverChannel = try await ServerBootstrap(group: eventLoopGroup)
.bind(unixDomainSocketPath: controlSocketURL.path()) { childChannel in
childChannel.eventLoop.makeCompletedFuture {
return try NIOAsyncChannel<ByteBuffer, ByteBuffer>(
wrappingChannelSynchronously: childChannel
)
}
}
try await withThrowingDiscardingTaskGroup { group in
try await serverChannel.executeThenClose { serverInbound in
for try await clientChannel in serverInbound {
group.addTask {
try await self.handleClient(clientChannel)
}
}
}
}
}
func handleClient(_ clientChannel: NIOAsyncChannel<ByteBuffer, ByteBuffer>) async throws {
self.logger.info("received new control socket connection from a client")
try await clientChannel.executeThenClose { clientInbound, clientOutbound in
self.logger.info("dialing to VM on port \(self.vmPort)...")
do {
guard let vmConnection = try await vm?.connect(toPort: self.vmPort) else {
throw RuntimeError.VMSocketFailed(self.vmPort, "VM is not running")
}
self.logger.info("running control socket proxy")
let vmChannel = try await ClientBootstrap(group: eventLoopGroup).withConnectedSocket(vmConnection.fileDescriptor) { childChannel in
childChannel.eventLoop.makeCompletedFuture {
try NIOAsyncChannel<ByteBuffer, ByteBuffer>(
wrappingChannelSynchronously: childChannel
)
}
}
try await vmChannel.executeThenClose { (vmInbound, vmOutbound) in
try await withThrowingDiscardingTaskGroup { group in
// Proxy data from a client (e.g. "tart exec") to a VM
group.addTask {
for try await message in clientInbound {
try await vmOutbound.write(message)
}
}
// Proxy data from a VM to a client (e.g. "tart exec")
group.addTask {
for try await message in vmInbound {
try await clientOutbound.write(message)
}
}
}
}
self.logger.info("control socket client disconnected")
} catch (let error) {
self.logger.error("control socket connection failed: \(error)")
}
}
}
}
@@ -36,7 +36,11 @@ class DockerConfigCredentialsProvider: CredentialsProvider {
process.launch()
inPipe.fileHandleForWriting.write("\(host)\n".data(using: .utf8)!)
do {
try inPipe.fileHandleForWriting.write(contentsOf: "\(host)\n".data(using: .utf8)!)
} catch {
throw CredentialsProviderError.Failed(message: "Failed to write host to Docker helper!")
}
inPipe.fileHandleForWriting.closeFile()
let outputData = try outPipe.fileHandleForReading.readToEnd()
+43
View File
@@ -0,0 +1,43 @@
import Foundation
import ArgumentParser
enum DiskImageFormat: String, CaseIterable, Codable {
case raw = "raw"
case asif = "asif"
var displayName: String {
switch self {
case .raw:
return "RAW"
case .asif:
return "ASIF (Apple Sparse Image Format)"
}
}
/// Check if the format is supported on the current system
var isSupported: Bool {
switch self {
case .raw:
return true
case .asif:
if #available(macOS 15, *) {
return true
} else {
return false
}
}
}
}
extension DiskImageFormat: ExpressibleByArgument {
init?(argument: String) {
self.init(rawValue: argument.lowercased())
}
static var allValueStrings: [String] {
return allCases.map { $0.rawValue }
}
}
+1 -1
View File
@@ -17,7 +17,7 @@ fileprivate var urlSession: URLSession = {
}()
class Fetcher {
static func fetch(_ request: URLRequest, viaFile: Bool = false, progress: Progress? = nil) async throws -> (AsyncThrowingStream<Data, Error>, HTTPURLResponse) {
static func fetch(_ request: URLRequest, viaFile: Bool = false) async throws -> (AsyncThrowingStream<Data, Error>, HTTPURLResponse) {
let task = urlSession.dataTask(with: request)
let delegate = Delegate()
@@ -0,0 +1,42 @@
import Foundation
import Network
import NIOPosix
import GRPC
import Cirruslabs_TartGuestAgent_Apple_Swift
import Cirruslabs_TartGuestAgent_Grpc_Swift
class AgentResolver {
static func ResolveIP(_ controlSocketURL: URL) async throws -> IPv4Address? {
do {
return try await resolveIP(controlSocketURL)
} catch let error as GRPCConnectionPoolError {
return nil
}
}
private static func resolveIP(_ controlSocketURL: URL) async throws -> IPv4Address? {
// Create a gRPC channel connected to the VM's control socket
let group = MultiThreadedEventLoopGroup(numberOfThreads: 1)
defer {
try! group.syncShutdownGracefully()
}
let channel = try GRPCChannelPool.with(
target: .unixDomainSocket(controlSocketURL.path()),
transportSecurity: .plaintext,
eventLoopGroup: group,
)
defer {
try! channel.close().wait()
}
// Invoke ResolveIP() gRPC method
let callOptions = CallOptions(timeLimit: .timeout(.seconds(1)))
let agentAsyncClient = AgentAsyncClient(channel: channel)
let resolveIPCall = agentAsyncClient.makeResolveIpCall(ResolveIPRequest(), callOptions: callOptions)
let response = try await resolveIPCall.response
return IPv4Address(response.ip)
}
}
+1 -1
View File
@@ -57,7 +57,7 @@ class DiskV1: Disk {
// Decompress the layers onto the disk in a single stream
let filter = try OutputFilter(.decompress, using: .lz4, bufferCapacity: Self.bufferSizeBytes) { data in
if let data = data {
disk.write(data)
try disk.write(contentsOf: data)
}
}
+22 -7
View File
@@ -44,7 +44,7 @@ class DiskV2: Disk {
let compressedData = try (data as NSData).compressed(using: .lz4) as Data
let compressedDataDigest = Digest.hash(compressedData)
try await retry(maxAttempts: 5, backoff: .exponentialWithFullJitter(baseDelay: .seconds(5), maxDelay: .seconds(60))) {
try await retry(maxAttempts: 5) {
if try await !registry.blobExists(compressedDataDigest) {
_ = try await registry.pushBlob(fromData: compressedData, chunkSizeMb: chunkSizeMb, digest: compressedDataDigest)
}
@@ -208,11 +208,26 @@ class DiskV2: Disk {
diskWritingOffset = try zeroSkippingWrite(disk, rdisk, fsBlockSize, diskWritingOffset, data)
}
try await registry.pullBlob(diskLayer.digest) { data in
try filter.write(data)
var rangeStart: Int64 = 0
// Update the progress
progress.completedUnitCount += Int64(data.count)
try await retry(maxAttempts: 5) {
try await registry.pullBlob(diskLayer.digest, rangeStart: rangeStart) { data in
try filter.write(data)
// Update the progress
progress.completedUnitCount += Int64(data.count)
// Update the current range start
rangeStart += Int64(data.count)
}
} recoverFromFailure: { error in
if error is URLError {
print("Error pulling disk layer \(index + 1): \"\(error.localizedDescription)\", attempting to re-try...")
return .retry
}
return .throw
}
try filter.finalize()
@@ -253,7 +268,7 @@ class DiskV2: Disk {
if chunk != actualContentsOnDisk {
try disk.seek(toOffset: offset)
disk.write(chunk)
try disk.write(contentsOf: chunk)
}
}
@@ -267,7 +282,7 @@ class DiskV2: Disk {
// is zeroed via truncate(2)
if chunk != zeroChunk {
try disk.seek(toOffset: offset)
disk.write(chunk)
try disk.write(contentsOf: chunk)
}
offset += UInt64(chunk.count)
+8
View File
@@ -14,6 +14,9 @@ let nvramMediaType = "application/vnd.cirruslabs.tart.nvram.v1"
let uncompressedDiskSizeAnnotation = "org.cirruslabs.tart.uncompressed-disk-size"
let uploadTimeAnnotation = "org.cirruslabs.tart.upload-time"
// Manifest labels
let diskFormatLabel = "org.cirruslabs.tart.disk.format"
// Layer annotations
let uncompressedSizeAnnotation = "org.cirruslabs.tart.uncompressed-size"
let uncompressedContentDigestAnnotation = "org.cirruslabs.tart.uncompressed-content-digest"
@@ -66,6 +69,11 @@ struct OCIManifest: Codable, Equatable {
struct OCIConfig: Codable {
var architecture: Architecture = .arm64
var os: OS = .darwin
var config: ConfigContainer?
struct ConfigContainer: Codable {
var Labels: [String: String]?
}
func toJSON() throws -> Data {
try Config.jsonEncoder().encode(self)
+37 -13
View File
@@ -20,6 +20,7 @@ enum HTTPCode: Int {
case Ok = 200
case Created = 201
case Accepted = 202
case PartialContent = 206
case Unauthorized = 401
case NotFound = 404
}
@@ -28,14 +29,26 @@ extension Data {
func asText() -> String {
String(decoding: self, as: UTF8.self)
}
func asTextPreview(limit: Int = 1000) -> String {
guard count > limit else {
return asText()
}
return "\(asText().prefix(limit))..."
}
}
extension AsyncThrowingStream<Data, Error> {
func asData() async throws -> Data {
func asData(limitBytes: Int64? = nil) async throws -> Data {
var result = Data()
for try await chunk in self {
result += chunk
if let limitBytes, result.count > limitBytes {
return result
}
}
return result
@@ -159,7 +172,7 @@ class Registry {
body: manifestJSON)
if response.statusCode != HTTPCode.Created.rawValue {
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing manifest", code: response.statusCode,
details: data.asText())
details: data.asTextPreview())
}
return Digest.hash(manifestJSON)
@@ -170,7 +183,7 @@ class Registry {
headers: ["Accept": ociManifestMediaType])
if response.statusCode != HTTPCode.Ok.rawValue {
throw RegistryError.UnexpectedHTTPStatusCode(when: "pulling manifest", code: response.statusCode,
details: data.asText())
details: data.asTextPreview())
}
let manifest = try OCIManifest(fromJSON: data)
@@ -196,7 +209,7 @@ class Registry {
headers: ["Content-Length": "0"])
if postResponse.statusCode != HTTPCode.Accepted.rawValue {
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing blob (POST)", code: postResponse.statusCode,
details: data.asText())
details: data.asTextPreview())
}
// Figure out where to upload the blob
@@ -217,7 +230,7 @@ class Registry {
)
if response.statusCode != HTTPCode.Created.rawValue {
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing blob (PUT) to \(uploadLocation)",
code: response.statusCode, details: data.asText())
code: response.statusCode, details: data.asTextPreview())
}
return digest
}
@@ -240,7 +253,7 @@ class Registry {
// always accept both statuses since AWS ECR is not following specification
if response.statusCode != HTTPCode.Created.rawValue && response.statusCode != HTTPCode.Accepted.rawValue {
throw RegistryError.UnexpectedHTTPStatusCode(when: "streaming blob to \(uploadLocation)",
code: response.statusCode, details: data.asText())
code: response.statusCode, details: data.asTextPreview())
}
uploadedBytes += chunk.count
// Update location for the next chunk
@@ -259,14 +272,26 @@ class Registry {
case HTTPCode.NotFound.rawValue:
return false
default:
throw RegistryError.UnexpectedHTTPStatusCode(when: "checking blob", code: response.statusCode, details: data.asText())
throw RegistryError.UnexpectedHTTPStatusCode(when: "checking blob", code: response.statusCode, details: data.asTextPreview())
}
}
public func pullBlob(_ digest: String, handler: (Data) async throws -> Void) async throws {
let (channel, response) = try await channelRequest(.GET, endpointURL("\(namespace)/blobs/\(digest)"), viaFile: true)
if response.statusCode != HTTPCode.Ok.rawValue {
let body = try await channel.asData().asText()
public func pullBlob(_ digest: String, rangeStart: Int64 = 0, handler: (Data) async throws -> Void) async throws {
var expectedStatusCode = HTTPCode.Ok
var headers: [String: String] = [:]
// Send Range header and expect HTTP 206 in return
//
// However, do not send Range header at all when rangeStart is 0,
// because it makes no sense and we might get HTTP 200 in return
if rangeStart != 0 {
expectedStatusCode = HTTPCode.PartialContent
headers["Range"] = "bytes=\(rangeStart)-"
}
let (channel, response) = try await channelRequest(.GET, endpointURL("\(namespace)/blobs/\(digest)"), headers: headers, viaFile: true)
if response.statusCode != expectedStatusCode.rawValue {
let body = try await channel.asData(limitBytes: 4096).asTextPreview()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pulling blob", code: response.statusCode,
details: body)
}
@@ -329,7 +354,6 @@ class Registry {
var (channel, response) = try await authAwareRequest(request: request, viaFile: viaFile, doAuth: doAuth)
if doAuth && response.statusCode == HTTPCode.Unauthorized.rawValue {
_ = try await channel.asData()
try await auth(response: response)
(channel, response) = try await authAwareRequest(request: request, viaFile: viaFile, doAuth: doAuth)
}
@@ -391,7 +415,7 @@ class Registry {
let (data, response) = try await dataRequest(.GET, authenticateURL, headers: headers, doAuth: false)
if response.statusCode != HTTPCode.Ok.rawValue {
throw RegistryError.AuthFailed(why: "received unexpected HTTP status code \(response.statusCode) "
+ "while retrieving an authentication token", details: data.asText())
+ "while retrieving an authentication token", details: data.asTextPreview())
}
await authenticationKeeper.set(try TokenResponse.parse(fromData: data))
+5
View File
@@ -127,6 +127,11 @@ struct UnsupportedHostOSError: Error, CustomStringConvertible {
[VZUSBScreenCoordinatePointingDeviceConfiguration(), VZMacTrackpadConfiguration()]
}
func pointingDevicesSimplified() -> [VZPointingDeviceConfiguration] {
// Only include the USB pointing device, not the trackpad
return [VZUSBScreenCoordinatePointingDeviceConfiguration()]
}
func pointingDevicesSuspendable() -> [VZPointingDeviceConfiguration] {
if #available(macOS 14, *) {
return [VZMacTrackpadConfiguration()]
+5
View File
@@ -42,4 +42,9 @@ struct Linux: Platform {
func pointingDevices() -> [VZPointingDeviceConfiguration] {
[VZUSBScreenCoordinatePointingDeviceConfiguration()]
}
func pointingDevicesSimplified() -> [VZPointingDeviceConfiguration] {
// Linux doesn't support trackpad, so just return the regular pointing devices
return pointingDevices()
}
}
+1
View File
@@ -7,6 +7,7 @@ protocol Platform: Codable {
func graphicsDevice(vmConfig: VMConfig) -> VZGraphicsDeviceConfiguration
func keyboards() -> [VZKeyboardConfiguration]
func pointingDevices() -> [VZPointingDeviceConfiguration]
func pointingDevicesSimplified() -> [VZPointingDeviceConfiguration]
}
protocol PlatformSuspendable: Platform {
+41 -34
View File
@@ -18,6 +18,7 @@ struct Root: AsyncParsableCommand {
Login.self,
Logout.self,
IP.self,
Exec.self,
Pull.self,
Push.self,
Import.self,
@@ -30,39 +31,6 @@ struct Root: AsyncParsableCommand {
])
public static func main() async throws {
// Initialize Sentry
if let dsn = ProcessInfo.processInfo.environment["SENTRY_DSN"] {
SentrySDK.start { options in
options.dsn = dsn
options.releaseName = CI.release
options.tracesSampleRate = Float(
ProcessInfo.processInfo.environment["SENTRY_TRACES_SAMPLE_RATE"] ?? "1.0"
) as NSNumber?
// By default only 5XX are captured
// Let's capture everything but 401 (unauthorized)
options.enableCaptureFailedRequests = true
options.failedRequestStatusCodes = [
HttpStatusCodeRange(min: 400, max: 400),
HttpStatusCodeRange(min: 402, max: 599)
]
}
}
defer { SentrySDK.flush(timeout: 2.seconds.timeInterval) }
SentrySDK.configureScope { scope in
scope.setExtra(value: ProcessInfo.processInfo.arguments, key: "Command-line arguments")
}
// Enrich future events with Cirrus CI-specific tags
if let tags = ProcessInfo.processInfo.environment["CIRRUS_SENTRY_TAGS"] {
SentrySDK.configureScope { scope in
for (key, value) in tags.split(separator: ",").compactMap({ parseCirrusSentryTag($0) }) {
scope.setTag(value: value, key: key)
}
}
}
// Add commands that are only available on specific macOS versions
if #available(macOS 14, *) {
configuration.subcommands.append(Suspend.self)
@@ -82,10 +50,43 @@ struct Root: AsyncParsableCommand {
// Set line-buffered output for stdout
setlinebuf(stdout)
// Parse and run command
do {
// Parse command
var command = try parseAsRoot()
// Initialize Sentry
if let dsn = ProcessInfo.processInfo.environment["SENTRY_DSN"] {
SentrySDK.start { options in
options.dsn = dsn
options.releaseName = CI.release
options.tracesSampleRate = Float(
ProcessInfo.processInfo.environment["SENTRY_TRACES_SAMPLE_RATE"] ?? "1.0"
) as NSNumber?
// By default only 5XX are captured
// Let's capture everything but 401 (unauthorized)
options.enableCaptureFailedRequests = true
options.failedRequestStatusCodes = [
HttpStatusCodeRange(min: 400, max: 400),
HttpStatusCodeRange(min: 402, max: 599)
]
}
}
defer { SentrySDK.flush(timeout: 2.seconds.timeInterval) }
SentrySDK.configureScope { scope in
scope.setExtra(value: ProcessInfo.processInfo.arguments, key: "Command-line arguments")
}
// Enrich future events with Cirrus CI-specific tags
if let tags = ProcessInfo.processInfo.environment["CIRRUS_SENTRY_TAGS"] {
SentrySDK.configureScope { scope in
for (key, value) in tags.split(separator: ",").compactMap({ parseCirrusSentryTag($0) }) {
scope.setTag(value: value, key: key)
}
}
}
// Run garbage-collection before each command (shouldn't take too long)
if type(of: command) != type(of: Pull()) && type(of: command) != type(of: Clone()){
do {
@@ -95,12 +96,18 @@ struct Root: AsyncParsableCommand {
}
}
// Run command
if var asyncCommand = command as? AsyncParsableCommand {
try await asyncCommand.run()
} else {
try command.run()
}
} catch {
// Not an error, just a custom exit code from "tart exec"
if let execCustomExitCodeError = error as? ExecCustomExitCodeError {
Foundation.exit(execCustomExitCodeError.exitCode)
}
// Capture the error into Sentry
SentrySDK.capture(error: error)
SentrySDK.flush(timeout: 2.seconds.timeInterval)
@@ -5,7 +5,7 @@ fileprivate func normalizeName(_ name: String) -> String {
return name.replacingOccurrences(of: ":", with: "\\:")
}
func completeMachines(_ arguments: [String]) -> [String] {
func completeMachines(_ arguments: [String], _ argumentIdx: Int, _ argumentPrefix: String) -> [String] {
let localVMs = (try? VMStorageLocal().list().map { name, _ in
normalizeName(name)
}) ?? []
@@ -15,12 +15,12 @@ func completeMachines(_ arguments: [String]) -> [String] {
return (localVMs + ociVMs)
}
func completeLocalMachines(_ arguments: [String]) -> [String] {
func completeLocalMachines(_ arguments: [String], _ argumentIdx: Int, _ argumentPrefix: String) -> [String] {
let localVMs = (try? VMStorageLocal().list()) ?? []
return localVMs.map { name, _ in normalizeName(name) }
}
func completeRunningMachines(_ arguments: [String]) -> [String] {
func completeRunningMachines(_ arguments: [String], _ argumentIdx: Int, _ argumentPrefix: String) -> [String] {
let localVMs = (try? VMStorageLocal().list()) ?? []
return localVMs
.filter { _, vmDir in (try? vmDir.state() == .Running) ?? false}
+60
View File
@@ -0,0 +1,60 @@
import Foundation
import System
struct State {
fileprivate let termios: termios
}
class Term {
static func IsTerminal() -> Bool {
var termios = termios()
return tcgetattr(FileHandle.standardInput.fileDescriptor, &termios) != -1
}
static func MakeRaw() throws -> State {
var termiosOrig = termios()
var ret = tcgetattr(FileHandle.standardInput.fileDescriptor, &termiosOrig)
if ret == -1 {
let details = Errno(rawValue: CInt(errno))
throw RuntimeError.TerminalOperationFailed("failed to retrieve terminal parameters: \(details)")
}
var termiosRaw = termiosOrig
cfmakeraw(&termiosRaw)
ret = tcsetattr(FileHandle.standardInput.fileDescriptor, TCSANOW, &termiosRaw)
if ret == -1 {
let details = Errno(rawValue: CInt(errno))
throw RuntimeError.TerminalOperationFailed("failed to set terminal parameters: \(details)")
}
return State(termios: termiosOrig)
}
static func Restore(_ state: State) throws {
var termios = state.termios
let ret = tcsetattr(FileHandle.standardInput.fileDescriptor, TCSANOW, &termios)
if ret == -1 {
let details = Errno(rawValue: CInt(errno))
throw RuntimeError.TerminalOperationFailed("failed to set terminal parameters: \(details)")
}
}
static func GetSize() throws -> (width: UInt16, height: UInt16) {
var winsize = winsize()
guard ioctl(STDOUT_FILENO, TIOCGWINSZ, &winsize) != -1 else {
let details = Errno(rawValue: CInt(errno))
throw RuntimeError.TerminalOperationFailed("failed to get terminal size: \(details)")
}
return (width: winsize.ws_col, height: winsize.ws_row)
}
}
+51 -30
View File
@@ -50,7 +50,8 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
audio: Bool = true,
clipboard: Bool = true,
sync: VZDiskImageSynchronizationMode = .full,
caching: VZDiskImageCachingMode? = nil
caching: VZDiskImageCachingMode? = nil,
noTrackpad: Bool = false
) throws {
name = vmDir.name
config = try VMConfig.init(fromURL: vmDir.configURL)
@@ -71,7 +72,8 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
audio: audio,
clipboard: clipboard,
sync: sync,
caching: caching
caching: caching,
noTrackpad: noTrackpad
)
virtualMachine = VZVirtualMachine(configuration: configuration)
@@ -99,16 +101,13 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// Download the IPSW
defaultLogger.appendNewLine("Fetching \(remoteURL.lastPathComponent)...")
let downloadProgress = Progress(totalUnitCount: 100)
ProgressObserver(downloadProgress).log(defaultLogger)
let request = URLRequest(url: remoteURL)
let (channel, response) = try await Fetcher.fetch(request, viaFile: true, progress: downloadProgress)
let (channel, response) = try await Fetcher.fetch(request, viaFile: true)
let temporaryLocation = try Config().tartTmpDir.appendingPathComponent(UUID().uuidString + ".ipsw")
defaultLogger.appendNewLine("Computing digest for \(temporaryLocation.path)...")
let digestProgress = Progress(totalUnitCount: response.expectedContentLength)
ProgressObserver(digestProgress).log(defaultLogger)
let progress = Progress(totalUnitCount: response.expectedContentLength)
ProgressObserver(progress).log(defaultLogger)
FileManager.default.createFile(atPath: temporaryLocation.path, contents: nil)
let lock = try FileLock(lockURL: temporaryLocation)
@@ -118,10 +117,9 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
let digest = Digest()
for try await chunk in channel {
let chunkAsData = Data(chunk)
fileHandle.write(chunkAsData)
digest.update(chunkAsData)
digestProgress.completedUnitCount += Int64(chunk.count)
try fileHandle.write(contentsOf: chunk)
digest.update(chunk)
progress.completedUnitCount += Int64(chunk.count)
}
try fileHandle.close()
@@ -145,6 +143,7 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
vmDir: VMDirectory,
ipswURL: URL,
diskSizeGB: UInt16,
diskFormat: DiskImageFormat = .raw,
network: Network = NetworkShared(),
additionalStorageDevices: [VZStorageDeviceConfiguration] = [],
directorySharingDevices: [VZDirectorySharingDeviceConfiguration] = [],
@@ -177,14 +176,15 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
_ = try VZMacAuxiliaryStorage(creatingStorageAt: vmDir.nvramURL, hardwareModel: requirements.hardwareModel)
// Create disk
try vmDir.resizeDisk(diskSizeGB)
try vmDir.resizeDisk(diskSizeGB, format: diskFormat)
name = vmDir.name
// Create config
config = VMConfig(
platform: Darwin(ecid: VZMacMachineIdentifier(), hardwareModel: requirements.hardwareModel),
cpuCountMin: requirements.minimumSupportedCPUCount,
memorySizeMin: requirements.minimumSupportedMemorySize
memorySizeMin: requirements.minimumSupportedMemorySize,
diskFormat: diskFormat
)
// allocate at least 4 CPUs because otherwise VMs are frequently freezing
try config.setCPU(cpuCount: max(4, requirements.minimumSupportedCPUCount))
@@ -226,15 +226,15 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
#endif
@available(macOS 13, *)
static func linux(vmDir: VMDirectory, diskSizeGB: UInt16) async throws -> VM {
static func linux(vmDir: VMDirectory, diskSizeGB: UInt16, diskFormat: DiskImageFormat = .raw) async throws -> VM {
// Create NVRAM
_ = try VZEFIVariableStore(creatingVariableStoreAt: vmDir.nvramURL)
// Create disk
try vmDir.resizeDisk(diskSizeGB)
try vmDir.resizeDisk(diskSizeGB, format: diskFormat)
// Create config
let config = VMConfig(platform: Linux(), cpuCountMin: 4, memorySizeMin: 4096 * 1024 * 1024)
let config = VMConfig(platform: Linux(), cpuCountMin: 4, memorySizeMin: 4096 * 1024 * 1024, diskFormat: diskFormat)
try config.save(toURL: vmDir.configURL)
return try VM(vmDir: vmDir)
@@ -250,6 +250,19 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
}
@MainActor
func connect(toPort: UInt32) async throws -> VZVirtioSocketConnection {
guard let socketDevice = virtualMachine.socketDevices.first else {
throw RuntimeError.VMSocketFailed(toPort, ", VM has no socket devices configured")
}
guard let virtioSocketDevice = socketDevice as? VZVirtioSocketDevice else {
throw RuntimeError.VMSocketFailed(toPort, ", expected VM's first socket device to have a type of VZVirtioSocketDevice, got \(type(of: socketDevice)) instead")
}
return try await virtioSocketDevice.connect(toPort: toPort)
}
func run() async throws {
do {
try await sema.waitUnlessCancelled()
@@ -302,7 +315,8 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
audio: Bool = true,
clipboard: Bool = true,
sync: VZDiskImageSynchronizationMode = .full,
caching: VZDiskImageCachingMode? = nil
caching: VZDiskImageCachingMode? = nil,
noTrackpad: Bool = false
) throws -> VZVirtualMachineConfiguration {
let configuration = VZVirtualMachineConfiguration()
@@ -343,7 +357,11 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
configuration.pointingDevices = platformSuspendable.pointingDevicesSuspendable()
} else {
configuration.keyboards = vmConfig.platform.keyboards()
configuration.pointingDevices = vmConfig.platform.pointingDevices()
if noTrackpad {
configuration.pointingDevices = vmConfig.platform.pointingDevicesSimplified()
} else {
configuration.pointingDevices = vmConfig.platform.pointingDevices()
}
}
// Networking
@@ -355,17 +373,19 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
// Clipboard sharing via Spice agent
if clipboard && vmConfig.os == .linux {
if clipboard {
let spiceAgentConsoleDevice = VZVirtioConsoleDeviceConfiguration()
let spiceAgentPort = VZVirtioConsolePortConfiguration()
spiceAgentPort.name = VZSpiceAgentPortAttachment.spiceAgentPortName
spiceAgentPort.attachment = VZSpiceAgentPortAttachment()
let spiceAgentPortAttachment = VZSpiceAgentPortAttachment()
spiceAgentPortAttachment.sharesClipboard = true
spiceAgentPort.attachment = spiceAgentPortAttachment
spiceAgentConsoleDevice.ports[0] = spiceAgentPort
configuration.consoleDevices.append(spiceAgentConsoleDevice)
}
// Storage
let attachment: VZDiskImageStorageDeviceAttachment = try VZDiskImageStorageDeviceAttachment(
var attachment = try VZDiskImageStorageDeviceAttachment(
url: diskURL,
readOnly: false,
// When not specified, use "cached" caching mode for Linux VMs to prevent file-system corruption[1]
@@ -394,15 +414,16 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
//
// A dummy console device useful for implementing
// host feature checks in the guest agent software.
if !suspendable {
let consolePort = VZVirtioConsolePortConfiguration()
consolePort.name = "tart-version-\(CI.version)"
let consolePort = VZVirtioConsolePortConfiguration()
consolePort.name = "tart-version-\(CI.version)"
let consoleDevice = VZVirtioConsoleDeviceConfiguration()
consoleDevice.ports[0] = consolePort
let consoleDevice = VZVirtioConsoleDeviceConfiguration()
consoleDevice.ports[0] = consolePort
configuration.consoleDevices.append(consoleDevice)
}
configuration.consoleDevices.append(consoleDevice)
// Socket device
configuration.socketDevices = [VZVirtioSocketDeviceConfiguration()]
try configuration.validate()
+8 -1
View File
@@ -25,6 +25,7 @@ enum CodingKeys: String, CodingKey {
case macAddress
case display
case displayRefit
case diskFormat
// macOS-specific keys
case ecid
@@ -54,12 +55,14 @@ struct VMConfig: Codable {
var macAddress: VZMACAddress
var display: VMDisplayConfig = VMDisplayConfig()
var displayRefit: Bool?
var diskFormat: DiskImageFormat = .raw
init(
platform: Platform,
cpuCountMin: Int,
memorySizeMin: UInt64,
macAddress: VZMACAddress = VZMACAddress.randomLocallyAdministered()
macAddress: VZMACAddress = VZMACAddress.randomLocallyAdministered(),
diskFormat: DiskImageFormat = .raw
) {
self.os = platform.os()
self.arch = CurrentArchitecture()
@@ -67,6 +70,7 @@ struct VMConfig: Codable {
self.macAddress = macAddress
self.cpuCountMin = cpuCountMin
self.memorySizeMin = memorySizeMin
self.diskFormat = diskFormat
cpuCount = cpuCountMin
memorySize = memorySizeMin
}
@@ -124,6 +128,8 @@ struct VMConfig: Codable {
display = try container.decodeIfPresent(VMDisplayConfig.self, forKey: .display) ?? VMDisplayConfig()
displayRefit = try container.decodeIfPresent(Bool.self, forKey: .displayRefit)
let diskFormatString = try container.decodeIfPresent(String.self, forKey: .diskFormat) ?? "raw"
diskFormat = DiskImageFormat(rawValue: diskFormatString) ?? .raw
}
func encode(to encoder: Encoder) throws {
@@ -142,6 +148,7 @@ struct VMConfig: Codable {
if let displayRefit = displayRefit {
try container.encode(displayRefit, forKey: .displayRefit)
}
try container.encode(diskFormat.rawValue, forKey: .diskFormat)
}
mutating func setCPU(cpuCount: Int) throws {
+9 -4
View File
@@ -24,7 +24,7 @@ extension VMDirectory {
}
let configFile = try FileHandle(forWritingTo: configURL)
try await registry.pullBlob(configLayers.first!.digest) { data in
configFile.write(data)
try configFile.write(contentsOf: data)
}
try configFile.close()
@@ -79,7 +79,7 @@ extension VMDirectory {
}
let nvram = try FileHandle(forWritingTo: nvramURL)
try await registry.pullBlob(nvramLayers.first!.digest) { data in
nvram.write(data)
try nvram.write(contentsOf: data)
}
try nvram.close()
@@ -87,11 +87,15 @@ extension VMDirectory {
try manifest.toJSON().write(to: manifestURL)
}
func pushToRegistry(registry: Registry, references: [String], chunkSizeMb: Int, diskFormat: String, concurrency: UInt) async throws -> RemoteName {
func pushToRegistry(registry: Registry, references: [String], chunkSizeMb: Int, diskFormat: String, concurrency: UInt, labels: [String: String] = [:]) async throws -> RemoteName {
var layers = Array<OCIManifestLayer>()
// Read VM's config and push it as blob
let config = try VMConfig(fromURL: configURL)
// Add disk format label automatically
var labels = labels
labels[diskFormatLabel] = config.diskFormat.rawValue
let configJSON = try JSONEncoder().encode(config)
defaultLogger.appendNewLine("pushing config...")
let configDigest = try await registry.pushBlob(fromData: configJSON, chunkSizeMb: chunkSizeMb)
@@ -121,7 +125,8 @@ extension VMDirectory {
layers.append(OCIManifestLayer(mediaType: nvramMediaType, size: nvram.count, digest: nvramDigest))
// Craft a stub OCI config for Docker Hub compatibility
let ociConfigJSON = try OCIConfig(architecture: config.arch, os: config.os).toJSON()
let ociConfigContainer = OCIConfig.ConfigContainer(Labels: labels)
let ociConfigJSON = try OCIConfig(architecture: config.arch, os: config.os, config: ociConfigContainer).toJSON()
let ociConfigDigest = try await registry.pushBlob(fromData: ociConfigJSON, chunkSizeMb: chunkSizeMb)
let manifest = OCIManifest(
config: OCIManifestConfig(size: ociConfigJSON.count, digest: ociConfigDigest),
+197 -4
View File
@@ -2,6 +2,25 @@ import Foundation
import Virtualization
import CryptoKit
// MARK: - Disk Image Info Structures
struct DiskImageInfo: Codable {
let sizeInfo: SizeInfo?
let size: UInt64?
enum CodingKeys: String, CodingKey {
case sizeInfo = "Size Info"
case size = "Size"
}
}
struct SizeInfo: Codable {
let totalBytes: UInt64?
enum CodingKeys: String, CodingKey {
case totalBytes = "Total Bytes"
}
}
struct VMDirectory: Prunable {
enum State: String {
case Running = "running"
@@ -26,6 +45,9 @@ struct VMDirectory: Prunable {
var manifestURL: URL {
baseURL.appendingPathComponent("manifest.json")
}
var controlSocketURL: URL {
baseURL.appendingPathComponent("control.sock")
}
var explicitlyPulledMark: URL {
baseURL.appendingPathComponent(".explicitly-pulled")
@@ -139,14 +161,34 @@ struct VMDirectory: Prunable {
try vmConfig.save(toURL: configURL)
}
func resizeDisk(_ sizeGB: UInt16) throws {
if !FileManager.default.fileExists(atPath: diskURL.path) {
FileManager.default.createFile(atPath: diskURL.path, contents: nil, attributes: nil)
}
func resizeDisk(_ sizeGB: UInt16, format: DiskImageFormat = .raw) throws {
let diskExists = FileManager.default.fileExists(atPath: diskURL.path)
if diskExists {
// Existing disk - resize it
try resizeExistingDisk(sizeGB)
} else {
// New disk - create it with the specified format
try createDisk(sizeGB: sizeGB, format: format)
}
}
private func resizeExistingDisk(_ sizeGB: UInt16) throws {
// Check if this is an ASIF disk by reading the VM config
let vmConfig = try VMConfig(fromURL: configURL)
if vmConfig.diskFormat == .asif {
try resizeASIFDisk(sizeGB)
} else {
try resizeRawDisk(sizeGB)
}
}
private func resizeRawDisk(_ sizeGB: UInt16) throws {
let diskFileHandle = try FileHandle.init(forWritingTo: diskURL)
let currentDiskFileLength = try diskFileHandle.seekToEnd()
let desiredDiskFileLength = UInt64(sizeGB) * 1000 * 1000 * 1000
if desiredDiskFileLength < currentDiskFileLength {
let currentLengthHuman = ByteCountFormatter().string(fromByteCount: Int64(currentDiskFileLength))
let desiredLengthHuman = ByteCountFormatter().string(fromByteCount: Int64(desiredDiskFileLength))
@@ -158,6 +200,157 @@ struct VMDirectory: Prunable {
try diskFileHandle.close()
}
private func resizeASIFDisk(_ sizeGB: UInt16) throws {
guard let diskutilURL = resolveBinaryPath("diskutil") else {
throw RuntimeError.FailedToResizeDisk("diskutil not found in PATH")
}
// First, get current disk image info to check current size
let infoProcess = Process()
infoProcess.executableURL = diskutilURL
infoProcess.arguments = ["image", "info", "--plist", diskURL.path]
let infoPipe = Pipe()
infoProcess.standardOutput = infoPipe
infoProcess.standardError = infoPipe
do {
try infoProcess.run()
infoProcess.waitUntilExit()
let infoData = infoPipe.fileHandleForReading.readDataToEndOfFile()
if infoProcess.terminationStatus != 0 {
let output = String(data: infoData, encoding: .utf8) ?? "Unknown error"
throw RuntimeError.FailedToResizeDisk("Failed to get ASIF disk info: \(output)")
}
// Parse the plist using PropertyListDecoder
do {
let diskImageInfo = try PropertyListDecoder().decode(DiskImageInfo.self, from: infoData)
// Extract current size from the decoded structure
var currentSizeBytes: UInt64?
// Try to get size from Size Info -> Total Bytes first
if let totalBytes = diskImageInfo.sizeInfo?.totalBytes {
currentSizeBytes = totalBytes
} else if let size = diskImageInfo.size {
// Fallback to top-level Size field
currentSizeBytes = size
}
guard let currentSizeBytes = currentSizeBytes else {
throw RuntimeError.FailedToResizeDisk("Could not find size information in disk image info")
}
let desiredSizeBytes = UInt64(sizeGB) * 1000 * 1000 * 1000
if desiredSizeBytes < currentSizeBytes {
let currentLengthHuman = ByteCountFormatter().string(fromByteCount: Int64(currentSizeBytes))
let desiredLengthHuman = ByteCountFormatter().string(fromByteCount: Int64(desiredSizeBytes))
throw RuntimeError.InvalidDiskSize("new disk size of \(desiredLengthHuman) should be larger " +
"than the current disk size of \(currentLengthHuman)")
} else if desiredSizeBytes > currentSizeBytes {
// Resize the ASIF disk image using diskutil
try performASIFResize(sizeGB)
}
// If sizes are equal, no action needed
} catch let error as RuntimeError {
throw error
} catch {
let outputString = String(data: infoData, encoding: .utf8) ?? "Unable to decode output"
throw RuntimeError.FailedToResizeDisk("Failed to parse disk image info: \(error). Output: \(outputString)")
}
} catch {
throw RuntimeError.FailedToResizeDisk("Failed to get disk image info: \(error)")
}
}
private func performASIFResize(_ sizeGB: UInt16) throws {
guard let diskutilURL = resolveBinaryPath("diskutil") else {
throw RuntimeError.FailedToResizeDisk("diskutil not found in PATH")
}
let process = Process()
process.executableURL = diskutilURL
process.arguments = [
"image", "resize",
"--size", "\(sizeGB)G",
diskURL.path
]
let pipe = Pipe()
process.standardOutput = pipe
process.standardError = pipe
do {
try process.run()
process.waitUntilExit()
let data = pipe.fileHandleForReading.readDataToEndOfFile()
if process.terminationStatus != 0 {
let output = String(data: data, encoding: .utf8) ?? "Unknown error"
throw RuntimeError.FailedToResizeDisk("Failed to resize ASIF disk image: \(output)")
}
} catch {
throw RuntimeError.FailedToResizeDisk("Failed to execute diskutil resize: \(error)")
}
}
private func createDisk(sizeGB: UInt16, format: DiskImageFormat) throws {
switch format {
case .raw:
try createRawDisk(sizeGB: sizeGB)
case .asif:
try createASIFDisk(sizeGB: sizeGB)
}
}
private func createRawDisk(sizeGB: UInt16) throws {
// Create traditional raw disk image
FileManager.default.createFile(atPath: diskURL.path, contents: nil, attributes: nil)
let diskFileHandle = try FileHandle.init(forWritingTo: diskURL)
let desiredDiskFileLength = UInt64(sizeGB) * 1000 * 1000 * 1000
try diskFileHandle.truncate(atOffset: desiredDiskFileLength)
try diskFileHandle.close()
}
private func createASIFDisk(sizeGB: UInt16) throws {
guard let diskutilURL = resolveBinaryPath("diskutil") else {
throw RuntimeError.FailedToCreateDisk("diskutil not found in PATH")
}
let process = Process()
process.executableURL = diskutilURL
process.arguments = [
"image", "create", "blank",
"--format", "ASIF",
"--size", "\(sizeGB)G",
"--volumeName", "Tart",
diskURL.path
]
let pipe = Pipe()
process.standardOutput = pipe
process.standardError = pipe
do {
try process.run()
process.waitUntilExit()
if process.terminationStatus != 0 {
let data = pipe.fileHandleForReading.readDataToEndOfFile()
let output = String(data: data, encoding: .utf8) ?? "Unknown error"
throw RuntimeError.FailedToCreateDisk("Failed to create ASIF disk image: \(output)")
}
} catch {
throw RuntimeError.FailedToCreateDisk("Failed to execute diskutil: \(error)")
}
}
func delete() throws {
let lock = try lock()
+15
View File
@@ -49,6 +49,7 @@ extension Error {
}
enum RuntimeError : Error {
case Generic(_ message: String)
case VMConfigurationError(_ message: String)
case VMDoesNotExist(name: String)
case VMMissingFiles(_ message: String)
@@ -59,6 +60,8 @@ enum RuntimeError : Error {
case DiskAlreadyInUse(_ message: String)
case FailedToOpenBlockDevice(_ path: String, _ explanation: String)
case InvalidDiskSize(_ message: String)
case FailedToCreateDisk(_ message: String)
case FailedToResizeDisk(_ message: String)
case FailedToUpdateAccessDate(_ message: String)
case PIDLockFailed(_ message: String)
case PIDLockMissing(_ message: String)
@@ -75,6 +78,8 @@ enum RuntimeError : Error {
case SuspendFailed(_ message: String)
case PullFailed(_ message: String)
case VirtualMachineLimitExceeded(_ hint: String)
case VMSocketFailed(_ port: UInt32, _ explanation: String)
case TerminalOperationFailed(_ message: String)
}
protocol HasExitCode {
@@ -84,6 +89,8 @@ protocol HasExitCode {
extension RuntimeError : CustomStringConvertible {
public var description: String {
switch self {
case .Generic(let message):
return message
case .VMConfigurationError(let message):
return message
case .VMDoesNotExist(let name):
@@ -104,6 +111,10 @@ extension RuntimeError : CustomStringConvertible {
return "failed to open block device \(path): \(explanation)"
case .InvalidDiskSize(let message):
return message
case .FailedToCreateDisk(let message):
return message
case .FailedToResizeDisk(let message):
return message
case .FailedToUpdateAccessDate(let message):
return message
case .PIDLockFailed(let message):
@@ -136,6 +147,10 @@ extension RuntimeError : CustomStringConvertible {
return message
case .VirtualMachineLimitExceeded(let hint):
return "The number of VMs exceeds the system limit\(hint)"
case .VMSocketFailed(let port, let explanation):
return "Failed to establish a VM socket connection to port \(port): \(explanation)"
case .TerminalOperationFailed(let message):
return message
}
}
}
+8 -5
View File
@@ -27,12 +27,12 @@ class VMStorageOCI: PrunableStorage {
return digest
}
func open(_ name: RemoteName) throws -> VMDirectory {
func open(_ name: RemoteName, _ accessDate: Date = Date()) throws -> VMDirectory {
let vmDir = VMDirectory(baseURL: vmURL(name))
try vmDir.validate(userFriendlyName: name.description)
try vmDir.baseURL.updateAccessDate()
try vmDir.baseURL.updateAccessDate(accessDate)
return vmDir
}
@@ -180,6 +180,10 @@ class VMStorageOCI: PrunableStorage {
let transaction = SentrySDK.startTransaction(name: name.description, operation: "pull", bindToScope: true)
let tmpVMDir = try VMDirectory.temporaryDeterministic(key: name.description)
// Open an existing VM directory corresponding to this name, if any,
// marking it as outdated to speed up the garbage collection process
_ = try? open(name, Date(timeIntervalSince1970: 0))
// Lock the temporary VM directory to prevent it's garbage collection
let tmpVMDirLock = try FileLock(lockURL: tmpVMDir.baseURL)
try tmpVMDirLock.lock()
@@ -196,7 +200,7 @@ class VMStorageOCI: PrunableStorage {
}
try await withTaskCancellationHandler(operation: {
try await retry(maxAttempts: 5, backoff: .exponentialWithFullJitter(baseDelay: .seconds(5), maxDelay: .seconds(60))) {
try await retry(maxAttempts: 5) {
// Choose the best base image which has the most deduplication ratio
let localLayerCache = try await chooseLocalLayerCache(name, manifest, registry)
@@ -213,8 +217,7 @@ class VMStorageOCI: PrunableStorage {
try await tmpVMDir.pullFromRegistry(registry: registry, manifest: manifest, concurrency: concurrency, localLayerCache: localLayerCache, deduplicate: deduplicate)
} recoverFromFailure: { error in
if error is URLError {
print("Error: \(error.localizedDescription)")
print("Attempting to re-try...")
print("Error pulling image: \"\(error.localizedDescription)\", attempting to re-try...")
return .retry
}
@@ -0,0 +1,67 @@
import XCTest
@testable import tart
final class DiskImageFormatTests: XCTestCase {
func testRawFormatIsAlwaysSupported() throws {
XCTAssertTrue(DiskImageFormat.raw.isSupported)
}
func testASIFFormatSupport() throws {
// ASIF should be supported on macOS 15+
if #available(macOS 15, *) {
XCTAssertTrue(DiskImageFormat.asif.isSupported)
} else {
XCTAssertFalse(DiskImageFormat.asif.isSupported)
}
}
func testFormatFromString() throws {
XCTAssertEqual(DiskImageFormat(rawValue: "raw"), .raw)
XCTAssertEqual(DiskImageFormat(rawValue: "asif"), .asif)
XCTAssertNil(DiskImageFormat(rawValue: "invalid"))
}
func testCaseInsensitivity() throws {
XCTAssertEqual(DiskImageFormat(argument: "ASIF"), .asif) // case insensitive
XCTAssertEqual(DiskImageFormat(argument: "Raw"), .raw) // case insensitive
}
func testAllValueStrings() throws {
let allValues = DiskImageFormat.allValueStrings
XCTAssertTrue(allValues.contains("raw"))
XCTAssertTrue(allValues.contains("asif"))
XCTAssertEqual(allValues.count, 2)
}
func testVMConfigDiskFormatSerialization() throws {
// Test that VMConfig properly serializes and deserializes disk format
let config = VMConfig(
platform: Linux(),
cpuCountMin: 2,
memorySizeMin: 1024 * 1024 * 1024,
diskFormat: .asif
)
XCTAssertEqual(config.diskFormat, .asif)
// Test JSON encoding/decoding
let encoder = JSONEncoder()
let data = try encoder.encode(config)
let decoder = JSONDecoder()
let decodedConfig = try decoder.decode(VMConfig.self, from: data)
XCTAssertEqual(decodedConfig.diskFormat, .asif)
}
func testVMConfigDefaultDiskFormat() throws {
// Test that VMConfig defaults to raw format
let config = VMConfig(
platform: Linux(),
cpuCountMin: 2,
memorySizeMin: 1024 * 1024 * 1024
)
XCTAssertEqual(config.diskFormat, .raw)
}
}
+24 -1
View File
@@ -13,7 +13,7 @@ brew install go
Finally, run the following command from this (`benchmark/`) directory:
```shell
go run cmd/main.go fio --image ghcr.io/cirruslabs/macos-sonoma-base:latest --prepare 'sudo purge && sync'
go run cmd/main.go fio --image ghcr.io/cirruslabs/macos-sequoia-base:latest --prepare 'sudo purge && sync'
```
You can also enable the debugging output to diagnose issues:
@@ -185,3 +185,26 @@ sync test Tart (--root-disk-opts="sync=none")
sync test Tart (--root-disk-opts="caching=cached") 0 B/s 35 MB/s 0 IOPS 15.67 kIOPS 0s ± 0s 11.319µs ± 24.771µs 51.731µs ± 42.208µs
sync test Tart (--root-disk-opts="sync=none,caching=cached") 0 B/s 17 MB/s 0 IOPS 7.39 kIOPS 0s ± 0s 21.23µs ± 81.749µs 113.239µs ± 191.266µs
```
### March 23, 2025
Host:
* Hardware: Mac mini (Apple M2 Pro, 8 performance and 4 efficiency cores, 32 GB RAM, `Mac14,12`)
* OS: macOS Sequoia 15.3.2
* Xcode: 16.2
Guest:
* Hardware: [Virtualization.Framework](https://developer.apple.com/documentation/virtualization)
* OS: macOS Sonoma 15.3.2
* Xcode: 16.2
```
Name Executor Time
XcodeBenchmark (d869315) local 2m19s
XcodeBenchmark (d869315) Tart 3m59s
XcodeBenchmark (d869315) Tart (--root-disk-opts="sync=none") 3m48s
XcodeBenchmark (d869315) Tart (--root-disk-opts="caching=cached") 3m35s
XcodeBenchmark (d869315) Tart (--root-disk-opts="sync=none,caching=cached") 3m14s
```
+6 -2
View File
@@ -1,21 +1,24 @@
module github.com/cirruslabs/tart/benchmark
go 1.22.1
toolchain go1.24.1
require (
github.com/avast/retry-go/v4 v4.5.1
github.com/dustin/go-humanize v1.0.1
github.com/google/uuid v1.6.0
github.com/gosuri/uitable v0.0.4
github.com/shirou/gopsutil v3.21.11+incompatible
github.com/spf13/cobra v1.8.0
github.com/stretchr/testify v1.9.0
go.uber.org/zap v1.27.0
golang.org/x/crypto v0.21.0
golang.org/x/crypto v0.35.0
)
require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/fatih/color v1.16.0 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
@@ -23,7 +26,8 @@ require (
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/sys v0.18.0 // indirect
golang.org/x/sys v0.30.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
+13 -6
View File
@@ -7,6 +7,8 @@ github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkp
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM=
github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE=
github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY=
github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gosuri/uitable v0.0.4 h1:IG2xLKRvErL3uhY6e1BylFzG+aJiwQviDDTfOKeKTpY=
@@ -26,26 +28,31 @@ github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJ
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/shirou/gopsutil v3.21.11+incompatible h1:+1+c1VGhc88SSonWP6foOcLhvnKlUeu/erjjvaPEYiI=
github.com/shirou/gopsutil v3.21.11+incompatible/go.mod h1:5b4v6he4MtMOwMlS0TUMTu2PcXUg8+E1lC7eC3UO/RA=
github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
golang.org/x/crypto v0.21.0 h1:X31++rzVUdKhX5sWmSOFZxx8UW/ldWx55cbf08iNAMA=
golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
golang.org/x/crypto v0.35.0 h1:b15kiHdrGCHrP6LvwaQ3c03kgNhhiMgvlhxHQhmg2Xs=
golang.org/x/crypto v0.35.0/go.mod h1:dy7dXNW32cAb/6/PRuTNsix8T+vJAqvuIy5Bli/x0YQ=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.18.0 h1:FcHjZXDMxI8mM3nwhX9HlKop4C0YQvCVCdwYl2wOtE8=
golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58=
golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.29.0 h1:L6pJp37ocefwRRtYPKSWOWzOtWSxVajvz2ldH/xi3iU=
golang.org/x/term v0.29.0/go.mod h1:6bl4lRlvVuDgSf3179VpIxBF0o10JUpXWOnI7nErv7s=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+1 -48
View File
@@ -4,8 +4,6 @@ import (
"encoding/json"
"fmt"
executorpkg "github.com/cirruslabs/tart/benchmark/internal/executor"
"github.com/cirruslabs/tart/benchmark/internal/executor/local"
"github.com/cirruslabs/tart/benchmark/internal/executor/tart"
"github.com/dustin/go-humanize"
"github.com/gosuri/uitable"
"github.com/spf13/cobra"
@@ -46,57 +44,12 @@ func run(cmd *cobra.Command, args []string) error {
_ = logger.Sync()
}()
var executorInitializers = []struct {
Name string
Fn func() (executorpkg.Executor, error)
}{
{
Name: "local",
Fn: func() (executorpkg.Executor, error) {
return local.New(logger)
},
},
{
Name: "Tart",
Fn: func() (executorpkg.Executor, error) {
return tart.New(cmd.Context(), image, nil, logger)
},
},
{
Name: "Tart (--root-disk-opts=\"sync=none\")",
Fn: func() (executorpkg.Executor, error) {
return tart.New(cmd.Context(), image, []string{
"--root-disk-opts",
"sync=none",
}, logger)
},
},
{
Name: "Tart (--root-disk-opts=\"caching=cached\")",
Fn: func() (executorpkg.Executor, error) {
return tart.New(cmd.Context(), image, []string{
"--root-disk-opts",
"caching=cached",
}, logger)
},
},
{
Name: "Tart (--root-disk-opts=\"sync=none,caching=cached\")",
Fn: func() (executorpkg.Executor, error) {
return tart.New(cmd.Context(), image, []string{
"--root-disk-opts",
"sync=none,caching=cached",
}, logger)
},
},
}
table := uitable.New()
table.AddRow("Name", "Executor", "B/W (read)", "B/W (write)", "I/O (read)", "I/O (write)",
"Latency (read)", "Latency (write)", "Latency (sync)")
for _, benchmark := range benchmarks {
for _, executorInitializer := range executorInitializers {
for _, executorInitializer := range executorpkg.DefaultInitializers(cmd.Context(), image, logger) {
if prepare != "" {
shell := "/bin/sh"
+2
View File
@@ -2,6 +2,7 @@ package command
import (
"github.com/cirruslabs/tart/benchmark/internal/command/fio"
"github.com/cirruslabs/tart/benchmark/internal/command/xcode"
"github.com/spf13/cobra"
)
@@ -14,6 +15,7 @@ func NewCommand() *cobra.Command {
cmd.AddCommand(
fio.NewCommand(),
xcode.NewCommand(),
)
return cmd
@@ -0,0 +1,13 @@
package xcode
type Benchmark struct {
Name string
Command string
}
var benchmarks = []Benchmark{
{
Name: "XcodeBenchmark (d869315)",
Command: "git clone https://github.com/devMEremenko/XcodeBenchmark.git && cd XcodeBenchmark && git reset --hard d86931529ada1df2a1c6646dd85958c360954065 && xcrun simctl list && sh benchmark.sh",
},
}
@@ -0,0 +1,49 @@
package xcode
import (
"fmt"
"regexp"
"time"
)
type Output struct {
Started time.Time
Ended time.Time
}
func ParseOutput(s string) (*Output, error) {
// Ensure that the build has succeeded
matched, err := regexp.MatchString("(?m)^\\*\\* BUILD SUCCEEDED \\*\\*.*$", s)
if err != nil {
return nil, fmt.Errorf("failed to parse output: regexp failed: %v", err)
}
if !matched {
return nil, fmt.Errorf("failed to parse output: \"** BUILD SUCCEEDED **\" string " +
"not found on a separate line, make sure you have Xcode installed")
}
re := regexp.MustCompile("Started\\s+(?P<started>.*)\\n.*Ended\\s+(?P<ended>.*)\\n")
matches := re.FindStringSubmatch(s)
if len(matches) != re.NumSubexp()+1 {
return nil, fmt.Errorf("failed to parse output: cannot find Started and Ended times")
}
startedRaw := matches[re.SubexpIndex("started")]
started, err := time.Parse(time.TimeOnly, startedRaw)
if err != nil {
return nil, fmt.Errorf("failed to parse started time %q: unsupported format", startedRaw)
}
endedRaw := matches[re.SubexpIndex("ended")]
ended, err := time.Parse(time.TimeOnly, endedRaw)
if err != nil {
return nil, fmt.Errorf("failed to parse ended time %q: unsupported format", startedRaw)
}
return &Output{
Started: started,
Ended: ended,
}, nil
}
@@ -0,0 +1,37 @@
package xcode_test
import (
"fmt"
"github.com/cirruslabs/tart/benchmark/internal/command/xcode"
"github.com/stretchr/testify/require"
"testing"
"time"
)
func TestParseOutput(t *testing.T) {
result, err := xcode.ParseOutput(`** BUILD SUCCEEDED ** [219.713 sec]
System Version: 14.6
Xcode 15.4
Hardware Overview
Model Name: Apple Virtual Machine 1
Model Identifier: VirtualMac2,1
Total Number of Cores: 4
Memory: 8 GB
✅ XcodeBenchmark has completed
1️⃣ Take a screenshot of this window (Cmd + Shift + 4 + Space) and resize to include:
- Build Time (See ** BUILD SUCCEEDED ** [XYZ sec])
- System Version
- Xcode Version
- Hardware Overview
- Started 13:46:20
- Ended 13:50:02
- Date Thu Jan 16 13:50:02 UTC 2025
2️⃣ Share your results at https://github.com/devMEremenko/XcodeBenchmark
`)
require.NoError(t, err)
fmt.Println(result)
require.Equal(t, 222*time.Second, result.Ended.Sub(result.Started))
}
+108
View File
@@ -0,0 +1,108 @@
package xcode
import (
"fmt"
executorpkg "github.com/cirruslabs/tart/benchmark/internal/executor"
"github.com/gosuri/uitable"
"github.com/spf13/cobra"
"go.uber.org/zap"
"go.uber.org/zap/zapio"
"os"
"os/exec"
)
var debug bool
var image string
var prepare string
func NewCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "xcode",
Short: "run XCode benchmarks",
RunE: run,
}
cmd.Flags().BoolVar(&debug, "debug", false, "enable debug logging")
cmd.Flags().StringVar(&image, "image", "ghcr.io/cirruslabs/macos-sequoia-xcode:latest", "image to use for testing")
cmd.Flags().StringVar(&prepare, "prepare", "", "command to run before running each benchmark")
return cmd
}
func run(cmd *cobra.Command, args []string) error {
config := zap.NewProductionConfig()
if debug {
config.Level = zap.NewAtomicLevelAt(zap.DebugLevel)
}
logger, err := config.Build()
if err != nil {
return err
}
defer func() {
_ = logger.Sync()
}()
table := uitable.New()
table.AddRow("Name", "Executor", "Time")
for _, benchmark := range benchmarks {
for _, executorInitializer := range executorpkg.DefaultInitializers(cmd.Context(), image, logger) {
if prepare != "" {
shell := "/bin/sh"
if shellFromEnv, ok := os.LookupEnv("SHELL"); ok {
shell = shellFromEnv
}
logger.Sugar().Infof("running prepare command %q using shell %q",
prepare, shell)
cmd := exec.CommandContext(cmd.Context(), shell, "-c", prepare)
loggerWriter := &zapio.Writer{Log: logger, Level: zap.DebugLevel}
cmd.Stdout = loggerWriter
cmd.Stderr = loggerWriter
if err := cmd.Run(); err != nil {
return fmt.Errorf("failed to run prepare command %q: %v", prepare, err)
}
}
logger.Sugar().Infof("initializing executor %s", executorInitializer.Name)
executor, err := executorInitializer.Fn()
if err != nil {
return err
}
logger.Sugar().Infof("running benchmark %q on %s executor", benchmark.Name,
executorInitializer.Name)
stdout, err := executor.Run(cmd.Context(), benchmark.Command)
if err != nil {
return err
}
output, err := ParseOutput(string(stdout))
if err != nil {
return err
}
duration := output.Ended.Sub(output.Started)
logger.Sugar().Infof("Xcode benchmark duration: %s", duration)
table.AddRow(benchmark.Name, executorInitializer.Name, duration)
if err := executor.Close(); err != nil {
return fmt.Errorf("failed to close executor %s: %w",
executorInitializer.Name, err)
}
}
}
fmt.Println(table.String())
return nil
}
@@ -0,0 +1,57 @@
package executor
import (
"context"
"github.com/cirruslabs/tart/benchmark/internal/executor/local"
"github.com/cirruslabs/tart/benchmark/internal/executor/tart"
"go.uber.org/zap"
)
type Initializer struct {
Name string
Fn func() (Executor, error)
}
func DefaultInitializers(ctx context.Context, image string, logger *zap.Logger) []Initializer {
return []Initializer{
{
Name: "local",
Fn: func() (Executor, error) {
return local.New(logger)
},
},
{
Name: "Tart",
Fn: func() (Executor, error) {
return tart.New(ctx, image, nil, logger)
},
},
{
Name: "Tart (--root-disk-opts=\"sync=none\")",
Fn: func() (Executor, error) {
return tart.New(ctx, image, []string{
"--root-disk-opts",
"sync=none",
}, logger)
},
},
{
Name: "Tart (--root-disk-opts=\"caching=cached\")",
Fn: func() (Executor, error) {
return tart.New(ctx, image, []string{
"--root-disk-opts",
"caching=cached",
}, logger)
},
},
{
Name: "Tart (--root-disk-opts=\"sync=none,caching=cached\")",
Fn: func() (Executor, error) {
return tart.New(ctx, image, []string{
"--root-disk-opts",
"sync=none,caching=cached",
}, logger)
},
},
}
}
+20
View File
@@ -7,11 +7,14 @@ import (
"fmt"
"github.com/avast/retry-go/v4"
"github.com/google/uuid"
"github.com/shirou/gopsutil/mem"
"go.uber.org/zap"
"go.uber.org/zap/zapio"
"golang.org/x/crypto/ssh"
"io"
"net"
"runtime"
"strconv"
"strings"
"time"
)
@@ -37,6 +40,23 @@ func New(ctx context.Context, image string, runArgsExtra []string, logger *zap.L
return nil, err
}
vmStat, err := mem.VirtualMemory()
if err != nil {
return nil, err
}
cpus := strconv.Itoa(runtime.NumCPU())
memory := strconv.FormatUint(vmStat.Total/1024/1024, 10)
logger.Info("Setting resources", zap.String("cpus", cpus), zap.String("memory", memory))
setResourcesArguments := []string{
"set", tart.vmName,
"--cpu", cpus,
"--memory", memory,
}
if err := Cmd(ctx, tart.logger, setResourcesArguments...); err != nil {
return nil, err
}
vmRunCtx, vmRunCancel := context.WithCancel(ctx)
tart.vmRunCancel = vmRunCancel
+2 -1
View File
@@ -9,4 +9,5 @@
"MD033": false # Inline HTML
"MD041": false # First line in file should be a top level heading
"MD045": false # OK not to have a description for an image
"MD046": false # Code block style [Expected: fenced; Actual: indented]
"MD046": false # Code block style [Expected: fenced; Actual: indented]
"MD059": false # It's OK to have "here" links
Binary file not shown.

After

Width:  |  Height:  |  Size: 104 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 155 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 118 KiB

@@ -0,0 +1,72 @@
---
draft: false
date: 2025-06-01
search:
exclude: true
authors:
- edigaryev
categories:
- announcement
---
# Bridging the gaps with the Tart Guest Agent
We're introducing a new improvement for the Tart usability experience: a [Tart Guest Agent](https://github.com/cirruslabs/tart-guest-agent).
This agent provides automatic disk resizing, seamless clipboard sharing for macOS guests (a [long-awaited](https://github.com/cirruslabs/tart/issues/14) feature), and the ability to run commands, without SSH and networking, using the new `tart exec` command.
As of recently, we include this agent in all non-vanilla Cirrus Labs images, so you likely won't need to do anything to benefit from these usability improvements.
Read on to learn why we chose to implement the agent from scratch in Golang, and which features we plan to add next.
<!-- more -->
## Existing solutions
Tart uses the Virtualization.Framework, and the latter implemented a SPICE client some time ago, however, one piece was missing: the agent that runs inside the guest.
The original [SPICE `vdagent` implementation](https://gitlab.freedesktop.org/spice/linux/vd_agent) only supports Linux. While [a fork](https://github.com/utmapp/vd_agent) from the UTM project adds macOS support, the long-term viability of maintaining this fork without upstreaming changes is uncertain.
Moreover, if we were to add some extra functionality (as we did), there would be more than one agent binary to ship and install, which complicates maintenance and makes it harder to explain to users why we need a bunch of agent binaries.
In the end, we decided to go with our own solution, one that would easily accomodate future ideas.
## Rolling our own agent
After carefully inspecting the [`vdagent` protocol](https://www.spice-space.org/agent-protocol.html) we've realized that the clipboard sharing is actually a small subset of the whole protocol, making it relatively simple to implement.
Thanks to Golang, we were able to implement the protocol much faster than we could have with a lower-level language like C (with all due respect), which requires manual memory management and complex event loops.
As for the command execution via `tart exec`, we've decided to go with gRPC with a rather simple protocol:
![An visualization of gRPC protocol used by the Tart Guest Agent](../images/tart-guest-agent-grpc-protocol.png)
For each `tart exec` invocation a new gRPC `Exec` bidirectional stream is established with the agent running inside a VM. After the gRPC stream is established, `tart exec` sends a command to execute to the guest and streams the I/O. Once the command terminates, `tart exec` collects the process exit code and quits with exactly that exit code.
Using gRPC simplifies `tart exec` implementation because of code generation and forms a nice bridge between the host and the guest which allows us to easily expand the protocol later down the road when we decide to introduce new features.
Thanks to [gRPC Swift](https://github.com/grpc/grpc-swift), which is built on top of [SwiftNIO](https://github.com/apple/swift-nio), we get [`async/await`](https://docs.swift.org/swift-book/documentation/the-swift-programming-language/concurrency/) support for free, further simplifying the `tart exec` logic.
As for the Tart Guest Agent, the final result is a Golang binary that [can be customized](https://github.com/cirruslabs/tart-guest-agent?tab=readme-ov-file#guest-agent-for-tart-vms) depending on the execution context:
* launchd global daemon — runs as a privileged user (`root`), has no clipboard access
* `--resize-disk` — resizes the disk when there's a free space at the end of a disk (assuming that one previously ran `tart set --disk-size`)
* launchd global agent — runs as a normal user (`admin`), has clipboard access
* `--run-vdagent` — clipboard sharing
* `--run-rpc` — `tart exec` and new functionality in the future
We’ve also introduced `--run-daemon` (which implies `--resize-disk`) and `--run-agent` (which implies both `--run-vdagent` and `--run-rpc`) to help run the most appropriate functionality based on the given context.
## Future plans
First, we'd like to thank our paid clients, without whom this feature wouldn't have been possible.
[Become one now](../../licensing.md) and enjoy higher allowances for Tart VMs and Orchard workers—while helping ensure that our roadmap aligns with your company's needs.
In the near future we plan to implement:
* Linux support — to provide seamless experience for Linux guests too
* a new `tart ip` resolver — to provide a more robust IP retrieval facility for Linux guests, which often struggle to populate the host's ARP table with their network activity
* `tart cp` command — to copy files from/to guest VMs
Stay tuned, and feel free to send us feedback on [GitHub](https://github.com/cirruslabs/tart) and [Twitter](https://x.com/cirrus_labs)!
+132 -4
View File
@@ -5,6 +5,16 @@ title: Frequently Asked Questions
description: Advanced configuration and troubleshooting tips for advanced configurations.
---
## Troubleshooting crashes
If you experience a crash or encounter another error while using the tart executable, you can collect debug information to assist with troubleshooting. Run the following command in a separate terminal window to gather logs from the Tart process and the macOS Virtualization subsystem:
```shell
log stream --predicate='process=="tart" OR process CONTAINS "Virtualization"' > tart.log
```
While the events are being streamed, attempt to reproduce the issue. Once the issue is reproduced, stop the streaming by pressing Ctrl+C. Then, attach the tart.log file to your report.
## VM location on disk
Tart stores all its files in `~/.tart/` directory. Local images that you can run are stored in `~/.tart/vms/`.
@@ -13,12 +23,12 @@ Remote images are pulled into `~/.tart/cache/OCIs/`.
## Nested virtualization support?
Tart is limited by functionality of Apple's `Virtualization.Framework`. At the moment `Virtualization.Framework`
doesn't support nested virtualization.
supports nested virtualization only on M3 or M4 chips running macOS 15 (Sequoia). By default, it is disabled, but can be enabled by passing the `--nested` flag to `tart run`.
## Connecting to a service running on host
To connect from within a virtual machine to a service running on the host machine
please first make sure that the service is binded to `0.0.0.0`.
please first make sure that the service is bound to `0.0.0.0`.
Then from within a virtual machine you can access the service using the router's IP address that you can get either from `Preferences -> Network`
or by running the following command in the Terminal:
@@ -29,7 +39,7 @@ netstat -nr | grep default | head -n 1 | awk '{print $2}'
Note: that accessing host is only possible with the default NAT network. If you are running your virtual machines with
[Softnet](https://github.com/cirruslabs/softnet) (via `tart run --net-softnet <VM NAME>)`, then the network isolation
is stricter and it's not only possible to access the host.
is stricter and it's not possible to access the host.
## Changing the default NAT subnet
@@ -67,6 +77,57 @@ sudo rm /var/db/dhcpd_leases
And no worries, this file will be re-created on the next `tart run`.
## Unsupported DHCP client identifiers
Due to the limitations of the macOS built-in DHCP server, `tart ip` is unable to correctly report the IP addresses for VMs using DHCP client identifiers that are not based on VMs link-layer addresses (MAC addresses).
By default, when [no `--resolver=arp` is specified](#resolving-the-vms-ip-when-using-bridged-networking), `tart ip` reads the `/var/db/dhcpd_leases` file and tries to find the freshest entry that matches the VM's MAC address (based on the `hw_address` field).
However, things starts to break when the VM uses a [DUID-EN](https://metebalci.com/blog/a-note-on-dhcpv6-duid-and-prefix-delegation#duid-types) identifier, for example. One of the notorious examples of this being Ubuntu, using this type of identifier by default on latest versions. This results in the `/var/db/dhcpd_leases` entry for Ubuntu appearing as follows:
```ini
{
name=ubuntu
ip_address=192.168.64.3
hw_address=ff,f1:f5:dd:7f:0:2:0:0:ab:11:cb:fb:30:b0:97:b6:3a:67
identifier=ff,f1:f5:dd:7f:0:2:0:0:ab:11:cb:fb:30:b0:97:b6:3a:67
lease=0x678e2ce7
}
```
Because the macOS built-in DHCP server overwrites the `hw_address` with the `identifier`, it leaves no information about the VM's MAC address to the `tart ip`.
To avoid this issue, make sure that your VM only sends a DHCP client identifier (option 61) with link-layer address (MAC address) or that it doesn't send this option at all.
For the aforementioned Ubuntu, the solution is outlined in the section [How to integrate with Windows DHCP Server](https://netplan.readthedocs.io/en/stable/examples/#how-to-integrate-with-windows-dhcp-server) of Canonical Netplan's documentation:
```yaml
network:
version: 2
ethernets:
enp3s0:
dhcp4: yes
dhcp-identifier: mac
```
## Resolving the VM's IP when using bridged networking
When running `tart run` with `--net-bridged`, you need to invoke `tart ip` differently, because the macOS built-in DHCP server won't have any information about the VM's IP-address:
```shell
tart ip --resolver=arp <VM>
```
This causes the `tart ip` to consult the host's ARP table instead of the `/var/db/dhcpd_leases` file.
Note that this method of resolving the IP heavily relies on the level of VM's activity on the network, namely, exchanging ARP requests between the guest and the host.
This is normally not an issue for macOS VMs, but on Linux VMs you might need to install Samba, which includes a [NetBIOS name server](https://www.samba.org/samba/docs/current/man-html/nmbd.8.html) and exhibits the same behavior as macOS, resulting in the population of the ARP table of the host OS:
```shell
sudo apt-get install samba
```
## Running login/clone/pull/push commands over SSH
When invoking the Tart in an SSH session, you might get error like this:
@@ -97,7 +158,7 @@ or features supported. If there is some feature missing please don't hesitate to
Instead of Anka Registry, Tart can work with any OCI-compatible container registry. This provides a much more consistent
and scalable experience for distributing virtual machines.
Tart doesn't yet have an analogue of Anka Controller for managing long living VMs but [soon will be](https://github.com/cirruslabs/tart/issues/372).
Tart does have an analogue of Anka Controller for managing VMs across a cluster of Mac hosts called [Orchard](orchard/quick-start.md).
## Automatic pruning
@@ -116,3 +177,70 @@ export TART_NO_AUTO_PRUNE=
```shell
TART_NO_AUTO_PRUNE= tart pull ...
```
## Disk resizing
Disk resizing works on most cloud-ready Linux distributions out-of-the box (e.g. Ubuntu Cloud Images have the `cloud-initramfs-growroot` package installed that runs on boot) and on the rest of the distributions by running the `growpart` or `resize2fs` commands.
For macOS, however, things are a bit more complicated, and you generally have two options: automated and manual resizing.
For the automated option, you can use [Packer](https://www.packer.io/) with the [Packer builder for Tart VMs](https://developer.hashicorp.com/packer/integrations/cirruslabs/tart/latest/components/builder/tart). The latter has two has configuration directives related to the disk resizing behavior:
* [`disk_size_gb`](https://developer.hashicorp.com/packer/integrations/cirruslabs/tart/latest/components/builder/tart#configuration-reference) — controls the target disk size in gigabytes
* [`recovery_partition`](https://developer.hashicorp.com/packer/integrations/cirruslabs/tart/latest/components/builder/tart#configuration-reference) — controls what to do with the recovery partition when resizing the disk
* you can either keep, delete or relocate it to the end of the disk
For the manual approach, you have to remove the recovery partition first, repair the disk and the resize the APFS container.
To do this, first we'll need to identify the primary disk and the APFS containers by running the command below from within a VM:
```shell
diskutil list physical
```
For example, the output might look like this:
```plain
/dev/disk0 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *100.0 GB disk0
1: Apple_APFS_ISC Container disk1 524.3 MB disk0s1
2: Apple_APFS Container disk3 44.1 GB disk0s2
3: Apple_APFS_Recovery Container disk2 5.4 GB disk0s3
(free space) 50.0 GB -
```
In the output, you'll normally see:
* a single physical disk (`disk0`)
* APFS container with the system partition which we're going to resize (`disk0s2`)
* APFS container with the recovery partition which we're going to delete (`disk0s3`)
* `(free space)` which we'll put to use
To proceed, boot the VM in recovery mode using `tart run --recovery` and choose the "Options" item:
![](assets/images/faq/tart-run-recovery-options.png){width="640" .center}
When the recovery OS boots, open the Terminal app:
![](assets/images/faq/tart-run-recovery-terminal.png){width="720" .center}
In Terminal app, invoke the command below to remove the recovery partition:
```shell
diskutil eraseVolume free free disk0s3
```
Now, repair the disk:
```shell
yes | diskutil repairDisk disk0
```
Finally, resize the system APFS container to take all the remaining space:
```shell
diskutil apfs resizeContainer disk0s2 0
```
Now, you can shut down and `tart run` as you'd normally do.
+2 -2
View File
@@ -18,9 +18,9 @@ steps:
- command: uname -a
plugins:
- cirruslabs/tart#main:
image: ghcr.io/cirruslabs/macos-sonoma-base:latest
image: ghcr.io/cirruslabs/macos-sequoia-base:latest
```
This will run `uname -r` in a macOS Tart VM cloned from `ghcr.io/cirruslabs/macos-sonoma-base:latest`.
This will run `uname -r` in a macOS Tart VM cloned from `ghcr.io/cirruslabs/macos-sequoia-base:latest`.
See plugin's [Configuration section](https://github.com/cirruslabs/tart-buildkite-plugin#configuration) for the full list of available options.
+2 -2
View File
@@ -18,7 +18,7 @@ task:
name: hello
macos_instance:
# can be a remote or a local virtual machine
image: ghcr.io/cirruslabs/macos-sonoma-base:latest
image: ghcr.io/cirruslabs/macos-sequoia-base:latest
hello_script:
- echo "Hello from within a Tart VM!"
- echo "Here is my CPU info:"
@@ -50,7 +50,7 @@ exposes it via [`artifacts` instruction](https://cirrus-ci.org/guide/writing-tas
task:
name: Build
macos_instance:
image: ghcr.io/cirruslabs/macos-sonoma-xcode:latest
image: ghcr.io/cirruslabs/macos-sequoia-xcode:latest
build_script: swift build --product tart
binary_artifacts:
path: .build/debug/tart
+1 -1
View File
@@ -42,7 +42,7 @@ Now you can use Tart Images in your `.gitlab-ci.yml`:
```yaml
# You can use any remote Tart Image.
# Tart Executor will pull it from the registry and use it for creating ephemeral VMs.
image: ghcr.io/cirruslabs/macos-sonoma-base:latest
image: ghcr.io/cirruslabs/macos-sequoia-base:latest
test:
tags:
+5 -5
View File
@@ -7,7 +7,7 @@ description: Use Packer to build custom VM images, configure VMs and work with r
## Creating from scratch
Tart supports macOS and Linux virtual machines. All commands like `run` and `pull` work the same way regarding of the underlying OS a particular VM image has.
Tart supports macOS and Linux virtual machines. All commands like `run` and `pull` work the same way regardless of the underlying OS a particular VM image has.
The only difference is how such VM images are created. Please check sections below for [macOS](#creating-a-macos-vm-image-from-scratch) and [Linux](#creating-a-linux-vm-image-from-scratch) instructions.
### Creating a macOS VM image from scratch
@@ -16,8 +16,8 @@ Tart can create VMs from `*.ipsw` files. You can download a specific `*.ipsw` fi
use `latest` instead of a path to `*.ipsw` to download the latest available version:
```bash
tart create --from-ipsw=latest sonoma-vanilla
tart run sonoma-vanilla
tart create --from-ipsw=latest sequoia-vanilla
tart run sequoia-vanilla
```
After the initial booting of the VM, you'll need to manually go through the macOS installation process. As a convention we recommend creating an `admin` user with an `admin` password. After the regular installation please do some additional modifications in the VM:
@@ -72,8 +72,8 @@ packer {
}
source "tart-cli" "tart" {
vm_base_name = "ghcr.io/cirruslabs/macos-sonoma-base:latest"
vm_name = "my-custom-sonoma"
vm_base_name = "ghcr.io/cirruslabs/macos-sequoia-base:latest"
vm_name = "my-custom-sequoia"
cpu_count = 4
memory_gb = 8
disk_size_gb = 70
+6 -4
View File
@@ -1,12 +1,14 @@
## Architecture
Orchard cluster consists of two components:
Orchard cluster consists of three components:
* Controller — responsible for managing the cluster and scheduling of resources
* Controller — responsible for managing the cluster and scheduling of resources
* Worker — responsible for executing the VMs
* Client — responsible for creating, modifying and removing the resources on the Controller, can either be an Orchard CLI or [an API consumer](/orchard/integration-guide)
* Client — responsible for creating, modifying and removing the resources on the Controller, can either be an [Orchard CLI](/orchard/using-orchard-cli) or [an API consumer](/orchard/integration-guide)
Normally you deploy a single Controller that needs to be accessible to both the Clients and Workers. Then you can deploy the Workers, which can reside anywhere and be inaccessible to Clients directly, e.g. behind a NAT.
At the moment, only one Controller instance is currently supported, while you can deploy one or more Workers and run any number of Clients.
In terms of networking requirements, only Controller needs to be directly accessible from Workers and Clients, while Workers and Clients can be deployed and run anywhere (e.g. behind a NAT).
## Security
+48 -1
View File
@@ -14,6 +14,47 @@ For example to use a secure, random value:
ORCHARD_BOOTSTRAP_ADMIN_TOKEN=$(openssl rand -hex 32) orchard controller run
```
## Customization
Note that all the [Deployment Methods](#deployment-methods) essentially boil down to starting an `orchard controller run` command and keeping it alive.
This means that by introducing additional command-line arguments, you can customize the Orchard Controller's behavior. Below, we list some of the common scenarios.
### Customizing listening port
* `--listen` — address to listen on (default `:6120`)
### Customizing TLS
* `--controller-cert` — use the controller certificate from the specified path instead of the auto-generated one (requires --controller-key)
* `--controller-key` — use the controller certificate key from the specified path instead of the auto-generated one (requires --controller-cert)
* `--insecure-no-tls` — disable TLS, making all connections to the controller unencrypted
* useful when deploying Orchard Controller behind a load balancer/ingress controller
### Built-in SSH server
Orchard Controller can act as a simple SSH server that port-forwards connections to the VMs running in the Orchard Cluster.
This way you can completely skip the Orchard API when connecting to a given VM and only use the SSH client:
```shell
ssh -J <service account name>@orchard-controller.example.com <VM name>
```
To enable this functionality, pass `--listen-ssh` command-line argument to the `orchard controller run` command, for example:
```ssh
orchard controller run --listen-ssh 6122
```
Here's other command-line arguments associated with this functionality:
* `--ssh-host-key` — use the SSH private host key from the specified path instead of the auto-generated one
* `--insecure-ssh-no-client-auth` — allow SSH clients to connect to the controller's SSH server without authentication, thus only authenticating on the target worker/VM's SSH server
* useful when you already have strong credentials on your VMs, and you want to share these VMs to others without additionally giving out Orchard Cluster credentials
Check out our [Jumping through the hoops: SSH jump host functionality in Orchard](/blog/2024/06/20/jumping-through-the-hoops-ssh-jump-host-functionality-in-orchard/) blog post for more information.
## Deployment Methods
While you can always start `orchard controller run` manually with the required arguments, this method is not recommended due to lack of persistence.
@@ -31,7 +72,13 @@ gcloud compute addresses create orchard-ip --region=us-central1
export ORCHARD_IP=$(gcloud compute addresses describe orchard-ip --format='value(address)' --region=us-central1)
```
Once we have the IP address, we can create a new instance with Orchard Controller running inside a container:
Then, ensure that there exist a firewall rule targeting `https-server` tag and allowing access to TCP port 443. If that's not the case, create one:
```shell
gcloud compute firewall-rules create default-allow-https --direction=INGRESS --priority=1000 --network=default --action=ALLOW --rules=tcp:443 --source-ranges=0.0.0.0/0 --target-tags=https-server
```
Once we have the IP address and the firewall rule set up, we can create a new instance with Orchard Controller running inside a container:
```bash
gcloud compute instances create-with-container orchard-controller \
+2 -2
View File
@@ -51,8 +51,6 @@ Then, create a launchd job definition in `/Library/LaunchDaemons/org.cirruslabs.
<dict>
<key>Label</key>
<string>org.cirruslabs.orchard.worker</string>
<key>UserName</key>
<string>admin</string>
<key>Program</key>
<string>/opt/homebrew/bin/orchard</string>
<key>ProgramArguments</key>
@@ -60,6 +58,8 @@ Then, create a launchd job definition in `/Library/LaunchDaemons/org.cirruslabs.
<string>/opt/homebrew/bin/orchard</string>
<string>worker</string>
<string>run</string>
<string>--user</string>
<string>admin</string>
<string>--bootstrap-token</string>
<string>${BOOTSTRAP_TOKEN}</string>
<string>orchard.example.com</string>
+2 -2
View File
@@ -85,7 +85,7 @@ def main():
# Create VM
response = requests.post("http://127.0.0.1:6120/v1/vms", auth=basic_auth, json={
"name": vm_name,
"image": "ghcr.io/cirruslabs/macos-sonoma-base:latest",
"image": "ghcr.io/cirruslabs/macos-sequoia-base:latest",
"cpu": 4,
"memory": 4096,
"startup_script": {
@@ -144,7 +144,7 @@ func main() {
Meta: v1.Meta{
Name: vmName,
},
Image: "ghcr.io/cirruslabs/macos-sonoma-base:latest",
Image: "ghcr.io/cirruslabs/macos-sequoia-base:latest",
CPU: 4,
Memory: 4096,
StartupScript: &v1.VMScript{
+6 -6
View File
@@ -26,7 +26,7 @@ more information.
Now, let's create a Virtual Machine:
```shell
orchard create vm --image ghcr.io/cirruslabs/macos-sonoma-base:latest sonoma-base
orchard create vm --image ghcr.io/cirruslabs/macos-sequoia-base:latest sequoia-base
```
You can check a list of VM resources to see if the Virtual Machine we've created above is already running:
@@ -48,7 +48,7 @@ instance. Orchard Controller instance is secured by default and all API calls ar
To SSH into a VM, use the `orchard ssh` command:
```shell
orchard ssh vm sonoma-base
orchard ssh vm sequoia-base
```
You can specify the `--username` and `--password` flags to specify the username/password pair to use for the SSH
@@ -58,14 +58,14 @@ You can also execute remote commands instead of spawning a login shell, similarl
a command argument:
```shell
orchard ssh vm sonoma-base "uname -a"
orchard ssh vm sequoia-base "uname -a"
```
You can execute scripts remotely this way, by telling the remote command-line interpreter to read from the standard
input and using the redirection operator as follows:
```shell
orchard ssh vm sonoma-base "bash -s" < script.sh
orchard ssh vm sequoia-base "bash -s" < script.sh
```
### VNC
@@ -73,7 +73,7 @@ orchard ssh vm sonoma-base "bash -s" < script.sh
Similarly to `ssh` command, you can use `vnc` command to open Screen Sharing into a remote VM:
```shell
orchard vnc vm sonoma-base
orchard vnc vm sequoia-base
```
You can specify the `--username` and `--password` flags to specify the username/password pair to use for the VNC
@@ -84,7 +84,7 @@ protocol. By default, `admin`/`admin` is used.
The following command will delete the VM we've created above and clean-up the resources associated with it:
```shell
orchard delete vm sonoma-base
orchard delete vm sequoia-base
```
## Environment variables
+77
View File
@@ -0,0 +1,77 @@
## Installation
The easiest way to install Orchard CLI is through the [Homebrew](https://brew.sh/):
```shell
brew install cirruslabs/cli/orchard
```
Binaries and packages for other architectures can be found in [GitHub Releases](https://github.com/cirruslabs/orchard/releases).
## Setting up a context
The first step after installing the Orchard CLI is to configure its context. Configuring context is like pairing with the specified Orchard Controller, so that the commands like `orchard create vm`, `orchard ssh vm` will work.
To configure a context, `orchard context` has a subfamily of commands:
* `orchard context create <CONTROLLER ADDRESS>` — creates a new context to communicate with Orchard Controller available on the specified address
* `orchard context default <CONTROLLER ADDRESS>` — sets a context with a given Orchard Controller address as default (in case there's more than one context configured)
* `orchard context list` — lists all the configured contexts, indicating the default one
* `orchard context delete <CONTROLLER ADDRESS>` — deletes a context for the specified Orchard Controller address
Most of the time, you'll only need the `orchard context create`. For example, if you've deployed your Orchard Controller to `orchard-controller.example.com`, a new context can be configured like so:
```shell
orchard context create orchard-controller.example.com
```
`orchard context create` assumes port 6120 by default, so if you use a different port for the Orchard Controller, simply specify the port explicitly:
```shell
orchard context create orchard-controller.example.com:8080
```
When creating a new context you will be prompted for the service account name and token, which can be obtained from:
* `orchard controller run` logs
* if this is a first start
* `orchard get service-account`
* from an already configured Orchard CLI
## Using labels when creating VMs
Labels are useful if you want to restrict scheduling of a VM to workers whose labels include a subset of the VM's specified labels.
For example, you might have an Orchard Cluster consisting of the following workers:
* Mac Minis (`orchard worker run --labels location=DC1-R12-S4,model=macmini`)
* Mac Studios (`orchard worker run --labels location=DC1-R18-S8,model=macstudio`)
To create and run a VM specifically on Mac Studio machines, pass the `--labels` command-line argument to `orchard create vm` when creating a VM:
```shell
orchard create vm --labels model=macstudio <NAME>
```
When processing this VM, the scheduler will only place it on available Mac Studio workers.
## Using resources when creating VMs
Resources are useful if you want to restrict scheduling of a VM to workers that still have enough of the specified resource to fit the VM's requirements.
The difference between the labels is that the resources are finite and are automatically accounted by the scheduler.
To illustrate this with an example, let's say you have an Orchard Cluster consisting of the following workers:
* Mac Mini with 1 Gbps bandwidth (`orchard worker run --resources bandwidth-mbps=1000`)
* Mac Studio with 10 Gbps bandwidth (`orchard worker run --resources bandwidth-mbps=10000`)
VM created using the command below will only be scheduled on a Mac Studio with 10 Gbps bandwidth:
```shell
orchard create vm --resources bandwidth-mbps=7500 <NAME>
```
However, after this VM is scheduled, the 10 Gbps Mac Studio will only be able to accommodate one more VM (due to internal Apple EULA limit for macOS virtualization) with `bandwidth-mbps=2500` or less.
After the VM finishes, the unused resources will be available again.
+13 -9
View File
@@ -9,18 +9,18 @@ Try running a Tart VM on your Apple Silicon device running macOS 13.0 (Ventura)
```bash
brew install cirruslabs/cli/tart
tart clone ghcr.io/cirruslabs/macos-sonoma-base:latest sonoma-base
tart run sonoma-base
tart clone ghcr.io/cirruslabs/macos-sequoia-base:latest sequoia-base
tart run sequoia-base
```
??? info "Manual installation from a release archive"
It's also possible to manually install `tart` binary from the latest released archive:
```bash
curl -LO https://github.com/cirruslabs/tart/releases/latest/download/tart-arm64.tar.gz
tar -xzvf tart-arm64.tar.gz
./tart.app/Contents/MacOS/tart clone ghcr.io/cirruslabs/macos-sonoma-base:latest sonoma-base
./tart.app/Contents/MacOS/tart run sonoma-base
curl -LO https://github.com/cirruslabs/tart/releases/latest/download/tart.tar.gz
tar -xzvf tart.tar.gz
./tart.app/Contents/MacOS/tart clone ghcr.io/cirruslabs/macos-sequoia-base:latest sequoia-base
./tart.app/Contents/MacOS/tart run sequoia-base
```
Please note that `./tart.app/Contents/MacOS/tart` binary is required to be used in order to trick macOS
@@ -34,6 +34,10 @@ tart run sonoma-base
The following macOS images are currently available:
* macOS 15 (Sequoia)
* `ghcr.io/cirruslabs/macos-sequoia-vanilla:latest`
* `ghcr.io/cirruslabs/macos-sequoia-base:latest`
* `ghcr.io/cirruslabs/macos-sequoia-xcode:latest`
* macOS 14 (Sonoma)
* `ghcr.io/cirruslabs/macos-sonoma-vanilla:latest`
* `ghcr.io/cirruslabs/macos-sonoma-base:latest`
@@ -82,7 +86,7 @@ These credentials work both for logging in via GUI, console (Linux) and SSH.
If the guest VM is running and configured to accept incoming SSH connections you can conveniently connect to it like so:
```bash
ssh admin@$(tart ip sonoma-base)
ssh admin@$(tart ip sequoia-base)
```
!!! tip "Running scripts inside Tart virtual machines"
@@ -91,8 +95,8 @@ ssh admin@$(tart ip sonoma-base)
```bash
brew install cirruslabs/cli/sshpass
sshpass -p admin ssh -o "StrictHostKeyChecking no" admin@$(tart ip sonoma-base) "uname -a"
sshpass -p admin ssh -o "StrictHostKeyChecking no" admin@$(tart ip sonoma-base) < script.sh
sshpass -p admin ssh -o "StrictHostKeyChecking no" admin@$(tart ip sequoia-base) "uname -a"
sshpass -p admin ssh -o "StrictHostKeyChecking no" admin@$(tart ip sequoia-base) < script.sh
```
## Mounting directories
+1 -4
View File
@@ -1,7 +1,4 @@
source = [
".build/x86_64-apple-macosx/release/tart",
".build/arm64-apple-macosx/release/tart"
]
source = [ "dist/tart_darwin_all/tart.app/Contents/MacOS/tart" ]
bundle_id = "com.github.cirruslabs.tart"
apple_id {
+2
View File
@@ -46,6 +46,7 @@ plugins:
match_path: blog/posts/.*
date_from_meta:
as_creation: date
abstract_chars_count: -1
- social:
cards_layout_dir: docs/layouts
cards_layout: custom
@@ -102,6 +103,7 @@ nav:
- "Architecture and Security": orchard/architecture-and-security.md
- "Deploying Controller": orchard/deploying-controller.md
- "Deploying Workers": orchard/deploying-workers.md
- "Using Orchard CLI": orchard/using-orchard-cli.md
- "Managing the Cluster": orchard/managing-cluster.md
- "Integrating with the API": orchard/integration-guide.md
- "FAQ": faq.md
+1 -1
View File
@@ -1,7 +1,7 @@
#!/bin/sh
# helper script to build and run a signed tart binary
# usage: ./scripts/run-signed.sh run sonoma-base
# usage: ./scripts/run-signed.sh run sequoia-base
set -e