Commit Graph

296 Commits

Author SHA1 Message Date
Fedor Korotkov ff928ad77d
Optimize DiskV2 Deduplication (#878)
* Revert "Lowercase `tart.app` (#751)"

This reverts commit a9e2a19015.

* Optimize DiskV2 deduplication logic

In case we cloned `disk.img` from a local image, check if data at offset has the expected contents already.

* Hole punch only if needed

* Calculate hash only if needed

* subdataChunks optimization

* Reapply "Lowercase `tart.app` (#751)"

This reverts commit e74e9c845a.

* format

* Save at least 1GB on deduplication logic

* Build separately

* Revert "subdataChunks optimization"

This reverts commit e59382aeba.

* Another optimization

* Removed debug log

* reformat

* Revert "Hole punch only if needed"

This reverts commit 8c569fc5
2024-08-05 12:24:31 -04:00
Nikolay Edigaryev 33b5cfe2ed
tart run: delay tilde (~) expansion until we're dealing with local path (#880) 2024-08-05 15:42:10 +04:00
Nikolay Edigaryev 3892cdb00d
tart run: replace --sync with --root-disk-opts (#879)
* VZDiskImageSynchronizationMode's "description" field is a dead code

* Re-use the VZDiskImageSynchronizationMode extension

* tart run: replace --sync with --root-disk-opts

* VM: support root disk synchronization mode on macOS
2024-08-05 15:17:58 +04:00
Nicholas FitzRoy-Dale 5f2199ef3e
Support setting root disk synchronization mode (#875)
* Support setting root disk synchronization mode

Adds a new VMConfig parameter (tart get / tart set) called 'sync' which
can be set to 'full' (default), 'fsync', or 'none', corresponding with
the values of VZDiskImageSynchronizationMode and allowing a tradeoff
between data integrity and speed.

* Remove unused import

* Fix formatting

* Make root disk sync behaviour a commandline option
2024-08-05 13:12:43 +04:00
Nikolay Edigaryev 06cae1296e
tart run: support disabling disk synchronization for --disk (#872) 2024-07-25 20:15:07 +04:00
Nikolay Edigaryev 1b81b12760
tart pull: try to re-use APFS blocks by cloning the base image (#864)
* tart pull: try to re-use APFS blocks by cloning the base image

* Punch a hole when a zero chunk is detected

* Properly retrieve errno when hole punching operation fails

* tart pull: do not retry on RuntimeError

* Ensure that the holes we're about to punch are FS block size-aligned

* VMDirectory: remove unused static variables

* tart pull: log if we've found an image to deduplicate against

* Do not prematurely read contents from disk

* Only consider candidates with deduplicatedBytes more than 0

* APFS reuse UX/DX improvements (#870)

* Show how much deduplication happening

Improvement to the APFS deduplication logic which checks whether a disk image file `mayShareFileContent` with some other file, and then we put a custom attribute to track the deduplication since there is no way to get this information from APFS itself.

It's not 100% accurate but given that OCI cache is immutable the actual disk usage can only be lover than that.

* Use string attribute

* Update Sources/tart/URL+Prunable.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Added SizeOnDisk colume

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

---------

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2024-07-25 15:33:15 +00:00
Nikolay Edigaryev 4ed73bc775
--no-audio: only disable the source and sink (#869)
To prevent crashes in the guest when playing or recording audio.
2024-07-18 15:10:44 +00:00
Nikolay Edigaryev 2dc25ce478
tart push: support --concurrency command-line argument (#868)
* tart push: support --concurrency command-line argument

* LayerizerTests: specify "concurrency" argument
2024-07-18 17:52:55 +04:00
Nikolay Edigaryev 1e74e268a5
DiskV2: change layer size to 512 * 1024 * 1024 bytes (#866)
Needed to test https://github.com/cirruslabs/tart/pull/864.
2024-07-17 16:37:45 +00:00
Nikolay Edigaryev bff344fb7f
tart login: better error when an improperly formatted host is provided (#863)
* tart login: better error when an improperly formatted host is provided

* Revert old behavior w.r.t. URLComponents()
2024-07-15 18:36:09 +04:00
Nikolay Edigaryev ababe8cefc
tart pull: choose across multiple VM images to deduplicate against (#862)
This is accomplished by saving the OCI VM image manifests on "tart pull"
in "manifest.json" file and then using them on successive "tart pull"'s
to find the best candidate that results in the most de-duplication,
measured in bytes.
2024-07-15 18:36:01 +04:00
Fedor Korotkov ea5313698e
Do not prune running VMs (#861)
Also prevent pushing of a running VM

Fixes #860
2024-07-15 07:10:51 -04:00
Nikolay Edigaryev 5eccdf7412
Support customizing VM disks and mounting remote VMs in `tart run` (#847)
* Support remote VM names in --disk command-line argument

* tart set: introduce "--disk" to support replacing VM's disk contents

* Complete the code comment
2024-07-02 18:12:35 +04:00
Nikolay Edigaryev 63e3235d91
tart run: pick up --net-softnet-allow when using --net-host (#853) 2024-07-02 16:39:51 +04:00
Tor Arne Vestbø bf5081b3d9
Hook SIGUSR2 to requestStop (#842)
For macOS this brings up a dialog, asking the user if they are sure
they want to shut down, which makes this less useful for automated
graceful shutdowns, but it may behave better on Linux, and there
might be ways to instruct macOS to not ask the user, so it's still
a nice feature, and aligns with the SIGUSR1 for suspend, and SIGINT
for non-graceful shutdown.
2024-06-17 12:11:00 -04:00
Fedor Korotkov 017592075f
Drop Monterey Support (#843)
* Drop Monterey Support

People will still be able to run and SSH into Monterey VMs or use VNC but pointing devices/keyboard won't work.

Fixes #841

* Fixed x86 build
2024-06-17 15:20:21 +00:00
Fedor Korotkov dd46033812
Friendly decompression error message (#835)
* Friendly decompression error message

* Wrap FilterError
2024-05-31 14:47:41 +00:00
Nikolay Edigaryev 204002f776
VMStorageOCI: percent-encode the colon in RemoteName's host (#828)
* VMStorageOCI: percent-encode the colon in RemoteName's host

* Do not use String extensions and add a comment
2024-05-21 11:21:28 -04:00
Nikolay Edigaryev 7c386e3466
tart pull: try to re-use local VM image layers to speed-up the pulling (#825)
* Remove unused pullFromRegistry() method with "reference" argument

* tart pull: try to deduplicate disk layers to speed-up the pulling
2024-05-16 19:43:56 +04:00
Nikolay Edigaryev dbbd716214
tart push: use fixed size chunks to allow for better deduplication (#821) 2024-05-14 19:23:04 +04:00
William Theaker 13d5ddb4a4
Minor documentation improvements. (#819)
* Minor documentation improvements.

* Fix MD031

* Add sudo to mount instructions.
2024-05-13 12:44:26 -04:00
Fedor Korotkov 985db24474
Introduce `--random-mac` and `--random-serial` flags for `tart set` (#809)
To generate new MAC address and/or serial number for a given VM.
2024-05-02 18:27:49 +04:00
Nikolay Edigaryev 1d01bf63fb
tart run: resolve VM's IP using ARP when using --net-bridged and --vnc (#811) 2024-05-02 18:04:26 +04:00
Andrew Malchuk 3ff3850da2
Add support pasting clipboard from host for Linux VMs (#806)
* Added partial support pasting clipboard from host (only for Linux VMs)

* Added option "--no-clipboard" to run command
2024-05-02 09:48:25 +04:00
Tor Arne Vestbø c6e8d0bfd7
Gracefully stop vm on tart stop (#808)
* Give Virtualization.framework a chance to stop the VM on tart stop

We were letting the CancellationError bubble up all the way until
it terminated app, which meant we didn't hit the shutdown code
in run(), stopping the VM and the network.

We now catch CancellationError and proceed to gracefully shut down.

We only stop the VM if it's still running, as a VM that has been
stopped via the menu can't be stopped again.

* Gracefully shut down VM when Tart is quit via menu

Normally the quit action will result in AppKit calling exit(),
but we want to gracefully shut down the VM, so we use the same
path as for closing of the VM window, namely signal our own
process with SIGINT or SIGUSR1.

If that doesn't work we let AppKit terminate as before.

This fixes the "Warning: NSActivity <_NSActivityAssertion:
0x600001f785a0> was ended multiple times" warning seen on
the console when quitting Tart via the menu.

* Activate Tart after application finishes launching

This ensures that the VM window has been shown by the time we
activate, so that we consistently activate and bring the VM
window to the front.
2024-04-30 09:27:41 -04:00
Fedor Korotkov 755aad4d7c
Check all VMs for MAC collision (#801)
* Check all VMs for MAC collision

Before only suspendable VMs were getting checked. Not sure why. It makes sense to check all.

* Always acquire a lock
2024-04-25 09:26:42 -04:00
Nikolay Edigaryev 9f38441a42
Fix pathHasMode() and only check for S_IFBLK (#800) 2024-04-23 11:12:35 -04:00
Fedor Korotkov 3d46c4e6c2
Support all NBD schemas (#799)
See https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md#nbd-uri-scheme

Fixes #792
2024-04-23 18:17:42 +04:00
Nikolay Edigaryev e59221f6a0
tart run: do not require root to mount a block device (#798) 2024-04-23 17:03:02 +04:00
Fedor Korotkov c6e99345cd
Validate Suspendability (#797)
And show "Suspend" menu item based on `--suspnedable` flag

Fixes #796
2024-04-22 10:07:01 +00:00
Nikolay Edigaryev 79084555f6
tart pull: retry VM pull with exponential backoff (#788) 2024-04-12 21:32:03 +04:00
Fedor Korotkov 896d03ce0b
Fixed Swift Warning (#787)
* Fixed Swift Warning

Plus updated all the dependencies and Swift Tools.

Fixes #785

* Fixed race condition
2024-04-11 19:53:10 +04:00
Fedor Korotkov 99c91cbf87
Allow mounting NBD disks (#786)
* Allow mounting NBD disks

Fixes #759

* Apply suggestions from code review

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Removed unnecessary docs

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2024-04-11 17:20:58 +04:00
Bartek Pacia da8afa1348
Add shell completions (#780)
* add VM completion for run command

* add VM completion for stop command

* create ShellCompletions utilities

* add shell completions to some commands

* add shell completion for fqn command

* run command: fix tiny typo

* add shell completion for get command

* more shell completions

* remove unnecessary `try`

* refactor ShellCompletions file
2024-04-11 06:22:50 -04:00
Nikolay Edigaryev 97b7ffef52
tart stop: throw RuntimeError.VMNotRunning consistently and use enumeration instead of strings (#784)
* Use enumeration instead of just strings for VMDirectory state

* tart stop: throw RuntimeError.VMNotRunning consistently
2024-04-10 14:53:04 +00:00
Nikolay Edigaryev b7b3b702ac
Sentry: upgrade and attach command-line arguments (#774)
* Sentry: upgrade and attach command-line arguments

* Sentry's setContext(): explicitly pass a String
2024-04-02 18:31:16 +04:00
Tor Arne Vestbø 560dba79e4
Report operating system in tart get (#772)
Can be useful to know from outside the VM.
2024-03-31 14:18:26 -04:00
Tor Arne Vestbø 2b33b8f9e6
Report progress when downloading IPSW files (#768)
The URLSession async/await functions do not report progress through
the normal URLSessionTaskDelegate callbacks, as reported in:

 https://developer.apple.com/forums/thread/723015

We don't want to use URLSession.bytes, as that results in a much
slower download speed compared to URLSession.download, but we can
work around the lack of progress callbacks by observing the
progress on the URLSessionTask itself.

Fixes #767
2024-03-28 19:15:06 +04:00
Tor Arne Vestbø d8b010c79c
Support cancellation of installation process (#770)
We wrap the installation with a withTaskCancellationHandler, which
ensures that the SIGINT signal handling code in main() will trigger
a cancellation of the installer.

As the VZMacOSInstaller must be both created and interacted with
on the VM's queue, which in our case is the main queue, we need
to move the logic to a separate function tagged with @MainActor.
This makes sense either way, as it cleans up the code a bit.
2024-03-28 00:14:33 +04:00
Fedor Korotkov ae2d59e5c2
Revert "Do not magically set --no-graphics when --vnc is passed (#763)
* Revert "Do not magically set `--no-graphics` when `--vnc` is passed (#732)"

This reverts commit a48f4d4ec9.

* Mark `--graphics` as private
2024-03-19 08:28:27 +00:00
Fedor Korotkov ac5d0baa0c
Fixed `--no-graphics` mode (#755)
Regression introduced in #746
2024-03-12 10:42:35 +00:00
Nikolay Edigaryev ee27cc57bb
tart run: introduce --net-softnet-allow command-line argument (#753) 2024-03-11 22:17:34 +04:00
Fedor Korotkov 89ff5f6b65
Respect `name` in case of a single directory mount (#750)
Fixes #748
2024-03-11 15:01:06 +00:00
Nikolay Edigaryev 6bf39e73d0
Prefer USB keyboard and screen coordinate pointing devices (#747) 2024-03-11 13:16:40 +00:00
Tor Arne Vestbø 0b693f6bc9
Improve macOS app integration (#746)
* Improve macOS app integration

Tart is now a proper application bundle, with the name and icon
declared in the Info.plist, which we were missing.

This also allows us to declare the app as LSBackgroundOnly
as a default, which means that 'tart create' and similar
background commands will not show the application icon in
the dock, while 'tart run' will, thanks to it overriding
the activation policy of the app.

For now the logic of creating the Tart.app bundle is duplicated
between the CI packaging scripts and the run-signed.sh script.
Now that these scripts are growing, it makes sense to look
at whether we can share the logic somehow, e.g. by building
the application bundle directly during build, and packaging
that, instead of creating it as a post install step.

* Integration tests: fix DockerContainer import

To work around the breaking change in 4.0.0,
see 383b12e9d6.

* .cirrus.yml(Release (Dry Run)): no need to install Sentry CLI

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2024-03-11 09:05:56 -04:00
Fedor Korotkov 99bbd838a1
Introduce `--net-host` flag to enforce host-only network (#743)
* Experimental `--net-host-only` option

* Use Softnet's host networking
2024-03-01 13:57:39 +04:00
Fedor Korotkov 5c7743b7cd
Show both size and actual size of files (#742)
Right now we show only actual size of files on disk which excludes empty blocks of the recently introduced sparced format in #671. This makes impossible to get info about disk size that we just set via `tart set`.

Here is an example of `tart list` output before the change:

```
Source Name                                                                                                            Size State
local  sonoma-base                                                                                                     22   stopped
local  sonoma-vanilla                                                                                                  18   stopped
local  sonoma-xcode                                                                                                    67   stopped
local  ubuntu                                                                                                          1    stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base:latest                                                                     22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base@sha256:16c1593bbaf787b20b3c0bc094c5b6baf71c937d22c2e4596da85ac55c92e6cc    22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla:14.3                                                                    17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla@sha256:23c4e853d48d00a4333346d66a32b2b5aad900cc0dc10e7ecb9dbe67b6f587f4 17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode:latest                                                                    67   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode@sha256:d0cb8d01424a68b89e0f16f5371bf2152b2c115bd886341a6ba8da42121d1f41   67   stopped
oci    ghcr.io/cirruslabs/ubuntu:22.04                                                                                 1    stopped
oci    ghcr.io/cirruslabs/ubuntu@sha256:037763feb7a15d6077edeb7a097738c34313637d16036764b4c196d28d8b429c               1    stopped
```

And here is the output after the change:

```
Source Name                                                                                                            Disk Size State
local  sonoma-base                                                                                                     50   22   stopped
local  sonoma-vanilla                                                                                                  50   18   stopped
local  sonoma-xcode                                                                                                    90   67   stopped
local  ubuntu                                                                                                          20   1    stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base:latest                                                                     50   22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base@sha256:16c1593bbaf787b20b3c0bc094c5b6baf71c937d22c2e4596da85ac55c92e6cc    50   22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla:14.3                                                                    50   17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla@sha256:23c4e853d48d00a4333346d66a32b2b5aad900cc0dc10e7ecb9dbe67b6f587f4 50   17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode:latest                                                                    90   67   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode@sha256:d0cb8d01424a68b89e0f16f5371bf2152b2c115bd886341a6ba8da42121d1f41   90   67   stopped
oci    ghcr.io/cirruslabs/ubuntu:22.04                                                                                 20   1    stopped
oci    ghcr.io/cirruslabs/ubuntu@sha256:037763feb7a15d6077edeb7a097738c34313637d16036764b4c196d28d8b429c               20   1    stopped
```

Additionally, `tart get` will print actual size with a 3 decimal point precision which will help to track growth in disk images for our templates.

`tart get` before:

```
CPU Memory Disk Display  State
4   8192   67   1024x768 stopped
```

`tart get` after:

```
CPU Memory Disk Size   Display  State
4   8192   90   67.333 1024x768 stopped
```
2024-02-27 20:13:38 +04:00
Nikolay Edigaryev e2d6c13ed0
DHCP MAC-address resolver: handle duplicate leases (#740) 2024-02-23 16:49:06 +04:00
Nikolay Edigaryev 3f17884ac2
Introduce "tart fqn" command (#735)
* tart pull: experimental --json-digest option

* Introduce "tart fqn" command

* Revert "tart pull: experimental --json-digest option"

This reverts commit 842052f5bd.
2024-02-19 21:13:45 +00:00
Fedor Korotkov 7dcebf9c04
Allow to override VirtioFS tag for shared directories (#733) 2024-02-19 20:51:25 +00:00