Otherwise we start to periodically get RST's from GitHub, possibly
because of too many connection opens, which has an effect of cancelling
previously received bytes.
These RST's can be observed in tcpdump/Wireshark or Console, emitted
from the libusrtcp.dylib library, com.apple.network subsystem, for the
Tart process:
>tcp_input [C59.1.1.1:3] flags=[R] seq=1805021659, ack=0, win=0 state=CLOSED rcv_nxt=1805021659, snd_una=1752355607
You can also observe the "Received Bytes" in "Activity Monitor" for
the Tart process while pulling ghcr.io/cirruslabs/macos-runner:sequoia,
and this value will periodically decrease.
* DiskV2: avoid allocating zero chunk on each zeroSkippingWrite() call
* Increase hole granularity size from 64 KiB to 4 MiB
* Fetcher: never write to disk, thanks to URLSessionDataDelegate
* Get a fresh instance of executor for each benchmark invocation
And don't pre-initialize all of the executors at once, as this
might reach the maximum number of VMs limit in case we want to
test multiple Tart executors.
* Run benchmarks on Tart with different --root-disk-opts options
* Fix TestTart
* benchmark fio: introduce --prepare command-line argument
To be able to specify --prepare='sudo purge && sync', similarly to
Hyperfine[1].
[1]: https://github.com/sharkdp/hyperfine
* Benchmark Tart with --root-disk-opts=caching=cached separately too
* Add Ars Technica recommended benchmarks
* Tart executor: log SSH session standard output and standard error
* Reduce file I/O size from 16 to 10 GB to avoid "No space left on device"
* Remove random writing tests to make space for more read/read-write tests
* Add some "randrw"-style fio benchmarks
* Show latency in benchmark results
* Add sync benchmark and show read/write/sync latency
* README.md: add new benchmark results
Was looking into performance and was wondering about Game Mode on Sonoma.
This change is unrelated. Just found they have a category for tools like Tart.
To make sure we won't prune then immediately after. Useful for when scenarios similar to Cirrus CLI when we make sure that several images are up-to-date before every request for task execution.
* Revert "Lowercase `tart.app` (#751)"
This reverts commit a9e2a19015.
* Optimize DiskV2 deduplication logic
In case we cloned `disk.img` from a local image, check if data at offset has the expected contents already.
* Hole punch only if needed
* Calculate hash only if needed
* subdataChunks optimization
* Reapply "Lowercase `tart.app` (#751)"
This reverts commit e74e9c845a.
* format
* Save at least 1GB on deduplication logic
* Build separately
* Revert "subdataChunks optimization"
This reverts commit e59382aeba.
* Another optimization
* Removed debug log
* reformat
* Revert "Hole punch only if needed"
This reverts commit 8c569fc5
* VZDiskImageSynchronizationMode's "description" field is a dead code
* Re-use the VZDiskImageSynchronizationMode extension
* tart run: replace --sync with --root-disk-opts
* VM: support root disk synchronization mode on macOS
* Support setting root disk synchronization mode
Adds a new VMConfig parameter (tart get / tart set) called 'sync' which
can be set to 'full' (default), 'fsync', or 'none', corresponding with
the values of VZDiskImageSynchronizationMode and allowing a tradeoff
between data integrity and speed.
* Remove unused import
* Fix formatting
* Make root disk sync behaviour a commandline option
* tart pull: try to re-use APFS blocks by cloning the base image
* Punch a hole when a zero chunk is detected
* Properly retrieve errno when hole punching operation fails
* tart pull: do not retry on RuntimeError
* Ensure that the holes we're about to punch are FS block size-aligned
* VMDirectory: remove unused static variables
* tart pull: log if we've found an image to deduplicate against
* Do not prematurely read contents from disk
* Only consider candidates with deduplicatedBytes more than 0
* APFS reuse UX/DX improvements (#870)
* Show how much deduplication happening
Improvement to the APFS deduplication logic which checks whether a disk image file `mayShareFileContent` with some other file, and then we put a custom attribute to track the deduplication since there is no way to get this information from APFS itself.
It's not 100% accurate but given that OCI cache is immutable the actual disk usage can only be lover than that.
* Use string attribute
* Update Sources/tart/URL+Prunable.swift
Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
* Added SizeOnDisk colume
---------
Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
---------
Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
Nikolay Edigaryev
Fedor Korotkov
Nicholas FitzRoy-Dale