Oyl
Faster YAML, less memory
Oyl is a YAML 1.2 library in C from Tab Computing. It's correct on the whole YAML Test Suite, 4 to 12 times faster than libyaml, and hardened against hostile input. We help teams put it to work wherever YAML costs them time and memory.
Case study: Helm repository indexes
Helm reads a chart repository's whole index.yaml when it adds the repository and when it pulls a chart. Big indexes make that expensive. A 65 MB index ran Helm out of memory at 2 GB (helm#9931). One Argo CD operator measured about 900 MB per chart pull from a 40 MB index, with up to 30 running at a time (argo-cd#23402). Flux's source controller has been killed for running out of memory at 1 GiB (flux2#4635). And when Argo CD fetches a chart with helm pull --repo, Helm parses the index twice: once to check it, then again from a temporary copy.
We rebuilt Helm with Oyl reading the index. The output is the same; the cost is a fraction:
| Command | Helm 4.3.1 | With Oyl |
|---|---|---|
helm repo add | 2.81 s · 1,368 MiB | 0.82 s · 556 MiB |
helm pull | 2.76 s · 1,043 MiB | 0.75 s · 369 MiB |
helm pull --repo, as Argo CD runs it | 5.73 s · 1,446 MiB | 1.54 s · 592 MiB |
| Command | Helm 4.3.1 | With Oyl |
|---|---|---|
helm repo add | 1.13 s · 635 MiB | 0.36 s · 272 MiB |
helm pull | 1.14 s · 554 MiB | 0.33 s · 194 MiB |
helm pull --repo, as Argo CD runs it | 2.26 s · 625 MiB | 0.66 s · 328 MiB |
Helm 4.3.1 built from source twice, the second time with the YAML step of its index loader replaced by Oyl's YAMLToJSONStrict, followed by the same strict JSON decode. Indexes served from a local web server; time and peak memory per process, median of three runs. Both builds print identical helm search repo -l -o json output, and Helm's repository tests pass with Oyl. The 64.5 MB index was merged with Helm's own code from 19 public indexes, to match the size in helm#9931.
What Oyl is
- Correct. It produces the expected output for all 308 valid cases in the YAML Test Suite and rejects all 94 invalid ones.
- Fast. It parses real-world files 4 to 12 times faster than libyaml, the C library most languages' YAML support is built on, and beats libfyaml and rapidyaml on all 12 inputs in its benchmark, with all four measured as event parsers.
- Compatible. Its YAML-to-JSON output matches Kubernetes' own converter,
sigs.k8s.io/yaml, byte for byte, about seven times faster. - Hardened. A fuzzer has thrown more than 13 billion inputs at it under AddressSanitizer and UndefinedBehaviorSanitizer. Its hash tables are keyed with a secret, so crafted input can't flood them.
- Flexible. It reads YAML 1.1 and go-yaml's dialect as well as 1.2, and can rewrite a YAML 1.1 file in place so it means the same thing under 1.2, keeping every comment.
- Shipped. Debian, Arch and RPM packages, a Rust crate, a Go module and a Crystal shard, tested on Linux, macOS and Windows. Open source under the MIT license.
How we can help
Speed up your YAML path
We put Oyl behind the YAML step that's costing you, in Go, Rust, Crystal or C, and check its output byte for byte against what you use today.
Helm and Argo CD with big indexes
A Helm build with Oyl's index loader, a drop-in for Argo CD repo servers and CI runners that read large chart repositories.
A YAML performance review
We find where parsing and converting YAML cost you time and memory, down to the CPU's counters, and fix what we find, sometimes without Oyl at all.
Support and custom work
Priority fixes and upgrades for Oyl in production, and new bindings, dialects or schemas when you need them.
Who's behind it
Oyl is built by Thomas Sawyer at Tab Computing. Thomas has worked with YAML since 2002, when he joined the yaml-core mailing list while the spec was still a draft, and he took part in the design discussions for YAML 1.1 and 1.2. He has maintained Ruby Facets since 2004. He built Oyl with AI as a coding partner, and wrote about it in A Quarter Inch.
Tell us what you're parsing
What reads your YAML, how big does it get, and what does it cost you now? Thomas reads every message and replies by email, usually within two business days. Or write to contact@tabcomputing.com.