OCaml
@ocaml.org
1.2K followers 15 following 410 posts
https://ocaml.org
Posts Media Videos Starter Packs
ocaml.org
OCaml Weekly News
Simon Cruanes announced Hello camels, I’m happy to announce the release of moonpool 0.9 and moonpool-lwt 0.9. Moonpool is a concurrency and parallelism library that provides lightweight fibers and a concept of Runner.t that they can be dispatched on. Multiple runners can co-exist inside a program. This release is a fairly large one. First, Moonpool now requires OCaml >= 5.0 (no more 4.xx compat), which removes the need for a preprocessor and makes await generally available on every Runner.t. Some sub-libraries are now deprecated (moonpool-io, moonpool.sync in favor of picos, etc.). The biggest improvement is moonpool-lwt. It now wraps Lwt_main.run and turns it into a Moonpool.Runner.t, meaning that Lwt, Lwt_io, Lwt_unix, and all the libraries built on top can now directly be used from Moonpool. Lwt promises can be turned into moonpool futures and conversely; fibers can be spawned in the Lwt_engine loop from any thread (to perform IO and call lwt libraries) and be awaited from other threads, too. Documentation: https://c-cube.github.io/moonpool/moonpool/index.html , https://c-cube.github.io/moonpool/moonpool-lwt/Moonpool_lwt/index.html Example echo server module M_lwt = Moonpool_lwt let ( let@ ) = ( @@ ) let str_of_sockaddr = function | Unix.ADDR_UNIX s -> s | Unix.ADDR_INET (addr, port) -> Printf.sprintf "%s:%d" (Unix.string_of_inet_addr addr) port let main ~port () : unit = (* never resolved *) let lwt_fut, _lwt_prom = Lwt.wait () in let handle_client client_addr (ic, oc) : _ Lwt.t = (* spawn a new fiber in the lwt thread *) let@ () = M_lwt.spawn_lwt in Printf.printf "got new client on %s\n%!" (str_of_sockaddr client_addr); let buf = Bytes.create 1024 in let continue = ref true in while !continue do let n = Lwt_io.read_into ic buf 0 (Bytes.length buf) |> M_lwt.await_lwt in if n = 0 then continue := false else ( Lwt_io.write_from_exactly oc buf 0 n |> M_lwt.await_lwt; Lwt_io.flush oc |> M_lwt.await_lwt; ) done; Printf.printf "done with client on %s\n%!" (str_of_sockaddr client_addr); in Printf.printf "listening on port=%d\n%!" port; let addr = Unix.ADDR_INET (Unix.inet_addr_any, port) in let _server = Lwt_io.establish_server_with_client_address addr handle_client |> M_lwt.await_lwt in M_lwt.await_lwt lwt_fut (* never returns *) let () = let port = ref 1234 in let opts = [ "-p", Arg.Set_int port, " port"; ] |> Arg.align in Arg.parse opts ignore "echo server"; M_lwt.lwt_main @@ fun _ -> main ~port:!port () Run it as echo_server -p 1234 and use nc localhost 1234 to connect. It will echo lines sent to it. We can reuse Lwt_io.establish_server_with_client_address just fine, and use direct style to implement the client handler inside a single Moonpool fiber (via Moonpool_lwt.spawn_lwt that runs its argument in the lwt event loop). Small server with a thread pool for compute a variation on the previous one, with a thread pool on which CPU bound tasks can be run: module M_lwt = Moonpool_lwt let ( let@ ) = ( @@ ) let str_of_sockaddr = function | Unix.ADDR_UNIX s -> s | Unix.ADDR_INET (addr, port) -> Printf.sprintf "%s:%d" (Unix.string_of_inet_addr addr) port (* don't do this at home *) let rec fib n = if n <= 2 then 1 else fib (n-1) + fib (n-2) let main ~port ~tpool () : unit = (* never resolved *) let lwt_fut, _lwt_prom = Lwt.wait () in let handle_client client_addr (ic, oc) : _ Lwt.t = (* spawn a new fiber in the lwt thread *) let@ () = M_lwt.spawn_lwt in Printf.printf "got new client on %s\n%!" (str_of_sockaddr client_addr); let continue = ref true in while !continue do match Lwt_io.read_line ic |> M_lwt.await_lwt with | exception End_of_file -> continue := false | line -> let input = int_of_string @@ String.trim line in (* run fib(input) in the thread pool and suspend until it's done *) let fib_input = Moonpool.Fut.spawn ~on:tpool (fun () -> fib input) |> Moonpool.Fut.await in Lwt_io.write oc (Printf.sprintf "%d\n" fib_input) |> M_lwt.await_lwt; Lwt_io.flush oc |> M_lwt.await_lwt; done; Printf.printf "done with client on %s\n%!" (str_of_sockaddr client_addr); in Printf.printf "listening on port=%d\n%!" port; let addr = Unix.ADDR_INET (Unix.inet_addr_any, port) in let _server = Lwt_io.establish_server_with_client_address addr handle_client |> M_lwt.await_lwt in M_lwt.await_lwt lwt_fut (* never returns *) let () = let port = ref 1234 in let j = ref 8 in let opts = [ "-j", Arg.Set_int j, " thread pool size"; "-p", Arg.Set_int port, " port"; ] |> Arg.align in Arg.parse opts ignore "echo server"; let@ tpool = Moonpool.Ws_pool.with_ ~num_threads:!j () in M_lwt.lwt_main @@ fun _ -> main ~port:!port ~tpool () Note how the computation is done by starting a task in the tpool argument (a moonpool Runner.t provided to the main, by default a work stealing pool of 8 threads that can be set via -j <number of threads>) and then await-ed from the lwt handler. While the computation is running, the lwt client handler is suspended and doesn’t prevent other clients from making progress. To test this one, use nc localhost 1234 and write (small) integers to get fib(n) computed. To see it work in parallel, open top or htop and run: for i in `seq 1 200`; do nc localhost 1234 <<< '35' & done First release candidate for OCaml 5.4.0 Archive: https://discuss.ocaml.org/t/first-release-candidate-for-ocaml-5-4-0/17338/1 octachron announced The release of OCaml 5.4.0 is imminent. As a final step, we are publishing a release candidate to check that everything is in order before the release in the upcoming week. If you find any bugs, please report them on the OCaml's issue tracker. Compared to the second beta, this release candidate only contains a fix in the TSAN mode, and one metadata fix in the changelog itself. The full change log for OCaml 5.4.0 is available on GitHub. Happy hacking, Florian Angeletti for the OCaml team. Fine-tuned compiler configuration If you want to tweak the configuration of the compiler, you can switch to the option variant with: opam update opam switch create <switch_name> ocaml-variants.5.4.0~rc1+options <option_list> where <option_list> is a space-separated list of ocaml-option-* packages. For instance, for a flambda and no-flat-float-array switch: opam switch create 5.4.0~rc1+flambda+nffa ocaml-variants.5.4.0~rc1+options ocaml-option-flambda ocaml-option-no-flat-float-array All available options can be listed with opam search ocaml-option. Announcing the OCaml Zulip at ocaml.zulipchat.com Archive: https://discuss.ocaml.org/t/announcing-the-ocaml-zulip-at-ocaml-zulipchat-com/17339/1 ancolie announced Dear OCaml community, There has been a recent renewed interest in maintaining an open, organized and synchronous communication channel, and the OCaml Zulip has been revived. It is freely readable without an account at ocaml.zulipchat.org, and can be accessed through various means of authentication, including Github accounts. On Zulip, we have full access to our data at all time, and should the company change its policy, the data can be retrieved and the current version of Zulip server is self-hostable. In the meantime, we have been graciously offered sponsorship as an open community and can enjoy all features of the platform for free, and we thank Zulip for that. The platform can be accessed either on the web (one tab per server), or on the desktop and mobile client, which allow for managing multiple organizations. Talking about multiple organizations, there are already many OCaml, programming languages and verification related Zulip servers, such as Rocq, Types, Why3, Catala, Bytecode alliance, Owi or Aeneas. Check-out the full list of open to the public communities for more. Finally, we would like to emphasize that any governance team or project is welcome to host their discussions on the Zulip, where a channel can be created and admin rights granted. Cheers! PS: For anyone already on the Zulip, as part of this effort, the URL was migrated from caml.zulipchat.org to ocaml.zulipchat.org and you may have to remove the server and login again. An impressive macrobenchmark for eio Archive: https://discuss.ocaml.org/t/an-impressive-macrobenchmark-for-eio/17344/1 conroj said While wandering around the web I came across a link to a slide deck by our own @kayceesrk. On slide #35 is a macrobenchmark showing an EIO-based network server, and its throughput is pretty favorable compared to the Rust implementation’s. Taking this at face value, it seems like quite an achievement - not only because GC is (supposedly) a handicap for OCaml, but also because this seems like a major improvement over a similar benchmark from 2022. I couldn’t find links to deeper discussion of these results, so I thought I would ask some of the obvious follow-up questions: Is OCaml’s tail latency on par with Rust’s in these scenarios? Are both the “OCaml eio” and “Rust Hyper” results using similar kernel capabilities, e.g. io_uring? (The slide seems to suggest so, but just confirming.) Do these results generalize to different levels of concurrency, request/response sizes, etc? Either way, kudos for raising OCaml’s profile as a platform for scalable computing! Anil Madhavapeddy replied They both used io_uring, yes. The OCaml bindings are at https://github.com/ocaml-multicore/ocaml-uring ; but note that there are several levels of io_uring usage possible depending on your tolerance for ranges of Linux kernel support (I’m just adding zero-copy transmit support at the moment for a project involving a petabyte of data). OCaml’s tail latency will be worse than Rust’s due to having a GC, but not terribly so. As for generalization, those tests were run on a pre-5.0 version of OCaml, so the whole test suite would have to be rebased against the released versions. A good and useful exercise if someone would like to have a go at it! Thomas Leonard also replied I think the benchmarks are from https://github.com/ocaml-multicore/retro-httpaf-bench I don’t think the Rust ones are using uring, but I’m not sure. I suspect that all the non-OCaml ones could do with a fan of that language optimising them a bit. In my experience, whether the Rust or Eio version gets better throughput depends on e.g. the number of connections, and tail latency was always better with Rust. But the basic result is that OCaml is competitive with Rust here. nim-ocaml Archive: https://discuss.ocaml.org/t/ann-nim-ocaml/17346/1 Florent Monnier announced I read a paper maybe not this one, but it seems it talks about the same thing https://arxiv.org/html/2506.04657v1. At the beginning I thought it's about the new programming language, but then chatgpt explained me that in fact it's a small game with stones. We put a given number of stones in the middle, and each player can take 1, 2 or 3 stones from the stack. There are two variants of the game, the one that only has one stone in front of him at the end wins or not. So I tryed to make a nim-ocaml to play against its Random.state, here below: let () = Random.self_init () ; let n = 13 + (Random.int 23) in let _n = ref n in let run = ref true in while !run do Printf.printf "%d\n" !_n; if !_n <= 1 then run := false ; let line = read_line () in begin try let d = int_of_string line in _n := !_n - d ; with _ -> Printf.printf "please input an integer number\n%!"; end; let b = Random.int 2 + Random.int 3 in Printf.printf "b played: %d\n" b; _n := !_n - b ; done; Printf.printf "done!\n" ; Or later: $ wget http://decapode314.free.fr/ocaml2/nim/nim.ml $ \ocaml nim.ml 23 3 b played: 3 17 7 b played: 0 10 3 b played: 2 5 2 b played: 2 1 0 b played: 0 done! Another version to play against your collegue at the pause: $ wget http://decapode314.free.fr/ocaml2/nim/.gil/nim.ml.0 Call for Contributions: BOB 2026 (Berlin, March 13 - Deadline Nov 17) Archive: https://discuss.ocaml.org/t/call-for-contributions-bob-2026-berlin-march-13-deadline-nov-17/17348/1 Michael Sperber announced OCaml contributions are spot-on for BOB - send us some! BOB Conference 2026 - Call for Contributions Looking for Speakers You are actively engaged in advanced software engineering methods, solve ambitious problem with software and are open to cutting-edge innovation? Attend this conference, meet people that share your goals, and get to know the best software tools and technologies available today. We strive to offer a day full of new experiences and impressions that you can use to immediately improve your daily life as a software developer. If you share our vision and want to contribute, submit a proposal for a talk or tutorial! NOTE: The conference fee will be waived for presenters. Travel expenses will not be covered (for exceptions see “Speaker Grants”). We are looking for talks about best-of-breed software technology, e.g.: functional programming persistent data structures and databases event-based modelling and architecture “fancy types” (dependent types, gradual typing, linear types, …) formal methods for correctness and robustness abstractions for concurrency and parallelism metaprogramming probabilistic programming math and programming controlled side effects program synthesis AI beyond vibecoding and chatbots linked data symbolic AI next-generation IDEs effective abstractions for data analytics … everything really that isn’t mainstream, but you think should be … including rough ideas that are worth discussing. Presenters should provide the audience with information that is practically useful for software developers. Challenges Furthermore, we seek contributions on successful approaches for solving hard problems, for example: bias in machine-learning systems digital transformation in difficult settings accessibility systems with critical reliability requirements ecologically sustainable software development We’re especially interested in experience reports. Other topics are also relevant, e.g.: introductory talks on technical background overviews of a given field demos and how-tos Organisation Direct questions to konferenz at bobkonf dot de Proposal deadline: November 17, 2025 Notification: December 5, 2025 Program: December 12, 2025 Speaker Grants BOB has Speaker Grants available to support speakers from groups under-represented in technology. We specifically seek women speakers, speakers of color, and speakers who are not able to attend the conference for financial reasons. ocp-indent 1.9.0 Archive: https://discuss.ocaml.org/t/ann-ocp-indent-1-9-0/17349/1 Nathan Rebours announced Here at OCamlPro we’re happy to announce the (long awaited) release of ocp-indent.1.9.0. The full release notes are available here if you want the detailed version. 1.9.0 contains mostly bug fixes, better and more consistent indentation of fun _ -> and |>, compatibility with cmdliner.1.3.0 and above (it works with 2.0.0) and a new utility tool: ocp-indent-gen-rules for those of you who would like to try ocp-indent in a dune fmt like workflow. This last bit is documented here. This is a feature that some of us wanted internally at OCamlPro so we decided to ship it with the tool as an experiment. We’d really like to hear if this fits your ocp-indent usage so please don’t hesitate to try it out and give us some feedback. We’re also interested in hearing how you use ocp-indent in general and what you expect from it. Reach out if you have any request! We’ve also updated the repo to fit the more recent development standards. We migrated the test suite to dune cram tests and re-enabled them in opam. Hopefully this should make contributing to ocp-indent a smoother experience! Also be aware that we’ll do our best to maintain ocp-indent more actively from now on. We’d like to thank our external contributors for this release: @dbuenzli, @nojb, @bcc32 and @Julow. Happy indenting! Sketch.sh now supports OCaml 5.3.0 Archive: https://discuss.ocaml.org/t/ann-sketch-sh-now-supports-ocaml-5-3-0/17352/1 Javier Chávarri announced The interactive OCaml sketchbook sketch.sh has added support for OCaml 5.3.0. Support for 5.3.0 Storing and running sketches using the compiler version 5.3.0 is now possible, this functionality has been added to the already existing support for versions 4.06.1 and 4.13.1. This new version brings support for OCaml 5’s effect handlers and multicore capabilities. Since sketch.sh runs in the browser using JavaScript via js_of_ocaml, the multicore capabilities are simulated using continuation-passing style. Here you can see a sketch showcasing effects: Effects Example - Sketch.sh. While support for intermediate versions is technically possible, it will require adding a mechanism to support choosing the version of the compiler for the current sketch (see issue #375). Existing sketches and forks Previously existing sketches remain in their original compiler version, while newly created sketches will be on 5.3.0 by default. For now, the only way to "migrate" a sketch to a newer version of the compiler is by copying its content and pasting it in a new sketch. Forked sketches inherit the compiler version of the upstream sketch. Reporting features and issues Please let us know in case you have a feature request, or if you encounter any issues or bugs. Also, don't hesitate to reach out via Reason Discord or Discuss DMs if you would like to contribute or participate in the project in some way. There are a lot of opportunities to do so, both on the frontend and backend sides. OUPS meetup october 2025 Archive: https://discuss.ocaml.org/t/oups-meetup-october-2025/17353/1 ancolie announced The next OUPS meetup will take place on Monday, 13th of October 2025. It will start at 6:30pm at the 4 place Jussieu in Paris. It will be in the in the Esclangon building (amphi Astier). Please, register on meetup as soon as possible to let us know how many pizza we should order. For more details, you may check the OUPS’ website . Moreover, we'd like to announce that the organizing team moved to the OCaml Zulip. Feel free to contact us there if you'd like to suggest talks. This time we’ll have the following talks: What's the deal with modular implicits ? – Samuel Vivien Modular implicits est une extension d'OCaml présentée en 2014 comme une solution à l'absence de type classe en OCaml. Cependant malgré l'ancienneté de cette proposition cette fonctionnalité n'est toujours pas disponible dans OCaml. Nous ferons un tour d'horizon de modular implicits pour rappeler comment cette fonctionnalité marche, ce qui as déjà été implémenté dans le compilateur mais aussi ce qu'il reste à faire ainsi que les problématiques liés au typage des implicites. Flambda2: Abstractions without Cost – Guillaume Bury Surprise. After the talks there will be some pizzas offered by the OCaml Software Foundation and later on we’ll move to a pub nearby as usual. New releases of Merlin (5.6) and OCaml-LSP (1.24.0) Archive: https://discuss.ocaml.org/t/ann-new-releases-of-merlin-5-6-and-ocaml-lsp-1-24-0/17354/1 Xavier Van de Woestyne announced We are pleased to announce new releases of Merlin (5.6-504 and 5.6-503) and OCaml-LSP (1.24.0, for 5.4, and 1.23.1)! This release of Merlin offers, firstly, support for OCaml 5.4. It improves support for OpenBSD (for merlin-reader), improves typing recovery in the handling of mutual recursion, and adds a new feature to the protocol: locate-types. It works similarly to locate-type, except that it allows you to distinguish between several locatable types in an expression like this: (int, Foo.t) result enabling the location of: int, Foo.t and ('a, 'b) result. In addition, the Vim client has been fixed for the use of project-wide-occurrences. The release of OCaml LSP also mainly concerns support for 5.4 and several bug fixes. As with every version upgrade, we are eager to hear user feedback. Try out these new releases on your 5.4 switches and don't hesitate to report any issues you encounter (Merlin, OCaml LSP)! Merlin Changelog Merlin 5.6-504 (& 5.6-503) merlin binary Add locate-types command (#1951) merlin library Fix merlin_reader for OpenBSD (#1956) Improve recovery of mutually recursive definitions (#1962, #1963, fixes #1953) Support for OCaml 5.4 (#1974) (only for 5.6-504) vim plugin Fix error when :MerlinOccurrencesProjectWide fails to gather code previews (#1970) test suite Add more short-paths tests cases (#1904) Other OCaml News
dlvr.it
Reposted by OCaml
sabine.sh
Hey everyone, we're down to 7 PRs.

In particular, that means all of the cookbook PRs waiting for review were addressed and we're now ready to take more cookbook contributions! 🧡🐫

github.com/ocaml/ocaml....
github.com
ocaml.org
Backstage OCaml: You Can Try the Experimental Branch of Merlin That Uses Domains and Effects
The Merlin team is excited to share that you can now try out an experimental branch of Merlin that leverages OCaml 5's domains and effects! This is Merlin-domains, and we'd love for you to test it and share your feedback. What is Merlin-domains? Merlin-domains is an experimental branch that uses domains and effects to implement two optimisations to improve performance in large buffers: partial typing and cancellation. As a reminder, Merlin is the editor service that powers OCaml's IDE features—if you're using the OCaml Platform extension with VS Code or ocaml-eglot with Emacs, you're already using Merlin under the hood through OCaml LSP Server. Why This Matters While Merlin has had relatively few performance complaints over the years, in some contexts like very large files, the parsing-typing-analysis mechanism could sometimes cause slowdowns. The experimental branch addresses this in a clever way. When you run an analysis command on a very large file, the type-checker will progress up to the location that makes the analysis possible, run the analysis phase, return the result, and then continue typing the file. This separation is made possible through control flow management enabled by effects, with two domains interacting with each other. The result? Analysis phases become much more efficient! This is a great example of migrating a regular OCaml application to take advantage of multicore. Learn More at Lambda World Want to understand the technical details? Sonja Heinze and Carine Morel will present their talk "When magic meets multicore - OCaml and its elegant era of parallelism" at Lambda World, where they'll dive into how this experimental branch works internally. How to Test It Currently, the branch is in its incubation phase. To test it, pin the branch in the switches where you want to experiment: opam pin add https://github.com/ocaml/merlin#merlin-domains Although this experimental branch passes the test suite, your feedback is very important to help collect potential bugs we may have missed. The team has added a Bug/Merlin-domains label to organize tickets related to this branch. What's Next The goal is for this branch to eventually become the main branch, so that all users can benefit from these improvements. The rest of the ecosystem depending on Merlin, including OCaml LSP Server, will be adapted to take full advantage of these new features. We need you! Try out merlin-domains with your real-world OCaml projects and share your experience on the Discuss thread. Your testing and feedback will help shape the future of Merlin!
dlvr.it
ocaml.org
Backstage OCaml: You Can Try the Experimental Branch of Merlin That Uses Domains and Effects
The Merlin team is excited to share that you can now try out an experimental branch of Merlin that leverages OCaml 5's domains and effects! This is Merlin-domains, and we'd love for you to test it and share your feedback. What is Merlin-domains? Merlin-domains is an experimental branch that uses domains and effects to implement two optimisations to improve performance in large buffers: partial typing and cancellation. As a reminder, Merlin is the editor service that powers OCaml's IDE features—if you're using the OCaml Platform extension with VS Code or ocaml-eglot with Emacs, you're already using Merlin under the hood through OCaml LSP Server. Why This Matters While Merlin has had relatively few performance complaints over the years, in some contexts like very large files, the parsing-typing-analysis mechanism could sometimes cause slowdowns. The experimental branch addresses this in a clever way. When you run an analysis command on a very large file, the type-checker will progress up to the location that makes the analysis possible, run the analysis phase, return the result, and then continue typing the file. This separation is made possible through control flow management enabled by effects, with two domains interacting with each other. The result? Analysis phases become much more efficient! This is a great example of migrating a regular OCaml application to take advantage of multicore. Learn More at Lambda World Want to understand the technical details? Sonja Heinze and Carine Morel will present their talk "When magic meets multicore - OCaml and its elegant era of parallelism" at Lambda World, where they'll dive into how this experimental branch works internally. How to Test It Currently, the branch is in its incubation phase. To test it, pin the branch in the switches where you want to experiment: opam pin add https://github.com/ocaml/merlin#merlin-domains Although this experimental branch passes the test suite, your feedback is very important to help collect potential bugs we may have missed. The team has added a Bug/Merlin-domains label to organize tickets related to this branch. What's Next The goal is for this branch to eventually become the main branch, so that all users can benefit from these improvements. The rest of the ecosystem depending on Merlin, including OCaml LSP Server, will be adapted to take full advantage of these new features. We need you! Try out merlin-domains with your real-world OCaml projects and share your experience on the Discuss thread. Your testing and feedback will help shape the future of Merlin!
dlvr.it
ocaml.org
OCaml @ocaml.org · 11d
A second foray into agentic coding
Continuing the previous theme of dabbling with matters agentic. Previously, I’d quite assiduously kept my fingers away from files. This time, I wanted to try something exploratory, switching to the agent for things I was actively stuck on. I was still (very) curious at the latent remaining bug in Lucas’s excellent work. There were some corners which had been cut in the prototype, and I had a brief foray into this problem, with a view this time to ensuring artefact equivalence between what OCaml’s build system would produce and what our altered driver program was doing. If you have a pre-built compiler and a clean (of binary artefacts) OCaml source tree, you can actually build the bytecode compiler in just three, ahem, short commands (I’m intentionally glossing over all the generated source files): $ ocamlc -I utils -I parsing -I typing -I bytecomp -I file_formats -I lambda -I middle_end -I middle_end/closure -I middle_end/flambda -I middle_end/flambda/base_types -I driver -I runtime -g -strict-sequence -principal -absname -w +a-4-9-40-41-42-44-45-48 -warn-error +a -bin-annot -strict-formats -linkall -a -o compilerlibs/ocamlcommon.cma utils/config.mli utils/build_path_prefix_map.mli utils/format_doc.mli utils/misc.mli utils/identifiable.mli utils/numbers.mli utils/arg_helper.mli utils/local_store.mli utils/load_path.mli utils/profile.mli utils/clflags.mli utils/terminfo.mli utils/ccomp.mli utils/warnings.mli utils/consistbl.mli utils/linkdeps.mli utils/strongly_connected_components.mli utils/targetint.mli utils/int_replace_polymorphic_compare.mli utils/domainstate.mli utils/binutils.mli utils/lazy_backtrack.mli utils/diffing.mli utils/diffing_with_keys.mli utils/compression.mli parsing/location.mli parsing/unit_info.mli parsing/asttypes.mli parsing/longident.mli parsing/parsetree.mli parsing/docstrings.mli parsing/syntaxerr.mli parsing/ast_helper.mli parsing/ast_iterator.mli parsing/builtin_attributes.mli parsing/camlinternalMenhirLib.mli parsing/parser.mli parsing/pprintast.mli parsing/parse.mli parsing/printast.mli parsing/ast_mapper.mli parsing/attr_helper.mli parsing/ast_invariants.mli parsing/depend.mli typing/annot.mli typing/value_rec_types.mli typing/ident.mli typing/path.mli typing/type_immediacy.mli typing/outcometree.mli typing/primitive.mli typing/shape.mli typing/types.mli typing/data_types.mli typing/rawprinttyp.mli typing/gprinttyp.mli typing/btype.mli typing/oprint.mli typing/subst.mli typing/predef.mli typing/datarepr.mli file_formats/cmi_format.mli typing/persistent_env.mli typing/env.mli typing/errortrace.mli typing/typedtree.mli typing/signature_group.mli typing/printtyped.mli typing/ctype.mli typing/out_type.mli typing/printtyp.mli typing/errortrace_report.mli typing/includeclass.mli typing/mtype.mli typing/envaux.mli typing/includecore.mli typing/tast_iterator.mli typing/tast_mapper.mli typing/stypes.mli typing/shape_reduce.mli file_formats/cmt_format.mli typing/cmt2annot.mli typing/untypeast.mli typing/includemod.mli typing/includemod_errorprinter.mli typing/typetexp.mli typing/printpat.mli typing/patterns.mli typing/parmatch.mli typing/typedecl_properties.mli typing/typedecl_variance.mli typing/typedecl_unboxed.mli typing/typedecl_immediacy.mli typing/typedecl_separability.mli lambda/debuginfo.mli lambda/lambda.mli typing/typeopt.mli typing/typedecl.mli typing/value_rec_check.mli typing/typecore.mli typing/typeclass.mli typing/typemod.mli lambda/printlambda.mli lambda/switch.mli lambda/matching.mli lambda/value_rec_compiler.mli lambda/translobj.mli lambda/translattribute.mli lambda/translprim.mli lambda/translcore.mli lambda/translclass.mli lambda/translmod.mli lambda/tmc.mli lambda/simplif.mli lambda/runtimedef.mli file_formats/cmo_format.mli middle_end/internal_variable_names.mli middle_end/linkage_name.mli middle_end/compilation_unit.mli middle_end/variable.mli middle_end/flambda/base_types/closure_element.mli middle_end/flambda/base_types/var_within_closure.mli middle_end/flambda/base_types/tag.mli middle_end/symbol.mli middle_end/flambda/base_types/set_of_closures_id.mli middle_end/flambda/base_types/set_of_closures_origin.mli middle_end/flambda/parameter.mli middle_end/flambda/base_types/static_exception.mli middle_end/flambda/base_types/mutable_variable.mli middle_end/flambda/base_types/closure_id.mli middle_end/flambda/projection.mli middle_end/flambda/base_types/closure_origin.mli middle_end/clambda_primitives.mli middle_end/flambda/allocated_const.mli middle_end/flambda/flambda.mli middle_end/flambda/freshening.mli middle_end/flambda/base_types/export_id.mli middle_end/flambda/simple_value_approx.mli middle_end/flambda/export_info.mli middle_end/backend_var.mli middle_end/clambda.mli file_formats/cmx_format.mli file_formats/cmxs_format.mli bytecomp/instruct.mli bytecomp/meta.mli bytecomp/opcodes.mli bytecomp/bytesections.mli bytecomp/dll.mli bytecomp/symtable.mli driver/pparse.mli driver/compenv.mli driver/main_args.mli driver/compmisc.mli driver/makedepend.mli driver/compile_common.mli utils/config.ml utils/build_path_prefix_map.ml utils/format_doc.ml utils/misc.ml utils/identifiable.ml utils/numbers.ml utils/arg_helper.ml utils/local_store.ml utils/load_path.ml utils/clflags.ml utils/profile.ml utils/terminfo.ml utils/ccomp.ml utils/warnings.ml utils/consistbl.ml utils/linkdeps.ml utils/strongly_connected_components.ml utils/targetint.ml utils/int_replace_polymorphic_compare.ml utils/domainstate.ml utils/binutils.ml utils/lazy_backtrack.ml utils/diffing.ml utils/diffing_with_keys.ml utils/compression.ml parsing/location.ml parsing/unit_info.ml parsing/asttypes.ml parsing/longident.ml parsing/docstrings.ml parsing/syntaxerr.ml parsing/ast_helper.ml parsing/ast_iterator.ml parsing/builtin_attributes.ml parsing/camlinternalMenhirLib.ml parsing/parser.ml parsing/lexer.mli parsing/lexer.ml parsing/pprintast.ml parsing/parse.ml parsing/printast.ml parsing/ast_mapper.ml parsing/attr_helper.ml parsing/ast_invariants.ml parsing/depend.ml typing/ident.ml typing/path.ml typing/primitive.ml typing/type_immediacy.ml typing/shape.ml typing/types.ml typing/data_types.ml typing/rawprinttyp.ml typing/gprinttyp.ml typing/btype.ml typing/oprint.ml typing/subst.ml typing/predef.ml typing/datarepr.ml file_formats/cmi_format.ml typing/persistent_env.ml typing/env.ml typing/errortrace.ml typing/typedtree.ml typing/signature_group.ml typing/printtyped.ml typing/ctype.ml typing/out_type.ml typing/printtyp.ml typing/errortrace_report.ml typing/includeclass.ml typing/mtype.ml typing/envaux.ml typing/includecore.ml typing/tast_iterator.ml typing/tast_mapper.ml typing/stypes.ml typing/shape_reduce.ml file_formats/cmt_format.ml typing/cmt2annot.ml typing/untypeast.ml typing/includemod.ml typing/includemod_errorprinter.ml typing/typetexp.ml typing/printpat.ml typing/patterns.ml typing/parmatch.ml typing/typedecl_properties.ml typing/typedecl_variance.ml typing/typedecl_unboxed.ml typing/typedecl_immediacy.ml typing/typedecl_separability.ml typing/typeopt.ml typing/typedecl.ml typing/value_rec_check.ml typing/typecore.ml typing/typeclass.ml typing/typemod.ml lambda/debuginfo.ml lambda/lambda.ml lambda/printlambda.ml lambda/switch.ml lambda/matching.ml lambda/value_rec_compiler.ml lambda/translobj.ml lambda/translattribute.ml lambda/translprim.ml lambda/translcore.ml lambda/translclass.ml lambda/translmod.ml lambda/tmc.ml lambda/simplif.ml lambda/runtimedef.ml bytecomp/meta.ml bytecomp/opcodes.ml bytecomp/bytesections.ml bytecomp/dll.ml bytecomp/symtable.ml driver/pparse.ml driver/compenv.ml driver/main_args.ml driver/compmisc.ml driver/makedepend.ml driver/compile_common.ml $ ocamlc -I utils -I parsing -I typing -I bytecomp -I file_formats -I lambda -I middle_end -I middle_end/closure -I middle_end/flambda -I middle_end/flambda/base_types -I driver -I runtime -g -strict-sequence -principal -absname -w +a-4-9-40-41-42-44-45-48 -warn-error +a -bin-annot -strict-formats -a -o compilerlibs/ocamlbytecomp.cma bytecomp/bytegen.mli bytecomp/printinstr.mli bytecomp/emitcode.mli bytecomp/bytelink.mli bytecomp/bytelibrarian.mli bytecomp/bytepackager.mli driver/errors.mli driver/compile.mli driver/maindriver.mli bytecomp/instruct.ml bytecomp/bytegen.ml bytecomp/printinstr.ml bytecomp/emitcode.ml bytecomp/bytelink.ml bytecomp/bytelibrarian.ml bytecomp/bytepackager.ml driver/errors.ml driver/compile.ml driver/maindriver.ml $ ocamlc -I utils -I parsing -I typing -I bytecomp -I file_formats -I lambda -I middle_end -I middle_end/closure -I middle_end/flambda -I middle_end/flambda/base_types -I driver -I runtime -g -compat-32 -o ocamlc -strict-sequence -principal -absname -w +a-4-9-40-41-42-44-45-48 -warn-error +a -bin-annot -strict-formats compilerlibs/ocamlcommon.cma compilerlibs/ocamlbytecomp.cma driver/main.mli driver/main.ml I wanted to try a different angle on the Load_path, and this time produced a function which predicts the files in the tree. The rules for this were pretty easy for me to define, and I wasn’t sure I could face watching Claude special-case everything. 130 lines of verifiably correct hacked OCaml later, I had my load path function. A little bit more code later, those three commands above were translated into an OCaml script (based on the ocamlcommon and ocamlbytecomp libraries) which should exactly the same build. It ran - and it built the compiler. ocamlc was, pleasingly, exactly the same. The .cma files, however, were not. For ocamlcommon.cma, that turned out to be me being sloppy with my commands. ocamlcommon.cma is linked with -linkall, but ocamlc -a foo.cma -linkall bar.cmo is not the same as ocamlc -a foo.cma -linkall bar.ml, because -linkall gets recorded in the .cmo file as well. Easy fix - but the files were still different. A bit more tweaking and I could see that actually the .cmo files were different. A bit more poking and checking with ocamlobjinfo and a few other flags and tricks, and I observed that: $ ocamlc -g -c utils/config.ml resulted in slightly different debug information from: $ console -g -c utils/config.mli utils/config.ml (it’s observably to do with the debug information - omit the -g and they’re all identical). Lots to suspect here, but time for… $ claude ╭───────────────────────────────────────────────────╮ │ ✻ Welcome to Claude Code! │ The problem was easy to state, but not quite so quick to come up with a conclusive explanation. Claude, like most of these models, appears not to have been trained on this old cartoon, and very merrily buzzes along for a few rounds of investigation, followed by a highly dubious explanation for how it was probably something to do with marshalling and, mumble mumble, the final binaries are the same so this bug is probably OK. Hmm. A few rounds of, “no, this needs to be equivalent as otherwise it’s not reproducible” (“You’re so right!”), and we had a lot of test programs, a frequent need for reminders that debugging OCaml’s Marshalling format was possibly not going to help, but we weren’t very much closer to an answer. Stepping back, I re-framed the problem, instead asking Claude to produce a program which would give a textual dump of the debug information in each file, so we could compare it. This was interesting - especially the occasional hallucinations at having analysed “all the fields”, but we got there. What was interesting was that we were struggling to perceive differences between anything. Claude at this point was desperate to delve into the runtime code and start doing hex-dumps of the marshal format to see what was actually different. I appear to be a little older than Claude, and was more reticent about this approach. I suggested we look at the polymorphic hash of some of these fields instead. At this point, we started to see some differences - Claude’s inferences at this point were working well, and there was a strong suggestion to add all sorts of accessor functions into the Types module to be able to introspect some of the values in more detail than normally intended (i.e. polymorphic hash was telling that us that some abstract values were different, but we wanted to see what the differences really were). Reader, I told it to use Obj.magic instead 🫣 However, what happened next was truly fascinating and definitely very efficient. The value being returned for one of the type IDs was simply not believable. It was far too high. Claude also correctly observed that it was in fact a block, and not an integer, which was what we were expecting. The human brain at this point cuts in, and looks at the type: Types.get_id: t -> int. No, that accessor looks right. Brain slowly whirring; look at the code: let get_id t = (repr t).id Oh - it’s not an accessor (in another life, I could possibly have performed Claude’s responses…). All I had to point out was that Types.get_id was not an accessor, it was normalising the result (to walk Tlink members of the type representation), and Claude was on it, replacing semi-elegant OCaml code with a sea of calls to Obj functions. But we had our answer - the type chain was different, if semantically equivalent and, more importantly, Claude then leaped to the problem. The internal Types.new_id reference isn’t reset between compilations 💥 A quick rebuild later, and the same debug information was given regardless of whether utils/config.mli was compiled at the same time as utils/config.ml. Go Claude. My contribution was keeping the explorations looking at relevant parts of the system, and not disappearing off on sometimes ridiculous and unbelievable tangents. Maybe it would have got there on its own, but who knows the tokens required and the GPUs scorched… Plug that back into my little script. ocamlcommon.cma still different. At this point, a line from Four Weddings and a Funeral could be heard loud and clear in the human mind. It’s the one which follows “Dear Lord, forgive me for what I am about to, ah, say in this magnificent place of worship…”. The fix was definitely working. But a quick bit of further experimentation revealed that including other .mli files before utils/config.ml (and there are a lot) was causing the information to change. So: $ claude -c As a human of hopefully normal emotional response to situations, the feeling of being back at square one would normally have meant I’d have at least needed a coffee before being able to face dusting off all the tools and scripts which had been constructed in the previous investigations. But here of course the LLM doesn’t care and was straight into using the tools previously constructed to look at the revised problem. A lot more Obj.magic-like investigations later looking at the shape of some debugging information, and Claude found another bit to reset, this time in Ctype. All the level information in the type-checker isn’t reset between compilations. Not a semantic issue, because the type checker uses those numbers relatively, but again they leak into the representation of some of the debugging information. And it was working 🥳 Next up was trying to put those fixes into something resembling a commit series that might one day be an acceptable PR. What I really wanted was a test. Claude was great for this, although it lacks anything approximating taste (and this is me writing…!). However, with no feelings to be hurt, the pointers were easy to issue and the results impressive - especially constructing a non-trivial ocamltest block. The result is previewable in dra27/ocaml#237 on my GitHub fork, and the test is entirely Claude’s. Having got to this stage, I extended the compiler with some of Lucas’s patches, and started passing just the .ml files for compilation, allowing the compiler to compile the .mli files on demand, as before. With some idle tinkering, I got to the end of “coreall”, which is the point in OCaml’s build process where ocamlc, the bytecode versions of everything in tools/ and ocamllex have all been compiled, along with the Standard Library. That was all being done from a single compiler process, where the OCaml script driving the compiler consisted mostly of the list of .ml files. Coupled with the predictive load path I’d already put together, at this stage the “plumbing” needed in the scheduler is just: let compile_file source_file () = Compenv.readenv Format.std_formatter (Before_compile source_file); let output_prefix = Compenv.output_prefix source_file in if Filename.extension source_file = ".mli" then Compile.interface ~source_file ~output_prefix else let start_from = Clflags.Compiler_pass.Parsing in Compile.implementation ~start_from ~source_file ~output_prefix let rec execute task = try task () with effect (Load_path.Missing path), k -> let file = Filename.chop_extension path ^ ".mli" in execute (compile_file file); execute (Effect.Deep.continue k) (as an aside, when it goes to being done with Domains I’ll possibly switch it to a shallow handler, because the call stack with the deep handlers isn’t as reasonable as I’d hoped for, but to be honest I just wanted to see it work!) Fascinatingly, all the artefacts (.cma and binaries) being produced were identical except for the Lazy module in the Standard Library! $ claude -c Claude was simultaneously amazing and useless at this. Amazing, because I was prompting some of this while cooking a meal, so being able to bark an instruction (actually, I hadn’t set it up for voice - I was just quickly typing) and then leave it to think for a minute or two was strangely efficient, because investigating this on my own would have taken too much continuous concentration. It was useless because we didn’t get anywhere near a believable explanation, despite various efforts at resetting things. Sometimes you just have to say /exit (and eat a meal…). However, after the aforementioned meal, I dug into it a bit further. The issue here was clearly to do with some state in the compiler - if ocamlcommon.cma or ocamlmiddleend.cma were compiled, then the Lazy module differed. Incidentally, at this point this wasn’t debug information which varied, it was the actual module, but it was still semantically the same. Claude had correctly identified that it was to do with the marshalling, and we had identified that there was a difference in string sharing (so not entirely useless, in fairness). I carried on poking and, with a little bit of jerry-rigging, managed to determine the relatively small set of files in flambda and in ocamlcommon whose compilation caused the change in Lazy. I was highly suspicious it was to do with compilation of lazy values. $ claude -c Feeding this information to Claude was a much better trick - the reasoning at this point would contradict its own tangents (“I should look at … but wait, the user has given me the list of affected files”). Impressively, we did hone in on the much more complex explanation for this third issue, which is to do with lazy values used in globals in the Matching module. In this particular case, if the compiler has compiled a file which matched on a lazy, causing Matching.code_force_lazy_block to be forced in the compiler and thus the CamlinternalLazy identified to be added to the current persistent environment, then a subsequent module (in this case lazy.ml in the Standard Library) which both pattern matches on a lazy and which also refers to CamlinternalLazy ends up with two extern’d string representations of CamlinternalLazy instead of one. The reason is that the forced code block in Matching still refers to a string used in a previous persistent environment. It’s not a semantic issue at all, but it manifests itself because the string is not shared when the subsequent file looks up the CamlinternalLazy identifier. It was a battle to update the test to show this behaviour, but in fairness that would have been a battle anyway! However, we got there too. Three reproducibility issues identified, and a viable PR produced - with tests!
dlvr.it