Fuzzing The Gleam Compiler

Can you find bugs in a compiler by generating random programs?

Published on: Tue Aug 25 2026

Introduction

I regularly check on Gleam’s changelog and issue tracker. I am very fond of this project and the people contributing to it. But every time I see an issue that relates to code generation or different outputs between the Erlang and JavaScript, it nags me that there was no way to basically “compute all the Gleam programs”, run them and see if there are any issues.

I imagine it like a chessboard, where you have a quasi-infinite number of possible positions on the board. But we want the chessboard to contain Gleam programs and we want an infinitely big database of those programs to see if they uncover untested edge cases.

My first attempt of doing something adjacent to this was actually prompting an LLM. I instructed it to read through loads of past Gleam issues and find more edge cases by “thinking hard about it”. It came up with all sorts of bit array combinations, nested anonymous functions, nested use patterns. Predictably, this approach did not yield many results.

$20 bucks of tokens later, it found exactly one issue, which was reported and fixed right away: https://github.com/gleam-lang/gleam/issues/5613. One is definitely more than zero. But there are plenty of issues with “LLM fuzzing”: it’s pricey, not deterministic and a bit like pulling the lever on a slot machine.

But there was another idea that I had avoided pursuing, because to be honest it just sounded like a lot of work: structure-aware fuzzing.

Structure-aware Fuzzing

Writing software is hard, and humans are not great at it. To help, we’ve built other software that can partially automate the search for bugs.

One of these programs is a fuzzer. They generate randomized inputs to feed into our program. The premise is that on a large scale, these random inputs will distribute in such a way that edge cases will be surfaced that we haven’t thought of yet.

Fuzzers can range from totally random scrambled bytes, to highly structured grammar-aware ASTs.

Feeding totally random bytes to a program is usually done for use cases that are working with images, files, network requests, protocols, etc. There are plenty of examples where fuzzing found real security flaws and bugs in open source software. For example, this finding by zzuf in Firefox, where flipping some bits in an image file would result in a browser crash: https://nvd.nist.gov/vuln/detail/CVE-2007-6715. But fuzzers have also uncovered real exploitable security flaws via buffer overflows.

There is a program by Google “OSS Fuzz” that continuously fuzzes a lot of important open source projects: https://google.github.io/oss-fuzz/

In our case, we are not working on a browser or network protocol. We have a compiler. And that opens up the possibility for structure-aware fuzzing. That means that we do not generate a stream of random bytes, but rather a stream of code in the form of source code or an AST.

Enter Gleam

There are a few things about Gleam that make it a particularly interesting candidate for fuzzing.

It generates code for two targets: JavaScript and Erlang. We can compare the output of the same program for both targets and flag any differences.

Gleam has a minimalistic syntax. At least compared to most other popular programming languages. We can generate valid programs that cover almost all concepts provided by the language with relatively little code.

Static types. Needless to say, this is an amazing feature that lets us ensure that a program will not crash at runtime. That doesn’t mean there can’t be any bugs in the type system. There have been issues related to type inference in the past. But as we will learn later on, each aspect of the language will require its own testing approach.

The functional nature and the fact that everything is an expression makes composing and structuring the programs very convenient.

Rust. This might be easy to overlook, but the fact that the Gleam compiler itself is written in Rust makes it very easy to integrate existing fuzzing tooling. We can test parts of the compiler without having to run a single .gleam file.

Resources I Used

We are going to dive into more technical aspects of the fuzzer. But I am not going to go into a lot of code or detail. If you would like to read more about that, do check out this post and blog by Nick Fitzgerald. It served as the main inspiration for this project: https://fitzgen.com/2020/08/24/writing-a-test-case-generator.html

Our fuzzer is going to be generation-based, not mutation-based. If you would like to understand the difference better, I recommend reading this article: https://fitzgen.com/2026/06/01/structure-aware-fuzzing-experiment.html

In the article, the author comes to the conclusion that, at least for wasm, the mutation-based approach found a lot more issues than the generation-based approach. So it is probably worth implementing for this project in the future!

For an even more in-depth dive into the topic, check out this resource: https://www.fuzzingbook.org/.

You can find the full code for the Gleam fuzzer in this branch of my Gleam fork: https://github.com/daniellionel01/gleam/tree/fuzzing

Phase 1: The Parser

An important design choice for the fuzzer: use the public compiler API. Even though there might not be any stability guarantees for the compiler API, this makes it easy to stay compatible with future versions of Gleam. It also avoids fiddling with implementation details, which is a good way to ensure we’re not creating any false positives or negatives.

To see some examples of how our parser catches and categorizes the outputs:

$ cargo run -p fuzzing-core --example classify
"pub fn main() { 1 }" -> compiled (js: 39B, ts: 32B, erl: 238B)
"pub fn main() { let f = fn(x) { x + 1 }; f(41) }" -> parse error
"pub fn main() { 1 +. \"x\" }" -> analysis rejected (javascript)
"pub fn main() {" -> parse error

Using the fuzz crate and some wrapper code, we can very quickly spam the Gleam compiler with randomly generated inputs (not structured yet), to see if we can crash the compiler instead of giving us an error message with more context.

We’re only going to run it for 1 second, because the output is quite large:

$ cargo +nightly fuzz run parse_only --fuzz-dir fuzzing-harness -- -max_total_time=1 -timeout=10

INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 302379076
INFO: Loaded 1 modules   (740945 inline 8-bit counters): 740945 [0x105eeac70, 0x105f9fac1),
INFO: Loaded 1 PC tables (740945 PCs): 740945 [0x105f9fac8,0x106aedfd8),
INFO:     2466 files found in fuzzing-harness/corpus/parse_only
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: seed corpus: files: 2466 min: 1b max: 4046b total: 425422b rss: 62Mb
#2467	INITED cov: 2434 ft: 8563 corp: 1249/171Kb exec/s: 0 rss: 108Mb
#2513	REDUCE cov: 2434 ft: 8563 corp: 1249/171Kb lim: 3764 exec/s: 0 rss: 108Mb L: 48/3753 MS: 1 EraseBytes-
#2645	REDUCE cov: 2434 ft: 8563 corp: 1249/171Kb lim: 3764 exec/s: 0 rss: 108Mb L: 8/3753 MS: 2 ChangeBit-EraseBytes-
#2656	REDUCE cov: 2434 ft: 8563 corp: 1249/171Kb lim: 3764 exec/s: 0 rss: 109Mb L: 2/3753 MS: 1 EraseBytes-
#2937	REDUCE cov: 2434 ft: 8563 corp: 1249/171Kb lim: 3764 exec/s: 0 rss: 109Mb L: 314/3753 MS: 1 EraseBytes-
#3183	NEW    cov: 2434 ft: 8578 corp: 1250/172Kb lim: 3764 exec/s: 0 rss: 110Mb L: 1054/3753 MS: 1 CopyPart-
#3591	REDUCE cov: 2434 ft: 8578 corp: 1250/172Kb lim: 3764 exec/s: 0 rss: 111Mb L: 99/3753 MS: 3 ShuffleBytes-CrossOver-EraseBytes-
#3934	NEW    cov: 2434 ft: 8585 corp: 1251/173Kb lim: 3764 exec/s: 0 rss: 112Mb L: 399/3753 MS: 3 CMP-CopyPart-CopyPart- DE: "\010\000\000\000\000\000\000\000"-

# ...

	NEW_FUNC[1/7]: 0x0001031cfb88 in _RINvNtCs3kGMwX4aip8_4core3ptr9drop_glueINtNtCshX1O598ANu2_5alloc3vec3VecINtNtNtCs846PmCUGaYz_10gleam_core3ast8constant8ConstantuEEEB1f_+0x0 (parse_only:arm64+0x1002abb88)
	NEW_FUNC[2/7]: 0x00010323bff4 in _RINvNtCs3kGMwX4aip8_4core3ptr9drop_glueINtNtNtCs846PmCUGaYz_10gleam_core3ast8constant8ConstantuEEBI_+0x0 (parse_only:arm64+0x100317ff4)
#16785	NEW    cov: 2483 ft: 8694 corp: 1262/177Kb lim: 3786 exec/s: 16785 rss: 144Mb L: 85/3753 MS: 1 CrossOver-
#18061	REDUCE cov: 2483 ft: 8694 corp: 1262/177Kb lim: 3797 exec/s: 18061 rss: 149Mb L: 403/3753 MS: 1 EraseBytes-
	NEW_FUNC[1/4]: 0x00010312c3b0 in _RINvMs_NtCs846PmCUGaYz_10gleam_core5parseINtB5_6ParserINtNtB5_5lexer5LexerINtBT_14NewlineHandlerINtNtNtNtCs3kGMwX4aip8_4core4iter8adapters3map3MapNtNtNtB1F_3str4iter11CharIndicesNCNvBT_14make_tokenizer0EEEE23parse_bit_array_segmentNtNtNtB7_3ast7untyped11UntypedExprNCNCNvB2_21parse_expression_units6_00NvB2_17expect_expressionNvB5_24bit_array_expression_intEB7_+0x0 (parse_only:arm64+0x1002083b0)
	NEW_FUNC[2/4]: 0x00010318893c in _RINvNtCs3kGMwX4aip8_4core3ptr9drop_glueINtNtCs846PmCUGaYz_10gleam_core3ast15BitArraySegmentNtNtBE_7untyped11UntypedExpruEEBG_+0x0 (parse_only:arm64+0x10026493c)

# ...

###### Recommended dictionary. ######
"\010\000\000\000\000\000\000\000" # Uses: 879
"\201\000" # Uses: 941
###### End of recommended dictionary. ######
Done 24887 runs in 2 second(s)

Sweet. Looking at some of the artifacts it produces, you can see what kind of inputs are generated:

fn
ar(n,n,n,///A#
o

ఌఌ「彸䕅䕅ⅅ䕅+�

">\u{000000000000000000.%\f0

fn
ar(n
ar:rn
a(

The nice thing about this is that it can test everything without running the gleam binary at all. It runs in-memory with the compiler pipeline in Rust.

And guess what! When I let this fuzzer run for quite a while, it actually found a regression on nightly, which did not happen on v1.18.1 (which was the latest version of Gleam at the time of writing this):

$ cargo +nightly fuzz run --fuzz-dir fuzzing-harness parse_only fuzzing-harness/artifacts/parse_only/crash-8b14db5e4bf152924501e0818787026e9f5ea229

=fuzzing-harness/artifacts/parse_only/ fuzzing-harness/artifacts/parse_only/crash-8b14db5e4bf152924501e0818787026e9f5ea229`
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3949856390
INFO: Loaded 1 modules   (750122 inline 8-bit counters): 750122 0x107773860, 0x10782aa8a),
INFO: Loaded 1 PC tables (750122 PCs): 750122 [0x10782aa90,0x10839cd30),
fuzzing-harness/target/aarch64-apple-darwin/release/parse_only: Running 1 inputs 1 time(s) each.
Running: fuzzing-harness/artifacts/parse_only/crash-8b14db5e4bf152924501e0818787026e9f5ea229

thread '<unnamed>' (23346909) panicked at /gleam/compiler-core/src/parse.rs:5226:52:
Token could not be converted to binop.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
==41428== ERROR: libFuzzer: deadly signal
    #0 0x000109a27654 in __sanitizer_print_stack_trace+0x28 (librustc-nightly_rt.asan.dylib:arm64+0x87654)
    #1 0x000106bcdf0c in fuzzer::PrintStackTrace()+0x30 (parse_only:arm64+0x1024a9f0c)
    #2 0x000106bc1f48 in fuzzer::Fuzzer::CrashCallback()+0x54 (parse_only:arm64+0x10249df48)
    #3 0x000181edb740 in _sigtramp+0x34 (libsystem_platform.dylib:arm64e+0x3740)
    #4 0x000181ed18d4 in pthread_kill+0x124 (libsystem_pthread.dylib:arm64e+0x68d4)
    # ... shortened ...

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
────────────────────────────────────────────────────────────────────────────────

Error: Fuzz target exited with exit status: 77

The key input was a pipeline |> in a const expression, like so: const b = 1 |> 2.

Once we find an issue, we also use git bisect on the issue. That way we can differentiate between a regression on nightly and an issue that is currently with the latest version of Gleam.

Of course we ideally not only let this run for 1 second, but many hours.

To be fair, the bugs found in this step are going to be nice to catch, but you are probably not going to find a code generation bug with this.

And at the time of posting this article, the libFuzzer package has been removed from the branch to keep the focus on type-safe programs. I have more targeted and efficient compiler crash tests in mind for a later stage of this project (namely [tree-splicer).

Phase 2: Type-safe Programs

To generate type-safe Gleam programs we are going to build a “smith”. We create our own simplified version of the Gleam AST that we then generate programs for in a probabilistic manner. That way, once the smith decides “we need an expression that resolves to an Int”, it can either provide a literal value like 3 or an anonymous function fn() { 3 }() or a variable, and so on and so forth. This is how we reliably create a lot of variety between programs to eventually discover new edge cases.

This is what one of those programs looks like:

pub const k_seed: Bool = False
pub const k_e: Int = 5
pub const k_golden: String = "data"

pub type V0 {
  Number(value: String, inner: List(Int))
}

fn walk(xs: List(Int), acc: Int) -> Int {
  case xs {
    [] -> acc
    [x, ..rest] -> walk(rest, acc + x)
  }
}

fn f0(arguments: #(Float, String), l: Int, item: #(Float, Bool)) -> List(Int) {
[]
}

fn f1(class: Int, acc: String) -> Int {
100
}

fn f2(pair: Int) -> Float {
case "" <> "abc" {
    "data" <> rest as whole -> fn(v1) { {
      let whole = []
      0.0
    } }(0.0)
    inner | "ab" <> inner -> {
      let self_ = 3.14
      fn(v2, v3) { self_ }("x", 4)
    }
    "b" <> b -> case fn(v4) { 42 }(2.0) {
      9 -> {
        1.0
      } *. {
        2.0
      }
      item -> 3.14
    }
  }
}

pub fn main() {
  let v = walk([5], k_e) - 100
  let z = f2(v)
  echo [2]
  echo {
    case "res" <> "data", f0(#(1.0, "data"), v, #(0.5, False)) {
      _, [] -> fn(v5, v6) { 10.0 }(False, "x")
      "b" <> _, [] -> z +. {
        2.0
      }
      k_seed, [2, h, ..] as whole -> 10.0
      _, _ -> f2(5)
    }
  } /. {
    0.5
  }
  echo {
    fn(v7) { 100.0 }(False)
  } *. {
    case "b", 2 {
      "ab", 1 -> {
        let rest = k_seed
        z
      }
      "x", 5 -> 2.0
      _, v8 -> z +. {
        0.25
      }
    }
  }
}

That is a whole lot of gibberish. And that is the point! We generate programs that combine together with randomly selected valid expressions to hopefully uncover combinations that cause incorrect behaviour on one or both targets.

But how do we know if one of our Gleam programs produces incorrect behaviour if it compiles successfully? This is where the two compilation targets come in. For example, if there is a bug in a program that contains a case expression, we can uncover it if the logic is correctly implemented in Erlang, because we would get a different output on the JavaScript target if we provide different values in each branch.

This is not 100% foolproof, but a solid approach to start with. We can still have bugs in the compiler that happen on both targets, so we can get false positives from this approach. As always: there is no silver bullet.

Before we look at our Gleam smith, I need to go on a slight tangent.

The echo Problem

Here is the thing. JavaScript and Erlang have different ideas about how to represent values at runtime. For example, you don’t have dedicated integer and float types in JavaScript. You have Number. Some values are represented differently when you convert them to a string and use Gleam’s built-in echo keyword.

Take this program:

pub type Wibble {
  Wibble(wobble: Int)
}

pub fn main() {
  echo "hello"
  echo <<1, 2, 3>>
  echo <<"a":utf8, "b":utf8, "c":utf8>>
  echo [1, 2, 3]
  echo Wibble(3)
  echo 1.2
  echo 1.0
}

Here is the output of that program on Erlang vs JavaScript:

$ gleam run --target erlang

src/app.gleam:6
"hello"
src/app.gleam:7
"\u{0001}\u{0002}\u{0003}"
src/app.gleam:8
"abc"
src/app.gleam:9
[1, 2, 3]
src/app.gleam:10
Wibble(3)
src/app.gleam:11
1.2
src/app.gleam:12
1.0

$ gleam run --target javascript

src/app.gleam:6
"hello"
src/app.gleam:7
<<1, 2, 3>>
src/app.gleam:8
<<97, 98, 99>>
src/app.gleam:9
[1, 2, 3]
src/app.gleam:10
Wibble(wobble: 3)
src/app.gleam:11
1.2
src/app.gleam:12
1

In JavaScript 1.0 is printed as 1, whereas in Erlang the bit array <<1, 2, 3>> is printed as "\u{0001}\u{0002}\u{0003}". The record Wibble does not contain any labels in Erlang. Not ideal if we want to compare the outputs directly in our fuzzer.

There are a few options I can think of to circumvent this issue:

• Intentionally not test the values we know to diverge in their echo output.
• Build a custom echo function in Gleam and inject it into the generated modules.
• Build a custom “parse this output from Gleam” in Rust, since we know from the generated Module AST which values are going to be echo’d in Gleam.

For this version I went with the third option. Since this was a very well constrained problem, I generated loads of test cases and the appropriate code to parse any echo output coming from a Gleam program and parse it into a Rust enum to predictably compare the outputs. This is definitely far from perfect, but it seems to work well for now.

Alright, now that we have everything in place, it’s just time to blast the fuzzer and let it generate 100k programs right? Well, this is where I need to go on my other tangent.

Duplicate Findings And Blocking Issues

Once you get the fuzzer going, it will generate a lot of programs of a similar shape. Consequently, if you do discover a bug, you will keep generating programs that reproduce that bug.

We again have a few ways to deal with this:

• Modify the Gleam smith code so that certain combinations of expressions will not be generated, until the issue is fixed.
• If there is a fix available (either by yourself or as a PR), provide and apply a patch to the Gleam fork running the fuzzer. That way you can keep running the fuzzer until a PR that fixes the issue is available on nightly and can be merged back into the fork.
• If your issue is an error message (not a value mismatch), you can usually find a string.contains phrase to skip this in your analysis.

At the time of writing this, I went with the third approach. I did try the patch approach, but this assumes that the PR or your fix is definitely correct and won’t introduce any new bugs. It’s better to rely on the exact state of Gleam that the official repository is in.

For example, I uncovered 2 code generation issues with JavaScript, so at the moment the fuzzer is completely ignoring any issues that generate a similar signature to the ones reported in the official repository. This does mean that I am potentially skipping over other code generation issues that have a similar error signature. I could introduce more heuristics and analysis of the program AST if it contains the exact expression that causes these issues. For a larger scale approach, this would definitely be a good idea, since right now I am running the fuzzer in batches of 100 programs and manually verify any findings it skips or flags as “new bug”. Out of 100 programs that is usually just a handful, so it’s still manageable for one person.

// https://github.com/gleam-lang/gleam/issues/6182
fn is_gleam_issue_6182(raw: &str) -> bool {
    raw.contains("SyntaxError: Unexpected token '&&'")
        || raw.contains("SyntaxError: Unexpected token ')'")
}

// https://github.com/gleam-lang/gleam/issues/6212
fn is_gleam_issue_6212(raw: &str) -> bool {
    raw.contains("TypeError:") && raw.contains("is not a function")
}

Using The Fuzzer

If you would like to play around with this yourself in the repository:

$ cargo run -p fuzzing-cli -- run 948
=== fuzz ===
gleam:       /daniellionel01/gleam/target/release/gleam
seed:        948
erlang exit: 0
nodejs exit: 0
match:       OK

--- values ---
[0] == erl=Bool(false)  js=Bool(false)
[1] == erl=String("bcx")  js=String("bcx")
[2] == erl=Bool(false)  js=Bool(false)

# A mismatch in the second `echo`'d value!

$ cargo run -p fuzzing-cli -- run 947
=== fuzz ===
gleam:       /daniellionel01/gleam/target/release/gleam
seed:        947
erlang exit: 0
nodejs exit: 0
match:       NO

--- values ---
[0] == erl=Int(2)  js=Int(2)
[1] != erl=Int(42)  js=Int(103)
[2] == erl=Bool(true)  js=Bool(true)

You can also print the program it is running to stdout:

$ cargo run -p fuzzing-cli -- print 947
pub const golden_value: String = "constructor"
pub const limit_value: Float = 2.0
pub const seed_value: Int = 42

fn yield(constructor: Float, prototype: Int, default: Float) -> List(Int) {
[7]
}
# ... lots more code ...

And fuzz batches of programs like so:

# seed 947 is a known bug, but there is no good heuristic to detect it right now.

$ cargo run -p fuzzing-cli -- batch 900 100
[fuzzing-cli] gleam: /daniellionel01/gleam/target/release/gleam
[fuzzing-cli] seeds 900..999
[fuzzing-cli] 10/100 run, 0 mismatches, 0 skipped
[fuzzing-cli] 20/100 run, 0 mismatches, 0 skipped
[fuzzing-cli] 30/100 run, 0 mismatches, 0 skipped
[fuzzing-cli] 40/100 run, 0 mismatches, 0 skipped
[fuzzing-cli] DIVERGENCE seed 947 -> erlang:0 nodejs:0
[fuzzing-cli] 50/100 run, 1 mismatches, 0 skipped
[fuzzing-cli] 60/100 run, 1 mismatches, 0 skipped
[fuzzing-cli] 70/100 run, 1 mismatches, 0 skipped
[fuzzing-cli] 80/100 run, 1 mismatches, 0 skipped
[fuzzing-cli] 90/100 run, 1 mismatches, 0 skipped
[fuzzing-cli] 100/100 run, 1 mismatches, 0 skipped
[fuzzing-cli] done: 100 programs, 1 mismatch(es), 0 skipped

And that’s it! I go through batches of 100 at a time, take a look at any crashes or differences in the Erlang vs JavaScript output and see if it is a known program shape, or something novel.

Findings By The Fuzzer

So far the fuzzer has already found 9 issues! One of them was closed and reported upstream to Erlang/OTP! That’s a really cool finding: https://github.com/erlang/otp/issues/11494

• #6179 — Unreachable branches cause Erlang codegen to panic
• #6180 — A type called Record gives a warning from Erlang
• #6181 — Matching on <<_:utf8>> takes the wrong branch on JavaScript
• #6182 — Matching on <<"":utf8>> generates incorrect JavaScript
• #6187 — Erlang throws on two consecutive echos with a list and empty string
• #6192 — Compiler crash on nightly when using |> in a const
• #6212 — Function called as wibble shadows a local on JavaScript
• #6213 — JS codegen uses the wrong variable when a let shadows an outer one

The fun thing is that we’re not even covering all of the potential expressions in Gleam with the current state of the fuzzer. It is one approach to fuzzing with a limited subset of Gleam programs. When we expand it to all possible expressions and statements in Gleam and introduce even more ways to test program correctness, this will become an incredibly useful tool for the Gleam community to harden the tool even more.

It is also worth noting that larger projects like esbuild, Babel and Elm, that compile to JavaScript, are still getting issues related to JavaScript code generation, even after having been around many many years (evanw/esbuild#4520, evanw/esbuild#4517, evanw/esbuild#4495, babel/babel#16009, babel/babel#15111, babel/babel#15145, elm/compiler#2360, elm/compiler#1223, elm/compiler#1636). Code generation, especially for JavaScript, is a really really hard problem. Especially when you consider that Gleam also compiles to Erlang, which could not be more different from a language like JavaScript, while still juggling both targets incredibly well!

Future Work & Emerging Problems

This project is in its infancy and there are a lot of things we can work on in the future:

• Something I wonder is how you would do fuzzing at scale. I’m sure other companies have dedicated servers and hardware for this. But parts of the fuzzer can also be run in GitHub Actions. I think a good workflow would be for this to be available for the Gleam core maintainers and be able to run a good amount of fuzzed programs when they are reviewing a PR and before they release a new RC.
• Right now, reviewing the findings of the fuzzer can be done manually and with the help of an LLM, depending on the quantity of programs. Because when you’re generating hundreds or even thousands of programs, a lot of them will run into the same issue once you find a new one. Finding a reliable approach to stop producing the same program shapes for known and not yet fixed issues seems like something that would yield a very high ROI (the investment being time in this case).
• I think implementing the fuzzer for the language server would uncover a lot of new edge cases, since it’s very susceptible to actions being triggered in unforeseen states.
• Metamorphic testing would be a good way to test the output of a program without having to compare both targets.
• There have been issues related to generics and type inference in the Gleam compiler that are definitely worth fuzzing for. This will require a more complex program smith than right now that takes into account those concepts.
• You can fuzz lots of inputs for official Gleam packages. Especially the standard library gleam_stdlib. There have been inconsistencies between behaviour on both targets before.
• I would even give the LLM fuzzing approach another shot, if I got my hands on physical hardware, like an outdated but powerful enough GPU to run a “good-enough” open weight model in a loop and just let it auto-discover new approaches 24/7 in a well-constructed harness.
• I think this paper would be incredibly cool to implement for this use case and might also help for deduplicating issue reports: https://www.cs.purdue.edu/homes/xyzhang/fall07/Papers/delta-debugging.pdf

Conclusion

I have a lot of fun writing code when I am working in a codebase where I can work on something without being anxious about the integration and testing of those changes.

And the best thing about a test suite is that it is usually additive. Every new edge case and bug fix can provide more safety in the future.

I am super excited for the future of Gleam, and contributing to it has been very rewarding.

Join the Community (especially Discord) if you haven’t yet!

https://gleam.run/community/

Cheers!