(that is, the piece of code to generate a random set of valid facts seemed not immediately obvious but maybe it's not that hard)
I've found that this is, hands-down, the hardest part of property-based testing.
That being said, we programmers implicitly do this kind of thinking all the time but don't explicitly encode it.
"Oh, the first argument has to be less than the second argument"
The other thing that I've found super useful is to use these types of tests to exercise variants of algorithms: "for every input, the SIMD-accelerated code must have the same results as the naive code"
This might be highly useful in this case as you could write "stupidly obvious" code that isn't as efficient as the :frog: but it's easy to verify it's correct.
are we talking about property based testing in general or the proptest crate specifically?
PBT usually runs along the lines of 1) roundtrip (encode . decode = id, decode . encode = id), 2) test against known to be correct algorithm, 3) running a verification algorithm on a decision algorithm, e.g.
is_sorted on the result of
sort + then also checking that its a permutation
some of my favorite PBT strategies are from https://fsharpforfunandprofit.com/pbt/
funny :P; ooh! diagrams are a nice strategy -- the aspiring category theorist in me likes this
the inputs seem a bit tough indeed to generate here, esp some of rustc’s liveness (if we need those facts) but some relations are easier than others; the test text parser already shows some of these validity rules / fact generation —definitely interesting
@lqd yeah graphs are usually hard to generate I find
@lqd have you tried the proptest crate?
(even reusing rustc might have been an option :)
I'm biased (since I wrote some of that code...) but it's pretty ergonomic to use
only quickcheck myself but I think niko has added some tests using it to datafrog
@lqd we're working on a book + shipping
#[derive(Arbitrary)] which should remove some of the boilerplate
I hope we’ll use it more then :)
@lqd it would be nice to get CoArbitrary to work as well to proptest some HoFs and such; I spoke to Koen about it a while back but it was sorta tricky to implement without GADTs so I put it on hold
I still love the post about using QuickCheck to test an entire C application
@Jake Goulding is that John's article?
Ah; nice one -- I really like this paper: https://dl.acm.org/citation.cfm?id=2364516
That link doesn't provide a way to read the article without paying, is that correct?
@Jake Goulding yeah sadly; here's a talk Koen held about it: https://www.youtube.com/watch?v=CH8UQJiv9Q4
@Jake Goulding Fortunately I found it in my stash :P p73-claessen.pdf
cough sci-hub cough