Video #32: Decodable Randomness: Part 2

Episode: Video #32 Date: Oct 1, 2018 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep32-decodable-randomness-part-2

Description

This week we compare our Decodable solution to building random structures with a composable solution involving the Gen type, exploring the differences and trade-offs of each approach. Along the way we’ll rediscover a familiar old friend with a brand new application.

Video

Cloudflare Stream video ID: f31cb08ca31510d02727a55b70060744 Local file: video_32_decodable-randomness-part-2.mp4 *(download with --video 32)*

Transcript

— 0:05

So what we are seeing is that we put in a lot of upfront work with Decodable , and got some impressive results out it because we can effortlessly generate a random value from pretty much any data type we define. However, the system is very rigid and will mostly create invalid values for your domain, like email addresses and positive user ids.

— 0:39

Maybe there’s another way. We saw last time that our Gen type was able to do things that the Swift 4.2 randomness APIs were not capable of, so maybe it can help us out again? Composing structure

— 1:00

Let’s first look at some of the work we did last time. struct Gen<A> { let run: () -> A } import Darwin let random = Gen(run: arc4random) extension Gen { func map(_ f: @escaping (A) -> B) -> Gen { return Gen { f(self.run()) } } } let uint64 = Gen<UInt64> { let lower = UInt64(random.run()) let upper = UInt64(random.run()) << 32 return lower + upper } func int(in range: ClosedRange<Int>) -> Gen<Int> { return Gen<Int> { var delta = UInt64( truncatingIfNeeded: range.upperBound &- range.lowerBound ) if delta == UInt64.max { return Int(truncatingIfNeeded: uint64.run()) } delta += 1 let tmp = UInt64.max % delta + 1 let upperBound = tmp == delta ? 0 : tmp var random: UInt64 = 0 repeat { random = uint64.run() } while random < upperBound return Int( truncatingIfNeeded: UInt64(truncatingIfNeeded: range.lowerBound) &+ random % delta ) } } func element<A>(of xs: [A]) -> Gen<A?> { return int(in: 0...(xs.count - 1)).map { idx in guard !xs.isEmpty else { return nil } return xs[idx] } } extension Gen { func array(count: Gen<Int>) -> Gen<[A]> { return Gen<[A]> { Array(repeating: (), count: count.run()) .map { self.run() } } } } We have a Gen type, which is generic over the value it generates. We have a base unit of randomness, derived from the arc4random function. We have a map function, which is used to compose and build all new kinds of generators, including a uint64 generator and our ability to recreate the int(in:) and element(of:) functionality of the Swift 4.2 APIs. We were even able to build all new notions of randomness, like the ability to generate a randomly-sized array of random values.

— 1:50

Let’s warm up a lil by creating a new generator, one that can generator random strings much like what we did with our ArbitraryDecoder : let string = int(in: Int(UInt8.min)...Int(UInt8.max)) .map { String(UnicodeScalar(UInt8($0))) } .array(count: int(in: 0...280) .map { $0.joined() }

— 3:30

And we can run I like any other generator. string.run() // "6!QwÉ…" string.run() // "AHO0 …" string.run() // "¿z3âO…"

— 3:34

And there we go! Random strings, just as we did with our ArbitraryDecoder , although this time we were able to build this reusable unit using composition and the Gen type. Composition allows us to focus on small units, which means we can further decompose things, which wasn’t something we could do within the constraints of our decoder.

— 3:51

For example, we can pull out the Int - UInt8 dance we did earlier by defining uint8(in:) . func uint8(in range: ClosedRange<UInt8>) -> Gen<UInt8> { return int(in: Int(range.lowerBound)...Int(range.upperBound)) .map(UInt8.init) }

— 4:15

And now, we can reuse that unit instead. let string = uint8(in: .min ... .max) .map { String(UnicodeScalar($0)) } .array(count: int(in: 0...280) .map { $0.joined() }

— 4:24

In addition to decomposition, we can look at the compositions themselves and explore how they glue together. For instance, we could decompose the string conversion into two discrete, point-free steps. .map(UnicodeScalar.init) .map(String.init)

— 4:45

And, as we’ve said in the past, the composition of the maps is the same as the map of the composition. .map(UnicodeScalar.init >>> String.init)

— 4:57

And this string generator is all we need to start generating users using Gen . So how can we create a random user using Gen ? Well, the simplest thing would be to create a User using the same generators. User( id: int(in: .min ... .max).run(), name: string.run(), email: string.run() ) // User( // id: 7705522375486543313, // name: "e,QeZGÒ½*»]μ]#TÈx", // email: "\r¢GU×Ô;uVsæ\rêXû)èCó•@´û>dóU…" // )

— 5:27

And this produces a random user, just as random as the kinds of users our decoder produced.

— 5:33

This was more work than what we did before, but now we have a bit more control of the randomness.

— 5:44

We can, for example, we can generate more appropriate ids by avoiding negative numbers. User( id: int(in: 1...1_000).run(), name: string.run(), email: string.run() ) // User( // id: 366, // name: "PoS<6]xóEÊ#ST²äÎÉç»äôù Uv8W\"|\…", // … // )

— 6:00

Now we get a user with a more reasonable id, but what about that name? Not quite so realistic for a user signup, though maybe reasonable for a hacker or spammer.

— 6:12

Let’s create a name generator, instead. What does it mean to generate a random name? Maybe we should start with some legally traditional names by limiting ourselves to latin letters and trying to generate first and last names.

— 6:36

We can start with the smallest unit: latin characters. let alpha = element(of: Array("abcdefghijklmnopqrstuvxcyz")) // Gen<Optional<Character>>

— 6:58

This produces a generator of optional characters, but we know it should always produce one safely, so let’s unwrap it using map . let alpha = element(of: Array("abcdefghijklmnopqrstuvxcyz")) .map { $0! } // Gen<Character>

— 7:10

Now we have a generator of characters, but we want a generator of strings. Last time we wrote a helper to produce randomly sized arrays of random elements from other generators. We should be able to do the same kind of thing to produce a helper that generates randomly-sized strings. extension Gen where A == Character { func string(count: Gen<Int>) -> Gen<String> { return self.map(String.init).array(count: count).map { $0.joined() } } } We can reuse the array helper, but arrays of characters don’t have a helper to be joined into a string, so we need to convert our generator of characters into a generator of strings.

— 8:41

From here we can create a new generator that generates single names. let namePart = alpha.string(count: int(in: 4...8)) // Gen<String>

— 8:59

And with this generator, we can build another generator that produces first and last names. let randomName = Gen<String> { namePart.run().capitalized + " " namePart.run().capitalized }

— 9:18

We’re doing that capitalization work twice, so let’s factor it out into a new generator. let capitalNamePart = namePart.map { $0.capitalized } let randomName = Gen<String> { capitalNamePart.run() + " " capitalNamePart.run() }

— 9:34

And we can plug this generator into our User . User( id: int(in: 1...1_000).run(), name: randomName.run(), email: string.run() ) // User( // id: 432, // name: "Aiqcbsoc Kqjdsla", // email: "ê{ô]+uéâs3cÏÜÂ•uÛ\"ûβZÏ*ã$&" // )

— 9:48

The name looks reasonable, but the email is still invalid. Let’s derive one more generator to fix that. let randomEmail = namePart.map { $0 + @pointfree.co" }

— 10:06

And we can plug that into our random user initialization. User( id: int(in: 1...1_000).run(), name: randomName.run(), email: randomEmail.run() ) // User( // id: 271, // name: "Spgahefb Mwdxjnf", // email: " [email protected] " // )

— 10:15

And there we go, some values that are much better and more appropriate than what we had before using the decoder.

— 10:25

We should even be able to extract our int(in:) call to be a more reusable generator. let randomId = int(in: 1...1_000) This gives our call site a bit more symmetry: User( id: randomId.run(), name: randomName.run(), email: randomEmail.run() )

— 10:39

We now have the ability to generate some better, domain-friendly values than we could have ever imagined with the decoder, and we were able to do so in a very lightweight and reusable manner using composition.

— 10:58

The last thing to do is to generate users in a more reusable fashion: let’s wrap it in a Gen . let randomUser = Gen { User( id: randomId.run(), name: randomName.run(), email: randomEmail.run() ) } // Gen<User>

— 11:11

And we can run it a bunch of times. print(randomUser.run()) print(randomUser.run()) print(randomUser.run()) print(randomUser.run()) // User( // id: 478, name: "Uedxqtjd Ibdp", email: " [email protected] " // ) // User( // id: 853, name: "Xkbbuy Haugz", email: " [email protected] " // ) // User( // id: 446, name: "Xruef Blhvprlw", email: " [email protected] " // ) // User( // id: 152, name: "Ezwbnl Tbxckrr", email: " [email protected] " // )

— 11:29

The Gen type powered all of this! The amount of work we had to do was pretty minimal and all built from small, composable, and very reusable units. Zipping structure

— 11:45

It’s all very simple. We have a random user generator that on the inside just fills all the fields with the result of other generators. This is all good and simple, but still slightly more work than we need to do.

— 12:02

We’ve already addressed this shape of problem previously. In our three episodes on zip , we identified the problem of what it means to take values that are wrapped up in a context and to create values from the values in those contexts. We did this with a Parallel type, with arrays, optionals, the Func type, and this is another one of those types. The Gen type is a context in which random values live, and we should be able to instantiate User from generator values in a much more seamless way.

— 12:46

It shouldn’t be too surprising, since Gen has map , that it also has zip . Let’s define it, starting with zip2 : func zip2<A, B>(_ a: Gen<A>, _ b: Gen) -> Gen<(A, B)> { return Gen<(A, B)> { (a.run(), b.run()) } }

— 13:24

And already we can use this with our previous generators to produce a generator of ids and names. zip2(randomId, randomName).run() // (773, "Ipbyiih Wiuqhyss") zip2(randomId, randomName).run() // (9, "Bfrzkch Wbsrxgt") zip2(randomId, randomName).run() // (849, "Yjebdcf Gadv")

— 13:38

With zip2 , we can build zip2(with:) , which applies a given transformation to the tuple being generated. func zip2<A, B, C>( with f: @escaping (A, B) -> C ) -> (Gen<A>, Gen) -> Gen<C> { return { zip2($0, $1).map(f) } }

— 14:14

With those defined, we probably also want zip3 . func zip3<A, B, C>( _ a: Gen<A>, _ b: Gen, _ c: Gen<C> ) -> Gen<(A, B, C)> { return zip2(a, zip2(b, c)).map { ($0, $1.0, $1.1) } } zip3 merely calls zip2 under the hood and maps the result to flatten the tuple.

— 14:39

And with zip3 , we probably also want zip3(with:) . func zip3<A, B, C, D>( with f: @escaping (A, B, C) -> D ) -> (Gen<A>, Gen, Gen<C>) -> Gen<D> { return { zip3($0, $1, $2).map(f) } } Which calls zip3 and maps the result with the transform function.

— 14:55

With zip3(with:) defined, we have precisely what we need to lift functions (A, B, C) -> D up into the world of generators of those types: (Gen<A>, Gen, Gen<C>) -> Gen<D> .

— 15:05

In particular, we can do the following: zip3(with: User.init) // (Gen<Int>, Gen<String>, Gen<String>) -> Gen<User> And we get back a function that, given a generator of int, a generator of string, and a generator of string, we can get back a generator of users.

— 15:17

So, if we pass along our generators, we’ll get a generator of random users. zip3(with: User.init)( randomId, randomName, randomEmail ) // Gen<User>

— 15:31

Let’s compare things to before. let randomUser = Gen { User( id: randomId.run(), name: randomName.run(), email: randomEmail.run() ) } Previously we had to do a little extra work: we had to instantiate the generator and user ourselves, and we had to run each generator manually. Whereas now we can very expressively describe a random user as zipping 3 other generators together with the user initializer. Strengthening structure

— 16:10

Let’s do something that wasn’t possible with our arbitrary decoder: let’s strengthen our user’s id to be a

UUID 16:24

We get a couple of compiler errors that we can comment out for now.

UUID 16:44

We also now get a runtime error from our arbitrary decoder, since it currently isn’t equipped to handle UUIDs.

UUID 17:06

Let’s fix the compiler errors that prevented us from generating a random user. We first need the ability to generate random UUIDs, which have the following format: "DEADBEEF-DEAD-BEEF-DEAD-BEEFDEADBEEF" It’s 8 hex values, followed by 4, followed by 4, followed by 4, and finally followed by 12, each section separated with a hyphen.

UUID 17:27

We can cook up a generator that generates random UUIDs using previous generators we defined.

UUID 17:34

Let’s start with a hex generator. let hex = element(of: Array("0123456789ABCDEF")).map { $0! } // Gen<Character>

UUID 18:00

We can now build a UUID string generator using this hex generator. let uuidString = Gen { hex.string(count: .init { 8 }).run() + "-" + hex.string(count: .init { 4 }).run() + "-" + hex.string(count: .init { 4 }).run() + "-" + hex.string(count: .init { 4 }).run() + "-" + hex.string(count: .init { 12 }).run() } // Gen<String>

UUID 18:46

We just need to package this string up into a UUID. let randomUuid = uuidString.map(UUID.init).map { $0! }

UUID 19:16

With this defined, we can finally use it to generate random users. zip3(with: User.init)( randomUuid, randomName, randomEmail ) print(randomUser.run()) print(randomUser.run()) print(randomUser.run()) // User( // id: 1DD02E51-5002-924D-87B1-558CD34142F0, // name: "Mfcag Urfvpp", // email: " [email protected] " // ) // User( // id: AD1493D8-17C1-EDC5-EF28-4E6D5E7190BD, // name: "Ovnl Qmxu", // email: " [email protected] " // ) // User( // id: A6F241E6-11C1-96D1-83DD-942BD6AD1765, // name: "Isjuqr Jbem", // email: " [email protected] " // )

UUID 19:38

That was very little work in order to get something going that the arbitrary decoder couldn’t do. If we were heavily reliant on the arbitrary decoder, we wouldn’t be able to use UUIDs at all! Strengthening structure again

UUID 19:52

We can take things even further! We can strengthen our types more while still leaving ourselves open to interface with randomness in a nice way.

UUID 20:12

In our Tagged episode, we had the ability to strengthen types by tagging them. We have Tagged available in our workspace, so let’s import it and strengthen our user id. import Tagged struct User: Decodable { typealias Id = Tagged<User, UUID> let id: Id let name: String let email: String }

UUID 20:32

Now we get some new compiler errors. User( id: randomUuid.run(), name: string.run(), email: string.run() ) Cannot convert value of type ‘UUID’ to ‘User.Id’

UUID 20:34

So how do we fix things? Well, we can just use map with User.Id.init . User( id: randomUuid.map(User.Id.init).run(), name: string.run(), email: string.run() ) zip3(with: User.init)( randomUuid.map(User.Id.init), randomName, randomEmail )

UUID 20:54

It all ran and looks the same, but under the hood the user id has been strengthened to be its own type, unique to UUID. It took almost no work! Because we have map on our generators, we were able to get random, tagged user ids, basically for free.

UUID 21:21

We’ve now gone through a couple steps to improve our users: we first strengthened an integer-based id to a UUID, and then we further strengthened the UUID to be tagged. Every step along the way was just a little bit of glue code to make it work, which was almost always just a matter of calling map . We were able to get a random UUID string by mapping on a hex generator. We were able to get a random UUID by mapping on the random UUID string generator. And we were able to get a random tagged user id by mapping on the random UUID generator.

UUID 21:54

These generators are so composable and interface with things so freely! Tagged has no idea of what Gen is, and Gen has no idea of what Tagged` is, but they play nicely together with very little work, which is powerful.

UUID 22:07

There are other fields we could have added, too, like a date field, and it would be just as nice to work with. It wouldn’t have the problems we had with the decoder. We could have a date generator that’s very dependably in the past, or a date generator that is very dependably in the future. What’s the point?

UUID 22:26

Alright, so this is all pretty cool, but maybe it’s time to stop and ask “what’s the point?” We started by exploring Decodable as a solution for generating random values, and it was seriously impressive that a protocol we typically only use for decoding API payloads is powerful enough to pluck random values of any decodable type out of thin air.

UUID 22:47

But as impressive as this machinery was, it was seriously complicated and not as flexible as we would have liked it to be, so instead we explored what it would mean to build generators of more complex data structures using the Gen type we introduced last week.

UUID 23:03

But what’s the point!? Why are we generating random users? What purpose could they possibly have? Is it worth building this machinery?

UUID 23:16

Swift 4.2 didn’t revamp the random APIs for nothing! Randomness is an important part of application development, whether we use it every day or not. Maybe you’re making a confetti animation for your app to celebrate a user’s birthday: it’s a bunch of randomly-sized and randomly colored views that are animated in random ways. That’s a bunch of randomness! Think about how that kind of code could be cleaned up using a Gen type, composing a randomBalloon generator out of randomColor , randomSize , etc.

UUID 23:59

Games also have a lot of randomness, and who knows what composability will unlock there?

UUID 24:09

There’s also the idea of fuzz testing, where you purposefully send very bizarre values through your program to see if something is going to crash or behave in unexpected ways. The arbitrary decoder we started building actually seems like a good fit for such a thing.

UUID 24:32

The biggest reason we wanted to start this study of randomness, though, is for the purpose of something called “property testing”, where you define a particular property that you want your application to have, and you verify that property by bombarding it with tons of random values and ensuring that the property holds true for every value.

UUID 25:00

This is a very powerful idea. It really helps exercise every little corner of your application, and the Gen type is really at the root of how you build that system.

UUID 25:13

It’s cool to see that testing is becoming a more and more important part of the Swift community, but we’re currently quite limited in the types of tests we typically write. Unit tests are written in such a way where we hand-write values to be tested against, with no variation between test runs, which is a lot of boilerplate for testing surface area. Property testing allows us to find bugs that we wouldn’t even think to test for.

UUID 25:41

Property testing is something we’ll explore soon, but we still have to explore another idea: seeding our random number generators for predictability so that we can reliably reproduce property failures. That’ll have to wait for another episode, though! Downloads Sample code 0032-arbitrary-pt2 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .