Video #42: The Many Faces of Flat‑Map: Part 1

Episode: Video #42 Date: Jan 7, 2019 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep42-the-many-faces-of-flat-map-part-1

Description

Previously we’ve discussed the map and zip operations in detail, and today we start completing the trilogy by exploring flatMap. This operation is precisely the tool needed to solve a nesting problem that map and zip alone cannot.

Video

Cloudflare Stream video ID: cd61f024fd18a9e596bd73c20d9cb2f5 Local file: video_42_the-many-faces-of-flat-map-part-1.mp4 *(download with --video 42)*

References

Transcript

— 0:05

We’re back for our first episode of 2019. Hope everyone had a nice break, and we’re ready to start some new material.

— 0:10

The title of this episode is “the many faces of flatMap ”, and if you’ve been a following Point-Free for awhile you may know that we’ve done this style of episode twice before. First we did “The Many Faces of Map” , where we showed that map is a very universal operation that goes far beyond the map that Swift defines on sequences and optionals. It’s in fact the unique function with its signature that satisfies a simple property, and that tells us that map isn’t something we invent but rather something we discover. It was there all along whether or not we knew it. This empowered us to define map on many new types, which unlocks a lot of nice compositions.

— 0:49

A few months after that we did a 3-part series of episodes called “The Many Faces of Zip” ( part 1 , part 2 , part 3 ), where we similarly showed that zip can also be generalized far beyond the zip that is defined on sequences in the Swift standard library. We could define zip on optionals, result types, function types, and even asynchronous values. We also saw that zip allows you to do map -like operations, but just with functions that take more than one argument, which is simply not possible with map alone. So zip unlocked something new for us.

— 1:16

Today’s episode completes a trilogy of operations, and honestly it’s kind of shocking that Point-Free launched nearly a year ago and this is our first time talking about it. I think a lot of people would assume this topic is the bread and butter of functional programming, and although quite important, in our 41 previous episodes so far we have shown that you can still do a lot without it.

— 1:32

We are of course talking about flatMap !

— 1:35

Swift ships with two flatMap methods, one on sequences and one on optionals, but the idea of flatMap is so much bigger than just that. It is a further generalization of the ideas of map and zip in that it can express things that are just not possible with map and zip alone. So today we begin to get comfortable with flatMap and expand our understanding of what its true purpose is.

— 1:58

We try to make episodes stand on their own as much as possible, but flatMap is so intimately related to map and zip that this just isn’t possible to do. We think you’ll get the most from this episode if you’ve seen our previous episodes on map and zip ( part 1 , part 2 , part 3 ) because we are building off of those ideas quite a bit. The need for flat‑map

— 2:19

So, let’s get started! The Swift standard library comes with a flatMap operation on both sequences and optionals, but before looking at that let’s first motivate the need for flatMap . Let’s see what would go wrong if we only had map and zip at our disposal.

— 2:33

Suppose we had two arrays and we wanted to construct a new array of all possible combinations of values from each of those arrays: func combos<A, B>(_ xs: [A], _ ys: [B]) -> [(A, B)] { fatalError() } combos([1, 2, 3], ["a", "b"]) // [(1, "a"), (1, "b"), (2, "a"), (2, "b"), (3, "a"), (3, "b")]

— 3:03

How could we implement that? Well, before we knew about map we may have been tempted to use forEach and mutation: func combos<A, B>(_ xs: [A], _ ys: [B]) -> [(A, B)] { var result: [(A, B)] = [] xs.forEach { x in ys.forEach { y in result.append((x, y)) } } return result }

— 3:26

However, we don’t typically like this style because of the amount of indirection. We create a mutable result array up top, and then mutate it inside a deep nesting of forEach ’s. Those two things are intimately related yet they are separated far apart.

— 3:44

Instead it would be better to use map , which allows us to perform transformations in a single unit: func combos<A, B>(_ xs: [A], _ ys: [B]) -> [(A, B)] { return xs.map { x in ys.map { y in (x, y) } } } ‘map’ produces ‘[T]’, not the expected contextual result type ‘[(A, B)]’

— 4:04

The code looks much nicer, but unfortunately it’s incorrect. By nesting two map operations we are getting a nested array of tuples, [[(A, B)]] : func combos<A, B>(_ xs: [A], _ ys: [B]) -> [(A, B)] { let tmp: [[(A, B)]] = xs.map { x in ys.map { y in (x, y) } } fatalError() }

— 4:39

Turns out, it is completely impossible to implement this function using only map . We just can’t do it. We will always end up with a nested structure.

— 4:50

But we also have the zip function, can that be used to define combos ? Well, zip already has the signature of combos , but it returns the wrong thing. It simultaneously walks both arrays and pairs off the elements together: zip([1, 2, 3], ["a", "b"]) // [(1, "a"), (2, "b")] // [(1, "a"), (2, "b") Notice that it needs to truncate the 3 since there’s no corresponding value in the second array. The resulting array is missing a lot of values that we would want in our combos array, so even zip doesn’t help us.

— 5:27

Let’s look at another example. Say we have a string of data that we’re loading from a file that holds a bunch of comma-delimited numbers that can run over many lines: let scores = """ 1,2,3,4 5,6 7,8,9 """

— 5:41

We want to parse every number from this string into a Swift array. Before we knew about map and zip and other functional operators, we may have written this code using forEach . var allScores: [Int] = [] scores.split(separator: "\n").forEach { line in line.split(separator: "," ).forEach { value in allScores.append(Int(value) ?? 0) } } allScores // [1, 2, 3, 4, 5, 6, 7, 8, 9]

— 6:16

Again we are resorting to mutation and forEach to get the job done, but there must be a better way.

— 6:21

However, map does not help us here: scores.split(separator: "\n").map { line in line.split(separator: "," ).map { value in Int(value) ?? 0 } } // [[1, 2, 3, 4], [5, 6], [7, 8, 9]]

— 6:36

We get a nested structure instead of a flat one.

— 6:46

This phenomenon isn’t only inherent to arrays. We can do the same with optionals.

— 6:51

Suppose we had an array of strings and we wanted to extract the first value from the array: let strings = ["42", "Blob", "functions"] strings.first // "42"

— 6:58

If you didn’t know this already, Swift has a lot of compiler magic when it comes to optionals, and in this case the playground is hiding that this value is actually optional. type(of: strings.first) // Optional<String>.Type

— 7:16

It is optional because the array could be empty and in that case there would be no string to return. Now say we wanted to perform a transformation on this string: we wanted to convert it into an integer. We’ve previously seen that map on optionals is a wonderful operation for safely unwrapping an optional and applying a transformation to the value on the inside, so that sounds exactly like what we want here: strings.first.map(Int.init) // 42

— 7:41

Now again Swift is hiding something here, and it’s hiding even more. Not only is this value optional, it’s doubly optional: type(of: strings.first.map(Int.init)) // Optional<Optional<String>>.Type

— 7:54

This is because the Int initializer is failable, since not all strings can be converted into integers, and hence returns an optional, and that optional gets packaged up in yet another optional since strings.first is optional. To make this crystal clear let’s recall that map on optionals is a way of lifting functions to work on optionals: ((A) -> B) -> (A?) -> B?

— 8:20

In our situation we have A = String and B = Int? so this signature becomes: Int.init: (String) -> Int? A = String B = Int? ((String) -> Int?) -> (String?) -> Int?? A function that takes functions on strings to optional integers, and lifts it to functions on optional strings to optional, optional integers.

— 8:42

This is why we have a double optional.

— 8:44

Again we have a nested structure where we didn’t want one. We don’t want the nesting because not only does it makes the value harder to use since we now have to dive two layers deep to get to it, any additional transformations will make this nesting even deeper.

— 8:58

It becomes really annoying to get at the value inside this double nested optional.

— 9:01

We can use if let , but we need to run it multiple times: once to unwrap the outer optional, and again to unwrap the inner. if let x = strings.first.map(Int.init), let y = x { type(of: y) // Int.Type }

— 9:12

We could also use if case let and explicitly pattern match into the nesting. if case let .some(.some(x)) = strings.first.map(Int.init) { type(of: x) // Int.Type }

— 9:25

We could even use a ?? shorthand syntax. if case let x?? = strings.first.map(Int.init) { type(of: x) // Int.Type }

— 9:25

Finally, we could use a switch statement to really account for all possibilities. switch strings.first.map(Int.init) { case let .some(.some(value)): print(value) case .some(.none): print(".some(.none)") case .none: print(".none") } Nesting problems on other types

— 9:44

But even worse, this nesting problem shows up in all situations where a generic container and a map operation is defined.

— 10:00

For example, in our “many faces of map” episode we defined map on a wide variety of types. That episode was a long time ago, and we encourage everyone to watch or rewatch it, but we’ll give a quick recap now.

— 10:13

First we looked at the Result type that is popular in the Swift community, and even soon to be a part of the standard library. It supports a map operation that allows you to transform the success value of a result: enum Result<A, E> { case success(A) case failure(E) func map(_ f: @escaping (A) -> B) -> Result<B, E> { switch self { case let .success(a): return .success(f(a)) case let .failure(e): return .failure(e) } } }

— 10:33

We can transform an existing success ful value using map . Result<Double, String>.success(42.0) .map { $0 + 1 } // .success(43.0)

— 10:41

However, if you map with a transformation that returns another result you are back in nested container territory. For example, let’s take a look at a function that runs a failable computation and packages up the return value in a Result : func compute(_ a: Double, _ b: Double) -> Result<Double, String> { guard a >= 0 else { return .failure("First argument must be non-negative.") } guard b != 0 else { return .failure("Second argument must be non-zero.") } return .success(sqrt(a) / b) }

— 11:06

We get failures for invalid inputs and successes for valid ones. compute(-1, 1729) // .failure("First argument must be non-negative.") compute(42, 0) // .failure("Second argument must be non-zero.") compute(42, 1729) // .success(0.0037482595132491965)

— 11:36

But what if we wanted to take the result of this computation and pipe it into another function that returns a result? compute(42, 1729) .map { compute($0, $0) } // .success(.success(16.333722542165852))

— 11:54

The playground makes it a little tough to see what we’re dealing with, so let’s print out the type of value. print( type( of: compute(42, 1729) .map { compute($0, $0) } ) ) // Result<Result<Double, String>, String>.Type

— 12:20

If we want to get at the value inside this nested structure we have to do a pretty complicated switch : switch compute(42, 1729).map({ compute($0, $0) }) { case let .success(.success(value)): print(value) case let .success(.failure(error)): print(error) case let .failure(error): print(error) }

— 12:50

We’ve also discussed a closely related value to Result called Validated , although we didn’t talk about it on our map episode. It represents a value that can fail in multiple ways. It’s not important to know all of the details of Validated for what we are talking about here, so we’ll paste in its definition: import NonEmpty enum Validated<A, E> { case valid(A) case invalid(NonEmptyArray<E>) func map(_ f: @escaping (A) -> B) -> Validated<B, E> { switch self { case let .valid(a): return .valid(f(a)) case let .invalid(e): return .invalid(e) } } }

— 13:14

The Validated type was most interesting to us because it supported a zip -like operation that was very powerful: it allowed us to accumulate multiple errors instead of just bailing on the first error like Swift’s throw API does. But, this type still has all of the same nesting problems that Result has: when we use map with transformations that validate, we end up with a nested validation.

— 13:37

We also defined map on functions. struct Func<A, B> { let run: (A) -> B func map<C>(_ f: @escaping (B) -> C) -> Func<A, C> { return Func<A, C> { a in f(self.run(a)) } } }

— 14:12

We saw that map on functions is the same as composing a new function onto an existing one: given a Func<A, B> mapping a function (B) -> C , we get a brand new Func<A, C> .

— 14:16

Our viewers may remember from our very first episode that we defined an operator for composition: >>> . This means that map on functions can be defined thusly: func map<C>(_ f: @escaping (B) -> C) -> Func<A, C> { // return Func<A, C> { a in // f(self.run(a)) // } return Func<A, C>(run: self.run >>> f) }

— 14:40

A simple example we considered was wrapping up a network request in one of these values, which effectively makes it a lazy value: let randomNumber = Func<Void, Int> { let number = try! String( contentsOf: URL( string: "https://www.random.org/integers/?num=1&min=1&max=235866&col=1&base=10&format=plain&rnd=new" )! ) .trimmingCharacters(in: .newlines) return Int(number)! }

— 14:58

And we can use the map operation to transform the result of a Func into a new Func lazily. randomNumber.map { $0 + 1 } // Func<(), Int>

— 15:15

To get a value from a Func , we must explicitly call run on it. randomNumber .map { $0 + 1 } .run(()) // 68405

— 15:31

But what if we wanted to chain the result of one of these lazy values onto the result of another? let words = Func<Void, [String]> { ( try! String( contentsOf: URL( fileURLWithPath: "/usr/share/dict/words" ) ) ) .split(separator: "\n") .map(String.init) }

— 15:57

Here we have another lazy value, representing a list of words. What if we wanted to take our random number and use it to retrieve a random word from our dictionary? randomNumber.map { number in words.map { words in words[number] } } // Func<(), Func<(), Substring>>

— 16:24

When we use map on our random number, and then further map onto our list of words, we end up with another nesting problem: we have a Func that nests another Func , and to get to our random word we need to call run twice: once to unwrap each nested Func . randomNumber.map { number in words.map { words in words[number] } }.run(()).run(()) // "unscreened"

— 16:50

Finally, we considered a type that at the time we gave an abstract name, but since then have renamed it to properly fit its semantics. It’s called Parallel , and it represents values that can be computed asynchronously: struct Parallel<A> { let run: (@escaping (A) -> Void) -> Void func map(_ f: @escaping (A) -> B) -> Parallel { return Parallel { callback in self.run { a in callback(f(a)) } } } }

— 17:15

This type is a concrete realization of callbacks we use in many places, like UIView animations and URLSession completions. All of those APIs could return a Parallel value instead of taking a callback and it would work essentially the same. Except it’s a little more composable because we have this map function on it.

— 17:40

And we can give it a spin by introducing this delay helper that we previously talked about. It’s just a simple function that gives you a parallel value that executes after some delay: func delay( by duration: TimeInterval, line: UInt = #line ) -> Parallel<Void> { return Parallel { callback in print("Delaying line \(line) by \(duration)") DispatchQueue.main.asyncAfter(deadline: .now() + duration) { callback(()) print("Executed line \(line)") } } } delay(by: 1).run { print("Executed after 1 second") } delay(by: 2).run { print("Executed after 2 seconds") } // Delaying line 212 by 1.0 // Delaying line 213 by 2.0 // Executed after 1 second // Executed line 212 // Executed after 2 seconds // Executed line 213

— 18:05

With this helper value we can easily create a delayed integer by mapping on the delayed void value: let aDelayedInt = delay(by: 3).map { 42 }

— 18:32

And then we get the delayed integer by running the parallel value: aDelayedInt.run { value in print("We got \(value)") } // Delaying line 215 by 3.0 // We got 42 // Executed line 215

— 18:51

This is a pretty silly example, but instead of delaying you could imagine we are making a network request for some data. The important idea here is that Parallel values are perfectly suited for when data is provided in an asynchronous manner.

— 19:03

Now the problem comes when we want to sequence two of these parallel values. For example, say we want to delay producing an integer like above, but then after that we want to delay getting another integer and do some computation with them: aDelayedInt.map { value in delayed(by: 1).map { value + 1729 } } // Parallel<Parallel<Int>>

— 19:33

We now have a nested parallel value. In order to get at the inner integer it is our job to manage running all of the parallels: aDelayedInt.map { value in delayed(by: 1).map { value + 1729 } }.run { innerParallel in innerParallel.run { value in print("We got \(value)") } } // Delaying line 215 by 3.0 // Delaying line 223 by 1.0 // Executed line 215 // We got 1771 // Executed line 223

— 20:12

Again we are seeing it can be really annoying to have to deal with this nesting, and it happens again and again. There is just currently no way to avoid it when all we have at our disposal is map and zip .

— 20:21

We’ve now shown that there is a pretty substantial problem that appears often and naturally when dealing with generic containers that have a map operation, and that is they tend to nest and become unwieldy to work with. So surely there must be a solution to this problem. Swift’s flat‑maps

— 20:23

Luckily there’s a fix, and Swift even ships with the solution for arrays and optionals: it’s flatMap ! This is an operation that is pretty similar to map , but has some extra functionality for further flattening the nesting that occurs when doing map s. That is indeed why it is called flatMap , it flattens the result of the map .

— 20:50

Many of our viewers may already be familiar with flatMap , but for the sake of completeness let’s give a quick tour of what Swift gives us.

— 21:04

The very first version of Swift shipped with a flatMap on sequences. extension Sequence { public func flatMap<SegmentOfResult>( _ transform: (Self.Element) throws -> SegmentOfResult ) rethrows -> [SegmentOfResult.Element] where SegmentOfResult : Sequence { } }

— 21:09

There’s a lot going on, but roughly it says that if you provide a transformation that goes from the current sequence’s elements to a new sequence called SegmentOfResult , then you will get back an array of the new sequence’s elements. There’s also some throws and rethrows stuff in there to make it play nicely with Swift’s error handling. This signature is kind of complicated, but that’s mostly due to Swift’s lack of certain type features that prevent it from properly expressing the signature. For example, ideally we’d be able to return a sequence of the same type as the one we are flatMap ping on, but that is not possible.

— 21:47

We can use this flatMap to fix some of the nesting problems we previously encountered when dealing with arrays. For example, the combos function is now simply: func combos<A, B>(_ xs: [A], _ ys: [B]) -> [(A, B)] { return xs.flatMap { x in ys.map { y in (x, y) } } } Very simple. By swapping our outer map with a flatMap , we’re able to return a non-nested array. combos([1, 2, 3], ["a", "b"]) // [(1, "a"), (1, "b"), (2, "a"), (2, "b"), (3, "a"), (3, "b")]

— 22:36

Also, the code that tried parsing a list of numbers from a string becomes: scores.split(separator: "\n").flatMap { line in line.split(separator: "," ).map { value in Int($0) ?? 0 } } // [1, 2, 3, 4, 5, 6, 7, 8, 9]

— 23:03

Both of these code snippets are similar: we map on the inside to transform some data into an array, and then flatMap on the outside so that we can flatten the doubly nested array. The flatMap operation on arrays is precisely what is needed to solve these problems.

— 23:14

Now I believe Swift had flatMap on sequences since day one, but it was a later release, I believe Swift 2, that we got flatMap on optionals: extension Optional { public func flatMap( _ transform: (Wrapped) throws -> U? ) rethrows -> U? } Again this has a bit of extra noise with the throws / rethrows stuff which is necessary to play nicely with Swift error handling.

— 23:47

We can use this flatMap to solve our nesting issues with optionals just like we used it for arrays: strings.first.flatMap(Int.init) // 42 type(of: strings.first.flatMap(Int.init)) // Optional<String>.Type

— 24:00

Now we have just an Optional<Int> instead of an Optional<Optional<Int>> . Again flatMap has solved our nesting problem. Till next time

— 24:05

We now know that flatMap is an important operation that solves a common problem. Important enough for Swift to provide this operation for arrays and optionals in the standard library.

— 24:20

However, all of the types we define for our own problems and applications, the ones that the Swift standard library knows nothing about, all have this nesting problem. The Result , Validated , Func , Parallel type and more. Can we define flatMap on them?

— 24:33

The answer is “yes we can and should be!” And as you define flatMap it uncovers interesting semantics of your types. It also will show interesting connections with zip , and that is what clarifies the purpose of the type. For example, why would you use Result over Validated ? Why would you use Func over Parallel ? It turns out that the way flatMap and zip relate to each other uncovers all of that.

— 25:02

So let’s define flatMap on our own types and see what it teaches us! References Railway Oriented Programming — error handling in functional languages Scott Wlaschin • Jun 4, 2014 This talk explains a nice metaphor to understand how flatMap unlocks stateless error handling. Note When you build real world applications, you are not always on the “happy path”. You must deal with validation, logging, network and service errors, and other annoyances. How do you manage all this within a functional paradigm, when you can’t use exceptions, or do early returns, and when you have no stateful data? This talk will demonstrate a common approach to this challenge, using a fun and easy-to-understand “railway oriented programming” analogy. You’ll come away with insight into a powerful technique that handles errors in an elegant way using a simple, self-documenting design. https://vimeo.com/97344498 A Tale of Two Flat‑Maps Brandon Williams & Stephen Celis • Mar 27, 2018 Up until Swift 4.1 there was an additional flatMap on sequences that we did not consider in this episode, but that’s because it doesn’t act quite like the normal flatMap . Swift ended up deprecating the overload, and we discuss why this happened in a previous episode: Note Swift 4.1 deprecated and renamed a particular overload of flatMap . What made this flatMap different from the others? We’ll explore this and how understanding that difference helps us explore generalizations of the operation to other structures and derive new, useful code! https://www.pointfree.co/episodes/ep10-a-tale-of-two-flat-maps Downloads Sample code 0042-the-many-faces-of-flatmap-pt1 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .