Video #43: The Many Faces of Flat‑Map: Part 2

Episode: Video #43 Date: Jan 16, 2019 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep43-the-many-faces-of-flat-map-part-2

Description

Now that we know that flatMap is important for flattening nested arrays and optionals, we should feel empowered to define it on our own types. This leads us to understanding its structure more in depth and how it’s different from map and zip.

Video

Cloudflare Stream video ID: 6aad96863c0fa1a4351562b1199d89dc Local file: video_43_the-many-faces-of-flat-map-part-2.mp4 *(download with --video 43)*

References

Transcript

— 0:05

We now know that flatMap is an important operation that solves a common problem. Important enough for Swift to provide this operation for arrays and optionals in the standard library.

— 0:20

However, all of the types we define for our own problems and applications, the ones that the Swift standard library knows nothing about, all have this nesting problem. The Result , Validated , Func , Parallel type and more. Can we define flatMap on them?

— 0:33

The answer is “yes we can and should be!” And as you define flatMap it uncovers interesting semantics of your types. It also will show interesting connections with zip , and that is what clarifies the purpose of the type. For example, why would you use Result over Validated ? Why would you use Func over Parallel ? It turns out that the way flatMap and zip relate to each other uncovers all of that.

— 1:02

So let’s define flatMap on our own types and see what it teaches us! Flat‑map on Array and Optional

— 1:08

Let’s get our feet wet by redefining flatMap on arrays even though the standard library gives us this operation, just to show how easy it is.

— 1:17

First we’ll take another look at how flatMap is actually defined in the standard library. extension Sequence { func flatMap<SegmentOfResult>( _ transform: (Self.Element) throws -> SegmentOfResult ) rethrows -> [SegmentOfResult.Element] where SegmentOfResult: Sequence }

— 1:24

There’s a lot going on in this signature. First: it’s operating on the Sequence protocol, not just on arrays. It flat-maps into another SegmentOfResult sequence. And we have a bunch of throws and rethrows noise to play nicely with Swift error handling.

— 1:47

Let’s greatly simplify this by defining flatMap directly on arrays. extension Array { func flatMap<NewElement>( _ f: (Element) -> [NewElement] ) -> [NewElement] { var result: [NewElement] = [] for element in self { result.append(contentsOf: f(element)) } return result } } We’re now flat-mapping into another array, rather than a sequence of any type, and we’ve removed throws and rethrows .

— 3:10

This function demonstrates how simple flatMap really is, and if Swift hadn’t shipped it in the standard library we really could have defined it ourselves.

— 3:17

Notice that we are using append(contentsOf:) instead of the plain append(_:) that we would have used if we were defining map . That is the key to performing the transformation and flattening operation at the same time.

— 3:32

Since that was so easily, let’s do the same for optionals: extension Optional { public func flatMap(_ transform: (Wrapped) throws -> U?) } We already have the signature from the standard library handy, so let’s flesh it out. extension Optional { func flatMap(_ transform: (Wrapped) -> U?) -> U? { switch self { case let .some(value): return transform(value) case .none: return Optional.none } } } We can slightly simplify the signature by getting rid of throws and rethrows , and in the body we can switch on the optional in order to unwrap a value and apply our transform function.

— 4:47

It was quite simple to define this flatMap as well! If Swift hadn’t given it to us, we could have very easily come across this implementation.

— 5:00

Notice that we did not write transform(value) in a .some case, as we would have if we were defining map . Again this is the key to simultaneously transforming and flattening the optional. Flat‑map on Result and Validated

— 5:19

Now that we are warmed up, let’s start with the Result type. Soon this type is going to be in the Swift standard library, and it will come with the flatMap operation. That means that Swift 5 will actually have 3 types with a flatMap operation defined, which really makes it start to seem like this is a very general pattern that many types can adopt.

— 5:43

But, since we aren’t yet on Swift 5, let’s define flatMap from scratch. We already have Result defined from earlier in the episode, but let’s bring it down here so that we can look at it with fresh eyes: enum Result<A, E> { case success(A) case failure(E) func map(_ f: @escaping (A) -> B) -> Result<B, E> { switch self { case let .success(a): return .success(f(a)) case let .failure(e): return .failure(e) } } }

— 5:51

To define flatMap on this we have to implement a method with a specific signature. In order to drive home the fact that this flatMap is closely related to all of the flatMap ‘s we have seen so far, I’m going to copy and paste the one from Optional that we just implemented above to see what needs to be changed: func flatMap(_ transform: (Wrapped) -> U) -> U? { switch self { case let .some(wrapped): return transform(wrapped) case .none: return Optional.none } }

— 6:13

First, instead of transforming into optionals we are using results, so let’s fix that: func flatMap(_ transform: (A) -> Result<U, E>) -> Result<U, E> { switch self { case let .some(wrapped): return transform(wrapped) case .none: return Optional.none } } First we can rename Wrapped to A to match our Result type’s generic parameter, and we can update our optionals to be results.

— 6:39

Take note that the E generic doesn’t get transformed at all, it just comes along for the ride.

— 6:46

The U generic seems a little out of place now, so let’s update it to B . func flatMap(_ transform: (A) -> Result<B, E>) -> Result<B, E> { switch self { case let .some(wrapped): return transform(wrapped) case .none: return Optional.none } } And now we can see that given a Result<A, E> , I can transform it into a Result<B, E> via a transformation from A to Result<B, E> .

— 7:03

Now we just need to update the body to work with Result ’s cases. func flatMap(_ transform: (A) -> Result<B, E>) -> Result<B, E> { switch self { case let .success(value): return transform(value) case let .failure(error): return .failure(error) } } The some becomes success and we can rename wrapped to value since we’re not dealing with the optional generic name anymore. And the none s becomes failure s where we unwrap the error and pass it along.

— 7:46

And this is all there is to implementing flatMap on Result . In the success case we just transform the value. And in the failure case we just pass the error along.

— 7:55

Maybe the best part of this implementation is how directly it followed the implementation on optionals. One way to think of this is that Optional is subset or specialized version of Result : where Optional allows you to express the idea of a value that’s present or not, Result allows you to express the idea of a value that’s present or an error message in its absence. So the implementation was mostly copying, pasting, and renaming.

— 8:27

To take this for a spin, remember we had this compute function we were playing with. It did a computation, but it could fail if you provided bad data to it. We wanted to invoke this function with some values, and then take the result of that to compute with the function again. When we used map we were led to a nested Result , which was difficult to work with.

— 8:45

Now we can just flatMap : compute(42, 1729) .flatMap { compute($0, $0) } // .success(16.333722542165852)

— 9:04

And we can chain on any number of these and we never get into the nested situation again: compute(42, 1729) .flatMap { compute($0, $0) } .flatMap { compute($0, $0) } .flatMap { compute($0, $0) } // success(2.0103482375314283) So we’ve now solved the nesting problem for Result . Anytime you need to transform a result via a function that also returns a result, you must use flatMap . It allows you to chain together multiple functions that can potentially return a failure.

— 9:21

Closely related to the Result type, and first discussed during our series of episodes on zip , is the Validated type. It’s like a result that can fail in multiple ways: import NonEmpty enum Validated<A, E> { case valid(A) case invalid(NonEmptyArray<E>) func map(_ f: @escaping (A) -> B) -> Validated<B, E> { switch self { case let .valid(a): return .valid(f(a)) case let .invalid(e): return .invalid(e) } } }

— 9:38

Defining flatMap on this type is almost identical to what we did for Result . In fact, I can copy and paste that implementation and make even fewer changes than were made between Optional and Result : func flatMap( _ transform: (A) -> Validated<B, E> ) -> Validated<B, E> { switch self { case let .valid(value): return transform(value) case let .invalid(error): return .invalid(error) } }

— 10:07

Again this is precisely the tool we need to flatten nested Validated values, and it is the thing we can use to chain together multiple functions that can potentially return invalid data. Just like flatMap on Result allowed us to chain multiple operations that could fail. And just like flatMap on Optional allowed us to chain multiple operations that could fail to nil . Flat‑map on Func and Parallel

— 10:31

We still have a few more generic containers to update. So let’s see how this chaining looks for them.

— 10:41

Can we define flatMap on our Func type, which was just a simple struct wrapper around functions from (A) -> B ? struct Func<A, B> { let run: (A) -> B func map<C>(_ f: @escaping (B) -> C) -> Func<A, C> { // return Func<A, C> { a in // f(self.run(a)) // } return Func<A, C>(run: self.run >>> f) } }

— 10:48

Let’s write out the signature of this flatMap from scratch because its generics work a little differently than our earlier examples: func flatMap<C>(_ f: @escaping (B) -> Func<A, C>) -> Func<A, C> { }

— 11:19

Now what can we do in here? Well we can start by just creating a stub of the return value we know we need to return: func flatMap<C>(_ f: @escaping (B) -> Func<A, C>) -> Func<A, C> { return Func<A, C> { a in } }

— 11:29

So somehow we need to turn that A into a C . Well, we can feed it into self.run , which turns a B , and that can be fed into f , which returns a Func<A, C> , and that can also be run with the a value: func flatMap<C>(_ f: @escaping (B) -> Func<A, C>) -> Func<A, C> { return Func<A, C> { a -> C in let b = self.run(a) let funcAC = f(b) return funcAC.run(a) } }

— 12:05

And that’s all we need to do to implement flatMap . We can clean things up a bit by inlining everything. func flatMap<C>(_ f: @escaping (B) -> Func<A, C>) -> Func<A, C> { return Func<A, C> { a in return f(self.run(a)).run(a) } }

— 12:12

And now with everything on one line it’s easier to see that calling run twice is precisely what gets us out of that nesting problem we saw earlier.

— 12:21

It looks pretty strange, but those two runs are exactly what we had to do earlier when we had a nested Func that appeared from loading a random number and the local dictionary in our usr directory: randomNumber.map { number in words.map { words in words[number] } } .run(()) .run(())

— 12:30

To fix this, we can replace our outer map with a flatMap and only call run once. randomNumber.flatMap { number in words.map { words in words[number] } } .run(())

— 12:38

And so now we’ve solved the nesting problem for functions and lazy values. You can chain together multiple lazy values in a synchronous manner by using flatMap .

— 12:49

Finally, we are left with parallel, the last of the types that we were able to define a map and zip operation on. struct Parallel<A> { let run: (@escaping (A) -> Void) -> Void func map(_ f: @escaping (A) -> B) -> Parallel { return Parallel { callback in self.run { a in callback(f(a)) } } } } Does it support a flatMap too? Well of course it does, let’s get the signature in place. func flatMap(_ f: @escaping (A) -> Parallel) -> Parallel { return Parallel { callback in } }

— 13:27

So technically this is already compiling, but it certainly isn’t right. It’s just returning a parallel value that never executes. We need to do something in this block.

— 13:37

Well, one of the only things we have that is immediately executable is self.run , which takes a callback that is invoked with a value in A once it is ready. So let’s start there. func flatMap(_ f: @escaping (A) -> Parallel) -> Parallel { return Parallel { callback in self.run { a in } } }

— 13:53

Now that we have a value in a we can apply our function f , and that gives us a new parallel value. func flatMap(_ f: @escaping (A) -> Parallel) -> Parallel { return Parallel { callback in self.run { a in let parallelB = f(a) } } }

— 14:00

To get access to that B value, we need to run that new parallel value as well. func flatMap(_ f: @escaping (A) -> Parallel) -> Parallel { return Parallel { callback in self.run { a in let parallelB = f(a) parallelB.run { b in } } } }

— 14:07

We’ve now gotten our B value, which is exactly what we can feed into the callback that defines the parallel value we are trying to return: func flatMap(_ f: @escaping (A) -> Parallel) -> Parallel { return Parallel { callback in self.run { a in let parallelB = f(a) parallelB.run { b in callback(b) } } } }

— 14:12

And we could simplify some of this by going point-free, which means to eliminate the usage of variables and instead rely on function composition: func flatMap(_ f: @escaping (A) -> Parallel) -> Parallel { return Parallel { callback in self.run { a in f(a).run(callback) } } }

— 14:27

And as we saw with Func , we’re able to flatten our nesting problem by calling run twice.

— 14:37

This is a lot to grok at once, but it is simply saying that to flatMap one parallel value onto another we just run the first parallel, and when it finishes we run the second parallel. So we are sequencing the two parallels together.

— 15:01

And this flatMap is what allows us to flatten the transformation we considered earlier, where we had to call run twice to unwrap both parallels: aDelayedInt.map { value in delay(by: 1).map { value + 1729 } } .run { innerParallel in innerParallel.run { value in print("We got \(value)") } }

— 15:11

Now we can simply call flatMap on our outer parallel, and get rid of the nested calls to run . aDelayedInt.flatMap { value in delay(by: 1).map { value + 1729 } } .run { value in print("We got \(value)") }

— 15:23

And this flatMap , just like the others, can be chained along and along without ever increasing the nesting. aDelayedInt .flatMap { int in delayed(by: 1).map { int + 1729 } } .flatMap { int in delayed(by: 1).map { int + 1729 } } .flatMap { int in delayed(by: 1).map { int + 1729 } } .flatMap { int in delayed(by: 1).map { int + 1729 } } .run { value in print("We got \(value)") }

— 15:54

We’ve now defined flatMap on Func and Parallel and both times it again solves the nesting problem. It’s not even a surprise this time around. But what is surprising is that in trying to solve this nesting problem, the solution related the problem to the idea of chaining things along. On Array it chained together operations that return arrays. On Optional it chained together operations that may fail into nil . On Result it chained together operations that may fail with an explicit error. On Validated it chained together operations that may fail validation with several errors. On Func it chained together lazy operations. And on Parallel it chained together operations that run via callback.

— 16:52

That’s basically flatMap in a nutshell: First, it’s a precise tool for flattening structures that naturally occur when we are performing mapping operations. Second, and most interesting, is that by solving the nesting problem we were naturally led to ability for us to sequence our values together. The structure of flat‑map

— 17:04

There’s something really wonderful lurking in the shadows here, but if all we knew about flatMap was what came with the standard library, we would be missing out on it entirely. It all boils down to the shape of flatMap . Let’s really get at that structure and dissect it.

— 17:24

Let’s go back to the flatMap on arrays. As we saw before, the signature of flatMap in the standard library is pretty messy, because it operates on the Sequence protocol, returns a completely different type of sequence, and has a bunch of throws and rethrows stuff.

— 17:37

Let’s clear away some of that clutter and focus on the signature as it pertains to arrays. flatMap: ([A], (A) -> [B]) -> [B]

— 17:57

I’ve written this signature in a different format so that we can concentrate more on its shape, but it’s identical to the method signature. You can think of it as a function with two arguments: the array you are transforming ( self ) and the transformation function. And then it returns a new array that is the result of mapping the given array and then flattening.

— 18:11

Now this is certainly clearer to understand than the mess of a signature that we showed for flatMap from the standard library, but it can be made even nicer. If you’ve been watching our series for awhile you may remember this trick from our episode on higher-order functions . In that episode we claimed that functions can be made more composable by putting their configuration arguments first and the data to be operated on last. This is because you are more likely to have the configuration early on, and so can partially apply the function with just those arguments, and then you are left with something that just operates on data.

— 19:00

So, let’s apply that idea to this signature by flipping the arguments since the transformation is kind of like configuration: flatMap: ((A) -> [B], [A]) -> [B]

— 19:13

And finally let’s curry this signature so that we can partially apply it: flatMap: ((A) -> [B]) -> (([A]) -> [B])

— 19:24

Interesting. From this perspective it’s kind of like flatMap is a means to lift functions that return arrays up to functions that operate on arrays.

— 19:42

We’ve previously used “lift” as a way to describe the map operation too. We said that map allows us to lift functions up to functions on arrays. Both of those descriptions sound very similar, so let’s put their signatures side-by-side: map: ((A) -> B ) -> (([A]) -> [B]) flatMap: ((A) -> [B]) -> (([A]) -> [B])

— 19:57

Wow ok, very similar signatures. In fact, the only difference is that flatMap works with transformation functions that go into an array, not just any type.

— 20:18

What about zip ? We’ve also used that “lifting” phrase to describe what zip does: it allows you to life functions that take multiple arguments up to functions on arrays. map: ((A) -> C ) -> (([A]) -> [C]) zip(with:): ((A, B) -> C ) -> (([A], [B]) -> [C]) flatMap: ((A) -> [C]) -> (([A]) -> [C])

— 20:46

So we see that map is simply a way to lift functions up to the world of arrays. And then zip generalizes that to let you lift functions that take multiple arguments up to functions between arrays. And then flatMap allows you to further generalize that to be able to lift functions that map into arrays.

— 21:16

But the amazing thing we have uncovered after many, many episodes is that all of these operations are defined on so many more types than just arrays. In fact, arrays are the only types for which Swift defines all 3 operations, so there’s a lot we are missing out on by using only what Swift gives us.

— 21:34

If we copy-and-paste these signatures and make a few small edits, we immediately uncover map , zip , and flatMap for optionals: map: ((A) -> C ) -> ((A?) -> C?) zip(with:): ((A, B) -> C ) -> ((A?, B?) -> C?) flatMap: ((A) -> C?) -> ((A?) -> C?)

— 21:45

So with just a small change we see the structure of these operations on optionals, and what’s interesting is that while Optional comes with map and flatMap , it does not come with zip , but we can see that it indeed exists.

— 22:02

But then I can also swap out the optionals for results and get: map: ((A) -> C ) -> ((Result<A, E>) -> Result<C, E>) zip(with:): ((A, B) -> C ) -> ((Result<A, E>, Result<B, E>) -> Result<C, E>) flatMap: ((A) -> Result<C, E>) -> ((Result<A, E>) -> Result<C, E>)

— 22:30

And if we can do it for results then certainly we can do it for validated values: map: ((A) -> C ) -> ((Validated<A, E>) -> Validated<C, E>) zip(with:): ((A, B) -> C ) -> ((Validated<A, E>, Validated<B, E>) -> Validated<C, E>) flatMap: ((A) -> Validated<C, E>) -> ((Validated<A, E>) -> Validated<C, E>)

— 22:46

And how about Func : map: ((B) -> D ) -> ((Func<A, B>) -> Func<A, D>) zip(with:): ((B, C) -> D ) -> ((Func<A, B>, Func<A, C>) -> Func<A, D>) flatMap: ((B) -> Func<A, D>) -> ((Func<A, B>) -> Func<A, D>)

— 22:53

And finally Parallel : map: ((A) -> C ) -> ((Parallel<A>) -> Parallel<C>) zip(with:): ((A, B) -> C ) -> ((Parallel<A>, Parallel) -> Parallel<C>) flatMap: ((A) -> Parallel<C>) -> ((Parallel<A>) -> Parallel<C>)

— 23:04

All of these signatures are basically the same thing if we squint. We have three different ways to lift functions up to work with a specific context. Whether the context is arrays, optionals, results, validated values, functions, and lazy values, these are the ways we can purely transform the underlying value of a context, take many values of a context and zip them together, or even take transformations that produce these contextual values and chain them along.

— 23:34

We can speak of these signatures even more abstractly by just replacing all of the various generic containers with an F : map: ((A) -> C ) -> ((F<A>) -> F<C>) zip(with:): ((A, B) -> C ) -> ((F<A>, F) -> F<C>) flatMap: ((A) -> F<C>) -> ((F<A>) -> F<C>)

— 23:49

All of our previous examples are just specializations of these three signatures. Just substitute your favorite generic container in for F and you immediately get the signatures for map , zip , and flatMap .

— 24:11

For example: F<A> = Array<A> F<A> = Optional<A> F<A> = Result<A, E> F<A> = Validated<A, E> F<A> = Func<A0, A> F<A> = Parallel<A>

— 24:21

Each of these operations accomplish a very specific and very important task.

— 24:27

map allows you to purely transform the underlying type of your generic container while leaving the structure fixed. So, for example, you can transform an array of integers into an array of strings, but only using a pure function that maps ints to strings, it can’t do anything else.

— 24:45

zip allows you to combine multiple generic container values into a single generic container, but there are no dependencies between the values in the containers. We can only purely transform an A and a B into a C . This is sometimes known as a “context independent” transformation since we can only use A and B in this transformation; we do not get access to F<A> or F at all.

— 24:52

flatMap , on the other hand, allows you to sequence your generic containers, and this is a dependent operation since to get from one step to the next you must have access to the previous value. This is a power that map and zip don’t have: we can transform a context by unwrapping its value and return a brand new context.

— 25:22

And this is a powerhouse of a trio of operations. They each do something very precise, and you can intuitively understand what they do regardless of the type you are operating on as long as you adhere to its structure. The signature needs to be exactly the same, regardless of the type you’re operating on. If you have a type that comes with these operations, you immediately know that map purely transforms the underlying value, zip allows you to combine multiple values into a single one, and flatMap allows you to chain together multiple values along a function.

— 26:17

It’s really wonderful to see how this trio of operations work together to provide a nice interface for dealing with all of these different generic types. What’s the point?

— 26:27

We’ve gone from a bunch of more specific types that we originally introduced in a quite abstract way in our episode on map and gone straight back into abstraction again. Given any generic container we can now intuitively know what it means if it has map or zip or flatMap . But let’s bring it back to earth. We usually ask “what’s the point?” after diving deep into a topic, so let’s resurface and explore why it’s important to know if map or zip or flatMap exists on a structure. What does it buy us in the real world?

— 27:12

We’ll explore that next time! References SE-0235: Add Result to the Standard Library Nov 7, 2018 The Swift evolution review of the proposal to add a Result type to the standard library. It discussed many functional facets of the Result type, including which operators to include (including map and flatMap ), and how they should be defined. https://forums.swift.org/t/se-0235-add-result-to-the-standard-library/17752 Railway Oriented Programming — error handling in functional languages Scott Wlaschin • Jun 4, 2014 This talk explains a nice metaphor to understand how flatMap unlocks stateless error handling. Note When you build real world applications, you are not always on the “happy path”. You must deal with validation, logging, network and service errors, and other annoyances. How do you manage all this within a functional paradigm, when you can’t use exceptions, or do early returns, and when you have no stateful data? This talk will demonstrate a common approach to this challenge, using a fun and easy-to-understand “railway oriented programming” analogy. You’ll come away with insight into a powerful technique that handles errors in an elegant way using a simple, self-documenting design. https://vimeo.com/97344498 A Tale of Two Flat‑Maps Brandon Williams & Stephen Celis • Mar 27, 2018 Up until Swift 4.1 there was an additional flatMap on sequences that we did not consider in this episode, but that’s because it doesn’t act quite like the normal flatMap . Swift ended up deprecating the overload, and we discuss why this happened in a previous episode: Note Swift 4.1 deprecated and renamed a particular overload of flatMap . What made this flatMap different from the others? We’ll explore this and how understanding that difference helps us explore generalizations of the operation to other structures and derive new, useful code! https://www.pointfree.co/episodes/ep10-a-tale-of-two-flat-maps Downloads Sample code 0043-the-many-faces-of-flatmap-pt2 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .