Video #45: The Many Faces of Flat‑Map: Part 4

Episode: Video #45 Date: Jan 28, 2019 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep45-the-many-faces-of-flat-map-part-4

Description

Continuing our 3-part answer to the all-important question “what’s the point?”, we show that the definitions of map, zip and flatMap are precise and concisely describe their purpose. Knowing this we can strengthen our APIs by not smudging their definitions when convenient.

Video

Cloudflare Stream video ID: d7bb6d6b514e47926c4288dd261ca73a Local file: video_45_the-many-faces-of-flat-map-part-4.mp4 *(download with --video 45)*

References

Transcript

— 0:05

Now we see map , zip and flatMap are an important trio of operations and each does one thing and does it well. We should now be able to convince ourselves that we shouldn’t be smudging their definitions just to suit our needs. We will likely come across functions with signatures that look a lot like flatMap and may even be tempted to call it flatMap , but doing so can destroy all of our intuitions around what flatMap is. Right now we have 6 types that we are very familiar with and all of the operations behave roughly the same, even for very different types.

— 1:00

And this lesson is an important one, but just 10 or 11 months ago we had a decisive moment in Swift history where the community got to learn from this and put it to real-world use. We actually had an entire episode dedicated to this back then, but now we are even in a better position to appreciate it, so let’s briefly recall the problem. Flat‑map vs. compact‑map

— 1:25

Prior to Swift 4.1 there was a flatMap on Sequence that didn’t have the shape of the flatMap we’ve come to love. extension Sequence { public func flatMap<ElementOfResult>( _ transform: (Element) throws -> ElementOfResult? ) rethrows -> [ElementOfResult] }

— 1:38

Why is this the wrong shape? Well, let’s convert it to our more streamlined notation: flatMap: ([A], (A) -> B?) -> [B]

— 2:00

And let’s refactor this notation in the same manner as before, where we shift some arguments around to prefer configuration up front (in this case the transform function), and data last (in this case the array). flatMap: ((A) -> B?, [A]) -> [B])

— 2:13

And finally we can curry the function so that the configuration can be partially applied. flatMap: ((A) -> B?) -> (([A]) -> [B]) And now this shows us that this “ flatMap ” lifts functions from A to optional B up to functions from arrays of A s to arrays of non-optional B s.

— 2:35

And if we wanted to further genericize it to something broader than arrays we could use F again: flatMap: ((A) -> B?) -> ((F<A>) -> F)

— 2:49

This is saying that given a transformation from A to B? we can somehow magically lift that to a function (F<A>) -> F . We have no other information of how F interacts with optionals. In the case that F is arrays it’s clear, we can just filter out the nil values from the array. But in the general case it isn’t clear what this would mean.

— 3:12

But more important, this operation has nothing to do with sequencing values. It doesn’t give us a tool to sequence two F values together with a transformation, and so there is no intuition we can reuse from our understanding of how flatMap works on the 6 other types we have discussed. And we now have to wonder whenever we see flatMap , are we talking about an operation that sequences computations or are we doing an operation that filters out optionals?

— 3:39

This is just a needless muddying of the concepts in order to avoid having a unique name for this operation. And fortunately a name change was proposed and accepted for Swift 4.1, and this operation is now known as compactMap . Result’s throwing maps and flat‑maps

— 4:22

Even more recently there was another situation in which knowledge of this structure helped the Swift community avoid another misuse of this operation. When the Result type was being proposed for Swift, there was a desire for the map and flatMap operations to take a throwing transformation.

— 4:45

Now, in one way we could argue that the Result type is already a type designed for dealing with error handling, and so mixing in throws seems to be awkward because we are mixing two different error models.

— 4:56

But we can more precisely describe why this is not right for us to do. It starts with the observation that there is an equivalence between functions that throw and functions that return a result. We can construct functions to convert between each of those styles: func fromThrowing<A, B>( _ f: @escaping (A) throws -> B ) -> (A) -> Result<B, Swift.Error> { return { a in do { return .success(try f(a)) } catch let error { return .failure(error) } } } func toThrowing<A, B>( _ f: @escaping (A) -> Result<B, Swift.Error> ) -> ((A) throws -> B) { return { a in switch f(a) { case let .success(value): return value case let .failure(error): throw error } } }

— 5:33

If you take a throwing function and apply fromThrowing and then apply toThrowing you will just end up right where you started. And same if you do the opposite, convert toThrowing and then back to fromThrowing .

— 5:41

So, knowing this equivalence, let’s see what this tells us about a version of map that takes a throwing transformation: extension Result { func map(_ f: (A) throws -> B) -> Result<B, E> { } }

— 6:05

Let’s convert that throwing function to the result version that we know is equivalent: extension Result { func map(_ f: (A) -> Result<B, Swift.Error>) -> Result<B, E> { } }

— 6:17

Well, this seems more like a flatMap than a map right? We are transforming A s into Result s and then flattening the result. The error parameter is a little different simply due to throws being untyped, but really this is just a flatMap . So again, our intuition around map is that it allows us to open up generic types, apply pure transformations on the data inside, and the wrap it back up in the generic container, but this map is chaining together multiple failable things. It has muddied the meaning of the operation.

— 6:50

Even worse, this signature can’t actually be implemented, because we can’t convert every Swift.Error to E . So we’d need to lock everything into untyped error handling. extension Result { func map( _ f: (A) -> Result<B, Swift.Error> ) -> Result<B, Swift.Error> { } }

— 7:02

And now that we’re changing the shape of the Result , this really is feeling like something other than a map .

— 7:09

And a flatMap with a throwing transformation doesn’t fare much better: extension Result { func flatMap(_ f: (A) throws -> Result<B, E>) -> Result<B, E> { } }

— 7:20

We can again use our equivalence to convert throws to Result . extension Result { func flatMap( _ f: (A) -> Result<Result<B, E>, Swift.Error> ) -> Result<B, Swift.Error> { } }

— 7:40

This one is even more bizarre. It’s allowing us to transform into a nested result, one of which has a typed error and one that doesn’t. Again this is muddying what should be a very simple operation: chaining together failable transformations.

— 7:54

And this one also doesn’t seem to be implementable. Relaxing E to Swift.Error may make it possible, but we’d still be left with a function that is very different than the flatMap we’ve come to know.

— 8:13

So we have another instance where knowing the importance of shape can instruct us on whether an operation should be named a certain way. Types with no flat‑map

— 9:00

On the flip side, sometimes there are types that simply don’t have a flatMap no matter how much we wish they did. We should be OK with that rather than forcing something that doesn’t work.

— 9:23

For example, Func has flatMap , but it only works on one of the generic parameters. What if we tried to implement flatMap on its other parameter? extension Func /* <A, B> */ { func flatMap<C>(_ f: @escaping (A) -> Func<C, B>) -> Func<C, B> { } }

— 10:00

This is like a flatMap that operates on the input of functions, whereas the other flatMap that we know and understand operates on the output.

— 10:09

This flatMap is not possible to implement. But that’s OK! In fact, it’s even more exciting. It means that perhaps there is something more to know about this shape, and perhaps there is a closely related operation that can be defined that is not exactly what flatMap is. We’ve already discussed this idea when we saw that some types carry an operation that looks like map , but it goes in the wrong direction. It was a contravariant map . It was an interesting operation on its own, but it wasn’t map and so we didn’t try to reuse that name.

— 10:50

Here we should do the same, where we explore different kinds of operations that work on the input of functions, and we may even get something that looks similar to flatMap , but even if it’s similar, we shouldn’t call it flatMap because it doesn’t have the exact signature or shape.

— 11:08

There are a lot of other types that don’t have flatMap , even though we might hope it could be useful.

— 11:20

For example, in our series of episodes where we discussed snapshot testing we came up with the following generic type for expressing values that can be diffed: struct Diffing<Value> { let toData: (Value) -> Data let fromData: (Data) -> Value let diff: (Value, Value) -> (String, [XCTAttachment])? } If you haven’t watched the snapshot testing episodes yet that’s OK, the details of this type aren’t important.

— 11:49

What is important is that flatMap cannot be defined on this type: extension Diffing { func flatMap<NewValue>( _ f: @escaping (Value) -> Diffing<NewValue> ) -> Diffing<NewValue> { fatalError("Not possible to implement.") } }

— 12:08

This could maybe be interesting to use to generate new diffing strategies from existing diffing strategies, but it’s not possible to implement this function.

— 12:18

In addition to the Diffing type we also had the Snapshotting type which described values that were capable of being snapshot into a diffable format: struct Snapshotting<Value, Format> { var pathExtension: String? let diffing: Diffing<Format> let snapshot: (Value) -> Format }

— 12:36

Again, this type does not support a flatMap operation in either the Value or Format generics.

— 12:43

We can’t implement flatMap to transform an existing value to a snapshotting of a new value. func flatMap<NewValue>( _ f: @escaping (Value) -> Snapshotting<NewValue, Format> ) -> Snapshotting<NewValue, Format> { fatalError("Not possible to implement.") }

— 12:57

And we also can’t implement flatMap to transform an existing format to a snapshotting of a new format. func flatMap<NewFormat>( _ f: @escaping (Format) -> Snapshotting<Value, NewFormat> ) -> Snapshotting<Value, NewFormat> { fatalError("Not possible to implement.") }

— 13:11

Both of these flatMap s are just not possible to implement, but we should be OK with that and comfortable with that. All it means is there’s maybe more to understand about these types and functions.

— 13:23

And it’s kind of interesting to explore these types that don’t have flatMap . It can make you curious as to what kinds of operations it does support. Renaming flat‑map

— 13:52

So now we see that a generic type carrying a flatMap operation is a pretty special thing. Not all types have it. And because it’s special, it deserves to have a name to distinguish it from other operations.

— 14:32

This is very important to understand because it is quite common to rename flatMap and other operations to seem “friendlier”. This usually feels like a good thing but can lead to situations in which things fall apart.

— 14:52

Perhaps the most common renaming is in the context of the Parallel type, also known as a Promise or Future , in which flatMap is renamed to then : extension Parallel { func then(_ f: @escaping (A) -> Parallel) -> Parallel { return self.flatMap(f) } }

— 15:03

Here I’ve called flatMap under the hood of then just to make it very clear that it’s only a rename. There is no new functionality here.

— 15:15

The idea behind this rename is that since flatMap is intimately related to sequencing of values, we could use “then” as a way of describing a sequence of steps.

— 15:27

We could rewrite some of our earlier parallel code to use then : Parallel { f in f(Bundle.main.path(forResource: "user", ofType: "json")!) } .map(URL.init(fileURLWithPath:)) .then { url in Parallel { f in f(try! Data(contentsOf: url)) } } .then { data in Parallel { f in f(try! JSONDecoder().decode(User.self, from: data)) } }

— 16:04

This isn’t the worst thing. It reads nicely, and concisely describes that we are sequencing values together. There are maybe only three real drawbacks:

— 16:13

By using this alternative name we lose out on a vast body of literature that uses the term flatMap .

— 16:40

And why rename flatMap to then for only Parallel ? Should we also use it for optionals, arrays, results, validated, and func? flatMap works in all of those cases so is it worth using a specialized name?

— 17:12

The biggest problem with this rename is that it is often accompanied with other renames.

— 17:18

For example, it’s very common to also use then for the map operation: extension Parallel { func then(_ f: @escaping (A) -> B) -> Parallel { return self.map(f) } }

— 17:26

Again I’ve called map under the hood just to make it clear that then is only a rename. With this defined our chain of then ’s become: Parallel { f in f(Bundle.main.path(forResource: "user", ofType: "json")!) } .then(URL.init(fileURLWithPath:)) .then { url in Parallel { f in f(try! Data(contentsOf: url) } } .then { data in Parallel { f in f(try! JSONDecoder().decode(User.self, from: data)) } }

— 17:45

We are now sacrificing conciseness and readability. Before when we saw flatMap or then we knew it meant a parallel value was being chained. That has real world implications because it means more work is being done in an async matter, like a network request or a long running computation. Now we don’t know unless we in inspect the body of the then to know if we are doing extra async work or if we are just doing a simple, pure transformation of data.

— 18:20

Let’s compare them side by side. Parallel { f in f(Bundle.main.path(forResource: "user", ofType: "json")!) } .then(URL.init(fileURLWithPath:)) .then { url in Parallel { f in f(try! Data(contentsOf: url) } } .then { data in Parallel { f in f(try! JSONDecoder().decode(User.self, from: data)) } } Parallel { f in f(Bundle.main.path(forResource: "user", ofType: "json")!) } .map(URL.init(fileURLWithPath:)) .flatMap { url in Parallel { f in f(try! Data(contentsOf: url) } } .flatMap { data in Parallel { f in f(try! JSONDecoder().decode(User.self, from: data)) } }

— 18:29

In the block of code using map and flatMap we get a clear delineation between pure transformations and transformations that do more work, whereas with then , not so much.

— 18:39

In the end it’s not that then is a bad name to use for flatMap . The issue is that it’s usually only applied to the parallel/promise/future type, whereas we were able to use flatMap for a wide variety of additional types. And it usually comes alongside other renames because it “reads well”, though it may muddy our understanding of code.

— 19:01

We want to convince everyone that the intuitions they can build with map , zip , and flatMap , and their signatures being solidified in stone, is reason enough to give them distinct names used across types rather than miss out and define transform functions with names which we can’t share as intuitions across types. Implicit flattening

— 19:23

The final thing we want to briefly mention when it comes to smudging definitions, is that maybe we’re wrong to even explore flatMap as a means to solve the nesting problem. Maybe the compiler should handle this kind of flattening for us automatically by not allowing for nesting at all.

— 19:45

In fact this is precisely what Kotlin does when it comes to nullability. Kotlin doesn’t have an Optional type. It only allows you to decorate types with ? to make it nullable. In Swift, Optional is its own type: an enum. In Kotlin, nullability is a feature of the compiler.

— 20:09

Because of this, Kotlin has no concept of optional nesting. No matter how many ? s you annotate a type with, it’s the same kind of nullability regardless, implicitly flattening it for you.

— 20:22

So what’s the problem? Well, why stop with nullability? Why not disallow nested arrays, nested results, arrays of optionals, optionals of arrays, etc.?

— 20:39

Well it’d probably be pretty difficult to do and might not be worth baking so much compiler magic into a language, but why definitively come up with how all nested types can be flattened when we have a single operation, flatMap , that allow us to solve the problem in a domain-specific way depending on what types we’re dealing with.

— 21:05

I can be surprising how often languages special-case this kind of implicit flattening. In addition to Kotlin, TypeScript is another language in which nullability is merely an annotation, and nested nullability doesn’t exist.

— 21:21

Even beyond that, JavaScript comes with a Promise type that has no map or flatMap , but instead has overloaded then s, and because it’s a dynamic language, there’s just no way of getting a nested Promise . The language will always implicitly flatten it for you.

— 21:39

Special-casing can seem convenient, but we’re missing out on these abstractions and shared intuitions in the process.

— 21:54

Last time we copied and pasted code and made it work with a whole bunch of types: optionals, results, validated, func, parallel. With each special-casing we miss out on this ability to share. Till next time…

— 22:05

So let’s talk the third and final part of “what’s the point?”. We’ve now spent a bunch of time getting comfortable with the idea of flatMap , justifying why we should use it, and why we should build an intuition for it. Once we did that, we convinced ourselves that the signature of flatMap and its friends is so important that we’re going to defend it from anyone that may disparage it: you shouldn’t change its signature, it’s there for a reason.

— 22:36

The reason we’ve done all this work is that now we can build off that foundation and ask very complex questions: questions that may have been seemingly intractable had we not taken this deep journey of discovery.

— 23:07

We’re going to look at composition of functions when it comes to flatMap . We saw that map had a wonderful property: the map of the compositions is the same as the composition of the map s. What that meant was that if you have a big chain of map s, you can collapse all that into a single map and call it once with the composition of all the units of work. Is there a version of this for flatMap ? There is!

— 23:33

Next, we know that flatMap can flatten nested containers, like optionals of optionals and results of results, but what about nested containers of different types, like an array of results, or array of parallels, etc. Is there anything we can discover with those kinds of nested containers.

— 23:58

Finally, what is the precise relationship between map , zip , and flatMap ? Can some operations be derived from others, what does it say about types that can do so, and is there some kind of hierarchy between these things?

— 24:16

These are some pretty complicated questions that we want to ask and we can finally answer them! References rename ELF.then to ELF.flatMap Apple • Jan 21, 2019 Apple’s Swift NIO project has a type EventLoopFuture that can be thought of as a super charged version of the Parallel type we’ve used many times on this series. It comes with a method that has the same signature as flatMap , but originally it was named then . This pull-request renames the method to flatMap , which brings it inline with the naming for Optional , Array and Result in the standard libary. https://github.com/apple/swift-nio/pull/760 SE-0235: Add Result to the Standard Library Nov 7, 2018 The Swift evolution review of the proposal to add a Result type to the standard library. It discussed many functional facets of the Result type, including which operators to include (including map and flatMap ), and how they should be defined. https://forums.swift.org/t/se-0235-add-result-to-the-standard-library/17752 Railway Oriented Programming — error handling in functional languages Scott Wlaschin • Jun 4, 2014 This talk explains a nice metaphor to understand how flatMap unlocks stateless error handling. Note When you build real world applications, you are not always on the “happy path”. You must deal with validation, logging, network and service errors, and other annoyances. How do you manage all this within a functional paradigm, when you can’t use exceptions, or do early returns, and when you have no stateful data? This talk will demonstrate a common approach to this challenge, using a fun and easy-to-understand “railway oriented programming” analogy. You’ll come away with insight into a powerful technique that handles errors in an elegant way using a simple, self-documenting design. https://vimeo.com/97344498 A Tale of Two Flat‑Maps Brandon Williams & Stephen Celis • Mar 27, 2018 Up until Swift 4.1 there was an additional flatMap on sequences that we did not consider in this episode, but that’s because it doesn’t act quite like the normal flatMap . Swift ended up deprecating the overload, and we discuss why this happened in a previous episode: Note Swift 4.1 deprecated and renamed a particular overload of flatMap . What made this flatMap different from the others? We’ll explore this and how understanding that difference helps us explore generalizations of the operation to other structures and derive new, useful code! https://www.pointfree.co/episodes/ep10-a-tale-of-two-flat-maps Downloads Sample code 0045-the-many-faces-of-flatmap-pt4 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .