Video #36: Advanced Protocol Witnesses: Part 2

Episode: Video #36 Date: Nov 5, 2018 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep36-advanced-protocol-witnesses-part-2

Description

We complete our dictionary for translating Swift protocol concepts into concrete datatypes and functions. This includes protocol inheritance, protocol extensions, default implementations and protocols with associated types. Along the way we will also show how concrete types can express things that are currently impossible with Swift protocols.

Video

Cloudflare Stream video ID: c0d48ab9c32adfe2298412080f47861a Local file: video_36_advanced-protocol-witnesses-part-2.mp4 *(download with --video 36)*

References

Transcript

— 0:05

It’s amazing that we can recover conditional conformance with just plain functions. It seems like such a complex feature, and it definitely is from the perspective of protocols and type systems, but at the value level it is so simple.

— 0:46

A complicated compiler feature can be reframed as function composition. Still, we have that compiler feature and we can use it, even if it took four years to get there. And we used it to great benefit with our Tagged and NonEmpty types. Still, there are plenty of things that you can’t do these days with protocols, but you can do them with witnesses. Tuple witnesses

— 1:17

Recall that currently in Swift, version 4.2 as of today, it is not possible to make tuples conform to protocols. For example, we can’t add a sum property to tuples of integers that adds the components together: extension (Int, Int) { var sum: Int { return self.0 + self.1 } } Non-nominal type ‘(Int, Int)’ cannot be extended

— 1:36

We get an error about tuples being nominal types, and those can’t conform to protocols.

— 1:43

Further, Swift doesn’t support generics in extensions, which means it’s not possible to conform a tuple to the Equatable protocol: extension (A, B): Equatable where A: Equatable, B: Equatable { static func ==(lhs: (A, B), rhs: (A, B)) -> Bool { return lhs.0 == rhs.0 && lhs.1 == rhs.1 } }

— 2:08

Heck we can’t even make Void , the empty tuple, equatable! extension Void: Equatable { static func ==(lhs: Void, rhs: Void) -> Bool { return true } }

— 2:20

This is a limitation of Swift, in which so-called “non-nominal” types are a lil different from the types that you and I are allowed to define in Swift. We are going to have a lot to say about nominal types in a future episode of Point-Free, but for today we are going to fix this problem.

— 2:35

It turns out that witness do not have this nominal type problem at all, and it’s because we are just using structs and generics. As we’ve seen in this episode, the Equating type is a concrete datatype that represents the Equatable protocol, and we can very easily create an equating witness for Void : extension Equating where A == Void { static let void = Equating { _, _ in true } }

— 3:09

Now we can create an array of Void elements and check if that array is equatable. Equating.array(of: .void).equals([(), ()], [(), ()]) // true Equating.array(of: .void).equals([(), ()], [()]) // false

— 3:25

This is impossible with protocols because we can’t extend the Void type to be Equatable . [(), ()] == [()] Error ‘<Self where Self Equatable> (Self.Type) -> (Self, Self) -> Bool’ requires that ‘()’ conform to ‘Equatable’ Yikes!

— 3:58

And more generally, we can derive a witness for equating a tuple as long as we are given witnesses to equate each of the components of the tuple: extension Equating { static func tuple( _ a: Equating<A>, _ b: Equating ) -> Equating<(A, B)> { return Equating<(A, B)> { lhs, rhs in a.equals(lhs.0, rhs.0) && b.equals(lhs.1, rhs.1) } } }

— 5:08

The compiler’s happy, so how can we use this? Let’s create an Equating instance of (Int, String) tuples where the string is equated by its character count. Equating.tuple(.int, stringCount) // Equating<(Int, String)>

— 5:24

And there we have it! Let’s give it a spin. Equating.tuple(.int, stringCount).equals((1, "Blob"), (1, "Blob")) // true Equating.tuple(.int, stringCount).equals((1, "Blob"), (1, "Blob Jr")) // false Equating.tuple(.int, stringCount).equals((1, "Blob"), (2, "Bolb")) // false

— 5:58

This is completely impossible in today’s Swift protocols, but is possible with concrete datatypes, and has been since Swift version 1! Function witnesses

— 6:16

Tuples aren’t the only non-nominal types in Swift, functions are also non-nominal. This means you can’t extend functions to add functionality or conform them to protocols. For example, functions whose input and output are of the same type can be made to conform to the Combinable protocol, but you can’t actually express that in Swift: extension (A) -> A: Combinable { func combine(other f: @escaping (A) -> A) -> (A) -> A { return { a in other(self(a)) } } } We get the same kinds of problems we had when we tried to extend tuples.

— 6:57

The workaround for this is to wrap functions in a struct so that we can then conform that wrapper to the protocol. struct Endo<A>: Combinable { let call: (A) -> A func combine(with other: Endo) -> Endo { return Endo { a in other.call(self.call(a)) } } }

— 7:22

Defining a whole new type just to wrap something seems needlessly heavy, and having to do that every time you want to conform a function to a protocol is going to get tiresome pretty quickly and prevents you from have truly generic abstractions.

— 7:35

However, witnesses have no such problems! We can very easily construct witness values for functions. In particular, we can construct a combining witness for functions of the form (A) -> A : extension Combining { static var endo: Combining<(A) -> A> { return Combining<(A) -> A> { f, g in { a in g(f(a)) } } } }

— 8:21

And this is really just function composition under the hood! We can even express this using our >>> operator. extension Combining { static var endo: Combining<(A) -> A> { // return Combining<(A) -> A> { f, g in // { a in g(f(a)) } // } return Combining<(A) -> A>(combine: >>>) } }

— 8:41

This construction allows you to combine functions as long as they have the same input and output.

— 8:46

We also had this concept of EmptyInitializing , which allowed us to create witnesses to pluck values of types out of thin air. Let’s see if we can create such a value for functions (A) -> A : extension EmptyInitializing { static var identity: EmptyInitializing<(A) -> A> { return EmptyInitializing<(A) -> A> { { $0 } } } }

— 9:42

Now that we’ve created witnesses on (A) -> A functions for both Combining and EmptyInitializing , we can use them with our generic algorithms.

— 9:53

For example, here’s an array of a bunch of such functions: let endos: [(Double) -> Double] = [ { $0 + 1.0 }, { $0 * $0 }, sin, { $0 * 1000.0 } ]

— 10:01

We can reduce all of these down to a single function using our reduce helper from before: endos.reduce(EmptyInitializing.identity, Combining.endo) // (Double) -> Double

— 10:43

Let’s pass a value along. endos.reduce(EmptyInitializing.identity, Combining.endo)(3) // -287.9033166650653

— 10:59

We’ve now unlocked a generic algorithm, reduce , to be used with functions, and that was previously impossible with protocols unless we introduced another type and performed a wrapping and unwrapping dance. Protocol inheritance

— 11:17

We’ve now uncovered some wonderful features of witnesses that are impossible to implement with protocols, but there are more features of protocols that we should figure out how they look like in the witness world, like “protocol inheritance.”

— 11:34

An interesting feature that protocols have is that you can inherit from an existing protocol to add new features to it. For example, the Comparable protocol inherits from the Equatable protocol and adds the functionality that determines when one value of a type is less than another value. What does this mechanism look like with concrete types and witnesses?

— 12:12

Here’s what the Comparable protocol looks like: protocol Comparable: Equatable { static func < (lhs: Self, rhs: Self) -> Bool } It inherits from Equatable and adds a requirement: the < function.

— 12:22

We already have an Equating witness, so how can we add this additional comparing functionality to it? struct Comparing<A> { let equating: Equating<A> let lessThan: (A, A) -> Bool } We just need to provide an equating witness alongside our comparison function. That’s all protocol inheritance is! Just some nested concrete types.

— 13:15

And we can easily create a witnesses for this concrete type: let intAsc = Comparing(equating: .int, compare: <) let intDesc = Comparing(equating: .int, compare: >)

— 13:46

There are completely valid reasons to compare integers in ascending or descending order, and we were able to create two separate witnesses to do just that, but with protocols, we can only conform to Comparable once, so we’re forced to pick!

— 14:21

Pullbacks also play nicely with this. We can reuse the pullback from Equating : struct Comparing<A> { … func pullback(_ f: @escaping (B) -> A) -> Comparing { return Comparing( equating: self.equating.pullback(f), lessThan: { lhs, rhs in self.lessThan(f(lhs), f(rhs)) } ) }

— 15:48

And now we can instantly pullback the comparing witness on integers to one on users where we only compare their ids: struct User { let id: Int, name: String } intAsc.pullback(get(\User.id)) // Comparing<User> intDesc.pullback(get(\User.id)) // Comparing<User> And now we get a really lightweight way to sort arrays of users by just pulling back the comparison of their ids.

— 16:40

We can even go deeper and create a Comparing against how short or long a user’s name is. intAsc.pullback(get(\User.name.count)) // Comparing<User>

— 16:55

Three different ways of comparing users, all built from the notion of comparing integers. Again we’re seeing functionality that is completely impossible in the protocol world and hidden from us. We would never think to do things like this. Protocol extensions

— 17:11

Another interesting feature of protocols is that you can extend them to add new functionality that is derived from the existing functionality. This is known as protocol extensions. For example, in the Swift standard library the Equatable protocol is extended to provide a != operator:

— 18:00

What would this look like for our Equating concrete type? Well, it just an extension of the struct to add this function! extension Equating { var notEquals: (A, A) -> Bool { return { lhs, rhs in !self.equals(lhs, rhs) } } }

— 19:07

And this is exactly what protocol extensions are for concrete data types: regular ole extensions!

— 19:30

There’s another type of protocol extension that is really handy that allows you to specify default implementations so that conforming types don’t have to, unless they want to. As an example, there’s a really popular protocol in the community that expresses the idea of types that can be used as reusable cells for table views and collection views: public protocol Reusable { static var reuseIdentifier: String { get } }

— 20:29

Now there’s a very obvious default reuseIdentifier to use for any type in which we just convert the name of the type into a string: public extension Reusable { static var reuseIdentifier: String { return String(describing: self) } }

— 20:51

With this, types can conform to this type without doing any extra work, unless they want to: class UserCell: UITableViewCell {} class EpisodeCell: UITableViewCell {} extension UserCell: Reusable {} extension EpisodeCell: Reusable {} UserCell.reuseIdentifier // "UserCell" EpisodeCell.reuseIdentifier // "EpisodeCell"

— 21:20

What we have here is a protocol with the idea of having a reuse identifier, but with a default implementation so types conforming to it don’t have to write this boilerplate every time.

— 21:32

What does that look like for concrete types? struct Reusing<A> { let reuseIdentifier: () -> String } Here we have a struct with the notion of returning a reuse identifier, but how can we insert that default implementation? struct Reusing<A> { let reuseIdentifier: () -> String init( reuseIdentifier: @escaping () -> String = { String(describing: A.self) } ) { self.reuseIdentifier = reuseIdentifier } }

— 23:07

So protocols with default implementations are the same as initializers with default arguments!

— 23:15

And now we can construct a witness to the Reusable protocol without doing any real work: Reusing<UserCell>() // Reusing<UserCell> Now we have a “reusing” witness on user cells, and we can pluck out a reuse identifier from it: Reusing<UserCell>().reuseIdentifier() // "UserCell" And the same works for reusing episode cells. Reusing<EpisodeCell>().reuseIdentifier() // "EpisodeCell" Protocol extensions with defaults directly correspond to initializers with default values.

— 23:53

This concrete type is kind interesting. It has a shape that we have seen before. Notice that the type A doesn’t appear anywhere in the body of the struct at all, which we called a “phantom type.” It’s what allows us to provide type safe wrapper around types so that you can distinguish for example, two strings that have different semantic meanings, like an email from an identifier. We further made a generic version of this type and called it Tagged in a previous episode. That is all this type is. It’s a tagged string! Associated types

— 24:55

Perhaps the most complicated feature of protocols is that of associated types. The moment you introduce an associated type into a protocol you lose the ability to use that protocol like you would most other types. For example, you can’t have an array of values that conform to that protocol: let collections: [Collection] protocol ‘Collection’ can only be used as a generic constraint because it has Self or associated type requirements Swift cannot yet make sense of this expression, although sometime soon Swift may gain this ability through a proposal known as “generalized existentials.” We will probably talk about that sometime soon on this series, but we don’t even have to go down that road just yet. As we have seen with all of the other terms in this dictionary correspondence we are building between protocols and concrete datatypes, the ideas in the datatype world seem to be simpler. So, what do associated types look like over in the datatype world?

— 26:30

There’s a really simple protocol with an associated type in Swift called RawRepresentable . This protocol expresses the idea of types that can be converted to another type in a possibly lossy manner. public protocol RawRepresentable { associatedtype RawValue public init?(rawValue: Self.RawValue) public var rawValue: Self.RawValue { get } }

— 27:02

You’ve probably most likely come across RawRepresentable with enums, in which simple enums can get a conformance for free: enum Directions: String { case down = "D" case left = "L" case right = "R" case up = "U" } Directions.down.rawValue // "D" Directions(rawValue: "D") // .some(Directions.down) Directions(rawValue: "X") // nil

— 27:44

Note that RawRepresentable doesn’t show up in the definition of Directions . The compiler is synthesizing a conformance for us.

— 28:01

Let’s translate this to a concrete datatype: struct RawRepresenting<A, RawValue> { let convert: (RawValue) -> A? let rawValue: (A) -> RawValue }

— 29:24

When it’s written this way, doesn’t it seem so simple? This is simply expressing that we can sometimes convert a RawValue into an A , though sometimes this will fail, but we can always convert an A into a RawValue .

— 29:42

What are some examples? Well, integers are raw representably by strings, for you can sometimes convert a string into an integer, but you can always extract a string from an integer: extension RawRepresenting where A == Int, RawValue == String { static var stringToInt = RawRepresenting( convert: { Int($0) }, rawValue: { "\($0)" } ) }

— 30:54

We can even clean up the code a bit more by going point-free. extension RawRepresenting where A == Int, RawValue == String { static var stringToInt = RawRepresenting( convert: Int.init, rawValue: String.init(describing:) ) }

— 31:11

So it’s pretty easy to construct these things. It’s kinda a bummer that we lose out on the compiler synthesized RawRepresentable conformances for simple enums, but it turns out it’s even possible to recover that! extension RawRepresenting where A: RawRepresentable, A.RawValue == RawValue { static var rawRepresentable: RawRepresenting { return RawRepresenting( convert: A.init(rawValue:), rawValue: { $0.rawValue } ) } }

— 32:50

We can go fully point-free here, as well, using our get helper from Overture . extension RawRepresenting where A: RawRepresentable, A.RawValue == RawValue { static var rawRepresentable: RawRepresenting { return RawRepresenting( convert: A.init(rawValue:), rawValue: get(^\.rawValue) ) } }

— 33:02

Now we can pluck out RawRepresenting witnesses from RawRepresentable types! RawRepresenting<Directions, String>.rawRepresentable

— 33:22

It’s kind of subtle what we’re seeing here but it’s worth repeating: what we’ve done here is taken a protocol from Swift and translated it into a concrete datatype, but we didn’t give up any of the default conformances or synthesized conformances or any of the other fun features Swift gives us. We’re able to recover everything in a very easy, lightweight way! We could have done the same for Equatable and the Equating witness. We can create witnesses from existing protocols in such a simple way. What’s the point?

— 34:08

So we’ve now basically gone down a laundry list of powerful features that protocols have and shown how to translate these ideas to concrete types and witnesses. We’ve nearly completed the dictionary. There are probably still a few small things to consider, but it’s mostly complete!

— 34:30

However, now that we have nearly completed this Rosetta Stone we must ask: what’s the point? Why do we keep translating protocol ideas over to structs when Swift has given us protocols and everyone tells us protocol-oriented programming is wonderful!

— 34:58

Well, even as just an exercise, translating protocols to concrete datatypes has revealed just how simple some of these seemingly complex features are. We could clear away the fog and see that a protocol feature is really just functions and composition and generics in disguise! Even the dreaded “ Self or associated type” error becomes much less dreadful when you realize that it’s just a couple hidden generics. That’s empowering, but it doesn’t really answer the question of whether we should write code this way!

— 35:57

Well, we alluded to this last time that knowing the pitfalls of protocols and how to solve those problems with concrete types can actually better influence API design. I think by default most people reach for protocols when design an API, but we want to show that isn’t really necessary and can in fact be limiting by these pitfalls. And we are going to really show this in our next series of episodes where we design a library from scratch using these principles.

— 36:51

OK, now that we have all of this out of the way, I think we are finally ready to tackle a real world use case: designing a library from scratch. I’m really excited for this next series of episodes. It has really challenged the way we think about library design, and what came out the other end is truly fantastic. Until next time! References Protocol-Oriented Programming in Swift Apple • Jun 16, 2015 Apple’s eponymous WWDC talk on protocol-oriented programming: At the heart of Swift’s design are two incredibly powerful ideas protocol-oriented programming and first class value semantics. Each of these concepts benefit predictability, performance, and productivity, but together they can change the way we think about programming. Find out how you can apply these ideas to improve the code you write. https://developer.apple.com/videos/play/wwdc2015/408/ Modern Swift API Design Apple • Jan 2, 2019 As of WWDC 2019, Apple no longer recommends that we “start with a protocol” when designing our APIs. A more balanced approach is discussed instead, including trying out concrete data types. Fast forward to 12:58 for the discussion. Note Every programming language has a set of conventions that people come to expect. Learn about the patterns that are common to Swift API design, with examples from new APIs like SwiftUI, Combine, and RealityKit. Whether you’re developing an app as part of a team, or you’re publishing a library for others to use, find out how to use new features of Swift to ensure clarity and correct use of your APIs. https://developer.apple.com/videos/play/wwdc2019/415/?time=778 Pullback We use the term pullback for the strange, unintuitive backwards composition that seems to show up often in programming. The term comes from a very precise concept in mathematics. Here is the Wikipedia entry: In mathematics, a pullback is either of two different, but related processes precomposition and fibre-product. Its “dual” is a pushforward. https://en.wikipedia.org/wiki/Pullback Protocols with Associated Types Alexis Gallagher • Dec 15, 2015 This talk by Alexis Gallagher shows why protocols with associated types are so complicated, and tries to understand why Swift chose to go with that design instead of other alternatives. https://www.youtube.com/watch?v=XWoNjiSPqI8 Protocol Oriented Programming is Not a Silver Bullet Chris Eidhof • Nov 24, 2016 An old article detailing many of the pitfalls of Swift protocols, and how often you can simplify your code by just using concrete datatypes and values. Chris walks the reader through an example of some networking API library code, and shows how abstracting the library with protocols does not give us any tangible benefits, but does increase the complexity of the code. http://chris.eidhof.nl/post/protocol-oriented-programming/ Value-Oriented Programming Matt Diephouse • Jul 29, 2018 Matt gives another account of protocol-oriented programming gone awry, this time by breaking down the famous WWDC talk where a shape library is designed using protocols. By rewriting the library without protocols Matt ends up with something that can be tested without mocks, can be inspected at runtime, and is more flexible in general. https://matt.diephouse.com/2018/08/value-oriented-programming/ Scrap your type classes Gabriella Gonzalez • May 2, 2012 Haskell’s notion of protocols are called “type classes,” and the designers of Swift have often stated that Swift’s protocols took a lot of inspiration from Haskell. This means that Haskellers run into a lot of the same problems we do when writing abstractions with type classes. In this article Gabriella Gonzalez lays down the case for scrapping type classes and just using simple datatypes. http://www.haskellforall.com/2012/05/scrap-your-type-classes.html Haskell Antipattern: Existential Typeclass Luke Palmer • Jan 24, 2010 A Haskell article that demonstrates a pattern in the Haskell community, and why it might be an anti-pattern. In a nutshell, the pattern is for libraries to express their functionality with typeclasses (i.e. protocols) and provide Any* wrappers around the protocol for when you do not want to refer to a particular instance of that protocol. The alternative is to replace the typeclass with a simple concrete data type. Sound familiar? https://lukepalmer.wordpress.com/2010/01/24/haskell-antipattern-existential-typeclass/ Some news about contramap Brandon Williams • Oct 29, 2018 A few months after releasing our episode on Contravariance we decided to rename this fundamental operation. The new name is more friendly, has a long history in mathematics, and provides some nice intuitions when dealing with such a counterintuitive idea. https://www.pointfree.co/blog/posts/22-some-news-about-contramap Contravariance Julie Moronuki & Chris Martin This article describes the ideas of contravariance using the Haskell language. In many ways exploring functional programming concepts in Haskell is “easier” because the syntax is sparse and allows you to focus on just the core ideas. https://typeclasses.com/contravariance Protocol Witnesses: App Builders 2019 Brandon Williams • May 3, 2019 Brandon gave a talk about “protocol witnesses” at the 2019 App Builders conference. The basics of scraping protocols is covered as well as some interesting examples of where this technique really shines when applied to snapshot testing and animations. Note Protocol-oriented programming is strongly recommended in the Swift community, and Apple has given a lot of guidance on how to use it in your everyday code. However, there has not been a lot of attention on when it is not appropriate, and what to do in that case. We will explore this idea, and show that there is a completely straightforward and mechanical way to translate any protocol into a concrete datatype. Once you do this you can still write your code much like you would with protocols, but all of the complexity inherit in protocols go away. Even more amazing, a new type of composition appears that is difficult to see when dealing with only protocols. We will also demo a real life, open source library that was originally written in the protocol-oriented way, but after running into many problems with the protocols, it was rewritten entirely in this witness-oriented way. The outcome was really surprising, and really powerful. https://www.youtube.com/watch?v=3BVkbWXcFS4 Some news about contramap Brandon Williams • Oct 29, 2018 A few months after releasing our episode on Contravariance we decided to rename this fundamental operation. The new name is more friendly, has a long history in mathematics, and provides some nice intuitions when dealing with such a counterintuitive idea. https://www.pointfree.co/blog/posts/22-some-news-about-contramap Downloads Sample code 0036-advanced-protocol-witnesses-pt2 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .