Video #33: Protocol Witnesses: Part 1

Episode: Video #33 Date: Oct 15, 2018 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep33-protocol-witnesses-part-1

Description

Protocols are a great tool for abstraction, but aren’t the only one. This week we begin to explore the tradeoffs of using protocols by highlighting a few areas in which they fall short in order to demonstrate how we can recover from these problems using a different tool and different tradeoffs.

Video

Cloudflare Stream video ID: b3cfeff95b203973584a2f4fb9062908 Local file: video_33_protocol-witnesses-part-1.mp4 *(download with --video 33)*

References

Transcript

— 0:05

Protocols are great! I use protocols. You use protocols. We all use protocols! They allows us to code to an interface rather than an implementation, and we’ve heard many times that this is a powerful idea. It gives you the ability to completely swap out implementations if you wanted, which is what many people do with their dependencies. They use live versions in production, but then substitute mock versions while testing.

— 0:25

However, protocols can’t possibly be the be-all and end-all of code reuse and extensibility in our Swift code. For one thing, they have a lot of problems that have to be dealt with! I’m sure everyone has come across the dreaded “Self or associated type requirements” error, and that’s always annoying. Also, protocols are quite rigid in that a type can conform to a given protocol in only one single way. Sometimes it’s completely valid and even technically correct to allow a type to conform to a protocol in multiple ways.

— 0:56

It turns out that basically every Swift protocol can be directly translated to a struct representation, and we’ve even done this process in a previous episode but we just didn’t dissect it. Doing this translation fixes a lot of the problems with protocols, but also introduces a few new quirks. Even cooler, this translation process is completely algorithmic. We can describe a set of steps that allows you to sit down and translate any protocol into a struct via a step-by-step process. What’s a protocol?

— 1:35

It sounds weird to say, but we actually haven’t really defined a single protocol on this series. So, maybe we should just give a quick tour of what protocols are and how they are used.

— 1:49

Protocols are a way of abstractly describing what a type can do by specifying a collection of function signatures and properties. A protocol doesn’t actually contain any implementations, it’s only the signatures.

— 2:00

For example, you can have a protocol that expresses the idea of types that are capable of being described as strings: protocol Describable { var describe: String { get } }

— 2:15

You may already be familiar with this protocol under a different name, CustomStringConvertible , but we just wanted to create a fresh one for the sake of discussion.

— 2:24

You can then extend any existing type to make it conform to this protocol, and Swift will aid you in fleshing out its implementation: extension Int: Describable { var describe: String { return "\(self)" } } 2.describe // "2" Not the most exciting stuff, but stay with us… Empty-initializable

— 2:47

Protocols can hold a lot more than just properties. They can hold static properties, methods, static methods, and even initializers. For example, there’s a large class of types out there for which it makes sense to initialize them without giving any data. We could write a protocol to represent such types: protocol EmptyInitializable { init() }

— 3:12

And we can make some types conform to this: extension String: EmptyInitializable { } extension Array: EmptyInitializable { } extension Int: EmptyInitializable { } All of these types already have empty initializers, so conforming to EmptyInitializable doesn’t require any additional work.

— 3:48

An example of something that doesn’t have an empty initializer is the Optional type. extension Optional: EmptyInitializable { init() { self = nil } }

— 4:02

This allows you to abstract over the idea of things that can be initialized out of thin air. How might that be useful? Where, there’s a really great function in Swift called reduce , and it allows you to accumulate state over a collection. It takes an initial state to start with, and an accumulation function that will be invoked with each value in the collection and each successive accumulated state. For example: [1, 2, 3].reduce(0, +) // 6

— 4:34

Notice that you have to start with an initial value. Well, what if we are reducing an “empty initializable” value? Then we could skip the initial value entirely: extension Array { func reduce<Result: EmptyInitializable>( _ accumulation: (Result, Element) -> Result ) -> Result { return try self.reduce(Result(), accumulation) } }

— 5:42

And now we can use it: [1, 2, 3].reduce(+) // 6 [[1, 2], [], [3, 4]].reduce(+) // [1, 2, 3, 4] ["Hello", " ", "Blob"].reduce(+) // "Hello Blob"

— 6:14

That’s kinda cool. We are now seeing that we can write generic algorithms by using protocols. Combinable

— 6:20

Protocols can also express far more complicated concepts. Like what if we wanted a protocol that represents types that are capable of combining with other values of the same type in order to produce another value? We can use Self with a capital S to reference the type that is conforming to the protocol, and so we can write such a protocol like so: protocol Combinable { func combine(with other: Self) -> Self }

— 7:02

Then we could conform to this protocol like so: extension Int: Combinable { func combine(with other: Int) -> Int { return self + other } } extension String: Combinable { func combine(with other: String) -> String { return self + other } } extension Array: Combinable { func combine(with other: Array) -> Array { return self + other } } Each of these types has a natural operation for being combined. Integers can be added, and strings and arrays can be concatenated.

— 7:27

What about optionals? extension Optional: Combinable { func combine(with other: Optional) -> Optional { return self ?? other } } It also has a combining operation using the coalescing operator, which will return the first non-nil value.

— 7:50

Where can we use this? Well, this combine function has a similar signature to the accumulation function for reduce , except the values we are accumulating and the result have the same type. So maybe we can make another reduce overload that doesn’t even take an accumulation function, and instead delegate it to this combine function: extension Array where Element: Combinable { func reduce(_ initial: Element) -> Element { return self.reduce(initial) { $0.combine(with: $1) } } }

— 8:55

Now whenever we are reducing arrays of combinable values we don’t even have to specify an accumulation function: [1, 2, 3].reduce(0) // 6 [[1, 2], [], [3, 4]].reduce([]) // [1, 2, 3, 4] [nil, nil, 3].reduce(nil) // 3 Protocol composition

— 9:33

And then, the really cool part of protocols is that they can be combined in a really lightweight way so that we can make an even simpler reduce for those times that we are reducing over something that is both combinable and empty initializable: extension Array where Element: Combinable & EmptyInitializable { func reduce() -> Element { return self.reduce(Element()) { $0.combine(with: $1) } } } [1, 2, 3].reduce() // 6 [[1, 2], [], [3, 4]].reduce() // [1, 2, 3, 4] [nil, nil, 3].reduce() // 3

— 10:47

That’s pretty cool, and that’s the power of a well formed abstraction. We have a few small pieces that represent one thing, and represent it well, and then we could write generic algorithms that simplify our use of those abstractions. The problem with protocols

— 11:11

However, that doesn’t mean that everything is sunshine and rainbows in the world of protocols. They have some drawbacks. All of our viewers have probably run up against the most egregious of problems with protocols, which is the “Self or associated type requirements” error. This happens whenever you try to use a protocol like the Combinable one as a plain type, like say an array of combinable values. You can’t do that.

— 11:38

We’ll have more to say about that and ways to fix it in a future episode, but for now we want to describe another deficiency of protocols, and that’s that there is only one way to conform to them.

— 11:59

Sometimes it is completely valid to have a type conform to a protocol in multiple ways. For example, Int could have conformed to EmptyInitializable and Combinable in a way that describes multiplication instead of addition. extension Int: Combinable { func combine(with other: Int) -> Int { return self * other } } Redundant conformance of ‘Int’ to protocol ‘Combinable’

— 12:36

We can’t have both conformances at once, so let’s get rid of the one that uses addition.

— 12:48

But now we have a problem: [1, 2, 3].reduce() // 0 Our reduce function on an array of Int s is returning zero because the value it starts with is zero, and zero multiplied against 1 and 2 and 3 is going to also return zero.

— 13:02

So we also have to update the EmptyInitializable conformance with a more suitable initial value. extension Int: EmptyInitializable { init() { self = 1 } }

— 13:16

And this fixes our expectations with our reduce function. [1, 2, 3].reduce() // 6 Multiplying 1 and 2 and 3 is the same as adding 1 and 2 and 3, so let’s double-check to make sure it’s really using multiplication. [1, 2, 3, 4].reduce() // 24

— 13:35

Now it may seem like we were able to solve the problems we were having with multiple conformances, but really, we had to trample over and abandon our old work in the process, losing the original addition-based conformance because we can only support a single conformance at a time. Another example

— 13:51

That may not seem like a huge deal, but let’s look at an example of where this really becomes obviously unfortunate.

— 13:58

Let’s look at that Describable protocol again: protocol Describable { var description: String { get } }

— 14:01

Types conform to this protocol to provide a textual representation of the type. How about a fun example of a type that could conform to this protocol.

— 14:10

Here’s a struct that holds the connection information for accessing a Postgres database: struct PostgresConnInfo { let database: String let hostname: String let password: String let port: Int let user: String }

— 14:18

We can construct a value easily enough: let localhostPostgres = PostgresConnInfo( database: "pointfreeco_development", hostname: "localhost", password: "", port: 5432, user: "pointfreeco" )

— 14:23

There’s one very obvious conformance of this type to Describable in which we just list out all the fields: extension PostgresConnInfo: Describable { var describe: String { return """ PostgresConnInfo(\ database: "\(self.database)", \ hostname: "\(self.hostname)", \ password: "\(self.password)", \ port: "\(self.port)", \ user: "\(self.user)"\ ) """ } }

— 14:46

And we can try it out by printing our value’s description to the console. print(localhostPostgres.describe) // PostgresConnInfo(database: "pointfreeco_development", hostname: "postgres", password: "", port: "5432", user: "pointfreeco")

— 15:00

However, we may have also wanted a “prettier” version of this printing in which we add some newlines: extension PostgresConnInfo: Describable { var description: String { return """ PostgresConnInfo( database: "\(self.database)", hostname: "\(self.hostname)", password: "\(self.password)", port: "\(self.port)", user: "\(self.user)" ) """ } }

— 15:27

Now when we print it, we see a nice, readable output. print(localhostPostgres.description) // PostgresConnInfo( // database: "pointfreeco_development", // hostname: "postgres", // password: "", // port: "5432", // user: "pointfreeco" // )

— 15:39

I’m not sure which is the correct conformance for this protocol. Even more troubling, there’s a completely different approach to this conformance in which we actually print the connection string that we could use to establish a connection with an actual postgres database: extension PostgresConnInfo: Describable { var describe: String { return """ postgres://\(self.user):\(self.password)@\ \(self.hostname):\(self.port)/\(self.database) """ } } print(localhost.description) // postgres://pointfreeco:@localhost:5432/pointfreeco_development

— 16:10

This one is quite different from the others, and I’m not sure which one is the correct one. In reality, none of them are the correct one, they are all equally valid depending on the use case. But we can only conform to this protocol a single time, and so anywhere Describable is used it will be forced to use this conformance.

— 16:35

And this is one of the biggest problems with protocols: you can only conform your types to them a single time. However, often a type can conform in many different ways. Till next time…

— 16:45

OK, now that we’ve given a refresher on how protocols work in Swift, and showed one of their biggest limitations, what can we do to address that? It turns out that when you define a protocol in Swift and conform a type to that protocol, the compiler is doing something very special under the hood in order to track the relationship between those two things. We are going to give a precise definition to that construction, and we will even recreate it directly in Swift code. This will mean that we are going to do the work that the compiler could be doing for us for free, but by us taking the wheel for this we will get a ton of flexibility and composability out of it.

— 17:46

We’ll start with the Describable protocol, but we’ll instead define a generic struct…next week! References Protocol-Oriented Programming in Swift Apple • Jun 16, 2015 Apple’s eponymous WWDC talk on protocol-oriented programming: At the heart of Swift’s design are two incredibly powerful ideas protocol-oriented programming and first class value semantics. Each of these concepts benefit predictability, performance, and productivity, but together they can change the way we think about programming. Find out how you can apply these ideas to improve the code you write. https://developer.apple.com/videos/play/wwdc2015/408/ Modern Swift API Design Apple • Jan 2, 2019 As of WWDC 2019, Apple no longer recommends that we “start with a protocol” when designing our APIs. A more balanced approach is discussed instead, including trying out concrete data types. Fast forward to 12:58 for the discussion. Note Every programming language has a set of conventions that people come to expect. Learn about the patterns that are common to Swift API design, with examples from new APIs like SwiftUI, Combine, and RealityKit. Whether you’re developing an app as part of a team, or you’re publishing a library for others to use, find out how to use new features of Swift to ensure clarity and correct use of your APIs. https://developer.apple.com/videos/play/wwdc2019/415/?time=778 Protocols with Associated Types Alexis Gallagher • Dec 15, 2015 This talk by Alexis Gallagher shows why protocols with associated types are so complicated, and tries to understand why Swift chose to go with that design instead of other alternatives. https://www.youtube.com/watch?v=XWoNjiSPqI8 Protocol Oriented Programming is Not a Silver Bullet Chris Eidhof • Nov 24, 2016 An old article detailing many of the pitfalls of Swift protocols, and how often you can simplify your code by just using concrete datatypes and values. Chris walks the reader through an example of some networking API library code, and shows how abstracting the library with protocols does not give us any tangible benefits, but does increase the complexity of the code. http://chris.eidhof.nl/post/protocol-oriented-programming/ Value-Oriented Programming Matt Diephouse • Jul 29, 2018 Matt gives another account of protocol-oriented programming gone awry, this time by breaking down the famous WWDC talk where a shape library is designed using protocols. By rewriting the library without protocols Matt ends up with something that can be tested without mocks, can be inspected at runtime, and is more flexible in general. https://matt.diephouse.com/2018/08/value-oriented-programming/ Scrap your type classes Gabriella Gonzalez • May 2, 2012 Haskell’s notion of protocols are called “type classes,” and the designers of Swift have often stated that Swift’s protocols took a lot of inspiration from Haskell. This means that Haskellers run into a lot of the same problems we do when writing abstractions with type classes. In this article Gabriella Gonzalez lays down the case for scrapping type classes and just using simple datatypes. http://www.haskellforall.com/2012/05/scrap-your-type-classes.html Haskell Antipattern: Existential Typeclass Luke Palmer • Jan 24, 2010 A Haskell article that demonstrates a pattern in the Haskell community, and why it might be an anti-pattern. In a nutshell, the pattern is for libraries to express their functionality with typeclasses (i.e. protocols) and provide Any* wrappers around the protocol for when you do not want to refer to a particular instance of that protocol. The alternative is to replace the typeclass with a simple concrete data type. Sound familiar? https://lukepalmer.wordpress.com/2010/01/24/haskell-antipattern-existential-typeclass/ Protocol Witnesses: App Builders 2019 Brandon Williams • May 3, 2019 Brandon gave a talk about “protocol witnesses” at the 2019 App Builders conference. The basics of scraping protocols is covered as well as some interesting examples of where this technique really shines when applied to snapshot testing and animations. Note Protocol-oriented programming is strongly recommended in the Swift community, and Apple has given a lot of guidance on how to use it in your everyday code. However, there has not been a lot of attention on when it is not appropriate, and what to do in that case. We will explore this idea, and show that there is a completely straightforward and mechanical way to translate any protocol into a concrete datatype. Once you do this you can still write your code much like you would with protocols, but all of the complexity inherit in protocols go away. Even more amazing, a new type of composition appears that is difficult to see when dealing with only protocols. We will also demo a real life, open source library that was originally written in the protocol-oriented way, but after running into many problems with the protocols, it was rewritten entirely in this witness-oriented way. The outcome was really surprising, and really powerful. https://www.youtube.com/watch?v=3BVkbWXcFS4 Downloads Sample code 0033-protocol-witnesses-pt1 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .