Video #34: Protocol Witnesses: Part 2

Episode: Video #34 Date: Oct 22, 2018 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep34-protocol-witnesses-part-2

Description

Last time we covered some basics with protocols, and demonstrated one of their biggest pitfalls: types can only conform to a protocol a single time. Sometimes it’s valid and correct for a type to conform to a protocol in many ways. We show how to remedy this by demonstrating that one can scrap any protocol in favor of a simple datatype, and in doing so opens up a whole world of composability.

Video

Cloudflare Stream video ID: 80b38a90b75e3bd4a03d016f6ce071ea Local file: video_34_protocol-witnesses-part-2.mp4 *(download with --video 34)*

References

Transcript

— 0:05

OK, now that we’ve given a refresher on how protocols work in Swift, and showed one of their biggest limitations, what can we do to address that? It turns out that when you define a protocol in Swift and conform a type to that protocol, the compiler is doing something very special under the hood in order to track the relationship between those two things. We are going to give a precise definition to that construction, and we will even recreate it directly in Swift code. This will mean that we are going to do the work that the compiler could be doing for us for free, but by us taking the wheel for this we will get a ton of flexibility and composability out of it.

— 1:06

Let’s start slowly with the protocol we’ve just been talking about: Describable . We are going to de-protocolize this by creating a generic struct, where the generic parameter represents the type conforming to the protocol, and the struct will have fields corresponding to the requirements of the protocol.

— 1:28

So, Describable had one requirement that said that it could turn itself into a string. We will represent this as a generic struct that wraps a function from the generic parameter into string: struct Describing<A> { var describe: (A) -> String }

— 1:47

Let’s put this next to the Describable protocol so that you can see the similarities: protocol Describable { var description: String { get } } struct Describing<A> { var describe: (A) -> String }

— 2:13

From this type we create instances, which are called “witnesses” to the protocol conformance: let compactWitness = Describing<PostgresConnInfo> { conn in """ PostgresConnInfo(\ database: "\(conn.database)", \ hostname: "\(conn.hostname)", \ password: "\(conn.password)", \ port: "\(conn.port)", \ user: "\(conn.user)"\ ) """ }

— 2:54

You can use this witness directly by just invoking the description field directly: compactWitness.describe(localhostPostgres) // PostgresConnInfo(database: "pointfreeco_development", hostname: "localhost", password: "", port: "5432", user: "pointfreeco")

— 3:05

We get the same output as before, but because our witnesses are just values, we can create as many of them as we want. let prettyWitness = Describing<PostgressConnInfo> { """ PostgresConnInfo( database: "\($0.database)", hostname: "\($0.hostname)", password: "\($0.password)", port: "\($0.port)", user: "\($0.user)" ) """ }

— 3:32

And we can use each witness the same way. prettyWitness.description(localhostPostgres) // PostgresConnInfo( // database: "pointfreeco_development", // hostname: "localhost", // password: "", // port: "5432", // user: "pointfreeco" // )

— 3:39

We can keep going and define an additional witness that prints connection strings. let connectionWitness = Describing<PostgressConnInfo> { """ postgres://\($0.user):\($0.password)@\($0.hostname):\($0.port)/\ \($0.database) """ } connectionWitness.description(localhostPostgres) "postgres://pointfreeco:@localhost:5432/pointfreeco_development" Witnessing generic algorithms

— 4:00

But more interesting is how can we write generic algorithm that work over the shape of Describing ? Well, previously we would just make a generic function that restricts the generic by the Describable protocol. In the de-protocolized world we instead allow the generic to roam free, but require passing in an explicit witness of the Describing type: func print<A>(tag: String, _ value: A, _ witness: Describing<A>) { print("[\(tag)] \(witness.describe(value))") }

— 5:02

We can use this function by explicitly passing a witness alongside our value. print(tag: "debug", localhostPostgres, connectionWitness) // [debug] postgres://pointfreeco:@localhost:5432/pointfreeco_development print(tag: "debug", localhostPostgres, prettyWitness) // [debug] PostgresConnInfo( // database: "pointfreeco_development", // hostname: "localhost", // password: "", // port: "5432", // user: "pointfreeco" // )

— 5:30

It’s instructive to see how this function would be written when using protocols: func print<A: Describable>(tag: String, _ value: A) { print("[\(tag)] \(value.describe)") } print(tag: "debug", localhostPostgres) // [debug] postgres://pointfreeco:@localhost:5432/pointfreeco_development We get the connection string because that’s the conformance we gave our type, and it’s the only conformance we can have.

— 6:33

It’s not really that much different. There’s a pretty clear correspondence between functions that are generic over parameters conforming to protocols and functions that are un-constrained in their generics but take an explicit witness to those protocols. In the former we are a little more succinct, but in the latter we are allowed to provide many different witnesses to the protocol. De-protocolizing Combinable

— 7:00

Protocols with properties are not the only things that can be translated into structs. Pretty much any protocol can be turned into an explicit struct.

— 7:07

For example the Combinable protocol looks like this in struct form: protocol Combinable { func combine(with other: Self) -> Self } struct Combining<A> { let combine: (A, A) -> A } The extra A parameter in this function is due to the fact that combine(with:) is an instance method in the protocol, and hence has an implicit self that gets passed along. De-protocolizing EmptyInitializable

— 7:59

The EmptyInitializable protocol is a lil trickier to convert. Well, that is until you realize that initializers in Swift are just glorified static functions, and static functions are just like regular methods that have no implicit self passed in, and hence don’t take an A as an argument: protocol EmptyInitializable { init() } struct EmptyInitializing<A> { let create: () -> A }

— 8:33

And now we can recreate some of the generic algorithms we had before, but now we just have to explicitly pass in witnesses to the protocols: extension Array { func reduce( _ initial: Element, _ combining: Combining<Element> ) -> Element { return self.reduce(initial, combining.combine) } } We’ve traded out that where Element: Combinable constraint for explicitly passing in a witness.

— 9:38

What this allows us to do is create a witness and use it in our new generic reduce algorithm. let sum = Combining<Int>(combine: +) [1, 2, 3, 4].reduce(0, sum) // 10

— 10:11

And now we can do something that we couldn’t have done before with protocols, which is to define another witness on integers for products. let product = Combining<Int>(combine: *) [1, 2, 3, 4].reduce(1, product) // 24

— 10:33

We’re starting to see how witnesses allow you to decide what conformance you want to have. De-protocolizing protocol composition

— 10:41

And what about combining two protocols like we saw earlier when we did Combinable & EmptyInitializable ? Well, that just translates to passing along two witnesses: extension Array { func reduce( _ initial: EmptyInitializing<Element>, _ combining: Combining<Element> ) -> Element { return self.reduce(initial.create(), combining.combine) } }

— 11:27

Now we just need some empty-initializing witnesses. First, for addition: let zero = EmptyInitializer<Int> { 0 } [1, 2, 3, 4].reduce(zero, sum) // 10

— 11:50

And then, for multiplication: let one = EmptyInitializer<Int> { 1 } [1, 2, 3, 4].reduce(one, product) // 24

— 12:03

We’ve now recreated our generic algorithms, but at the call site we get to substitute in the different witnesses that we want to use rather than relying on the implicit conformance of a protocol. What’s the point?

— 12:17

And we could keep going. We could figure out how to translate more pieces of protocols into structs. Like, what do static properties and static functions look like? Or how do associated types work into the translation? And what if those associated types had constraints? What about method requirements in a protocol that themselves have generics? And what about protocol inheritance and extensions? And conditional conformance? And on and on and on…

— 12:43

It is possible to translate all of those things into structs (yes even conditional conformance), but before we get into all of that we should probably ask, “What’s the point?” What’s the point of going through the motions of translating protocols to structs when Swift has given us protocols, and encourages us to do “protocol-oriented programming.” Are there any benefits to understanding this correspondence?

— 13:16

Well, there are actually a few reasons! First, this process of translating protocols into structs and then passing around explicit witnesses is literally what the Swift compiler is doing under the hood. It’s nice to know that there isn’t any real magic happening behind the scenes, and that the concepts are super simple.

— 13:43

Second, explicit witnesses give us a whole new level composability with our conformances that was impossible to see when we are dealing with protocols. Witnessing composition

— 14:02

Let’s take another look at our Describing struct: struct Describing<A> { let describe: (A) -> String }

— 14:08

The struct is just a wrapper around a function (A) -> String , which has a very familiar shape that has a very important operation on it that we covered in a previous episode that really enhances the composability and transformability of the type. Does anyone remember it?

— 14:31

It’s contramap ! Contramap is a function that you can sometimes define on generic types, and its very similar to the map operation that we all know and love on arrays and optionals, except it flips things around a bit. Let’s define it: struct Describing<A> { let describe: (A) -> String func contramap<B>(_ f: @escaping (B) -> A) -> Describing <B> { return Describing<B> { b in self.describe(f(b)) } } } This is saying that if you tell me how to transform B s into A s, contramap can then transform Describing <A> s into Describing <B> s. This allows us to create all new witnesses of the protocol from existing witnesses.

— 15:44

Why on earth would you ever want to contramap a witness? What does that even mean? Well, going back to our Postgres connection type, recall that it contained a password field. That’s pretty sensitive info! There’s a pretty good chance we don’t want to always print that part of the connection. Well, with contramap it’s very easy to do that! let secureCompactWitness = compactWitness.contramap { PostgresConnInfo( database: $0.database, hostname: $0.hostname, password: "***", port: $0.port, user: $0.user ) }

— 16:49

Now we get to use our secure witness to describe our values. print(secureCompactWitness.describe(localhostPostgres)) // PostgresConnInfo(database: "pointfreeco_development", hostname: "localhost", password: "***", port: "5432", user: "pointfreeco")

— 17:18

We can do the same thing with the pretty witness. let securePrettyWitness = prettyWitness.contramap { PostgresConnInfo( database: $0.database, hostname: $0.hostname, password: "***", port: $0.port, user: $0.user ) } And now we have a witness that prints a connection in a nice, human-readable way, but also redacts the password.

— 17:37

Let’s take it for a spin. print(secureCompactWitness.describe(localhostPostgres)) // [debug] PostgresConnInfo( // database: "pointfreeco_development", // hostname: "localhost", // password: "***", // port: "5432", // user: "pointfreeco" // )

— 17:48

Even better, we can use some concepts from previous episodes to make this read really nicely. In our series of episodes on key paths and setters we came up with some nice helper functions that enhance the composability of setters. We even open sourced a library with all of those helpers, and more, called Overture . If we import that library we can rewrite this secure witness like so: import Overture let secureCompactWitness: Describing<PostgresConnInfo> = compactWitness.contramap(set(\.password, "***"))

— 18:44

When you use these nice functional pieces that glue together in nice ways, you can even leverage type inference to make things really succinct. let secureCompactWitness = compactWitness.contramap( set(\.password, "***") )

— 18:59

The securePrettyWitness can be defined the same way. let securePrettyWitness = prettyWitness.contramap( set(\.password, "***") )

— 19:16

This is really amazing. This type of composition and transformation is completely hidden in the protocol world. There is no version of this in protocols. Only witnesses get access to this composition.

— 19:30

We’re seeing that we’re defining witnesses easily, deriving all new witnesses out of existing witnesses, and we just have to pass them around where we want to use them. We don’t get to leverage the compiler’s implicit handling of protocols, but we get composition out of it.

— 19:43

And there’s even more composability to come. We’ve done a series of episodes on zip and its relation to map . While there’s map and there’s contramap, there’s zip and there’s contrazip! And this is yet another kind of composability we can use with witnesses, but we’ll have to save that for another episode.

— 20:04

Another point to be made is that perhaps we reach for protocols more often than we need to. A struct can be just fine and we get a lot of flexibility out of it! We can use this idea of struct representations of protocols to guide library design. I think that most of us would reach for protocols when designing a library, but maybe that isn’t necessary. Maybe we don’t need the full power and sugar of protocols that Swift gives us all the time, and maybe just a simple struct data type will do just fine.

— 20:38

In an upcoming episode we are going to apply this directly to the design of a library we have been working on for a long time. We started in the protocol-oriented way, and kept it like that for a long time, but we were continually held back by its rigidity and lack of composition. And then finally one day we decided to rewrite it using structs and by passing around explicit witnesses, and it opened up all types of cool possibilities.

— 21:18

And if Swift had not provided protocols for us its kinda nice to know that we wouldn’t have lost that much. We could still write generic algorithms over the shapes of abstractions, it just would have been a lil more cumbersome sometimes. There’s an article that looks at this same problem from the Haskell perspective called Scrap Your Type Classes (type classes are Haskell’s version of protocols) where the author, Gabriella Gonzalez, points out that you can replace all of your type classes with data types.

— 22:01

It’s really interesting to see how we can live without protocols. Protocols have their problems, some of which we’ve discussed, but they’ve also evolved over time with more advanced features, like conditional conformance. Well, now that we know we can replicate the functionality of protocols using structs, it might be surprising to know that we could have replicated the functionality of conditional conformance on day one when Swift was first released!

— 22:29

We want to dive deeper and show what it looks like to take more protocol-based code and convert it to use structs, but that’ll have to wait for future episodes. Till next time! References Protocol-Oriented Programming in Swift Apple • Jun 16, 2015 Apple’s eponymous WWDC talk on protocol-oriented programming: At the heart of Swift’s design are two incredibly powerful ideas protocol-oriented programming and first class value semantics. Each of these concepts benefit predictability, performance, and productivity, but together they can change the way we think about programming. Find out how you can apply these ideas to improve the code you write. https://developer.apple.com/videos/play/wwdc2015/408/ Modern Swift API Design Apple • Jan 2, 2019 As of WWDC 2019, Apple no longer recommends that we “start with a protocol” when designing our APIs. A more balanced approach is discussed instead, including trying out concrete data types. Fast forward to 12:58 for the discussion. Note Every programming language has a set of conventions that people come to expect. Learn about the patterns that are common to Swift API design, with examples from new APIs like SwiftUI, Combine, and RealityKit. Whether you’re developing an app as part of a team, or you’re publishing a library for others to use, find out how to use new features of Swift to ensure clarity and correct use of your APIs. https://developer.apple.com/videos/play/wwdc2019/415/?time=778 Protocols with Associated Types Alexis Gallagher • Dec 15, 2015 This talk by Alexis Gallagher shows why protocols with associated types are so complicated, and tries to understand why Swift chose to go with that design instead of other alternatives. https://www.youtube.com/watch?v=XWoNjiSPqI8 Protocol Oriented Programming is Not a Silver Bullet Chris Eidhof • Nov 24, 2016 An old article detailing many of the pitfalls of Swift protocols, and how often you can simplify your code by just using concrete datatypes and values. Chris walks the reader through an example of some networking API library code, and shows how abstracting the library with protocols does not give us any tangible benefits, but does increase the complexity of the code. http://chris.eidhof.nl/post/protocol-oriented-programming/ Value-Oriented Programming Matt Diephouse • Jul 29, 2018 Matt gives another account of protocol-oriented programming gone awry, this time by breaking down the famous WWDC talk where a shape library is designed using protocols. By rewriting the library without protocols Matt ends up with something that can be tested without mocks, can be inspected at runtime, and is more flexible in general. https://matt.diephouse.com/2018/08/value-oriented-programming/ Scrap your type classes Gabriella Gonzalez • May 2, 2012 Haskell’s notion of protocols are called “type classes,” and the designers of Swift have often stated that Swift’s protocols took a lot of inspiration from Haskell. This means that Haskellers run into a lot of the same problems we do when writing abstractions with type classes. In this article Gabriella Gonzalez lays down the case for scrapping type classes and just using simple datatypes. http://www.haskellforall.com/2012/05/scrap-your-type-classes.html Haskell Antipattern: Existential Typeclass Luke Palmer • Jan 24, 2010 A Haskell article that demonstrates a pattern in the Haskell community, and why it might be an anti-pattern. In a nutshell, the pattern is for libraries to express their functionality with typeclasses (i.e. protocols) and provide Any* wrappers around the protocol for when you do not want to refer to a particular instance of that protocol. The alternative is to replace the typeclass with a simple concrete data type. Sound familiar? https://lukepalmer.wordpress.com/2010/01/24/haskell-antipattern-existential-typeclass/ Protocol Witnesses: App Builders 2019 Brandon Williams • May 3, 2019 Brandon gave a talk about “protocol witnesses” at the 2019 App Builders conference. The basics of scraping protocols is covered as well as some interesting examples of where this technique really shines when applied to snapshot testing and animations. Note Protocol-oriented programming is strongly recommended in the Swift community, and Apple has given a lot of guidance on how to use it in your everyday code. However, there has not been a lot of attention on when it is not appropriate, and what to do in that case. We will explore this idea, and show that there is a completely straightforward and mechanical way to translate any protocol into a concrete datatype. Once you do this you can still write your code much like you would with protocols, but all of the complexity inherit in protocols go away. Even more amazing, a new type of composition appears that is difficult to see when dealing with only protocols. We will also demo a real life, open source library that was originally written in the protocol-oriented way, but after running into many problems with the protocols, it was rewritten entirely in this witness-oriented way. The outcome was really surprising, and really powerful. https://www.youtube.com/watch?v=3BVkbWXcFS4 Downloads Sample code 0034-protocol-witnesses-pt2 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .