Video #276: Shared State: File Storage, Part 2

Episode: Video #276 Date: Apr 22, 2024 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep276-shared-state-file-storage-part-2

Description

It’s incredible how easy @Shared makes it to persist complex data types to the file system, but currently it completely ruins our ability to test features that use it. Let’s fix that, and we will also explore what it means to derive a small piece of shared state from a bigger piece of shared state.

Video

Cloudflare Stream video ID: 4b5842e57e6f42dbf3f65e3067aa2163 Local file: video_276_shared-state-file-storage-part-2.mp4 *(download with --video 276)*

Transcript

— 0:05

We have now made some incredible improvements to the library. We have a .fileStorage persistence strategy that sits right alongside the .appStorage strategy we created last episode. Where .appStorage is great for persisting tiny bits of simple data in a lightweight way, .fileStorage is great for persisting large, complex data structures. And we’ve even built in some niceties in the strategy such as debouncing save tasks so that we don’t thrash the file system with disk writes, and we are even listening for changes on the disk so that we can detect external writes and update our shared state accordingly. Stephen

— 0:42

But what about testing? We saw with .appStorage we had to do a little bit of extra work to make sure that shared state didn’t bleed out from test to test, and I would imagine the same is true of .fileStorage . After all, the file system is a large, global, mutable blob of data that anyone can access and write to at any time. Surely that is going to complicate testing.

— 1:01

Let’s take a look. Making file storage testable

— 1:04

If we run the test suite for the shared state case study we will see something surprising. All of the tests seem to pass except for the testAlert test, which is the one that doesn’t even change shared state or really even assert on shared state.

— 1:22

And further, the failure message in the testAlert test is a little strange: A state change does not match expectation: … SharedState.State( _currentTab: .counter, _counter: CounterTab.State( _alert: AlertState( − title: "👎 The number 0 is not prime :(" + title: "👍 The number 10 is prime!" ), _stats: Shared.Storage(…), _isOn: Shared.Storage(…) ), _profile: ProfileTab.State(…), _stats: Shared.Storage(…) ) (Expected: −, Actual: +)

— 1:26

We are asserting that the message says “👎 The number 0 is not prime :(” because we don’t specifically an initial count for the shared state, and hence we would hope its 0, but in reality it is 10. And 10 is even a prime number, so that creates completely different alert messaging.

— 1:42

So, why is the count 10 instead of 0?

— 1:45

Well, 10 is the last number that I incremented up to in the simulator. That state was persisted to the file system, and now is bleeding over into our tests.

— 1:56

And this is of course happening because in our FileStorageKey persistence strategy we are writing out to the global, uncontrolled dependency of data loading: try? JSONDecoder().decode( Value.self, from: Data(contentsOf: self.url) ) …and saving: try? JSONEncoder().encode(value).write(to: url)

— 2:15

We need to control these dependencies if we want to have a shot at making this flavor of shared state testable. And this is exactly what we did for .appStorage . We say that having unencumbered access to UserDefaults.standard was a problem, and so we took control over that dependency.

— 2:29

Let’s do the same for .fileStorage . We will design a little dependency client that is capable of saving and loading data from a URL on disk. And we will take inspiration from our naming in the .appStorage strategy and we will call this DefaultFileStorage : struct DefaultFileStorage { var load: (URL) throws -> Data var save: (Data, URL) throws -> Void }

— 3:03

And we will make this dependency conform to the DependencyKey protocol so that we can specify the liveValue to use when running in the simulator and on device, which can just reach out to the live APIs that interact with the file system: struct DefaultFileStorage: DependencyKey { var load: (URL) throws -> Data var save: (Data, URL) throws -> Void static let liveValue = Self( load: { try Data(contentsOf: $0) }, save: { try $0.write(to: $1) } ) }

— 3:32

And let’s take more inspiration from the .appStorage strategy of last episode. We saw that it was helpful to provide a testValue for the underlying dependency that simply provided a user defaults that does not persist its changes to the system. This gave each test a little scratch defaults object that could be written to and read from, but it would never bleed out of that test.

— 3:48

Let’s do the same for the file system. We can provide a testValue of this dependency that pretends like you are reading from and writing to the real file system, but under the hood all its doing is keep track of URLs and Data values in a dictionary: static var testValue: Self { let fileSystem = LockIsolated<[URL: Data]>([:]) return Self( load: { url in guard let data = fileSystem[url] else { struct LoadError: Error {} throw LoadError() } return data }, save: { data, url in fileSystem.withValue { $0[url] = data } } ) }

— 5:06

Next we need to register the dependency with the system by adding a computed property to DependencyValues : extension DependencyValues { var defaultFileStorage: DefaultFileStorage { get { self[DefaultFileStorage.self] } set { self[DefaultFileStorage.self] = newValue } } }

— 5:29

And now we can hold onto that dependency in the FileStorageKey persistence strategy: public class FileStorageKey<Value: Codable>: PersistenceKey { @Dependency(\.defaultFileStorage) var fileStorage … }

— 5:42

And we can start using this dependency instead of reaching out to the global, uncontrolled dependencies. This includes when loading data: try? JSONDecoder().decode( Value.self, // from: Data(contentsOf: self.url) from: self.fileStorage.load(self.url) ) …and saving data: // try? JSONEncoder().encode(value).write(to: url) try? self.fileStorage.save(JSONEncoder().encode(value), url)

— 6:07

And hopefully that is all it takes to get a passing test suite again. We can run the tests and see that indeed they do pass!

— 6:19

This is because each test is now given its own scratch file system to work with. It can write whatever data it wants to in that file system, and it will never bleed over into another test.

— 6:29

Now there are a few improvements that we will want to make to this dependency before it’s ready for prime time, but we aren’t going to spend time on that right now. Derived shared state

— 6:34

Things are looking pretty incredible now. We have very simple tools for sharing state across the application and persisting state when something changes. Brandon

— 6:43

But let’s take things even further.

— 6:45

Right now we have a bunch of examples of holding onto shared state, but so far we have always held onto the full piece of shared state. What if a particular feature only needs access to a small piece of the shared state? And further, what if that feature doesn’t care if the state is persisted or how it’s persisted? It just wants access to a bit of state that is automatically in sync with a parent feature.

— 7:07

Let’s see what it takes to derive a whole new shared object from an existing shared object so that we can whittle state down to the bare essentials when passing data around to our features.

— 7:21

To explore this we are going to take a look at the sign up flow case study we built a few episodes back. It demonstrated how to create a moderately complex, multistep sign up flow, and it used shared state throughout in order for changes to be immediately visible to all features.

— 7:47

We are going to concentrate on particular step of the demo, which was the “topics” screen. This is where you get to select all the topics that are interesting to you. Currently in order to present this step you must provide it a shared piece of SignUpData : @Reducer struct TopicsFeature { @ObservableState struct State { … @Shared var signUpData: SignUpData } … }

— 8:01

But the feature only needs access to one tiny bit of state from this data type. It just needs the set of topics.

— 8:29

Wouldn’t it be great if we could whittle this down to a more precise domain by saying all we need is a shared set of topics? // @Shared var signUpData: SignUpData @Shared var topics: Set<SignUpData.Topic>

— 9:05

That allows this feature to be more honest with what it needs to do its job, and it means we have less to think about while working in the feature. We no longer have to worry about the topics feature making changes to parts of SignUpData that it shouldn’t because now it doesn’t even have access to any of that data.

— 9:24

This causes a few compiler errors, and so let’s see what kind of syntax we would like to have for passing around this kind of derived shared state, and then see what it takes to make it a reality.

— 9:36

The first error we have is where we try to construct a NavigationLink to the topics screen, and currently we are doing so by passing along the full shared sign up data: NavigationLink( "Next", state: SignUpFeature.Path.State.topics( TopicsFeature.State(signUpData: store.$signUpData) ) )

— 9:54

We now want to whittle this down to just the topics inside the shared sign up data. What if we could use simple dot-chaining to specify a property in signUpData that we want to derive shared state from: NavigationLink( "Next", state: SignUpFeature.Path.State.topics( TopicsFeature.State(topics: store.$signUpData.topics) ) )

— 10:11

And further, any changes made to this derived shared state should be immediately reflected in the parent shared state it was derived from, and vice-versa.

— 10:22

So, let’s make that syntax a reality.

— 10:25

In order to unlock such simple dot-chaining syntax we must add @dynamicMemberLookup to the Shared type: @propertyWrapper @dynamicMemberLookup public struct Shared<Value> { … }

— 10:44

And that forces us to provide a dynamic member subscript that takes a key path: public subscript<Member>( dynamicMember keyPath: WritableKeyPath<Value, Member> ) -> Shared<Member> { fatalError() }

— 11:27

We’ll fatal error for now just to see what else needs to change to get things compiling. For example in the preview we can now pass some shared topics along as a set: TopicsFeature.State(topics: Shared([])

— 12:09

Then in the topics reducer we no longer need to go through the signup data: if state/*.signUpData*/.topics.isEmpty { … }

— 12:23

And similarly in the topics view: Toggle( topic.rawValue, isOn: $store/*.signUpData*/.topics[contains: topic] ) … .interactiveDismissDisabled( store/*.signUpData*/.topics.isEmpty )

— 12:38

And finally where the topics destination is populated from the summary, we can project into the shared signup data’s topics: state.destination = .topics( TopicsFeature.State( topics: state.$signUpData.topics ) )

— 12:52

And now everything is compiling, though the dynamic member subscript is still just a fatal error.

— 13:12

In order to implement this subscript we need to return a Shared<Member> value, and to do that we need to use one of the two initializer available to us. But neither seems very useful:

— 13:29

One takes a concrete value and creates brand new storage under the hood. This is the initializer that is used when using the @Shared property wrapper without persistence and without a default value.

— 13:38

The other takes a concrete value and persistence strategy.

— 13:51

We want something quite a bit different from either of these. We want to create a whole new shared value that somehow holds onto the existing storage, along with a key path so that we can pluck a small part of the state in the underlying storage.

— 14:11

To accomplish this we will alter the Shared type a bit. It will no longer be thought of as a simple struct holding onto some underlying storage that directly holds onto the shared state. Instead it will hold onto a reference to some parent state and a key path that allows us to get an set a subset of state inside the Storage . And the key path must be type erased because we only want to be generic over the type of value Shared holds onto, not the type of value the root storage holds onto: @propertyWrapper @dynamicMemberLookup public struct Shared<Value> { let keyPath: AnyKeyPath let storage: Storage … }

— 15:11

We’ll update the existing initializers to start the key path as the identity key path on Value : public init(_ value: Value) { self.keyPath = \Value.self … } public init( wrappedValue value: Value, _ key: some PersistenceKey<Value> ) { self.keyPath = \Value.self … }

— 16:09

And then we can provide an initializer to allow creating shared state from an existing storage and a key path for diving into the storage’s state: init(storage: Storage, keyPath: AnyKeyPath) { self.storage = storage self.keyPath = keyPath }

— 16:24

And it can be internal because no one outside the library should ever need to use this API, nor should they be allowed because of the type erasure. We are definitely going to have to do some force casting at some point, and so the responsibility of type safety will be on us, and so this API is best left internal to the library where we can be extra careful.

— 16:44

With those changes I would hope that I could now implement the dynamic member subscript by passing along the storage and appending the key paths together: public subscript<Member>( dynamicMember keyPath: WritableKeyPath<Value, Member> ) -> Shared<Member> { Shared<Member>( storage: self.storage, keyPath: self.keyPath.appending(path: keyPath)! ) } However this does not work: Cannot convert parent type ‘Shared<Value>’ to expected type ‘Shared<Member>’

— 17:28

The problem is that the Storage held in Shared still retains all of its type information, and so we can’t pass a parent storage down to a child shared state. Nearly all of the type information needs to be erased from Storage in order for it to be held on by multiple Shared values. And this isn’t such a big deal because we are already holding onto a completely erased key path. So we just have to do the same for Storage .

— 18:14

To achieve this type erasure we are going need a protocol that the Storage type can conform to. We will start with it being very bare and then add onto it as we need. In fact, we will start with it just being an empty protocol for now: protocol StorageProtocol { }

— 18:25

The Storage type will conform to this protocol: @Perceptible class Storage: StorageProtocol { … }

— 18:31

And then we will erase the type information held in the storage property: let storage: any StorageProtocol let keyPath: AnyKeyPath

— 18:38

And the new initializer we added a moment ago will take some StorageProtocol instead of a concrete Storage object: init(storage: some StorageProtocol, keyPath: AnyKeyPath) { … }

— 18:47

This causes a lot of problems, but it also fixed one problem. Our implementation of the dynamic member subscript is now compiling just fine: public subscript<Member>( dynamicMember keyPath: WritableKeyPath<Value, Member> ) -> Shared<Member> { Shared<Member>( storage: self.storage, keyPath: self.keyPath.appending(path: keyPath)! ) } And that is because we erased the type information from storage .

— 19:09

Now let’s fix the other errors one-by-one. One easy problem to fix is with this line right here: sharedStates[key] = self.storage

— 19:22

This is complaining that we are trying to assign something that is required to be an AnyObject : Cannot assign value of type ‘any StorageProtocol’ to subscript of type ‘AnyObject?’ And that’s because the sharedStates global dictionary requires the stored values to be classes.

— 19:31

Sounds like we need the StorageProtocol to inherit from AnyObject so that it can only be applied to class types: protocol StorageProtocol: AnyObject {}

— 19:37

And that has already fixed one problem.

— 19:40

The next problem we have is in the wrappedValue of Shared : public var wrappedValue: Value { get { self.storage.value } set { self.storage.value = newValue } }

— 19:46

We can no longer reach into storage to grab the value because storage is now fully type erased with the StorageProtocol . But, we can add a new requirement to the protocol so that we can extract this information. It sounds like we need a property with a get and set , and we need an associated type: protocol StorageProtocol: AnyObject { associatedtype Value var value: Value { get set } }

— 20:19

Now we do have a value property to access on storage, but the Value associated type has been completely erased with the way we are holding onto storage : let storage: any StorageProtocol

— 20:25

And so storage.value is an Any , and that doesn’t help us much.

— 20:35

While we do want to erase the associated value type from the storage property held in Shared , we do want to be able to recover it at a later point, such as in this computed property. We want to be able to open this existential, see what kind of value it wraps on the inside, and then key path into that value.

— 20:54

Swift provides a tool that allows us to do exactly this, and its called primary associated types: protocol StorageProtocol<Value>: AnyObject { associatedtype Value var value: Value { get set } }

— 21:02

They are associated types that can be recovered from an existential later on if we need.

— 21:08

And the way we can recover the associated type is by defining a generic function that has statically known types and passing an existential through it: func open<Root>(_ storage: some StorageProtocol<Root>) { } open(self.storage)

— 21:57

This is a common trick when dealing with existentials. You can convert this to generics by simply defining a generic function and passing the existential to it.

— 22:05

This is a very powerful feature of Swift. Inside the open function we have an honest, static type that conforms to the StorageProtocol rather than the far weaker any StorageProtocol that we are holding onto in the Shared type.

— 22:18

This means we can access value on this storage to get an honest Root value out of it: func open<Root>(_ storage: some StorageProtocol<Root>) { storage.value as Root }

— 22:32

However, we don’t want a Root value in this getter. We want something of type Value . And the key path is supposed to get us access to that value, so maybe we can just apply it: func open<Root>(_ storage: some StorageProtocol<Root>) { storage.value[keyPath: self.keyPath] }

— 22:49

But now we are back in the type erased world. Since AnyKeyPath is fully erased, key pathing in with it has no choice but to hand us back something of type Any? . However, we know that this key is supposed to be one that goes from Root to Value . If it is not then we have messed something up somewhere.

— 23:25

So, let’s force cast to the shape we know it should be: func open<Root>(_ storage: some StorageProtocol<Root>) { storage.value[keyPath: self.keyPath] as! Value }

— 23:32

Now we have something of type Value , and that is what this function can return: func open<Root>( _ storage: some StorageProtocol<Root> ) -> Value { storage.value[keyPath: self.keyPath] as! Value }

— 23:36

And we can finish off the get by just returning the opening of storage : return open(self.storage)

— 23:59

And we can do something similar for the setter: set { func open<Root>(_ storage: some StorageProtocol<Root>) { storage.value[ keyPath: self.keyPath as! WritableKeyPath<Root, Value> ] = newValue } open(self.storage) }

— 24:54

OK, things are starting to look pretty good. We now have the ability for Shared to hold onto some nebulous, type erased reference of storage and we can use a key path to get a specific piece of data out of that storage, and to set some data in the storage.

— 25:08

We have one final compiler error when trying to define equality on the Shared type. Currently we just defer equality to the synthesized conformance on the Shared struct: extension Shared: Equatable where Value: Equatable {}

— 25:21

But now the struct is holding more exotic data, like existentials, and so the conformance can no longer be automatically synthesized for us.

— 25:30

So we would hope that we could do the work the synthesized conformance did, just manually, since we want to call down to the storage’s equality check: extension Shared: Equatable where Value: Equatable { public static func == (lhs: Self, rhs: Self) -> Bool { lhs.storage == rhs.storage } }

— 25:55

However, such a simple equality check against two any StorageProtocol existentials is no longer going to fly. Swift doesn’t know anything about the type of storage being used in the left-hand side or right-hand side. We may be trying to compare two completely different types of storage for all Swift knows.

— 26:25

So, we need a bit more existential magic. What we can do is add computed properties to Shared for accessing the underlying currentValue and snapshot in its storage: public struct Shared<Value> { … var currentValue: Value { func open<Root>( _ storage: some StorageProtocol<Root> ) -> Value { storage.currentValue[keyPath: self.keyPath] as! Value } return open(self.storage) } var snapshot: Value? { func open<Root>( _ storage: some StorageProtocol<Root> ) -> Value? { storage.snapshot[keyPath: self.keyPath] as? Value } return open(self.storage) } … }

— 29:49

And then we can define Equatable on Shared directly, rather than its storage: extension Shared: Equatable where Value: Equatable { public static func == (lhs: Shared, rhs: Shared) -> Bool { if SharedLocals.isAsserting { return lhs.snapshot ?? lhs.currentValue == rhs.currentValue } else { return lhs.currentValue == rhs.currentValue } } }

— 30:09

And with that everything is compiling, and we have finished implementing this feature. It’s quite an advanced feature, and uses advanced Swift techniques to implement, but it is extremely powerful.

— 30:34

Let’s go back to our sign up case study to see that everything compiles. This means we are now able to dot-chain onto any existing @Shared value in order to instantly derive a whole new @Shared value that is focused in on just one particular property.

— 30:55

And not only does everything compile, but everything also works exactly as it did before. We can go through the whole sign up flow to see that the topics screen does still make mutations to the sign up data even though it doesn’t directly have access to SignUpData at all. It only has access to a set of topics. It doesn’t know where or how it got that shared state, nor does it know if it’s persisted state or in-memory state. But none of that matters. It can read from it and write to it like any other piece of state, and every other feature in the app that has a piece shared state connected to those topics will observe those changes immediately.

— 32:20

And further, testing also works the exact same with derived shared state. You can write tests against these topics exactly like you would against any other kind of regular state or shared state, and it will also do exhaustive checking automatically. We aren’t going to dive into that right now, but we will see an example of that later in the series. Conclusion

— 32:40

We are now finally at the end of our shared state series. Over the last 9 episodes we have accomplished a ton.

— 32:46

First we demonstrated that sharing state in the Composable Architecture is definitely a real problem that requires tools to solve. We had to contort ourselves in strange ways by either using complex computed properties or strange dependencies just to get a single piece of state into multiple parts of the app. Stephen

— 33:05

Then we showed a really nice solution for sharing state: just stick a reference type in your feature’s state! Typically that would be a scary thing to do because reference types used to not play nicely with SwiftUI’s view invalidation mechanism, but that was solved with the new observation tools in Swift 5.9. Brandon

— 33:20

It was also a little scary to stick a reference type in state because reference types are notoriously difficult to test. And so that led us down a path to build all new tooling into the Composable Architecture that allows us to “snapshot” state before and after an action is run, and that gave us the ability to get test coverage on shared state, and even exhaustive test coverage. Stephen

— 33:41

Then we showed that while sharing state with a few features is powerful, there is a much bigger idea lurking in the shadows. Many times you also want to share state with an external system, such as user defaults or the file system. So we showed how with a little bit of upfront work we can automatically persist little bits of our state in the background, and users of the tool don’t have to think about those details at all. Brandon

— 34:04

And then finally we went the extra mile to make sure these persistence tools are well behaved. This includes making sure that they listen for changes in the external system so that they can be played back to the state held in your features, and it meant taking control over the dependencies of user defaults and the file system so that everything can remain 100% testable, and of course exhaustively testable. Stephen

— 34:28

Next week we will show how to use these tools in a more complex and real world application than the little case studies we played around with in this series. This includes the SyncUps app, which is our port of Apple’s Scrumdinger sample code to use more modern SwiftUI techniques. And we will take a look at our isowords app that we open sourced a few years ago. Both of those apps are quite complex, and will benefit greatly from the new shared state tools we have just built.

— 34:54

Until next time! Downloads Sample code 0276-shared-state-pt9 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .