Video #275: Shared State: File Storage, Part 1

Episode: Video #275 Date: Apr 15, 2024 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep275-shared-state-file-storage-part-1

Description

While user defaults is convenient for persisting simple bits of state, more complex data types should be saved to the file system. This can be tricky to get right, and so we take the time to properly handle all of the edge cases.

Video

Cloudflare Stream video ID: bbd70da9bcdb267700c09142f44fceb7 Local file: video_275_shared-state-file-storage-part-1.mp4 *(download with --video 275)*

Transcript

— 0:05

We have accomplished quite a bit in this series so far. We now have 2 primary methods of sharing state in a Composable Architecture application:

— 0:13

We can use a simple, unadorned @Shared property wrapper to represent a piece of state that is passed in from the parent feature, and shares its state with the parent.

— 0:21

Or we can provide a persistence strategy to the @Shared property wrapper that describes how we want the state to be persisted to an outside system. This makes it possible to globally share the state with any part of the application without needing to explicitly pass it around, and even persist the data to some external storage. Currently we have two persistence strategies: We have an in-memory persistence strategy that just doesn’t persist data at all. This means you will lose the data when the app re-launches, but that can be fine for many use cases. And we have a user defaults persistence strategy that saves simple data types to user defaults. We even went the extra mile to observe changes to user defaults so that if someone makes a change outside of the @Shared property wrapper we can be sure to update our state. Brandon

— 1:02

This is all great, but user defaults persistence is a bit limited. It only works for the most basic of data types, such as booleans, integers and strings, and therefore is appropriate for little scraps of data and user preferences.

— 1:16

There are many times that we need to persist far more significant chunks of data, and the easiest place to do that is directly on the device’s file system. And the easiest way to serialize the data to disk is via Swift’s Codable protocol and JSON serialization.

— 1:31

Let’s see what it takes to introduce a 3rd conformance to the PersistenceKey protocol and make file storage a reality in Composable Architecture features. Persistence to file storage

— 1:42

Let’s start by defining a whole new type that conforms to the PersistenceKey protocol. As we’ve seen from the other persistence strategies, it will need to be public, but it isn’t surfaced to the user in any real way, so we can underscore it: public struct FileStorageKey: PersistenceKey { }

— 1:56

It will be generic over the type of data it saves to the file system, just as the AppStorageKey persistence strategy was generic over the time of data it saves in user defaults: public struct FileStorageKey<Value>: PersistenceKey { }

— 2:03

And the value being persisted will need to be Codable so that we can use a JSON encoder and decoder: public struct FileStorageKey< Value: Codable >: PersistenceKey { }

— 2:14

Now let’s try implementing its requirements. First we have the load endpoint: public func load() -> Value? { }

— 2:20

This needs to load data from the file system, if possible, and then decode that data into the Value type. In order to do this we need to have a URL on the file system where we are loading from and saving to. This can be a property on the FileStorageKey type: public struct FileStorageKey< Value: Codable >: PersistenceKey { let url: URL … }

— 2:38

And now load can be implemented in a straightforward manner: public func load() -> Value? { try? JSONDecoder().decode( Value.self, from: Data(contentsOf: self.url) ) }

— 2:58

Similarly, save can also be implemented quite easily by first encoding the value to data, and then writing that data to the URL provided: public func save(_ value: Value) { try? JSONEncoder().encode(value).write(to: self.url) }

— 3:16

And finally let’s not worry about the updates stream. It’s purpose is to notify the Shared type of when something changes on the file system, but we won’t worry about that right now: public var updates: AsyncStream<Value> { .finished }

— 3:41

And it may seem hard to believe, but that right there is all it takes to get some basic persistence to the file system.

— 3:48

We can expose this new persistence strategy to the world by adding another static func to the PersistenceKey protocol: extension PersistenceKey { public static func fileStorage<Value>( _ url: URL ) -> Self where Self == FileStorageKey<Value> { Self(url: url) } }

— 4:37

And now we can start making use of it.

— 4:41

We’ve got a great use case for this in the shared state case study. It has a shared stats value that multiple features want to access and mutate. And currently we are sharing this state via the “in-memory” persistence strategy: @Shared(.inMemory("stats")) var stats = Stats()

— 4:54

This means this stats value is immediately available to any feature that wants it, and all features will see the exact same value. But, if we relaunch the app the stats will be cleared and we will start back at the defaults.

— 5:26

Let’s upgrade this to use the new .fileStorage persistence strategy. First we need the stats value to be Codable so that it can be serialized and deserialized: struct Stats: Codable, Equatable { … }

— 5:36

And we need a URL to specify where on the file system we want to save the data: extension URL { static let stats = Self .documentsDirectory .appending(path: "stats.json") }

— 5:57

And now we can replace all instances of the .inMemory("stats") with a .fileStorage(.stats) : @Shared(.fileStorage(.stats)) var stats = Stats()

— 6:18

It is a seemingly small change but with big ramifications.

— 6:27

The application now runs mostly as it did before. We can still increment and decrement, switch tabs to see the data is shared between features, and we can reset the stats to see that the first tab saw those changes.

— 6:45

But incredible, all of these changes to the data was persisted to disk. Let’s count up a few more times, and then relaunch the application. The demo is restored back to the exact state we left it in. We can make a few more changes, and then relaunch again.

— 7:09

Again all of the data is loaded exactly as we left it during the last run of the application. Debouncing persistence

— 11:25

It’s pretty incredible how easy it was to fit in an all new persistence strategy into the @Shared property wrapper. We now have 3 persistence strategies:

— 11:35

In-memory, which allows us to share any kind of data throughout the application, but it is not preserved across app launches.

— 11:42

App storage, which allows us to store very simple data in user defaults.

— 11:47

And file storage, which allows us to store Codable data in the file system. Stephen

— 11:52

So, this is looking great, but there are a few improvements we can make. For one thing, right now we are saving the data to disk after every single change to the value. This means if we did a loop to increment the counter 1,000 times, we would write to the disk 1,000 times.

— 12:07

This may have been fine to do in user defaults, but I don’t think it’s correct for the file system. Rather than thrashing the file system with a save on every little change, it would be better if we introduced some debouncing logic to the file storage strategy. We could decide to save to the file system only if 5 seconds have passed since the last time the value was changed.

— 12:25

Let’s see what it takes to accomplish that.

— 12:30

There are a few ways we can accomplish debouncing behavior in the FileStorageKey type, but no matter what we’re going to need to upgrade the type to be a class: public final class FileStorageKey<Value: Codable>: PersistenceKey { }

— 12:42

This is because we are going to need to track some internal state for debouncing, and we can’t do that with a struct alone.

— 12:49

This means we need to provide an explicit initializer, and we lose synthesized Hashable conformance, so we have to add all of that manually: init(url: URL) { self.url = url } public func hash(into hasher: inout Hasher) { hasher.combine(self.url) } public static func == ( lhs: FileStorageKey, rhs: FileStorageKey ) -> Bool { lhs.url == rhs.url }

— 13:10

Now we can store some mutable state in this type. There are a few ways to do this, but we will just go with using tools from Grand Central dispatch. That library comes with a type that represents a unit of work that can be enqueued on a DispatchQueue . So we will hold onto one of those units of work that represents us saving: public class FileStorageKey<Value: Codable>: PersistenceKey { … var saveWorkItem: DispatchWorkItem? … }

— 13:34

Then when saving we can cancel any in-flight work items being executed: public func save(_ value: Value) { self.saveWorkItem?.cancel() … }

— 13:44

Then construct a whole new work item to perform the saving work: self.saveWorkItem = DispatchWorkItem { [weak self] in guard let self else { return } try? JSONEncoder().encode(value).write(to: url) saveWorkItem = nil }

— 14:05

And then enqueue this work item onto a dispatch queue. But we don’t have a dispatch queue right now. Let’s create one that can be used for any shared state that is persisted with file storage: let saveQueue = DispatchQueue( label: "co.pointfree.save" )

— 14:22

And now we can enqueue on that queue, with a delay of, say, 5 seconds: public func save(_ value: Value) { self.saveWorkItem?.cancel() self.saveWorkItem = DispatchWorkItem { [weak self] in guard let self else { return } try? JSONEncoder().encode(value).write(to: url) saveWorkItem = nil } saveQueue.asyncAfter( deadline: .now() + 5, execute: self.saveWorkItem! ) }

— 14:36

This means that if two invocations of save happen back to back, the second invocation will cancel the first work item, and the saveQueue will know it doesn’t even need to wait around for 5 seconds to execute the item since it was cancelled.

— 14:50

And believe it or not, that’s all it takes. We can even prove this debouncing is actually happening by launching the app, incrementing, and instantly relaunching the app. We will see that the data did not save. But if we increment again, wait 5 seconds, and then re-launch the app, we will see that the data was saved. Saving on willResignActive

— 15:20

But of course it would be quite a bummer if we lost data just because the user decided to force quit the app before the 5 seconds elapsed. Luckily there is a straightforward way to fix this.

— 15:28

There is a way to be notified when the user is about to leave the app and put it in the background. This is done through NotificationCenter , and we can even use the callback-based API: NotificationCenter.default.addObserver( forName: <#NSNotification.Name?#>, object: <#Any?#>, queue: <#OperationQueue?#>, using: <#(Notification) -> Void#> )

— 15:48

The name of the notification we want to listen for is UIApplication.willResignActiveNotification : NotificationCenter.default.addObserver( forName: UIApplication.willResignActiveNotification, object: <#Any?#>, queue: <#OperationQueue?#>, using: <#(Notification) -> Void#> )

— 15:56

The object represents the object that posts the notification, so that you can narrow down the vast sea of notifications to only those associated with something specific. But in the case of Notification Center this is not used: NotificationCenter.default.addObserver( forName: UIApplication.willResignActiveNotification, object: nil, queue: <#OperationQueue?#>, using: <#(Notification) -> Void#> )

— 16:07

The queue is where we want to execute the trailing closure callback when the notification is posted. This is actually an OperationQueue , not a DispatchQueue , so we have a bit of impedance mismatch here. But we aren’t going to do any heavy work in the callback of this API anyway, so this isn’t super important for us: NotificationCenter.default.addObserver( forName: UIApplication.willResignActiveNotification, object: nil, queue: nil, using: <#(Notification) -> Void#> )

— 16:27

This API does highlight just how nice Swift’s new concurrency tools are when compared to the older tools.

— 16:41

In modern Swift concurrency we have the ability for caller to determine the global actor it wants an async unit of work to execute on with a very lightweight syntax. If we wanted to execute on the main thread we could do: using: { @MainActor in }

— 16:57

Or any global actor: using: { @SomeGlobalActor in }

— 17:01

This syntax marries the concept of what should be executed when a notification is posted with the context in which it should execute. The user of this kind of API has full control.

— 17:13

Whereas with the current Notification Center API we have to pass the context and closure as two separate units, and just hope that the Notification Center is going to do the right thing under the hood. Nothing is stopping the method from simply discarding the queue and executing our callback on any thread it wants. But in the modern concurrency style that becomes impossible. The API doesn’t get to choose the context the async closure is executed in. The caller does.

— 17:39

But those remarks aside, we can now provide a trailing closure which will be called when the notification is posted: NotificationCenter.default.addObserver( forName: UIApplication.willResignActiveNotification, object: nil, queue: nil ) { _ in }

— 17:47

In here we want to immediately perform any work item that is in-flight, and if none is in-flight it means that nothing has changed since last time we saved, and so there is nothing to do.

— 17:57

So, in this closure we will immediately execute the work item on the saveQueue , if it exists: NotificationCenter.default.addObserver( forName: UIApplication.willResignActiveNotification, object: nil, queue: nil ) { [weak self] _ in guard let self, let saveWorkItem else { return } saveQueue.async(execute: saveWorkItem) }

— 18:22

But, technically the work item was already put in the saveQueue , and so will be executed again once its time is up. So, after this execution is done, let’s also cancel the work item and nil out the state: saveQueue.async { self.saveWorkItem?.cancel() self.saveWorkItem = nil }

— 18:45

That is the basics of listening to the “will resign” notification. Technically this method also returns an object that we could hold onto in order to cancel at a later time. But since this FileStorageKey live for the entire lifetime of the app that doesn’t seem necessary.

— 19:02

We can now give this a spin to see that it works. We can increment the count, immediately relaunch the app, and we will see that the data did save. And so now we are saving every time the data changes in the @Shared property wrapper, but debounced by 5 seconds, and we are saving every time the app is put into the background. Observing external writes to file system

— 19:27

So it seems we have implemented all the most important parts of the file storage persistence strategy. We are now loading the initial data for a @Shared piece of state from disk, and when data changes in @Shared it is automatically persisted to disk. We’ve even baked in some smarts so that we don’t thrash the disk with too many writes, while also making sure we don’t lose data if someone force quits the app. Brandon

— 19:48

However, there is still a pretty important part missing from the .fileStorage persistence strategy, and that is the updates stream. Currently it is stubbed out as a stream that never emits and completes immediately, but this stream serves an important purpose. It’s our way of communicating to the @Shared property wrapper that something in the external system has changed, and so the @Shared data should change.

— 20:14

We used this to great effect with our user defaults persistence strategy. It allowed any part of the application to make a change to user defaults, and the @Shared property wrapper would instantly take notice and update its data.

— 20:28

We would like the same with file storage. After all, the file system is a global blob of mutable state that anyone can write to. If someone were to write over the stats.json file we are using to power our demo, then we would certainly want the UI to update accordingly.

— 20:45

So, let’s see what it takes to accomplish this.

— 20:48

Right now we are returning a .finished stream in the updates endpoint, but now it’s time to do some real work: public var updates: AsyncStream<Value> { AsyncStream { continuation in } }

— 21:13

In this stream we need to somehow subscribe to changes to the file system so that we can detect when a file changes.

— 21:26

It turns out that Grand Central Dispatch provides just the tool for this. It’s called DispatchSource , and it is capable of listening to writes to a particular file on disk. It’s an older style API so it will look a little strange at first: DispatchSource.makeFileSystemObjectSource( fileDescriptor: <#Int32#>, eventMask: <#DispatchSource.FileSystemEvent#>, queue: <#DispatchQueue?#> )

— 21:50

It takes 3 arguments. The first is a fileDescriptor , which is expressed as a raw Int32 . To get this value we use the C function called open , that takes the path we want to observe, and a flag for how we want to interact with the file handle. In this case we only care about observing changes to this file handle, and we won’t use it to read/write to the file, so we’ll use “event only.” fileDescriptor: open(self.url.path, O_EVTONLY),

— 22:32

The eventMask argument lets us decide which events we want to monitor: eventMask: .write,

— 22:40

And then finally the third argument is a dispatch queue that the dispatch source will use to notify us when an event occurs. We can use our saveQueue for this: queue: saveQueue

— 22:51

This method returns a source, so we will assign it: let source = DispatchSource.makeFileSystemObjectSource( fileDescriptor: open(self.url.path, O_EVTONLY), eventMask: .write, queue: saveQueue )

— 22:56

And we can set the event handler for the source: source.setEventHandler { }

— 23:04

This is the closure that will be called when the dispatch source detects something is written to the file we specified.

— 23:13

All we need to do is load the newest value from disk, and then pass it along to the continuation of the stream: source.setEventHandler { guard let value = self.load() else { return } continuation.yield(value) }

— 23:34

Then we have to explicitly tell the source to start listening for events by invoking the resume method: source.resume()

— 23:49

And finally we can make sure the source stays alive as long as the stream by capturing it in the onTermination callback of the continuation. We can even cancel the source and close the file descriptor at that point: continuation.onTermination = { _ in source.cancel() close(source.handle) }

— 24:12

And believe it or not, that is all it takes. We are now observing when anyone makes a change to the file directly, and that will cause the updates stream to emit, and the @Shared property wrapper will take care of updating its state, and that will cause the view to automatically update.

— 24:37

To verify this, let’s add a button to the UI that overwrites the stats.json file with some new stats that have completely random value in its fields: Button("Write directly to stats.json") { try? JSONEncoder().encode( Stats( count: .random(in: 1...1_000), maxCount: .random(in: 1...1_000), minCount: .random(in: 1...1_000), numberOfCounts: .random(in: 1...1_000) ) ) .write(to: .stats) }

— 25:57

When we tap this button we will see the UI immediately update with the freshest version of the stats. And this is happening even though we are not editing the state directly held in the feature. We are instead writing to a file on disk , but the library is able to correctly observe that change and do the right thing.

— 26:48

It’s pretty incredible. Fixing feedback loop

— 26:50

But there is one small problem here. We have a bit of a feedback loop between our saving logic and our updates logic. If we put a print statement just before we save the data to disk: print("SAVE!!!") try? JSONEncoder().encode(value).write(to: url)

— 27:30

Then run the app and tap the “+” button, we will see that after a few seconds it saves. But also, if we wait a few more seconds it says again. And it does it again, and again, and again.

— 27:50

The problem is that FileStorageKey is saving the current data to disk after it waits 5 seconds after the data changed, and that’s good. But that causes the dispatch source to observe those changes and the load the freshest data from disk. But then that triggers another 5 second debounce, which causes a save. And the cycle repeats.

— 28:41

We can break this cycle by introducing a _currentValue property to the Shared type’s Storage class, which does not have a didSet and does not notify the strategy that it should save. var _currentValue: Value

— 28:41

And then currentValue can wrap around this private storage as a computed property and add the did-set hook manually: var currentValue: Value { get { self._currentValue } set { self._currentValue = newValue self.persistenceKey?.save(newValue) } }

— 29:17

And now, when external updates are fed into the system, we will bypass this save by updating the underscored property directly: if let updates = persistenceKey?.updates { Task { @MainActor in for await value in updates { self._currentValue = value } } }

— 29:28

And that right there is enough to break the feedback loop. We can run the app and see that the repeating “SAVE!!!” is no longer being printed to the console. Next time: Testable file storage

— 29:58

We have now made some incredible improvements to the library. We have a .fileStorage persistence strategy that sits right alongside the .appStorage strategy we created last episode. Where .appStorage is great for persisting tiny bits of simple data in a lightweight way, .fileStorage is great for persisting large, complex data structures. And we’ve even built in some niceties in the strategy such as debouncing save tasks so that we don’t thrash the file system with disk writes, and we are even listening for changes on the disk so that we can detect external writes and update our shared state accordingly. Stephen

— 30:35

But what about testing? We saw with .appStorage we had to do a little bit of extra work to make sure that shared state didn’t bleed out from test to test, and I would imagine the same is true of .fileStorage . After all, the file system is a large, global, mutable blob of data that anyone can access and write to at any time. Surely that is going to complicate testing.

— 30:55

Let’s take a look…next time! Downloads Sample code 0275-shared-state-pt8 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .