Video #218: Modern SwiftUI: Effects, Part 2

Episode: Video #218 Date: Jan 2, 2023 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep218-modern-swiftui-effects-part-2

Description

We wrap up the “record meeting” screen by implementing two more side effects: speech recognition, and persistence. We’ll experience the pitfalls of interacting directly with these dependencies, and why we should care about controlling them.

Video

Cloudflare Stream video ID: 52aa0047763160aff7a0ef0c780c3a71 Local file: video_218_modern-swiftui-effects-part-2.mp4 *(download with --video 218)*

References

Transcript

— 0:05

We have now built all the behavior for the timer, and it turned out to be pretty complex! We had to create new ways for the child to communicate back to the parent, and we had to implement behavior to pause the timer whenever an alert is shown, and it was pretty cool to see how we were able to accomplish these features.

— 0:20

OK, we are finally ready to attack the speech recognition functionality.

— 0:25

We are going to do the absolute bare minimum of work to implement this behavior by cramming Speech framework code directly into our model. Longtime viewers of Point-Free will see the problems with this right away, but we want to show this approach as it is how Apple’s Scrumdinger was built, and it will make it easier to see the drawbacks. Speech recognition

— 0:43

Let’s dip our toe in the deep end by first asking the user for authorization when the screen first appears. We already have a method that is called when the view appears, and it’s even asynchronous, so maybe we can leverage that in order to suspend while the user is decided whether or not they want to grant us access.

— 0:56

At the call site we might hope it could look at simple as this: @MainActor func task() async { if await self.requestAuthorization() == .authorized { // Authorization granted } }

— 1:22

Perhaps our model as an async function called requestAuthorization , and once the user decides we will get a status back that we can check against.

— 1:40

Apple’s Speech framework does offer a method for asking for authorization, but it is modeled on the old style of completion callbacks. Luckily it’s quite straightforward to convert that type of API into one that plays nicely with async/await using withUnsafeContinuation : import Speech … private func requestAuthorization() async -> SFSpeechRecognizerAuthorizationStatus { await withUnsafeContinuation { continuation in SFSpeechRecognizer.requestAuthorization { status in continuation.resume(returning: status) } } }

— 2:19

That function compiles, and even the earlier theoretical code we sketched out compiles.

— 2:25

Unfortunately we can’t test this in the preview because the Speech framework just doesn’t work at all in Xcode previews. A permission alert isn’t shown, and in fact the requestAuthorization seems to be suspending forever because the timer isn’t even starting anymore.

— 2:41

This just shows why it’s so important to control dependencies and not naively sprinkle 3rd party framework code throughout your code. This preview was fully functional a moment ago, and now it’s just kinda broken.

— 2:52

So, let’s start up the app, start a new meeting, and then we will see the speech permission alert. So that’s pretty cool, but one interesting thing I’m already noticing is that the timer isn’t running. This alert isn’t even being powered off of our Destination enum. It’s completely handled by iOS.

— 3:18

The whole reason this is working is thanks to the magic of structured concurrency. We are awaiting for authorization before starting the timer, and so the correct behavior just kind naturally happens without having to do anything. That’s pretty amazing.

— 3:31

If we grant permission we will see the timer starts up.

— 3:37

Also recall that the behavior we are seeing now has dramatically improved from what we saw in Apple’s Scrumdinger application. In that app we noticed that when you start a meeting and get the alert, you would suddenly be popped back to the detail screen. Our best guess was that happens due to a bug with fire-and-forget navigation links, but because all of our navigation is driven off of state it doesn’t happen. So, that’s great.

— 3:58

OK, we’ve got our first bits of speech code in the feature. If we get into this if conditional, we want to officially start up the speech recognizer, and ideally it would be done in a way that could leverage the already provided structured concurrency context we are in: if await self.requestAuthorization() == .authorized { // await ... }

— 4:12

Now there’s one problem to grapple with here. We will technically have two long living asynchronous units of work. There’s the speech recognition and there’s the timer.

— 4:18

The way the code is structured now, we are going to await until the speech client task is finished, and then we will start the timer. That’s not what we want. Instead, we want to run these two units of work in parallel, and this is precisely what task groups allow for.

— 4:36

It looks roughly something like this: do { try await withThrowingTaskGroup(of: Void.self) { group in group.addTask { // Start speech task } group.addTask { // Start timer } } try await group.waitForAll() } catch { }

— 5:15

We’ve now incurred 2 more levels of indentation just to get these parallel tests set up, and already the timer has a bunch of indentation. Rather than trying to inline all of the work in each of these child tasks, let’s break them out to their own async functions: private func startSpeechRecognition() async throws { // TODO } private func startTimer() async throws { while true { try await Task.sleep(for: .seconds(1)) guard !self.isAlertOpen else { return } self.secondsElapsed += 1 if self.secondsElapsed.isMultiple( of: Int( self.standup.durationPerAttendee.components.seconds ) ) { if self.speakerIndex == self.standup.attendees.count - 1 { self.onMeetingFinished() self.dismiss = true break } self.speakerIndex += 1 } } }

— 6:02

Then we can call out to those async functions from the task method. But, we will first ask for authorization since we do want to wait for that before starting the timer: @MainActor func task() async { let authorizationStatus = await self.requestAuthorization() do { try await withThrowingTaskGroup(of: Void.self) { group in if authorizationStatus == .authorized { group.addTask { try await self.startSpeechRecognition() } } group.addTask { try await self.startTimer() } try await group.waitForAll() } } catch { } }

— 6:15

And since we already have the infrastructure in place, let’s also show an error if one is thrown. We won’t take the time to properly figure out what went wrong and give the user some options for fixing the error, but we will at least get something up: self.destination = .alert( AlertState(title: TextState("Something went wrong.")) )

— 6:39

I’m not exactly sure how we can test this behavior though. We would have to do something to make the speech recognizer throw an error, but that’s not clear how to do. This is another reason we should control dependencies, because then we could force it into this error state to see how our application handles it rather than trying to make the Speech framework error.

— 6:57

Still, this is looking pretty great, but we did start to get some warnings: Capture of ‘self’ with non-sendable type ‘RecordMeetingModel’ in a @Sendable closure

— 7:01

Recall that we cranked the concurrency warnings up to their max in the Xcode project settings, and so that is why we are seeing this. The RecordMeetingModel is not sendable, as it is just a class and has a whole bunch of mutable data inside, but the closure for addTask requires a @Sendable closure since it can execute concurrently.

— 7:20

The way to make our model Sendable is to mark it with @MainActor : @MainActor class RecordMeetingModel: ObservableObject { … }

— 7:27

With that, this file compiles with no warnings, but we do now have to add @MainActor to the StandupDetailModel and StandupsListModel .

— 7:59

OK, we now have a brand new asynchronous function for us to cram all of our speech recognition logic. In here we want to start up a new speech recognition task and get notified whenever the Speech framework transcribes some audio. The code to do this is decently complex. It requires setting up an AVAudioEngine , piping its output to a speech recognition task, and then listening for results that the speech engine emits. All of these pieces are well covered in Apple’s sample code for the Speech framework, so we aren’t going to dive into the details.

— 8:29

Instead, we are going to paste in a very lightweight wrapper that abstracts away the work we just described. We’ll start a new file called Speech.swift .

— 8:40

Then we will do a @preconcurrency import of Speech because it hasn’t yet been audited for concurrency issues and so it will cause a lot of warnings: @preconcurrency import Speech

— 8:52

And then we will paste in a little type that abstracts away the work of starting up a speech recognition task and listening for transcripts: actor Speech { private var audioEngine: AVAudioEngine? = nil private var recognitionTask: SFSpeechRecognitionTask? = nil private var recognitionContinuation: AsyncThrowingStream< SFSpeechRecognitionResult, Error >.Continuation? func startTask( request: SFSpeechAudioBufferRecognitionRequest ) -> AsyncThrowingStream< SFSpeechRecognitionResult, Error > { AsyncThrowingStream { continuation in self.recognitionContinuation = continuation let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setCategory( .record, mode: .measurement, options: .duckOthers ) try audioSession.setActive( true, options: .notifyOthersOnDeactivation ) } catch { continuation.finish(throwing: error) return } self.audioEngine = AVAudioEngine() let speechRecognizer = SFSpeechRecognizer( locale: Locale(identifier: "en-US") )! self.recognitionTask = speechRecognizer .recognitionTask(with: request) { result, error in switch (result, error) { case let (.some(result), _): continuation.yield(result) case (_, .some): continuation.finish(throwing: error) case (.none, .none): fatalError( """ It should not be possible to have \ both a nil result and nil error. """ ) } } continuation.onTermination = { [audioEngine, recognitionTask] _ in _ = speechRecognizer audioEngine?.stop() audioEngine?.inputNode.removeTap(onBus: 0) recognitionTask?.finish() } self.audioEngine?.inputNode.installTap( onBus: 0, bufferSize: 1024, format: self.audioEngine?.inputNode .outputFormat(forBus: 0) ) { buffer, when in request.append(buffer) } self.audioEngine?.prepare() do { try self.audioEngine?.start() } catch { continuation.finish(throwing: error) return } } } }

— 8:58

As you can see there’s quite a bit in there, but we don’t need to really know about the internals.

— 9:03

But, we can immediately start making use of the object in the startSpeechRecognizer method. We will construct a speech request, pass it to the startTask endpoint, and for try await it: private func startSpeechRecognition() async throws { let speech = Speech() let request = SFSpeechAudioBufferRecognitionRequest() for try await result in await speech.startTask( request: request ) { } }

— 9:40

Now, it may be hard to believe, but with this little bit of code we really are transcribing live audio from the device. Just to check it out let’s add a print statement to the for await so that we can see what is the best transcription so far: for try await result in await self.startTask(request) { print(result.bestTranscription.formattedString) }

— 9:54

If we run the app in the simulator and start a meeting, we will see a live stream of text in the console that roughly matches the words we are speaking.

— 10:02

So, we just need to keep track of the last best transcript so that when the meeting ends we can save it to the standup’s history. To do this we will add a little bit of private state to the model, which does not need to be @Published since the view does not care about it: class RecordMeetingModel: ObservableObject { private var transcript = "" … }

— 10:25

And we will assign that state whenever the speech client delivers a new result: for try await result in await self.startTask(request) { self.transcript = result.bestTranscription.formattedString }

— 10:35

Then we just need some way to communicate this transcript to the parent.

— 10:39

We already have the onMeetingFinished closure for communicating to the parent when the meeting finishes, so perhaps it should now take an argument so that we can send the transcript to the parent: var onMeetingFinished: (String) -> Void = unimplemented( "RecordMeetingModel.onMeetingFinished" )

— 10:49

Then we need to update the call sites of this closure to include the transcript: self.onMeetingFinished(self.transcript)

— 11:01

And finally we need to update where we override this closure so that it is handed the transcript, and then we can insert the meeting into the top of the history: case let .record(recordMeetingModel): recordMeetingModel.onMeetingFinished = { [weak self] transcript in guard let self else { return } self.standup.meetings.insert( Meeting( id: Meeting.ID(UUID()), date: Date(), transcript: transcript ), at: 0 ) self.destination = nil }

— 11:33

And just like that finished implementing the core speech feature of the application. There’s still more polish to apply, but this is basically it.

— 11:42

If we start a meeting and let it run out its timer, or if we end it early, we will see that we pop back to the detail screen and there’s a new history item in the list. We can even drill down to that historical meeting and we will see the transcript. So, that’s absolutely amazing.

— 12:01

However, as we mentioned before, the user experience of this flow wasn’t as good as it could be. When we animated back from the record meeting screen to the detail screen, there was no indication that a meeting was inserted into the history list. And it can be very easy to overlook that.

— 12:15

It would be better if when we popped back there was an animation of the meeting being inserted into the list. This is technically easy enough with SwiftUI: withAnimation { _ = self.standup.meetings.insert( Meeting( id: UUID(), date: Date(), transcript: transcript ), at: 0 ) }

— 12:20

But, is even that sufficient? If we run the app again, start the meeting and end the meeting, we will see a hint of an animation, but it happens in unison with the pop animation, and so can be easy to miss.

— 12:46

It would be far better if we waited a small amount of time so that the pop finishes, and then inserted the meeting with animation. Thanks to structured concurrency this is quite easy to do.

— 13:10

Essentially we want to sleep for a small amount of time before inserting with animation: try await Task.sleep(for: .milliseconds(100)) withAnimation { _ = self.standup.meetings.insert( Meeting( id: UUID(), date: Date(), transcript: transcript ), at: 0 ) }

— 13:22

Now there are two things wrong with this. First, we don’t have a throwing context to be doing try , and we don’t have an asynchronous context to be doing await .

— 13:31

For the try , we don’t actually want to throw here. The only time Task.sleep throws is when it is cancelled, and even if the asynchronous context is cancelled we still want to perform the insertion logic. So, let’s just ignore the cancellation error: try? await Task.sleep(for: .milliseconds(100))

— 13:43

And then, in order to await we need an asynchronous context, so let’s spin up an unstructured task: Task { try? await Task.sleep(for: .milliseconds(100)) withAnimation { _ = self.standup.meetings.insert( Meeting( id: UUID(), date: Date(), transcript: transcript ), at: 0 ) } }

— 13:53

With that, we should be able to test out the delay in animation. But, I don’t know about you, but I’m kinda get tired of needing to load up the whole app over and over just to test this one tiny flow.

— 14:02

I would love if I could start testing this in a preview. Then I could make a tweak, the preview would automatically refresh, and I could see the effect.

— 14:08

And we seem to have that ability. After all, I can very easily configure the StandupDetail preview to start in a very specific state. I can give it a standup with a single attendee and a 1 second duration, and have it drilled down to the record screen: struct StandupDetail_Previews: PreviewProvider { static var previews: some View { NavigationStack { var standup = Standup.mock let _ = standup.duration = .seconds(1) let _ = standup.attendees = [ Attendee(id: UUID(), name: "Blob"), ] StandupDetailView( model: StandupDetailModel( destination: .record( RecordMeetingModel(standup: standup) ), standup: standup ) ) } } }

— 15:02

It’s amazing to see that this is possible, and the preview does indeed start in that exact state. But sadly, it doesn’t really work. Due to the problem we saw earlier, the timer doesn’t actually start, and that’s because the Speech framework is just completely broken in previews.

— 15:17

Again, we are seeing why it is so important to control our dependencies. I can’t use the preview to play around with this one use case, so I guess we will have to go back to the simulator. At least we can edit the app entry point to start in the exact state we want.

— 15:44

Let’s launch the app, let the meeting finish, and see the pop animation with the new meeting being inserted after a 100 millisecond delay.

— 15:49

It looked a little better than before, but I don’t think our delay is quite long enough. Let’s up it a bit: try? await Task.sleep(for: .milliseconds(1000))

— 16:00

If we launch the app again and wait for the meeting to end we will see that it is way too long. Let’s split the difference: try? await Task.sleep(for: .milliseconds(400))

— 16:11

OK, much better, and that’s the delay we will go with. It’s a bummer that we had to resort to the simulator in order to iterate on such basic functionality of our feature. The whole point of Xcode previews is to be able to repeatedly make small tweaks and instantly see the result.

— 16:26

And even worse, because we are editing the actual, real entry point of the application, we will always have to remember to clean up this code later. We would never want to ship this code to the App Store.

— 16:36

This is why it is so important to control dependencies and not let them control you. It’s also good motivation for maintaining “preview” apps, which allow you to run portions of your application is standalone apps, that way you don’t have to make changes to the entry point of your production app. Persistence

— 16:50

There’s only one feature left to implement in this app in order to bring it to full parity with Apple’s Scrumdinger app, and that’s persistence. The Scrumdinger app implements persistence by saving whenever the app is backgrounded. That gets the job done, but it’s maybe not ideal. If your app crashes then you will not have saved any data since the last time the app was backgrounded.

— 17:11

So, we will take a different approach. We will save the data whenever the data changes, and we will see that introduces some interesting challenges with interesting solutions.

— 17:22

The most natural place to put the persistence logic is in the StandupsListModel . It is the root model that powers the root view, and it is only created a single time for the entire duration of the application. That makes it the perfect place to load up the previously saved standups and listen for changes in the standups so that we can save.

— 17:40

First, let’s create a little private helper for calculating the URL on disk where we will be saving and loading this data. It will need to be used in at least two spots, so it would be best to not have to repeat this logic: extension URL { static let standups = Self.documentsDirectory .appending(component: "standups.json") }

— 17:52

Then, in the initializer, we can try loading that json file and decoding it into an array of standups.

— 18:01

However, currently we support passing in an array of standups when initializing the model: standups: IdentifiedArrayOf<Standup> = []

— 18:06

This was great for starting up the app in a specific state so that you can test things. For example, what if you wanted to see how the app performs with 10,000 meetings. Well, this initializer makes that quite easy.

— 18:18

However, there are now two conflicting ways of providing the default standups. It can be passed into the initializer, but we also want to load it from disk. It’s sad, but I think we’re going to have to give up the initializer argument since persistence is an actual product requirement of our application: init(destination: Destination? = nil) { self.destination = destination self.standups = [] … }

— 19:07

And then we can load the standups from disk: do { self.standups = try JSONDecoder().decode( IdentifiedArray.self, from: Data(contentsOf: .standups) ) } catch { // TODO: Handle error }

— 19:44

We aren’t going to do the error handling now, but we do have some exercises for our viewers at the bottom of the episode page.

— 20:01

That takes care of the loading standups, but what about saving?

— 20:09

The standups array in the model is @Published , which means it is very easy for us to subscribe to any changes: self.$standups .sink { standups in }

— 20:33

And so inside this sink closure we could save the standups to disk: self.$standups .sink { standups in do { try JSONEncoder() .encode(standups) .write(to: .standups) } catch { } }

— 21:05

Also, there’s no reason to save the first emission of this publish since it represents the value we just loaded from disk. So, we can drop it using a Combine operator: self.$standups .dropFirst() .sink { standups in … }

— 21:17

Now we have a warning because this subscription creates a cancellable, and we have to keep that cancellable around if we want the subscription to keep living. Since we want to tie the subscription to the lifetime of the model, we need a cancellable store: private var cancellables: Set<AnyCancellable> = []

— 21:40

And assign it: self.$standups … .store(in: &self.cancellables)

— 21:44

That’s all it takes. We have now implemented persistence. We can launch the app, add a few meetings, record a meeting, and then force quit the app. When we relaunch the app we will see all the data has been restored.

— 22:24

Let’s make a quick improvement to this. It seems like it could be possible that items in the array change very quickly, causing us to save to the disk many times in rapid succession. It would be better if we could space saves out a bit. And since we are already dealing with a Combine publisher we can easily make use of the debounce operator: self.standupsCancellable = self.$standups .dropFirst() .debounce( for: .seconds(1), scheduler: DispatchQueue.main ) .sink { standups in … }

— 22:49

This will make it so that at least a second has to pass after the standups update before we will save. So we will never save more than once in a second interval. Next time: uncontrolled dependencies

— 22:58

Alright, so we have now introduced 3 pretty complex effects into our application. We are dealing with timers that can be paused while alerts are up, we’re requesting speech authorization from the user and starting up a speech recognition task in parallel with the timer, and on top of all of that we are listening to any changes made to the array of standups, debouncing them for a second, and then persisting the data. Oh, and we also load that data on launch.

— 23:22

Without these effects our little demo was nothing more than a “cute” toy. Sure we had some fun interactions like sheets, drill downs and alerts, but everything was implemented with just simple state mutation. There was no interaction with the outside world. These effects have added a whole new dimension of behavior to the demo and turned it into a full blown application.

— 23:42

But with that new behavior comes new challenges. We have opened up Pandora’s box of complexity and unknowability in our codebase. We already saw this in concrete terms where we saw we have effectively broken the “record meeting” preview due to the fact that it interacts with Apple’s Speech framework directly, which does not work in Xcode previews. And we saw that when we added persistence we destroyed our ability to open up the application or preview into a state with a bunch of standups stubbed in because now that data has to come from the disk.

— 24:13

And if those problems weren’t bad enough, we also don’t have any hope of writing unit tests for any of those code. The Speech framework doesn’t work at all in unit tests, and because we have a real life timer we are going to have to wait for real life time in our tests, which will slow down the tests. And because we are reading and writing to the real disk we are going to have to be careful to clean up after tests, or else that data will start to leak across tests, causing mystifying test failures.

— 24:40

This is what motivates us to finally consider properly controlling our dependencies on things like timers, the Speech framework, and disk access. Doing so allows us to fix all of these problems and more.

— 24:53

So, let’s quickly look at all the problems that crop up when dealing with uncontrolled dependencies, and then let’s fix them. References Getting started with Scrumdinger Apple Learn the essentials of iOS app development by building a fully functional app using SwiftUI. https://developer.apple.com/tutorials/app-dev-training/getting-started-with-scrumdinger SyncUps App Brandon Williams & Stephen Celis A rebuild of Apple’s “Scrumdinger” application that demosntrates how to build a complex, real world application that deals with many forms of navigation (e.g., sheets, drill-downs, alerts), many side effects (timers, speech recognizer, data persistence), and do so in a way that is testable and modular. https://github.com/pointfreeco/syncups Downloads Sample code 0218-modern-swiftui-pt5 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .