Video #122: Parsing Xcode Logs: Part 2

Episode: Video #122 Date: Oct 26, 2020 Access: Members Only 🔒 URL: https://www.pointfree.co/episodes/ep122-parsing-xcode-logs-part-2

Description

We finish up our XCTest log parser by parsing out the data associated with a test failure. Once done we will format the results in a pretty way and package everything up in a CLI tool we can run in our own projects.

Video

Cloudflare Stream video ID: 1b95a1f59c950a59c5e6c72033eb3082 Local file: video_122_parsing-xcode-logs-part-2.mp4 *(download with --video 122)*

References

Transcript

— 0:37

So we’ve done a little bit of clean up, but now let’s get back to where we were before this diversion. We were in the process of cooking up little parsers to help us parser the body of the test failure. Parsing the test case body

— 0:49

Using our new helpers we should now be able to write our test case body parser more simply. We can start by using prefix(upTo:) to capture everything through the body of a test case. let testCaseBody = Parser.prefix(upTo: "Test Case '-[")

— 1:12

But there’s more information in the test body that we want to extract, starting with the file name, so let’s zip the file name parser before matching this prefix. let testCaseBody = zip( fileName, .prefix(upTo: "Test Case '-[") )

— 1:53

And between the file name and the test case body there’s some additional context, which is the line number. We can extract this information out by introducing just a few more parsers: one that parses off a literal ":" , another that parses off an integer, and another that parses through to the beginning of the test failure using the token "] : " . let testCaseBody = zip( fileName, ":", .int, .prefix(through: "] : "), .prefix(upTo: "Test Case '-[") )

— 3:18

And we’ve now consumed the entirety of the test failure body, but we have a bunch of data we don’t care about. The only things we want to keep are the file name, line number and failure message, which we can package up into a tuple: let body = zip( fileName, ":", .int, .prefix(through: "] : "), .prefix(upTo: "Test Case '-[") ) .map { fileName, _, line, _, failureMessage in (fileName, line, failureMessage) }

— 4:25

Let’s give this parser a spin by running it: body.run( """ /Users/point-free/projects/swift-composable-architecture/Examples/VoiceMemos/VoiceMemosTests/VoiceMemosTests.swift:107: error: -[VoiceMemosTests.VoiceMemosTests testPermissionDenied] : XCTAssertTrue failed Test Case '-[VoiceMemosTests.VoiceMemosTests testPermissionDenied]' failed (0.003 seconds). """ ) // ("VoiceMemosTests.swift", 107, "XCTAssertTrue failed\n")

— 5:00

One thing to notice is that we have a trailing newline in the failure message, so we can drop that in our map : (fileName, line, failureMessage.dropLast())

— 5:44

The next chunk we need to parse is the line that concludes this particular test case running: Test Case '-[VoiceMemosTests.VoiceMemosTests testDeleteMemoWhilePlaying]' failed (0.008 seconds).

— 6:12

The only thing we want to extract out of this line is the amount of time the test took to run. Seems like we can consume everything up until we reach the " (" segment, then consume the double, and then finally consume everything until the end of the line. This can be done by zipping a few parsers up: let testCaseFinishedLine = zip( .prefix(through: " ("), .double, " seconds).\n" ) .map { _, time, _ in time }

— 7:21

We can take it for a spin and see that we can successfully extract the duration of a test: testCaseFinishedLine.run( """ Test Case '-[VoiceMemosTests.VoiceMemosTests testPermissionDenied]' failed (0.003 seconds). """ ) // (match 0.003, rest "")

— 7:47

And we now have all the parts to parse a single test result from the front of the logs string and package it up into a TestResult value: let testFailure: Parser<TestResult> = zip( testCaseStartedLine, testCaseBody, testCaseFinishedLine ) .map { testName, bodyData, time in TestResult.failed( failureMessage: bodyData.2, file: bodyData.0, line: bodyData.1, testName: testName, time: time ) }

— 9:52

And to see this, let’s give it a spin: let testResult: Parser<TestResult> = testFailure let testResults: Parser<[TestResult]> = testResult.zeroOrMore() testResults.run(logs) // (match [...], rest "...")

— 10:22

It’s looking good! We’re seeing that we’ve definitely parsed out a test failure.:

— 10:41

But we’re still missing the tests that passed, so let’s write a parser for them by reusing a couple parsers we’ve already built. let testPassed = zip( testCaseStartedLine, testCaseFinishedLine ) .map(TestResult.passed(testName:time:))

— 11:58

Now parsing out a test result is merely trying to parse out a failure, and if that fails, we can try to parse out a pass instead. let testResult: Parser<TestResult> = .oneOf( testFailed, testPassed )

— 12:37

When we check our logs to see what we’ve parsed out we see that we’ve successfully parsed three results: a failure and two passes.

— 12:57

So it seems to be working with our smaller sample of logs. Let’s now try it out with a much larger sample, including multiline failures.

— 13:18

It looks like we’ve parsed out four results, but that isn’t quite right because we have 8 tests, and in fact we’re seeing we parsed a passing test as a failure. It turns out we made a mistake when we defined the test name parser to make it a bit too lenient. It will parse everything through ".swift" , including passing tests. let fileName = Parser.prefix(through: ".swift")

— 14:47

In order to make it more strictly aware that it’s parsing a path, we need some way of anchoring ourselves. One simple way is to parse out a literal "/" before we parse out the rest of the path. let fileName = zip("/", .prefix(through: ".swift")) .map { _ in path

— 15:12

And if we run things again, we see that we successfully parse out 8 results, which is exactly the number we’d expect. Formatting test results

— 15:58

We now have an incredibly powerful parser that can extract all of the test case data from the test logs, and if we were to clean up this code a bit it fits in under 50 lines of code. And even better, the parser has been broken down into lots of little parsers where each parser is only a few lines long and accomplishes just a single task and does it well. All of the parsers are also operating at a pretty high level by leveraging the library of combinators we have created. We aren’t doing any messy index juggling or subscripting, which are the parts of parsing that are hard to get right.

— 16:46

So, we’ve accomplished half of what we set out to do: we can now parse test logs. But let’s take this to the next level. We are going to develop a CLI tool that we can pipe the output of swift test into that will nicely format our logs.

— 17:03

Before moving out of this playground, let’s cook up the code that will perform the formatting to turn a TestResult value into something nicely formatted.

— 17:19

Let’s remember how we want to format a test failure: VoiceMemoTests.swift:123, testDelete failed in 2.0 seconds. ┃ ┃ XCTAssertTrue failed ┃ ┗━━──────────────

— 17:26

We can start by just trying to format a single result: func format(result: TestResult) -> String { switch result { case let .failed(failureMessage: failureMessage, file: file, line: line, testName: testName, time: time): case let .passed(testName: testName, time: time): } }

— 17:50

The easier one of these cases to fill in is the passed since all we have is the test name and time: return "\(testName) passed in \(time) seconds."

— 18:08

For the failed case we want to create that nicely formatted message that shows the top-level info about the failure, and then a hanging bracket to fence off the failure message.

— 18:21

The top-level info can be constructed like so: var output = "\(file):\(line), \(testName) failed in \(time) seconds." return output

— 18:48

Then we can append some strings to start the bracket: output.append("\n") output.append(" ┃") output.append("\n")

— 19:00

Then we want to append the failure message, but we need to prefix each line of the message with the bracket and some spacing: output.append( failureMessage .split(separator: "\n") .map { " ┃ " + $0 } .joined(separator: "\n") )

— 19:44

And then finally we can close off the bracket: output.append("\n") output.append(" ┃") output.append("\n") output.append(" ┗━━──────────────") output.append("\n") return output

— 19:59

Let’s take it for a spin. When we hand it a test that passed it prints nice and succinctly: format(result: .passed(testName: "testPassed", time: 0.02)) // "testPassed passed in 0.02 seconds"

— 20:15

And when we hand it a failure it prints really nicely: print( format( result: .failed( failureMessage: "XCTAssertTrue failed", file: "VoiceMemosTests.swift", line: 123, testName: "testFailed" time: 0.03 ) ) ) //VoiceMemoTests.swift:123, testFailed failed in 0.03 seconds. // ┃ // ┃ XCTAssertTrue failed // ┃ // ┗━━────────────── Packaging up the CLI tool

— 20:43

Now we are ready to make our CLI tool. We’ll initialize a new Swift package for an executable by running this command: $ mkdir pretty-test-logs $ cd pretty-test-logs $ swift package init --type executable Creating executable package: pretty-test-logs Creating Package.swift Creating README.md Creating .gitignore Creating Sources/ Creating Sources/pretty-test-logs/main.swift Creating Tests/ Creating Tests/LinuxMain.swift Creating Tests/pretty-test-logsTests/ Creating Tests/pretty-test-logsTests/pretty_test_logsTests.swift Creating Tests/pretty-test-logsTests/XCTestManifests.swift

— 21:14

This just sets up a new SPM module that is set up to be run from the command line, and it does this by automatically creating a main.swift file which serves as the entry point for the tool.

— 21:27

We are going to paste everything in our playground into this main.swift file and comment out anything that doesn’t build.

— 21:51

We now need to figure out how we are going to get the text that is piped into this CLI tool. Ideally we’d like to use the tool like this: $ xcodebuild test \ -scheme VoiceMemos \ -destination platform="iOS Simulator,name=iPhone 1 Pro Max" \ | pretty-test-logs

— 22:11

This would allow swift test to run, sending its output directly to our tool. In order to do this we need to read from STDIN, which is done via the readLine function in Swift. It returns a string when there is something in STDIN to retrieve, and it returns nil when there is nothing left. So, we could use a while loop to pull from this function until we run out of lines: var stdinLogs = "" while let line = readLine() { stdinLogs.append(line) stdinLogs.append("\n") }

— 23:07

And then, once we’ve accumulated all of our logs, we can hand them to our testResults parser. testResults.run(stdinLogs).match?.forEach { result in print(format(result: result)) }

— 23:33

We can already give this tool a spin. We can copy all of the logs we’ve been dealing with to our pasteboard, and then pipe those logs directly into this tool from the command line: $ pbpaste | swift run pretty-test-logs

— 24:23

And it works!

— 24:43

Let’s try it out on a real project. I’m going to hop over to the directory where I have the Composable Architecture checked out. If I run swift test I get the big mess of logs: $ xcodebuild test \ -scheme VoiceMemos \ -destination platform="iOS Simulator,name=iPhone 11 Pro Max"

— 25:02

We’d like to pipe this into the CLI tool we just built. Ideally this tool is somewhere in our PATH so that we can access it from anywhere, but for now we can just explicitly write the path to it in our pretty-test-logs project directory: $ xcodebuild test \ -scheme VoiceMemos \ -destination platform="iOS Simulator,name=iPhone 11 Pro Max" \ | ~/pretty-test-logs/.build/x86_64-apple-macosx/debug/pretty-test-logs

— 25:42

And now we get some nicely printed logs! But everything passes. Let’s introduce some test failures so we can see what it looks like.

— 26:20

And now when we run the tool we get a much better print out of what passed and what failed.

— 26:30

We’re getting a lot of extra output, though, because xcodebuild prints a lot of things to STDERR. We can clean this up by redirecting STDERR to /dev/null . $ xcodebuild test \ -scheme VoiceMemos \ -destination platform="iOS Simulator,name=iPhone 11 Pro Max" \ 2>/dev/null \ | ~/pretty-test-logs/.build/x86_64-apple-macosx/debug/pretty-test-logs

— 26:58

And now when we run things we just get the pretty-printed output from our tool Till next time

— 26:30

So, that’s the basics of this parser and CLI tool. There’s a lot more that would need to be done before it’s production worthy, but we just wanted to help spark some creativity in everyone to see that there are lots of unstructured things out there that would be handy to parse so that we can do something interesting with them.

— 27:38

And that officially concludes our recap series to get everyone on the same page when it comes to parsing. Next week we’ll dig into some new material, starting with ergonomics that can greatly simplify how we interact with some of these combinators. And soon after that we will talk about generalizing parsing, which will show us how to squeeze out extra performance from our parsers, and then we’ll describe what it means to invert a parser.

— 28:12

Until next time! References Combinators Daniel Steinberg • Sep 14, 2018 Daniel gives a wonderful overview of how the idea of “combinators” infiltrates many common programming tasks. Note Just as with OO, one of the keys to a functional style of programming is to write very small bits of functionality that can be combined to create powerful results. The glue that combines the small bits are called Combinators. In this talk we’ll motivate the topic with a look at Swift Sets before moving on to infinite sets, random number generators, parser combinators, and Peter Henderson’s Picture Language. Combinators allow you to provide APIs that are friendly to non-functional programmers. https://vimeo.com/290272240 Parser Combinators in Swift Yasuhiro Inami • May 2, 2016 In the first ever try! Swift conference, Yasuhiro Inami gives a broad overview of parsers and parser combinators, and shows how they can accomplish very complex parsing. Note Parser combinators are one of the most awesome functional techniques for parsing strings into trees, like constructing JSON. In this talk from try! Swift, Yasuhiro Inami describes how they work by combining small parsers together to form more complex and practical ones. https://academy.realm.io/posts/tryswift-yasuhiro-inami-parser-combinator/ Regex Alexander Grebenyuk • Aug 10, 2019 This library for parsing regular expression strings into a Swift data type uses many of the ideas developed in our series of episodes on parsers. It’s a great example of how to break a very large, complex problem into many tiny parsers that glue back together. https://github.com/kean/Regex Regexes vs Combinatorial Parsing Soroush Khanlou • Dec 3, 2019 In this article, Soroush Khanlou applies parser combinators to a real world problem: parsing notation for a music app. He found that parser combinators improved on regular expressions not only in readability, but in performance! http://khanlou.com/2019/12/regex-vs-combinatorial-parsing/ Learning Parser Combinators With Rust Bodil Stokke • Apr 18, 2019 A wonderful article that explains parser combinators from start to finish. The article assumes you are already familiar with Rust, but it is possible to look past the syntax and see that there are many shapes in the code that are similar to what we have covered in our episodes on parsers. https://bodil.lol/parser-combinators/ Sparse John Patrick Morgan • Jan 12, 2017 A parser library built in Swift that uses many of the concepts we cover in our series of episodes on parsers. Note Sparse is a simple parser-combinator library written in Swift. https://github.com/johnpatrickmorgan/Sparse parsec Daan Leijen, Paolo Martini, Antoine Latter Parsec is one of the first and most widely used parsing libraries, built in Haskell. It’s built on many of the same ideas we have covered in our series of episodes on parsers, but using some of Haskell’s most powerful type-level features. http://hackage.haskell.org/package/parsec Parse, don’t validate Alexis King • Nov 5, 2019 This article demonstrates that parsing can be a great alternative to validating. When validating you often check for certain requirements of your values, but don’t have any record of that check in your types. Whereas parsing allows you to upgrade the types to something more restrictive so that you cannot misuse the value later on. https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ Ledger Mac App: Parsing Techniques Chris Eidhof & Florian Kugler • Aug 26, 2016 In this free episode of Swift talk, Chris and Florian discuss various techniques for parsing strings as a means to process a ledger file. It contains a good overview of various parsing techniques, including parser grammars. https://talk.objc.io/episodes/S01E13-parsing-techniques Downloads Sample code 0122-parsers-recap-pt4 Point-Free A hub for advanced Swift programming. Brought to you by Brandon Williams and Stephen Celis . Content Become a member The Point-Free Way Beta previews Gifts Videos Collections Free clips Blog More About Us Community Slack Mastodon Twitter BlueSky GitHub Contact Us Privacy Policy © 2026 Point-Free, Inc. All rights are reserved for the videos and transcripts on this site. All other content is licensed under CC BY-NC-SA 4.0 , and the underlying source code to run this site is licensed under the MIT License .