Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for xcuserdata #739

Merged
merged 12 commits into from
Feb 2, 2023
Merged

Add support for xcuserdata #739

merged 12 commits into from
Feb 2, 2023

Conversation

teameh
Copy link
Contributor

@teameh teameh commented Dec 26, 2022

👋 Hi I'm new here. Thanks for this project, it's great!

Short description 📝

XcodeProj doesn't support xcuserdata yet, this PR adds XCUserData

Background

XCSchemeManagement is already implemented, but it's not included in Sources/XcodeProj/Project/XcodeProj.swift. XcodeProj also currently does not feature the necessary XCUserData where XCSchemeManagement should be part of.

In XcodeGen we are trying to add support for scheme management via XCSchemeManagement that should result in three options for schemes: visibility, order hints and wether or not a scheme is shared.

When a scheme is not shared it should be written to xcuserdata instead of xcshareddata but since xcuserdata is not part of XcodeProj yet this is not possible at the moment and the shared: Bool property of XCSchemeManagement is not very useful.

209574381-98840d4a-ea19-4602-a838-9de86fbaac8d

Implementation 👩‍💻👨‍💻

  • Created XCUserData & XCUserDataTests
  • Changed XcodeProj.swift so schemes and breakpoints could be written to both shared and user data
  • Conformed XCSharedData to Writable because XcodeProj.swift was getting a bit bloated.
  • Conformed XCSchemeManagement to Writable
  • Added XcodeProjTests to verify changes did not break anything
  • Extracted func testReadWriteProducesNoDiff(file:line:path:initModel:) to testWrite.swift to dry up the tests a bit.

I'll add some review remarks and questions to the diff.

Questions

  • Coverage for the files inSources/XcodeProj/Project is a bit low. I found no tests for the code in XcodeProj.swift that is responsible for writing files to disk. XCSharedData is also lacking tests. I thought it would be too much out of scope for this PR, but maybe I can add those in another PR?

image

Remarks?

I had good fun working on this today! I hope you see the added value.

@@ -45,9 +45,9 @@
buildConfiguration = "Debug"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When adding tests that would read and then write schemes, breakpoints and a xcschememanagement.plist back to disk the tests failed because these changes popped up in the git diff. So I've added them to make sure that when these files are read in and written back to disk there is no diff. I think this is the correct behaviour.


/// Returns schemes folder path relative to the given path.
///
/// - Parameter path: parent folder of schemes folder (xcshareddata or xcuserdata)
/// - Returns: schemes folder path relative to the given path.
public static func schemesPath(_ path: Path) -> Path {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These methods now need the xcshareddata or xcuserdata path instead of the .xcodeproj path because they need to it to produce a relative path to that folder instead of a path always relative to the xcshareddata folder.

public static func schemesPath(path:)
public static func schemePath(path:schemeName:)
public static func debuggerPath(path:)
public static func breakPointsPath(path:)
public static func schemeManagementPath(path:)            <-- New

The same goes for the writing methods, they need a path to the xcshareddata or xcuserdata folder instead of the .xcodeproj path.

public static func writeSchemes(schemes:path:override:)
public static func writeBreakPoints(breakpoints:path:override:)

initModel: { try? XCUserData(path: $0) },
modify: { $0 },
assertion: {
assert(userData: $1, userName: "copy")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

testWrite copies the username1.xcuserdatad folder over to <tmpDir>copy.xcuserdatad thus changing the userName of the XCUserData to copy...

Copy link
Collaborator

@kwridan kwridan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for contributing this @teameh , the additional notes and tests were much appreciated to help review this 🙏

I think many of the pieces here are valuable additions, we'll need to figure out how to integrate it without introducing functional or source breaking changes.

/// - Throws: An error if the write fails.
public func write(path: Path) throws {
public func write(path: Path, override: Bool) throws {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this introduces a breaking change, we'll need to offer a default value for override

e.g.

https://github.com/tuist/tuist/blob/main/Sources/TuistGenerator/Writers/XcodeProjWriter.swift#L175

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 6f10238

try writeSchemes(path: path, override: override)
try writeBreakPoints(path: path, override: override)
try writeSharedData(path: path, override: override)
try writeUserData(path: path, override: override)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sadly introduces a few functional breaking changes ...

  • Any user specified local schemes will get wiped whenever the project is written to vs before they would be retained between project writes. The previous behaviour makes sense given custom user schemes wouldn't have been checked in anyways, and when moving to a project generation workflow the same semantics would ideally remain.
  • Any user created breakpoints will get wiped out on every re-generation! (This is most likely undesired)

I suspect xcuserdata was not written by default on purpose and was left up to the tools that use the XcodeProj library to decide how to deal with it, for example worth taking a look at Tuist's XcodeProjWriter and XcodeProjWriterTests which only append user schemes rather than wipe the entire directory.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any user specified local schemes will get wiped

Fixed in 0ef27bc

Any user created breakpoints will get wiped

See https://github.com/tuist/XcodeProj/pull/739/files#r1059789045

for example worth taking a look at Tuist's XcodeProjWriter and XcodeProjWriterTests

Nice. Yes that approach makes sense. Nice that Tuist could also improve after this PR:

// XcodeProj can manage writing of shared schemes, we have to manually manage the user schemes


// MARK: - Writable

public func write(path: Path, override: Bool) throws {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for XCUserData perhaps the override should only apply to the individual files rather than the wiping the entire directory ...

I could see this being extended to XCWorkspace later on as it has the same set of files, but in the XCWorkspace, there's a few additional files Xcode places there

App.xcodeproj/project.xcworkspace/xcuserdata/kas.xcuserdatad/UserInterfaceState.xcuserstate 

The UserInterfaceState.xcuserstate file holds some Xcode UI state like which source files were being viewed, tabs etc... we in fact had to address this in Tuist (by preserving xcuserdata rather than completely wipe it) for our project generation workflow as it was quite disruptive to developers having Xcode lose its state every time a re-generation would take place.

tuist/tuist#1006

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah, that makes sense. That sounds like a good approach.

I've pushed 0ef27bc that changes the behaviour to only override individual files and individual XCUserData directories when data is written.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sources/XcodeProj/Project/XcodeProj.swift Outdated Show resolved Hide resolved
Sources/XcodeProj/Project/XcodeProj.swift Show resolved Hide resolved
@teameh teameh force-pushed the xcuserdata-support branch 2 times, most recently from 08ed35a to 0ef27bc Compare January 1, 2023 19:44
/// If false will throw error if breakpoints file exists at the given path.
func writeBreakpoints(path: Path, override: Bool) throws {
try XcodeProj.debuggerPath(path).mkpath()
try breakpoints?.write(path: XcodeProj.breakpointsPath(path), override: override)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not so sure what to do about the breakpoints though.. since the breakpoints are all stored in a single file, I guess we'll need to:

  • Read in the current file (if it exists).
  • Append the new breakpoints
  • Filter out duplicates?

Would that be a good approach?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it's explicitly specified then overwriting should be ok (reading / merging is unnecessary complexity), the concern was if there were no breakpoints specified (nil) and that wipes any local ones. The approach you have followed here is good 👍

let schemesPath = XcodeProj.schemesPath(path)
if override, schemesPath.exists {
try schemesPath.delete()
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still happening, only now as part of XCSharedData.writeSchemes

@teameh
Copy link
Contributor Author

teameh commented Jan 3, 2023

(Rebased and signed commits with gpg key)

Copy link
Collaborator

@kwridan kwridan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates @teameh

/// If false will throw error if breakpoints file exists at the given path.
func writeBreakpoints(path: Path, override: Bool) throws {
try XcodeProj.debuggerPath(path).mkpath()
try breakpoints?.write(path: XcodeProj.breakpointsPath(path), override: override)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it's explicitly specified then overwriting should be ok (reading / merging is unnecessary complexity), the concern was if there were no breakpoints specified (nil) and that wipes any local ones. The approach you have followed here is good 👍

Sources/XcodeProj/Project/XCUserData.swift Outdated Show resolved Hide resolved
Sources/XcodeProj/Project/XCUserData.swift Outdated Show resolved Hide resolved
Sources/XcodeProj/Project/XcodeProj.swift Outdated Show resolved Hide resolved
Sources/XcodeProj/Project/XcodeProj.swift Outdated Show resolved Hide resolved
Sources/XcodeProj/Project/XcodeProj.swift Outdated Show resolved Hide resolved
Sources/XcodeProj/Project/XCUserData.swift Outdated Show resolved Hide resolved
@kwridan
Copy link
Collaborator

kwridan commented Jan 5, 2023

@all-contributors add @teameh for code

@allcontributors
Copy link
Contributor

@kwridan

I've put up a pull request to add @teameh! 🎉

@teameh teameh force-pushed the xcuserdata-support branch 2 times, most recently from c73599f to 9556a7b Compare January 5, 2023 13:05
@@ -17,6 +17,8 @@
</ActionContent>
</BreakpointActionProxy>
</Actions>
<Locations>
</Locations>
Copy link
Contributor Author

@teameh teameh Jan 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can prevent this addition if we add this change to XCBreakpointList.swift:

--- a/Sources/XcodeProj/Project/XCBreakpointList.swift
+++ b/Sources/XcodeProj/Project/XCBreakpointList.swift
@@ -300,7 +300,9 @@ public final class XCBreakpointList: Equatable, Writable {

                 let locations = AEXMLElement(name: "Locations", value: nil, attributes: [:])
                 self.locations.map { $0.xmlElement() }.forEach { locations.addChild($0) }
-                element.addChild(locations)
+                if !locations.children.isEmpty {
+                    element.addChild(locations)
+                }```

@teameh
Copy link
Contributor Author

teameh commented Jan 6, 2023

  • Adding some ignored schemes fixed the builds :)
  • Cleaned up some small bits

@teameh teameh requested a review from kwridan January 6, 2023 17:56
Copy link
Collaborator

@kwridan kwridan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all your efforts on this @teameh

Sources/XcodeProj/Project/XCSharedData.swift Outdated Show resolved Hide resolved
@teameh
Copy link
Contributor Author

teameh commented Jan 9, 2023

Thanks! I've removed the superfluous comments.

What do you think about #739 (comment)?

@kwridan kwridan requested review from a team, adellibovi, brentleyjones, danieleformichelli and fortmarek and removed request for a team and adellibovi January 20, 2023 18:33
@teameh
Copy link
Contributor Author

teameh commented Jan 31, 2023

Hey @kwridan, @adellibovi, @brentleyjones, @danyf90 & @fortmarek. If there's anything I can do to move this PR forward, please let me know. Also happy to hop on a quick call to elaborate the changes if you think that is helpful for one of you.

@kwridan
Copy link
Collaborator

kwridan commented Feb 1, 2023

Hey @kwridan, @adellibovi, @brentleyjones, @danyf90 & @fortmarek. If there's anything I can do to move this PR forward, please let me know. Also happy to hop on a quick call to elaborate the changes if you think that is helpful for one of you.

Apologies for the delays - trying to source a second reviewer is proving more challenging these days 😅, I have posted a message in the Tuist slack for visibility.

@danieleformichelli danieleformichelli removed their request for review February 1, 2023 08:33
Copy link
Member

@fortmarek fortmarek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have the most context on this part of the codebase but the changes look good, thanks for writing the extensive tests 🎉

Sources/XcodeProj/Errors/Errors.swift Outdated Show resolved Hide resolved
.compactMap { try? XCScheme(path: $0) }
schemeManagement = try? XCSchemeManagement(path: XCSchemeManagement.path(schemesPath))

breakpoints = try? XCBreakpointList(path: XCBreakpointList.path(XCDebugger.path(path)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we want to ignore the thrown errors here?

Copy link
Contributor Author

@teameh teameh Feb 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say we shouldn't ignore the error's here, at least print / log them. But we're not doing that anywhere in similar files, see for example the same line in XCSharedData: https://github.com/tuist/XcodeProj/blob/master/Sources/XcodeProj/Project/XCSharedData.swift#L41

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kwridan do you have any context why we'd want to ignore these erros? But if this is what we're doing elsewhere, I don't mind shipping as-is

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping the behaviour consistent is preferable 👍

Looking closely at the code, we may not be able to safely change this without changing the APIs / behaviour. Some of those nested elements are optional, for example you can have an user data folder without breakpoints or schemes - and the nested elements currently throw if the path doesn't exist (which isn't ideal :/) - resulting in making the API unusable unless you have a project with all the sub elements present.

Sources/XcodeProj/Project/XcodeProj.swift Outdated Show resolved Hide resolved
Copy link
Collaborator

@kwridan kwridan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks all for the feedback 🙏

.compactMap { try? XCScheme(path: $0) }
schemeManagement = try? XCSchemeManagement(path: XCSchemeManagement.path(schemesPath))

breakpoints = try? XCBreakpointList(path: XCBreakpointList.path(XCDebugger.path(path)))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping the behaviour consistent is preferable 👍

Looking closely at the code, we may not be able to safely change this without changing the APIs / behaviour. Some of those nested elements are optional, for example you can have an user data folder without breakpoints or schemes - and the nested elements currently throw if the path doesn't exist (which isn't ideal :/) - resulting in making the API unusable unless you have a project with all the sub elements present.

@kwridan kwridan merged commit 1370528 into tuist:main Feb 2, 2023
@teameh
Copy link
Contributor Author

teameh commented Feb 2, 2023

Awesome! Thanks for merging this 😃

@teameh
Copy link
Contributor Author

teameh commented Feb 21, 2023

Hey there, any plans for creating a new release for XcodeProj? Would be cool if we could use this for yonaskolb/XcodeGen#1142!

@kwridan
Copy link
Collaborator

kwridan commented Feb 22, 2023

It's already out as part of https://github.com/tuist/XcodeProj/releases/tag/8.9.0 🙂

@MustardseedX
Copy link

MustardseedX commented Feb 22, 2023 via email

@teameh
Copy link
Contributor Author

teameh commented Feb 23, 2023

haha wow thanks, I just missed it. Great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants