Tremblp/maya 112824/pull push 01 to dev #1789

ppt-adsk · 2021-10-28T00:43:45Z

This pull request introduces the edit as Maya data / merge back to USD workflow (a.k.a pull and push, respectively). This workflow is used to convert USD data to Maya data for editing, then back to USD once editing is done. It leverages existing USD import support for edit as Maya (pull), and USD export support for merge back to USD (push). In a similar way to the import and export prim readers and writers, the edit as Maya / merge back to USD uses prim updaters, created on a per type basis, to perform the conversions.

Once a USD hierarchy has been edited as Maya, it appears in the Outliner hierarchy as parented under its original USD parent. However, in the Maya data model a separate, hidden Maya hierarchy is created to contain the pulled objects in their Maya representation. Because the paths to these pulled objects are pure Maya paths, all existing Maya editors and tools can operate on pulled data, and they will appear in the Maya selection as normal Maya objects.

* Second demo * Added marking menu support for USD and DAG objects * Only show USD marking menu for pulled DAG objects * Added push animation support and push to USD marking menu * Bake animation only when pulled object is animated * Import MayaReference under common parent. Fix pull of MayaReference. * Move pull/pushBack workflow to updater. Python script is still available, but soon we should migrate the rest of the code * Added missing export symbols. * WIP - Shot based animation * Extend registry to support look-up by maya type * WIP - Shot based animation * Support for pulled object directly under proxy shape * Don't show pull menu when no USD prims are selected * WIP - Shot based animation * Added new PullObjectHierarchy to handle pickwalking. * MayaReference updater called from manager * Fix baking * Clang * WIP - Shot based animation * Hide Maya pulled object. * Remove complex Maya path stuff. * New UsdPathMappingHandler * Implement copy spec in mayareference updater * Implemented maya reference push back to payload in cache variant * WIP - Shot based animation * Register for new callback "addItemsToOutlinerNodePopupMenu" to add context menu items to Maya objects. * Moved "USD Layer Editor" context menu (on proxy) to plugin (from Maya). * AutoPull for updaters that declare a need for it * CopyBetween data models. Old script is now pretty much replaced. * WIP - Shot based animation * Register/unregister a Maya path mapping handler with the core library. * Refactor some common code in {Usd/ProxyShape}Hierarchy. * Optimize the path mapping handler with a cache and check ancestors of input host path. * Invisibile prims don't get notifications on _PropagateDirtyBits or Sync, so when a prim turns invisible make sure all reprs can draw correctly. * Clang format. * MAYA-111521: Fix error messages showing up when opening the "Applied Schema" category. * Basis Curves have the same issue meshes do. Invisible prims get no chances to update, so make sure all reprs will draw correctly when the prim turns invisible. * Update SchemaRegistry callsite for a USD function name change * Updating the build doc with the latest supported version of USD dev * Disable the new point snapping support for the time being. * Implement setMatrixCmd() for common transform API. * MAYA-112058 - AE Missing Information on Materials and texture file prims USD v21.02 in Maya 2022.1 * In Maya 2022.1 we need to hold onto the Ufe SceneItem to make sure it doesn't go stale. This is not needed in latest Maya. * MAYA-112126: make sure parented object is added to selection list after it is moved. * Address feedbacks. * MAYA-111835: Fix Grouping a prim twice that crashed Maya. * check for return status when calling MGlobal::executeCommand. * Simplify selection logic. * Revert "MAYA-112126: make sure parented object is added to selection list after it is moved." * Update ae_template.py Python 2 strings are ascii by default, whereas Python 2 strings as are unicode by default. This string had a unicode char (for apostrophe) that I replaced with the ascii one. * Working Dag pull / push with pull root and pull parent. * Parent transform onto pull parent, remove pull root on last child, and hide it in Outliner. * Update to latest changes from Krystian. * Node lock / unlock on pull / push, and copy operation support. * WIP - Shot based animation * Fixed crash from UFE test. * Respect edit target * Use new export roots implementation. * Ufe PathMappingHandler is now in UFE v3 * clang format * Small fix for Outliner callback * Fixing bad merge of test. * Two-step pull, pull clear, design wording and icons, and tests. (#251) * Two-step pull, pull clear, design wording and icons, and tests. * Addressed code review comments. * Addressed test code review feedback. * Edit as Maya tests conditional to UFE path mapper. * Edit as Maya depends on UFE v3 path mapping. * More edit as Maya dependence on UFE v3 path mapping. * More conditional compilation of edit as Maya code. * Yet more conditional compilation of edit as Maya code. * Conditional compilation of edit as Maya code part 6. * Conditional compilation of edit as Maya code part 7. * Conditional compilation of edit as Maya code part 8. * Conditional compilation of edit as Maya code part 9. * WIP: layer traversal on push copy specs. * Prim updater push copy spec code cleanup. * Production-quality changes for push traversal. * WIP changes for push traversal. * Split out DiscardEdits and PushEnd. * Layer traversal error reporting and discard edits fix. * Prim updater cleanup. * Debugging output cleanup. Co-authored-by: Krystian Ligenza <[email protected]> Co-authored-by: Sean Donnelly <[email protected]> Co-authored-by: krickw <[email protected]> Co-authored-by: Hamed Sabri <[email protected]> Co-authored-by: Dan McGarry <[email protected]>

ppt-adsk · 2021-10-28T01:04:21Z

lib/mayaUsd/commands/baseExportCommand.cpp

@@ -319,8 +319,7 @@ MStatus MayaUSDExportCommand::doIt(const MArgList& args)
            std::string rootPath = tmpArgList.asString(0).asChar();

            if (!rootPath.empty()) {
-                MDagPath rootDagPath;
-                UsdMayaUtil::GetDagPathByName(rootPath, rootDagPath);
+                MDagPath rootDagPath = UsdMayaUtil::nameToDagPath(rootPath);


Removed duplicate name to Dag path utility and its uses.

ppt-adsk · 2021-10-28T01:06:08Z

lib/mayaUsd/fileio/CMakeLists.txt

@@ -28,6 +25,19 @@ target_sources(${PROJECT_NAME}
        writeJobContext.cpp
 )

+# Edit as Maya requires UFE path mapping.


Currently, edit as Maya is conditional on UFE v3, because it requires UFE path mapping for proper Outliner support. However, it would theoretically be possible to back-port the UFE path mapping interface to UFE v2, as it is a purely additive interface change to UFE. This will be for future consideration.

ppt-adsk · 2021-10-28T01:24:20Z

lib/mayaUsd/fileio/inMemoryTranslation.py

@@ -0,0 +1,325 @@
+import maya.cmds as cmds


This file was written for an earlier prototype. Parts of it could be cleaned up, as they are no longer relevant.

We shouldn't need this file anymore. Let's remove it.

ppt-adsk · 2021-10-28T01:25:58Z

lib/mayaUsd/fileio/jobs/readJob.cpp

@@ -589,6 +589,11 @@ const MDagPath& UsdMaya_ReadJob::GetMayaRootDagPath() const { return mMayaRootDa

 double UsdMaya_ReadJob::timeSampleMultiplier() const { return mTimeSampleMultiplier; }

+const UsdMayaPrimReaderContext::ObjectRegistry& UsdMaya_ReadJob::GetNewNodeRegistry() const


Provide access to the import new Maya node registry, so that the edit as Maya code can be run on them.

Should it be available in python?

Sounds potentially useful, but I could not find a Python wrapper wrapper for a UsdMaya_ReadJob.

ppt-adsk · 2021-10-28T01:27:41Z

lib/mayaUsd/fileio/jobs/writeJob.cpp

@@ -819,4 +827,9 @@ bool UsdMaya_WriteJob::_CheckNameClashes(const SdfPath& path, const MDagPath& da
    return true;
 }

+const UsdMayaUtil::MDagPathMap<SdfPath>& UsdMaya_WriteJob::GetDagPathToUsdPathMap() const
+{
+    return mDagPathToUsdPathMap;


Provide access to the export Dag path to USD path map, so the merge to USD code can access it.

ppt-adsk · 2021-10-28T01:35:41Z

lib/mayaUsd/fileio/primUpdater.cpp


-const MDagPath& UsdMayaPrimUpdater::GetDagPath() const { return _dagPath; }
+    return SdfCopySpec(


Plain SdfCopySpec overload recurses to USD children, which is not useful for us as we want to control the primSpec traversal and provide virtual function callbacks. The new overload allows for single primSpec control, but requires complex functors to obtain this behavior.

ppt-adsk · 2021-10-28T01:38:35Z

lib/mayaUsd/fileio/primUpdaterManager.cpp

+    return true;
+}
+
+bool PrimUpdaterManager::Pull(const Ufe::Path& path)


Core of the review: edit as Maya (pull). The pull is done is two steps, first import, then a pull customization step is done on the pulled Maya data.

ppt-adsk · 2021-10-28T01:40:01Z

lib/mayaUsd/fileio/primUpdaterManager.cpp

+        UsdMayaPrimUpdaterContext context(srcProxyShape->getTime(), proxyStage, userArgs);
+
+        // FIXME  Re-implement with proper traversal.  PPT, 22-Oct-2021.
+#if 0


Copying was part of an earlier prototype, must be re-implemented with the newer architecture in this pull request.

ppt-adsk · 2021-10-28T02:01:10Z

lib/mayaUsd/utils/util.cpp

@@ -215,19 +215,6 @@ MStatus UsdMayaUtil::GetMObjectByName(const std::string& nodeName, MObject& mObj
    return status;
 }

-MStatus UsdMayaUtil::GetDagPathByName(const std::string& nodeName, MDagPath& dagPath)


Removed duplicated functionality.

ppt-adsk · 2021-10-28T02:01:32Z

lib/mayaUsd/utils/util.h

@@ -185,10 +185,6 @@ MStatus GetMObjectByName(const std::string& nodeName, MObject& mObj);
 MAYAUSD_CORE_PUBLIC
 UsdStageRefPtr GetStageByProxyName(const std::string& nodeName);

-/// Gets the Maya MDagPath for the node named \p nodeName.


Duplicated functionality, removed.

kxl-adsk · 2021-10-29T20:04:28Z

lib/mayaUsd/fileio/inMemoryTranslation.py

@@ -0,0 +1,325 @@
+import maya.cmds as cmds


We shouldn't need this file anymore. Let's remove it.

kxl-adsk · 2021-10-29T20:05:48Z

lib/mayaUsd/fileio/jobs/readJob.cpp

@@ -589,6 +589,11 @@ const MDagPath& UsdMaya_ReadJob::GetMayaRootDagPath() const { return mMayaRootDa

 double UsdMaya_ReadJob::timeSampleMultiplier() const { return mTimeSampleMultiplier; }

+const UsdMayaPrimReaderContext::ObjectRegistry& UsdMaya_ReadJob::GetNewNodeRegistry() const


Should it be available in python?

kxl-adsk · 2021-10-29T20:06:45Z

lib/mayaUsd/fileio/primUpdater.cpp

-    , _baseDagToUsdPaths(UsdMayaUtil::getDagPathMap(depNodeFn, usdPath))
+namespace {
+// Set name that will be used to hold all pulled objects
+// MOVED


If it's moved, why do we need it here still?

Move was incomplete, readPullInformation() needed to come along with it to the PrimUpdaterManager.

kxl-adsk · 2021-10-29T20:07:27Z

lib/mayaUsd/fileio/primUpdater.cpp

+// MOVED
+const MString kPullDGMetadataKey("Pull_UfePath");
+
+Ufe::Path usdToMaya(const std::string& pathStr)


Do we have a utility for it already? Looks like something useful in general.

From the call site it appears that this is just Ufe::PathString::path().

kxl-adsk · 2021-10-29T20:08:12Z

lib/mayaUsd/fileio/primUpdater.cpp

+    auto isDagPathAnimated = [](const MDagPath& dagPath) {
+        int     upstreamDependencies = -1;
+        MString pyCommand;
+        pyCommand.format(


We should convert this to C++ and expose any missing APIs.

Excellent idea, but I feel this is out of scope for this pull request, and would be best handled as a separate task. Entered internally to Autodesk as MAYA-114591.

kxl-adsk · 2021-10-29T20:21:34Z

lib/mayaUsd/ufe/PulledObjectHierarchyHandler.h

+
+//! \brief Maya run-time hierarchy handler with support for pulled Maya objects.
+/*!
+    TO DOC


Now would be a good time to update documentation.

kxl-adsk · 2021-10-29T20:23:03Z

lib/mayaUsd/ufe/UsdUIUfeObserver.cpp

+                // start with |
+                MString firstPath = paths[0];
+                if (firstPath.substringW(0, 0) != "|")
+                    return;


Why do we need to do this?

This is a verbatim change from dev --- I wonder why it is showing up here. See

#1790

The reason is that as originally written, non-absolute Maya paths (e.g. just a node name, "pSphere1") are given to Ufe::PathString::path(), and since we have not (yet) registered any UFE handlers for this case (see Ufe::PathString::setCreateSingleSegmentPathFn()), PathString::path() cannot decide which run-time to choose for the path, so it throws, the exception is not caught, and we crash. A better fix would be for Maya to register a handler with UFE for this case, and the problem would be solved for all UFE clients, not just this one.

kxl-adsk · 2021-10-29T20:26:40Z

lib/usd/translators/mayaReferenceUpdater.cpp

 {
    const MObject& parentNode = GetMayaObject();
-
    UsdMayaTranslatorMayaReference::UnloadMayaReference(parentNode);


This doesn't discard edits on MayaReference.

Quite true. If you feel this needs to be part of this pull request, I can have a look, otherwise we can consider implementing this in the next phase, which will be for Maya reference workflows.

kxl-adsk · 2021-10-29T20:28:49Z

plugin/adsk/scripts/mayaUSDRegisterStrings.mel

@@ -50,6 +50,7 @@ global proc mayaUSDRegisterStrings() {
    register("kMenuClear", "Clear");
    register("kMenuRevertToFile", "Revert to File");
    register("kMenuLayerEditor", "USD Layer Editor");
+    register("kContextMenuLayerEditor", "USD Layer Editor...");


kMenuLayerEditor and kContextMenuLayerEditor look very similar. Just checking that it's not a mistake to have both.

Another change that I'm afraid I don't know much about. Both are used in plugin/adsk/scripts/mayaUsdMenu.mel.

kxl-adsk · 2021-10-29T20:30:00Z

test/lib/ufe/testGroupCmd.py

@@ -155,6 +155,12 @@ def testUsdGroup(self):
        groupPath = ufe.Path([mayaPathSegment, usdUtils.createUfePathSegment("/Ball_set/Props/newGroup1")])
        self.assertEqual(globalSelection.front(), ufe.Hierarchy.createItem(groupPath))

+        # Group object (a.k.a parent) will be added to selection list. This behavior matches the native Maya group command.
+        globalSelection = ufe.GlobalSelection.get()


How is this change related to the rest of the PR?

It is not. I looked at the history, and this change was done as part of the work on this feature in our internal repository, so I would need to go back and trace the origin of this change further. Since it is a test-only change, it passes, and the test ensures we meet requirements, I suggest we leave it in.

ppt-adsk · 2021-11-01T22:22:50Z

Review comments as of 1-Nov-2021 addressed. Labeled as do not merge, because of a regression with Maya reference prim creation found internally, to be investigated.

pierrebai-adsk · 2021-11-02T13:14:19Z

lib/mayaUsd/fileio/primUpdater.cpp

+            "  upstream = cmds.evaluationManager(ust='^1s')\n"
+            "  for node in upstream:\n"
+            "    if cmds.nodeType(node) == 'mayaUsdProxyShape':\n"
+            "      countNonProxyShape -= 1\n"


I don't see where countNonProxyShape gets initialized so 1 can be subtracted from it.

Absolutely, this will generate a NameError exception. Will investigate, good catch.

pierrebai-adsk · 2021-11-02T13:18:33Z

lib/mayaUsd/fileio/primUpdater.h

@@ -49,71 +51,56 @@ class UsdMayaPrimUpdater
        Push = 1 << 0,
        Pull = 1 << 1,
        Clear = 1 << 2,
+        AutoPull = 1 << 3,
        All = Push | Pull | Clear


IDK if AutoPull should be in All?

I don't know either. I will assume the above is correct, until shown otherwise.

kxl-adsk

I'm accepting this, but there is still more cleanup to do in subsequent PRs.

kxl-adsk · 2021-11-03T14:36:31Z

lib/mayaUsd/python/wrapPrimUpdaterManager.cpp

+        .def("push", push)
+        .def("pull", pull)
+        .def("pullClear", discardEdits)


These three looks like a good candidates for cleanup

lib/usd/translators/mayaReferenceUpdater.cpp

ppt-adsk and others added 2 commits October 27, 2021 11:14

Clang format.

4e83794

ppt-adsk requested a review from kxl-adsk October 28, 2021 00:43

ppt-adsk assigned ecp-maya-devops-adsk Oct 28, 2021

ppt-adsk commented Oct 28, 2021

View reviewed changes

Fixed compilation errors.

63a5725

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Oct 28, 2021

Fix clang-format.

0e45628

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Oct 28, 2021

kxl-adsk mentioned this pull request Oct 28, 2021

Support MayaReference prim behavior in mayaUsdProxyShape #1759

Closed

Fix for older USD versions.

5e78625

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Oct 28, 2021

Revert channel box observer workaround to use dev branch fix.

f115535

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Oct 29, 2021

kxl-adsk reviewed Oct 29, 2021

View reviewed changes

ppt-adsk requested a review from pierrebai-adsk November 1, 2021 14:34

Addressed code review comments.

fa92815

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Nov 1, 2021

clang-format

02b7079

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Nov 1, 2021

ppt-adsk added the do-not-merge-yet Development is not finished, PR not ready for merge label Nov 1, 2021

pierrebai-adsk reviewed Nov 2, 2021

View reviewed changes

Duplicate from Maya to USD and vice versa, with new wording in menus.

ffec2dc

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Nov 3, 2021

kxl-adsk previously approved these changes Nov 3, 2021

View reviewed changes

Fix for on-exit macOS failure, and addressed code review feedback.

acd10e4

ppt-adsk dismissed kxl-adsk’s stale review via acd10e4 November 3, 2021 19:49

ppt-adsk removed the do-not-merge-yet Development is not finished, PR not ready for merge label Nov 3, 2021

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Nov 3, 2021

kxl-adsk previously approved these changes Nov 3, 2021

View reviewed changes

kxl-adsk added the workflows Related to in-context workflows label Nov 3, 2021

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Nov 4, 2021

UFE registration cleanup on program exit.

2eb60f2

ppt-adsk dismissed kxl-adsk’s stale review via 2eb60f2 November 4, 2021 21:03

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Nov 4, 2021

clang-format

c3c8d8f

ppt-adsk assigned ecp-maya-devops-adsk and unassigned ecp-maya-devops-adsk Nov 4, 2021

ppt-adsk added the ready-for-merge Development process is finished, PR is ready for merge label Nov 4, 2021

kxl-adsk approved these changes Nov 5, 2021

View reviewed changes

kxl-adsk merged commit 395ad31 into dev Nov 5, 2021

kxl-adsk deleted the tremblp/MAYA-112824/pull_push_01_to_dev branch November 5, 2021 00:15

jonjondev mentioned this pull request Nov 24, 2021

Fixed exiting callback failing to deregister on UFE global #1856

Merged

Ennosch mentioned this pull request Nov 25, 2021

Edit- Duplicate as Maya data feature bug & PrimUpdaterManager import error #1860

Closed

		@@ -589,6 +589,11 @@ const MDagPath& UsdMaya_ReadJob::GetMayaRootDagPath() const { return mMayaRootDa

		double UsdMaya_ReadJob::timeSampleMultiplier() const { return mTimeSampleMultiplier; }

		const UsdMayaPrimReaderContext::ObjectRegistry& UsdMaya_ReadJob::GetNewNodeRegistry() const


		const MDagPath& UsdMayaPrimUpdater::GetDagPath() const { return _dagPath; }
		return SdfCopySpec(

Tremblp/maya 112824/pull push 01 to dev #1789

Tremblp/maya 112824/pull push 01 to dev #1789

Conversation

ppt-adsk commented Oct 28, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ppt-adsk commented Nov 1, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kxl-adsk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ppt-adsk commented Oct 28, 2021 •

edited

Loading