Importing a publishable library into another publishable library fails

[wapplegate]

I've been working on a proof of concept using nx extensions. What I'd like to do is create multiple publishable libraries that will be hosted in an npm feed. In addition, the nx-workspace contains an app that would import these libraries.

So I have Lib1, Lib2, and Lib3. Lib1 contains core components + services. Lib2 imports Lib1 to use some of those dumb components. Finally, Lib2 is imported into the application in the workspace. Running the application and this all works correctly. I see my dumb components from Lib1 working correctly inside my Lib2 components. However, when I build Lib2 using "ng build Lib2" (which imports Lib1) to get it ready for publish on an npm feed, I get the following error:

File 'lib1.module.ts' is not under 'rootDir' 'lib2\src'. 'rootDir' is expected to contain all source files.

From doing research on the topic, it seems like this is an issue with ng-packgr potentially? Is importing one publishable library into another supported in an nx-workspace?

[thelgevold]

Hi. Can you please include a repro of this issue?

[wapplegate]

Sure thing. Just attached my actual project. There really aren't many modifications from the project that was generated. I added one application "huron-application" and 3 libraries "admin", "core", and "identity". The admin and core libraries build fine with ng build. The core library has some dumb components inside it which are imported and used by identity library. The identity.module.ts file imports the core module in order to use some of the dumb components. The identity library has one component "login-page" which is where the component from core is used. The identity library is then imported into the huron-application. If you run "ng serve huron-application", things work fine. Go to the /login url and you should see some text. If I try ng build identity you get the error I described above.
project.zip

[wapplegate]

I'm assuming what is happening is when the identity library is built, it needs to include the core library which has an alias in the tsconfig.json:

"paths": {
  "@huron/core": ["libs/core/src/index.ts"],
  "@huron/identity": ["libs/identity/src/index.ts"],
  "@huron/admin": ["libs/admin/src/index.ts"]
}

So, using the alias path of "huron/core" it goes outside the identity projects folder structure to get those files. I guess because it's outside the source directory of the identity library, ng-packagr doesn't know where to put them in relation to the identity library's source files. I would think this would be a supported scenario when using an nx workspace. Because I'd like to also host these 3 libraries on a private nuget feed.

[matheo]

I was able to build my libs using ng-packagr directly,
and passing a custom package.json with the external references like:

  "ngPackage": {
    "dest": "../../../../dist/libs/admin",
    "lib": {
      "entryFile": "index.ts",
      "umdModuleIds": {
        "@huron/core": "huron.core"
      }
    }
  }

I will be digging into the Angular build process with the bundled ng-packagr, because I saw that Nx removed the build command of the Angular CLI libraries.

[wapplegate]

@matheo Interesting, so you were able to build all 3 libraries, even the ones that import other libraries? I'm curious to know if the maintainers of nx feel my scenario is a legitimate one. Have you made any progress with your research? Thanks for taking the time to respond and check it out.

[matheo]

@Singularity222 currently my dep-graph looks like:

Soon I will add some testing utilities to the library and it will be widely used too.
I will be distributing the datepicker bundle in npm, the others will be private for the company.

Also, I'm using deep dependencies too like @coachcare/common/store enabling them in the tslint.json, and I would love an article from @vsavkin about best practices on libraries and enterprise repo architecture :)

[matheo]

@Singularity222 I cannot share the company repo but soon I would get some time to build a Nx playground repo and showup with this stuff. I still need to go deeper with the @angular-devkit/build-ng-packagr builder and contribute my findings back to nx ;)

[wapplegate]

Any maintainers have input on this issue? I want to introduce the extension into my company but the lack of any acknowledgment of this issue is a concern to me.

[matheo]

@Singularity222 we will figure this stuff together, don't worry
the main "problem" in a monorepo is versioning and tagging the libraries, what's your approach?

[wapplegate]

Well, I'm not really concerned about the versioning. The issue I was looking for some guidance on was using a publishable library within another publishable library. My plan was to have a few publishable libraries developers could use in their application. So they could use the "Core" library as a base for their Angular applications. This library would contain a lot of lower-level functionality components, like select menus and text-boxes. Then, if they wanted more advanced features, they could import the "Admin" or "Identity" libraries to get those extra features. However, because the "Core" library has these lower level components, I also wanted to use "Core" within "Admin" and "Identity". So that's the really what I've wanted to accomplish.

[matheo]

@Singularity222 I see, that's a weird architecture tho.
Right now I'm using ng-packagr secondary entrypoints and I'm exporting like:
@company/common exports the Angular Module
@company/common/components and @company/common/pipes as secondary exporters.
Perhaps they are in the same published library.

[wapplegate]

Not too familiar with that, but I'll do some research. Why do you think what I'm trying to do is weird? Legitimately asking. My thought process was the "Core" library would have a lot of general application components. Like buttons, tabs, cards and other dumb UI components. In addition to the "Core" library there would be other libraries with more advanced use cases. The "Identity" library would contain components and pages related to authentication, like a login page. Depending on the complexity of the application, a consumer of my libraries could bring in "Core" when they have a relatively simple application, or they can import "Core" and "Identity" for an application that is a bit more complex and requires user accounts.

Because the "Identity" library has a lot more advanced components, I wanted that library to import "Core" so it can use those buttons, tabs, and other styled components. Otherwise I have to program the same button twice in each one of my other libraries if I can't import "Core". See what I mean?

[matheo]

@Singularity222 Well, what I've learned building libraries is that your code must depend on sibling or child code, not from a parent. So your identity lib cannot be a child of core if it uses components or features of it.

I think you rather need many publishable libraries instead of one core with sublibraries that are publishable too. I'd suggest to code your Core library and use it inside your Angular App, so you will see when the compiler complains about circular dependencies and stuff, that you need to sort out with the correct code organizations and imports.

[wapplegate]

The "Core" library is imported into "Identity". "Identity" would not be imported into "Core" at all.

[matheo]

@Singularity222 then it's pretty easy, as I shown you, I have a dependency tree with many libraries, using common and api.

I will setup a demo repo this weekend hopefully, so you can see my solution manipulating ng-packagr directly ;)

[wapplegate]

Thanks I would appreciate that. I haven't been able to make it work myself.

[wapplegate]

So I was just able to get a publishable library to import and build another publishable library by passing a custom tsconfig.json file when building the importing library, and moving the built library files of the library you'd be importing into the node_modules folder. It looks like this is a technique used by folks using ng-packagr. It's a bit hacky, but I've tested most scenarios work. I'll post a little more detailed explanation when I have some time to write things up.

[NgxDev]

do you have some small sample repo with this working?
I just couldn't make it work, I've been all day at this. Wish it just worked out of the box. Hopefully, it will at some point.

@matheo I have no idea what "umdModuleIds" is for :)
@Singularity222 the approach you've found sounds better I believe, but I'm not sure how that custom tsconfig.json should look like, I just understand that it's for the importing library (ie the lib that imports stuff from other libs)

[wapplegate]

When I get back to my computer I’ll write something up for you. Probably won’t get to it until sometime tonight. It’s not too complicated. I don’t have a build process set-up, but I can show you how to get the importing library to build correctly and you can go from there. I’m no expert by any means, but it seems like people associated to the ng-packagr project have used this method to overcome the error.

…

On Mon, Jul 9, 2018 at 6:38 PM Marian Stoica ***@***.***> wrote: do you have some small sample repo with this working? I just can't make it work, I've been all day at this. @matheo <https://github.com/matheo> I have no idea what "umdModuleIds" is for :) @Singularity222 <https://github.com/Singularity222> the approach you've found sounds better I believe, but I'm not sure how that custom tsconfig.json should look like, I just understand that it's for the importing library (ie the lib that imports stuff from other libs) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#602 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA7d2LCYw84vLcsaEDJMApLjU85nPoMFks5uE9t2gaJpZM4U0Yhk> .

[NgxDev]

@Singularity222 just a reminder, I don't think you've managed to find time yet.
Would it be easier if I do a sample repo on this?
Thanks!

[zack9433]

[UPDATED] I fixed this error, checkout this commit zack9433/poc-workspace@7dfedf7

and run yarn build tp-core && yarn build tp-common

=========================================================================

I have encountered this problem too. This is a sample repo

https://github.com/zack9433/poc-workspace

I create two library tp-common and tp-core both are publishable and tp-common library import tp-core. When I run yarn build tp-common, there is an error happened

[stefan-karlsson]

Thanks to @zack9433 's post i managed to fix this issue for my project.

Version

nrwl/nx: "6.2.0"

Steps taken

Add npm scope to libraries

libs/cognito/package.json:
"name": "@npmScope/cognito",

libs/authenticator/package.json:
"name": "@npmScope/authenticator",

Add dist lib path to parent lib in root tsconfig.json

 "paths": {
      "@npmScope/cognito": [
        "libs/cognito/src/index.ts"
      ],
      "@npmScope/authenticator": [
        "dist/libs/authenticator",
        "libs/authenticator/src/index.ts"
      ]
    }

[latchezar-kostov-ascendlearning-com]

@zack9433 @stefan-karlsson That seems to fix the build issue. However, when I serve my application, my dependency lib (e.g. tp-core) is not properly resolved by the lib that depends on it (e.g. tp-common). I have to delete the dist folder in order to get this to work.

[zack9433]

@latchezar-kostov-ascendlearning-com currently i add a prestart npm script to remove dist folder to avoid resolved issue for serve application.

[wapplegate]

Sorry I've been absent from this thread. Had some personal issues come up the past few weeks. The way I actually "solved" this issue (which may be stupid compared to what is described above) is for the libraries that had a dependency on another library, I modified the tsconfig to remove the dependent library as an alias (and there might be another change or two). So to the library that was importing it, it seemed like a third-party library that would be found in the node_modules directory. When I would build the library that imported another, I had a script that would build the imported library first, and then place it in the node_modules folder. I saw this solution suggested in an ng-packagr thread. It has worked for me without issue. The build process is more complicated, but using ng serve requires no other modifications to be made. I just have an npm script that handles the entire build process for all libraries, then I publish them to our private npm feed. Has worked pretty well for me so far.

[kotva006]

I am running into the same issue, even after using @stefan-karlsson fix it appears that nx isn't aware of the dependency between libraries.
I have created a sample [project to demonstrate the issue](: https://github.com/kotva006/lib-issue)
I have 2 libraries abc, and xyz. abc imports xyz, and building abc causes an error because xyz isn't built before abc.

Steps to reproduce (run either command from the root of the project dir):
ng build --project=abc
npm run affected:build -- --all

Both of them fail on building abc (though xyz will build when the second command is ran after abc fails to build)
I believe that the build is supposed to know which libraries are dependencies and build them first.

[mohyeid]

@vsavkin I think this is a very common case and it would be greatly appreciated to have your input into this. Thanks!

[nadavsinai]

Were migrating a whole ecosystem of apps and libs from publishing to internal NPM server to a mono-repo, and obviously we're keeping the ability to publish, I have encountered this error and fixed it via introducing a tsconfig.lib.json to the workspace root folder, it includes the path aliases for all libs to be from the dist folder rather than the lib/ source. I then changed all the tsconfig.lib.json inside the libs to extend this tsconfig inplace of the regular one.
I adjusted the script by @conor-mac-aoidh a little so that libs build by reference order as described in the peerDependencies section of package.json
this works but is not ideal :

• paths in tsconfig.lib.json are to be manually created when creating a new lib
• all lib builds must occur (slow) and in order else it is easy to have source and compiled files out of sync.
• it may be impossible to create a cyclic dependency between libs (unverified)

I guess it is about ngPackager changing the baseUrl/rootDir of tsc not allowing the compiler to work with .ts files outside the rootDir

[devinshoemaker]

I am facing this issue as well. I am currently attempting to build a publishable library that imports a component from a non-publishable library and renders it to a div as a way of making a small separate bundle that I bring into a legacy app. I have tried a couple of the path based workarounds, but nothing has worked for me so far, and publishing these builds seems overkill. Being able to get this working would be hugely beneficial.

[dalemanthei]

Thanks to all the commenters here and other related threads, we have been able to publish a component library based on Angular Material. Like Material we have components that depend on other components that needs to be built in sequence.

In our scenario each component (think button, checkbox, etc) lives in it's own nx lib. Each of our libs (components) is built using ng-packagr controlled by npm scripts in package.json. I have created a demo repo which puts this all together showing 3 components with dependencies on the build order.

Here's the simple demo repo to build a library that could be published and installed from @dmv/demo: nx-ng-packagr-demo

Our production repo has 2 applications and a pile of components. This is an example of the dep-graph when we were writing the library:

Maybe this can help other folks trying to solve a similar problem.

[vsavkin]

Thank you so much for your patience folks.

@juristr is working on reworking how publishable libraries work to better support this use case. The initial PR landed into master and will be a part of the 8.12 release.

[nayfin]

I've actually had a pretty good experience. I just made a silly mistake when configuring.

[gund]

Actually I've found a workaround for building libraries in correct order.

However I'm not sure that this was intended behavior - but it always works exactly how it needs to.
So the trick is to use run-many command to build all your libraries like so:

nx run-many --all --target build

This will execute build target in all your libraries and applications in the order they depend upon each other (at least this is how it was for me). I've tested this with clear setup on different machines and the order was always correct (dependencies wise).

@vsavkin maybe you can confirm if this is by design - or just a side-effect that we should not rely upon?

[conor-mac-aoidh]

@gund I've tried this command and it does not seem to take account of the dependencies. It does not build the publishable libs in the order they depend on each other.

@dalemanthei That's a similar use-case to me. I just checked out your repository and tried running nx build link and it throws this error:

➜  nx-ng-packagr-demo git:(master) nx build link
Building Angular Package

------------------------------------------------------------------------------
Building entry point '@dmv/demo/link'
------------------------------------------------------------------------------
Compiling TypeScript sources through ngc
ERROR: libs/link/src/lib/link.module.ts(4,28): error TS2307: Cannot find module '@dmv/demo/core'.

How are you building the libs in order in this repository? Is there a custom command that I need to run?

[dalemanthei]

@conor-mac-aoidh We use a custom set of npm scripts invoked using npm run build-libs. The sequencing of the component builds is done manually in the npm scripts.

I suspect the nx build link is dependent on nx understanding the build dependencies. In our use case the npm run dep-graph doesn't work due to issues between published package path and code paths in the project. So the dep-graph works if our components were @dmv/link, @dmv/button, etc. Introducing the demo into the path to produce @dmv/demo/button breaks the dep-graph.

I will try to get a real README up this morning. Over the weekend I was thinking it needs to outline the various adjustments to get the repo working. Hopefully the build-libs command will get you going for now.

edit: I updated the README to document the changes we made.

[devinshoemaker]

@vsavkin is there any documentation around this new feature? I read the release blog post, however, I am still having issues. After updating and running the migrations, I get this error when building my library that depends on another library in the workspace.

Bundling...
rpt2: options error TS6059: File '/Users/devin/code/workspace/libs/react-components/src/index.ts' is not under 'rootDir' '/Users/devin/code/workspace/libs/entry-points/long-query-alert/src'. 'rootDir' is expected to contain all source files.
rpt2: options error TS6059: File '/Users/devin/code/workspace/libs/react-components/src/lib/react-components.tsx' is not under 'rootDir' '/Users/devin/code/workspace/libs/entry-points/long-query-alert/src'. 'rootDir' is expected to contain all source files.
'@workspace/react-components' is imported by libs/entry-points/long-query-alert/src/index.ts, but could not be resolved – treating it as an external dependency
'@workspace/react-components' is imported by libs/entry-points/long-query-alert/src/index.ts, but could not be resolved – treating it as an external dependency
No name was provided for external module '@workspace/react-components' in output.globals – guessing 'reactComponents'

So, it did not alert me that my other library needed to be built first (despite deleting my dist/ directory beforehand), and it did not update any tsconfigs to fix the build. Is this user error?

[devinshoemaker]

I found this PR which indicates that this fix was only applied for Angular projects. I am currently using React and am facing the same issue.

[dalemanthei]

@vsavkin @juristr Do you think the work being done to build dependent libraries will help with my challenge of building an angular material like library?

I created a demo repo. Based on this issue I made a couple of variations on the folder structure to improve on the dep-graph generation.

Ideally we could build all the components using a single nx command vs the npm scripts we have now. Of more value is being able to use the dep-graph and affected scope commands which seems possible based on changing the folder structure of the project.

[juristr]

Hey all, building publishable libs that reference other publishable libs should be solved now and work as expected starting from our 8.12 release.

A couple of notes here:

• publishable libraries can only depend on other publishable libraries. There is a new option in the nx enforce boundaries lint rule that checks for it
• you will need to build your dependent library first. Say lib-a depends on lib-b, you need to first build lib-b and then lib-a. The reason is that when building publishable libraries we depend on the built outcome of dependent libraries, not from src. We think that better represents the real world usage and thus helps potentially uncover problems before you publish them to npm. We will improve the dev ergonomics here in that we will automatically build all dependencies in the future. For now you can use the “run-many” command

Let us know if you run into any issues :)

[juristr]

@dalemanthei I didn’t look into the details of your demo repo, but I think the Nx run-many command could help you out here.

[snebjorn]

I have a @nrwl/node:lib (publishable) package that I import into a @nrwl/angular:lib (publishable). When I run nx build I get this warning.

WARNING: No name was provided for external module '@nx-test/libtest' in output.globals – guessing 'libtest'

> nx run libtest:build
Building Angular Package
******************************************************************************
It is not recommended to publish Ivy libraries to NPM repositories.
Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility
******************************************************************************

------------------------------------------------------------------------------
Building entry point '@nx-test/libtest'
------------------------------------------------------------------------------
Compiling TypeScript sources through ngc
Bundling to FESM2015
Bundling to FESM5
Bundling to UMD
WARNING: No name was provided for external module '@nx-test/libtest' in output.globals – guessing 'libtest'
Minifying UMD bundle
Writing package metadata
Built @nx-test/libtest

------------------------------------------------------------------------------
Built Angular Package
 - from: C:\Users\FOO\Repos\nx-test\libs\libtest
 - to:   C:\Users\FOO\Repos\nx-test\dist\libs\libtest
------------------------------------------------------------------------------

[smkart]

@juristr thanks for the fix , I have one concern of building dependent library , This is because in Angular when we create a library that is non publishable we will not be able to build those libraries, That case we are forced to create a library as publishable because we need to build it. Hope you understand my concern

Recommendation: I would recommend nx to build an library without building dependent library, Or may be an option in nx.json takes care of connecting dependency libraries

Regards,
Mani

[pburgmer]

I have a workspace with multiple publishable libraries. E.g. B depends on A.
Building A works. Building B doesn't work. Nx always fails, even if i run it with --with-deps.

Some of the project B's dependencies have not been built yet. Please build these libraries before: A

[seaBubble]

I'm still seeing this issue with @nrwl/angular 9.5

I'm running
nx run lib2:build --with-deps --prod
where lib2 is a publishable lib that depends on another publishable lib

And getting
error TS6059: File '[...]/lib1/src/index.ts' is not under 'rootDir' '[...]/lib2/src'. 'rootDir' is expected to contain all source files.

lib1 is built successfully on its own.

Is there an extra step to make this work?

[samratshaw]

@vsavkin Do you have any guide for how to achieve this in React-based app which is using Nx?

[AsimNet]

for anyone struggling, this worked for me:

URL in the image: https://github.com/trellisorg/platform/tree/master/packages/make-buildable

Credits: Benjamin Charity
https://nrwlcommunity.slack.com/archives/C016ASCS683/p1630157090012400

[github-actions]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.