Incompatible with electron-rebuild / electron-forge

[rkistner]

Previously reported in #53 #54 #55 #56 #57 #58 #61 #63. Creating one issue to keep track of this.

There are now workaround instructions here: https://github.com/journeyapps/node-sqlcipher#usage-with-electron-forge--electron-rebuild
It would still be great to find a solution that works without additional configuration.

There are multiple issues with electron-rebuild (which is also used by default with electron-forge) attempting to rebuild this module.

1. It does not support {napi_build_version} in the module_path. See Incorrect folder name when rebuilding "sqlite3" for arm64/x64 architecture electron/rebuild#554 / Cannot rebuild sqlite3 electron/rebuild#546.
2. It does not use the pre-built modules. See node-pre-gyp support in the same way than prebuild is supported electron/rebuild#367.
3. When building from source, there is something wrong in the build environment causing a compile error (Build fails on Windows with Electron 11: 'napi_is_detached_arraybuffer': identifier not found #63).

The workaround is to disable rebuilding of this module with electron-rebuild. However, there are still potential issues:

1. If the node version and electron versions don't match, the correct N-API version would not be installed automatically.
2. When cross-compiling for a different architecture or platform, the correct N-API version would not be installed automatically.

Disabling N-API v6 support would solve (1) above, but not (2). It would also remove worker_thread support, which is not ideal.

[Christilut]

I tried the workaround but that resulted in this:

14:49:03.383 > Error: ERR_CONNECTION_REFUSED (-102) loading 'http://localhost:3000/main_window'
    at rejectAndCleanup (electron/js2c/browser_init.js:205:1493)
    at Object.failListener (electron/js2c/browser_init.js:205:1706)
    at Object.emit (events.js:315:20)
    at Object.EventEmitter.emit (domain.js:483:12)

On Windows 10, node 12.20.1, electron 11

I'd be happy to try any suggestions. I'm not sure what to do about this here.

[1111mp]

electron-rebuild can compile successfully, but an error will be reported when starting:errno 3228369023


electron:11.0.4 node:14.0.0 electron-rebuild:2.3.4 windows platform. python 2.7.9 Visual Studio 2017(2015 compilation will not pass)
I also tried these versions:
electron: 11.1.1 11.2.0
node: 12.17.0 12.20.0
The same error occurs.

[rkistner]

@1111mp The error code doesn't say much unfortunately. There may be more details in the log files.

Also keep in mind that even though the compile succeeds, there will likely still be issues with it (e.g. placing the output in the wrong folder). I'd still recommend disabling the compile in electron-rebuild.

[1111mp]

0 info it worked if it ends with ok
1 verbose cli [
1 verbose cli   'D:\\software\\nodejs\\node.exe',
1 verbose cli   'D:\\software\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'run',
1 verbose cli   'start:main'
1 verbose cli ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart:main', 'start:main', 'poststart:main' ]
5 info lifecycle electron-client@~prestart:main: electron-client@
6 info lifecycle electron-client@~start:main: electron-client@
7 verbose lifecycle electron-client@~start:main: unsafe-perm in lifecycle true
8 verbose lifecycle electron-client@~start:main: PATH: D:\software\nvm\v14.0.0\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin;E:\webpack\electron_client\node_modules\.bin;C:\Users\s4243\AppData\Local\Temp\yarn--1610614630512-0.24730506172334077;E:\webpack\electron_client\node_modules\.bin;C:\Users\s4243\AppData\Local\Yarn\Data\link\node_modules\.bin;D:\software\libexec\lib\node_modules\npm\bin\node-gyp-bin;D:\software\lib\node_modules\npm\bin\node-gyp-bin;D:\software\nodejs\node_modules\npm\bin\node-gyp-bin;C:\Users\s4243\AppData\Local\Temp\yarn--1610614629003-0.5493713816293628;E:\webpack\electron_client\node_modules\.bin;C:\Users\s4243\AppData\Local\Yarn\Data\link\node_modules\.bin;D:\software\libexec\lib\node_modules\npm\bin\node-gyp-bin;D:\software\lib\node_modules\npm\bin\node-gyp-bin;D:\software\nodejs\node_modules\npm\bin\node-gyp-bin;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\windows\System32\OpenSSH\;D:\software\python27;D:\software\Git\cmd;D:\software\svn\bin;D:\software\nvm;D:\software\nodejs;E:\flutter\bin;C:\Program Files\MySQL\MySQL Server 8.0\bin;E:\redis\;C:\Users\s4243\AppData\Roaming\npm;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\windows\system32\config\systemprofile\.dnx\bin;C:\Program Files\Microsoft DNX\Dnvm\;C:\Users\s4243\AppData\Local\Yarn\bin;C:\Users\s4243\AppData\Roaming\npm;D:\scoop\shims;C:\Users\s4243\AppData\Local\Microsoft\WindowsApps;D:\software\Microsoft VS Code\bin;D:\software\python27;D:\software\nvm;D:\software\nodejs;C:\Users\s4243\AppData\Local\BypassRuntm;D:\software\PyCharm 2019.3.1\bin;
9 verbose lifecycle electron-client@~start:main: CWD: E:\webpack\electron_client
10 silly lifecycle electron-client@~start:main: Args: [
10 silly lifecycle   '/d /s /c',
10 silly lifecycle   'cross-env NODE_ENV=development electron -r ./.erb/scripts/BabelRegister ./src/main.dev.ts'
10 silly lifecycle ]
11 silly lifecycle electron-client@~start:main: Returned: code: 3228369023  signal: null
12 info lifecycle electron-client@~start:main: Failed to exec start:main script
13 verbose stack Error: electron-client@ start:main: `cross-env NODE_ENV=development electron -r ./.erb/scripts/BabelRegister ./src/main.dev.ts`
13 verbose stack Exit status 3228369023
13 verbose stack     at EventEmitter.<anonymous> (D:\software\nvm\v14.0.0\node_modules\npm\node_modules\npm-lifecycle\index.js:332:16)
13 verbose stack     at EventEmitter.emit (events.js:315:20)
13 verbose stack     at ChildProcess.<anonymous> (D:\software\nvm\v14.0.0\node_modules\npm\node_modules\npm-lifecycle\lib\spawn.js:55:14)
13 verbose stack     at ChildProcess.emit (events.js:315:20)
13 verbose stack     at maybeClose (internal/child_process.js:1051:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
14 verbose pkgid electron-client@
15 verbose cwd E:\webpack\electron_client
16 verbose Windows_NT 10.0.18362
17 verbose argv "D:\\software\\nodejs\\node.exe" "D:\\software\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "run" "start:main"
18 verbose node v14.0.0
19 verbose npm  v6.14.4
20 error code ELIFECYCLE
21 error errno 3228369023
22 error electron-client@ start:main: `cross-env NODE_ENV=development electron -r ./.erb/scripts/BabelRegister ./src/main.dev.ts`
22 error Exit status 3228369023
23 error Failed at the electron-client@ start:main script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 3228369023, true ]

import path from 'path';
import fs from 'fs';
import { execSync } from 'child_process';
import { remove } from 'lodash';
import { dependencies } from '../../src/package.json';

const nodeModulesPath = path.join(__dirname, '../../src/node_modules');

const modules = remove(
  Object.keys(dependencies || {}),
  (module) => module !== '@journeyapps/sqlcipher'
);

if (
  Object.keys(dependencies || {}).length > 0 &&
  fs.existsSync(nodeModulesPath)
) {
  const electronRebuildCmd = `../node_modules/.bin/electron-rebuild --parallel --force --only ${modules.join(
    ','
  )} --module-dir .`;
  // const electronRebuildCmd =
  //   '../node_modules/.bin/electron-rebuild --parallel --force --types prod,dev,optional --module-dir .';

  const cmd =
    process.platform === 'win32'
      ? electronRebuildCmd.replace(/\//g, '\\')
      : electronRebuildCmd;
  execSync(cmd, {
    cwd: path.join(__dirname, '../../src'),
    stdio: 'inherit',
  });
}

When I compile without electron-rebuild, this error still appears.

[rkistner]

Unfortunately those logs still do not contain enough info. If it is an issue with the native module, there should be an error with something like "Cannot find module", which could give a better indication of what the issue is.

[1111mp]

I created the simplest project that can reproduce the problem: https://github.com/1111mp/your-project-name
node 14.0.0
If you can, I hope you can take the time to verify.thank you very much.
If electron-rebuild is not needed to compile,can remove postinstall:
src/package.json "postinstall": "yarn electron-rebuild"

[rkistner]

That sample project works for me with node 14.0.0 (on Linux, don't have a Windows machine available currently). I ran yarn start, but my console output is completely different from yours.

Do check that you have this file installed:
src/node_modules/@journeyapps/sqlcipher/lib/binding/napi-v6-.../node_sqlite3.node

[1111mp]

I'm pretty sure there is a corresponding file:

Maybe I should test it on linux.
When I use electron 9.4.0 version and @journeyapps/sqlcipher version of 4.1.0, there is no problem at all (on windows platform)

[JustYummy]

It seems like some problem with windows, on Mac it works fine. but it crash on Windows.Testing Project

[ajmasia]

Hi guys!

I have been able to compile the library in windows using the 5.0.0 version with node 10.17.0. The point is that the library works, that is, the database is created and encrypted, but when it is used in this way, the main window of electron closes without showing any error. You can try this test repo. This occurs both in production and development. When you remove the encryption the project work fine using the database:

Using encryption:

Without using encryption:

Using a mac I have tested and it works perfectly. It just stops working using windows

Any idea what is happening? @rkistner @1111mp @Christilut?

Thanks!

Repo: https://github.com/ajmasia/electron-sqlite-playground

[1111mp]

@ajmasia At present I don't know how to solve this problem. It will only happen on the windows platform.
On windows, you can reduce the version of electron and @journeyapps/sqlcipher to avoid this problem:
electron: 9.* and @journeyapps/sqlcipher: 4.1.0 .

[ajmasia]

Thanks @1111mp for your answer. I thought version 4.1.1 only allowed up to electron 8.1.1. I have tested with version 9.4.1 and works fine on both mac and windows, but takes longer to rebuild 😢

[1111mp]

@ajmasia I am currently using electron 9.4.0 and @journeyapps/sqlcipher 4.1.0 in the official environment, and no problems have been found so far.The electron-rebuild task speed is also normal.
I don't know if I use the latest version and packaged the windows package on the mac can it run normally on windows. I haven't tested it like this yet. I will take the time to test it.
If you don't use any ORM, then you can try this:better-sqlite3

[ajmasia]

Thanks @1111mp, I will check it 😄

[ajmasia]

Hi @rkistner @1111mp I have thoroughly tested electron 9.4.1 with @journeyapps/sqlcipher: 4.1.0 and it works perfectly on all three platforms locally (win/mac/linux). Now when I try to do a build in the Github CI it gives me a 403 error getting the binaries. I'm using windows-lates virtual-environments/Windows2019-Readme.md at main · actions/virtual-environments. Any idea?

The funny thing is that on my local machine it downloads the binaries without a problem 😢

Node version: 12.16.1
Yarn version: 1.22.10

Thanks!

[ajmasia]

Hi @rkistner @1111mp I have thoroughly tested electron 9.4.1 with @journeyapps/sqlcipher: 4.1.0 and it works perfectly on all three platforms locally (win/mac/linux). Now when I try to do a build in the Github CI it gives me a 403 error getting the binaries. I'm using windows-lates virtual-environments/Windows2019-Readme.md at main · actions/virtual-environments. Any idea?

The funny thing is that on my local machine it downloads the binaries without a problem 😢

Node version: 12.16.1
Yarn version: 1.22.10

Thanks!

Fixed, was an issue with new Github actions windows image :-)
You can see a solution here, Yarn install process fails using windows-latest image · Issue #2665 · actions/virtual-environments

[Christilut]

Did anyone get this working in a production build with electron-forge?

I'm using the electron 9.4.1, sqlcipher 4.1.0 and node 12.20.0 combination that works in development but when using electron-forge make the resulting build has this error:

ERROR in ./node_modules/node-pre-gyp/lib/node-pre-gyp.js
Module not found: Error: Can't resolve './' in '/Users/chris/workspace/my-app/node_modules/node-pre-gyp/lib'
 @ ./node_modules/node-pre-gyp/lib/node-pre-gyp.js 52:13-32 184:38-57
 @ ./node_modules/@journeyapps/sqlcipher/lib/sqlite3.js

Any ideas?

[marek8623]

Hi everyone,
my package.json includes:

"@journeyapps/sqlcipher": "^5.0.0",
"cordova-electron": "^3.0.0",
"electron-rebuild": "^2.0.0"

but I am getting the following error:

Cannot find module ’/Documents/app/platforms/electron/build/mac/Application.app/Contents/Resources/app.asar/node_modules/@journeyapps/sqlcipher/lib/binding/napi-v3-darwin-x64/node_sqlite3.node

Could someone help me what are the latest versions that I need to get rid of this error?

Thanks

[TheBlueAssasin]

@ajmasia I am currently using electron 9.4.0 and @journeyapps/sqlcipher 4.1.0 in the official environment, and no problems have been found so far.The electron-rebuild task speed is also normal. I don't know if I use the latest version and packaged the windows package on the mac can it run normally on windows. I haven't tested it like this yet. I will take the time to test it. If you don't use any ORM, then you can try this:better-sqlite3

I tried using electron-builder using electron 16.0.5 and @journeyapps/sqlcipher 5.3.1. It builds fine, but when I run the application on Linux 64 bit OS. I get a Trace/breakpoint trap error after running a couple of queries. I haven't been able to find a rule about which queries break the application though. It is a bit random.
It works well with the versions you specified.