Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: create V8 code cache after script is run #21567

Closed
wants to merge 1 commit into from

Conversation

joyeecheung
Copy link
Member

@joyeecheung joyeecheung commented Jun 27, 2018

This patch makes it possible to generate the code cache
for the builtins directly from the original script object
(instead of compiling a new one) and after the script has
been run (via NativeModule.require). Before this patch
only the top level functions (the wrapped ones)
are included in the cache, after this patch the inner
functions in those modules will be included as well.

Also blacklists modules from dependencies like V8 and
node-inspect since we cannot guarantee that they are suitable
to be executed directly.

Refs: #21563

Performance stats

Ideally this should improve the startup time now that we include more into the cache,
but here is what I get now:

                       confidence improvement accuracy (*)   (**)  (***)
 misc/startup.js dur=1                -0.18 %       ±1.89% ±2.52% ±3.28%

Before this patch, the logs of tools/generate_code_cache.js

See logs
out/Release/node --expose-internals tools/generate_code_cache.js out/Release/obj/gen/node_code_cache.cc
Generated cache for 'internal/bootstrap/cache', size = 1.84KB, total = 1.84KB
Generated cache for 'async_hooks', size = 4.95KB, total = 6.79KB
Generated cache for 'assert', size = 8.68KB, total = 15.47KB
Generated cache for 'buffer', size = 17.08KB, total = 32.55KB
Generated cache for 'child_process', size = 5.73KB, total = 38.27KB
Generated cache for 'console', size = 10.13KB, total = 48.41KB
Generated cache for 'constants', size = 1.16KB, total = 49.56KB
Generated cache for 'crypto', size = 7.79KB, total = 57.35KB
Generated cache for 'cluster', size = 1.00KB, total = 58.35KB
Generated cache for 'dgram', size = 8.73KB, total = 67.09KB
Generated cache for 'dns', size = 7.60KB, total = 74.69KB
Generated cache for 'domain', size = 6.75KB, total = 81.44KB
Generated cache for 'events', size = 10.35KB, total = 91.79KB
Generated cache for 'fs', size = 27.88KB, total = 119.66KB
Generated cache for 'http', size = 2.26KB, total = 121.92KB
Generated cache for 'http2', size = 1.66KB, total = 123.59KB
Generated cache for '_http_agent', size = 3.20KB, total = 126.78KB
Generated cache for '_http_client', size = 6.70KB, total = 133.48KB
Generated cache for '_http_common', size = 4.01KB, total = 137.48KB
Generated cache for '_http_incoming', size = 2.22KB, total = 139.70KB
Generated cache for '_http_outgoing', size = 9.00KB, total = 148.70KB
Generated cache for '_http_server', size = 10.34KB, total = 159.05KB
Generated cache for 'https', size = 3.86KB, total = 162.91KB
Generated cache for 'inspector', size = 3.34KB, total = 166.25KB
Generated cache for 'module', size = 840.00B, total = 167.07KB
Generated cache for 'net', size = 21.53KB, total = 188.60KB
Generated cache for 'os', size = 6.59KB, total = 195.20KB
Generated cache for 'path', size = 8.84KB, total = 204.03KB
Generated cache for 'perf_hooks', size = 11.95KB, total = 215.98KB
Generated cache for 'process', size = 776.00B, total = 216.73KB
Generated cache for 'punycode', size = 3.83KB, total = 220.56KB
Generated cache for 'querystring', size = 5.75KB, total = 226.31KB
Generated cache for 'readline', size = 9.76KB, total = 236.07KB
Generated cache for 'repl', size = 10.91KB, total = 246.98KB
Generated cache for 'stream', size = 2.78KB, total = 249.77KB
Generated cache for '_stream_readable', size = 12.66KB, total = 262.43KB
Generated cache for '_stream_writable', size = 13.59KB, total = 276.02KB
Generated cache for '_stream_duplex', size = 3.02KB, total = 279.04KB
Generated cache for '_stream_transform', size = 2.66KB, total = 281.70KB
Generated cache for '_stream_passthrough', size = 1.29KB, total = 282.99KB
Generated cache for '_stream_wrap', size = 880.00B, total = 283.85KB
Generated cache for 'string_decoder', size = 3.37KB, total = 287.22KB
Generated cache for 'sys', size = 1008.00B, total = 288.20KB
Generated cache for 'timers', size = 9.07KB, total = 297.27KB
Generated cache for 'tls', size = 5.08KB, total = 302.35KB
Generated cache for '_tls_common', size = 2.50KB, total = 304.85KB
Generated cache for '_tls_wrap', size = 12.74KB, total = 317.59KB
Generated cache for 'trace_events', size = 3.39KB, total = 320.98KB
Generated cache for 'tty', size = 4.25KB, total = 325.23KB
Generated cache for 'url', size = 8.33KB, total = 333.56KB
Generated cache for 'util', size = 19.30KB, total = 352.87KB
Generated cache for 'v8', size = 7.41KB, total = 360.28KB
Generated cache for 'vm', size = 7.10KB, total = 367.38KB
Generated cache for 'worker_threads', size = 1.21KB, total = 368.59KB
Generated cache for 'zlib', size = 11.30KB, total = 379.89KB
Generated cache for 'internal/assert', size = 2.68KB, total = 382.57KB
Generated cache for 'internal/async_hooks', size = 8.59KB, total = 391.16KB
Generated cache for 'internal/buffer', size = 9.52KB, total = 400.69KB
Generated cache for 'internal/cli_table', size = 2.39KB, total = 403.08KB
Generated cache for 'internal/child_process', size = 10.37KB, total = 413.45KB
Generated cache for 'internal/cluster/child', size = 3.28KB, total = 416.73KB
Generated cache for 'internal/cluster/master', size = 5.93KB, total = 422.66KB
Generated cache for 'internal/cluster/round_robin_handle', size = 1.99KB, total = 424.65KB
Generated cache for 'internal/cluster/shared_handle', size = 1.41KB, total = 426.05KB
Generated cache for 'internal/cluster/utils', size = 1.34KB, total = 427.39KB
Generated cache for 'internal/cluster/worker', size = 1.77KB, total = 429.16KB
Generated cache for 'internal/crypto/certificate', size = 2.13KB, total = 431.29KB
Generated cache for 'internal/crypto/cipher', size = 5.23KB, total = 436.52KB
Generated cache for 'internal/crypto/diffiehellman', size = 4.80KB, total = 441.32KB
Generated cache for 'internal/crypto/hash', size = 3.41KB, total = 444.73KB
Generated cache for 'internal/crypto/pbkdf2', size = 2.54KB, total = 447.27KB
Generated cache for 'internal/crypto/random', size = 2.72KB, total = 449.98KB
Generated cache for 'internal/crypto/scrypt', size = 2.68KB, total = 452.66KB
Generated cache for 'internal/crypto/sig', size = 3.25KB, total = 455.91KB
Generated cache for 'internal/crypto/util', size = 3.34KB, total = 459.25KB
Generated cache for 'internal/constants', size = 2.87KB, total = 462.12KB
Generated cache for 'internal/dns/promises', size = 5.73KB, total = 467.85KB
Generated cache for 'internal/dns/utils', size = 3.60KB, total = 471.45KB
Generated cache for 'internal/encoding', size = 23.34KB, total = 494.80KB
Generated cache for 'internal/errors', size = 33.81KB, total = 528.61KB
Generated cache for 'internal/error-serdes', size = 3.19KB, total = 531.80KB
Generated cache for 'internal/fixed_queue', size = 4.05KB, total = 535.84KB
Generated cache for 'internal/freelist', size = 1.38KB, total = 537.23KB
Generated cache for 'internal/fs/promises', size = 9.45KB, total = 546.67KB
Generated cache for 'internal/fs/read_file_context', size = 2.10KB, total = 548.77KB
Generated cache for 'internal/fs/streams', size = 4.63KB, total = 553.41KB
Generated cache for 'internal/fs/sync_write_stream', size = 1.98KB, total = 555.39KB
Generated cache for 'internal/fs/utils', size = 10.46KB, total = 565.85KB
Generated cache for 'internal/fs/watchers', size = 3.74KB, total = 569.59KB
Generated cache for 'internal/http', size = 1.41KB, total = 571.01KB
Generated cache for 'internal/inspector_async_hook', size = 2.02KB, total = 573.03KB
Generated cache for 'internal/linkedlist', size = 1.37KB, total = 574.40KB
Generated cache for 'internal/modules/cjs/helpers', size = 5.29KB, total = 579.69KB
Generated cache for 'internal/modules/cjs/loader', size = 23.36KB, total = 603.05KB
Generated cache for 'internal/modules/esm/loader', size = 2.91KB, total = 605.95KB
Generated cache for 'internal/modules/esm/create_dynamic_module', size = 1.66KB, total = 607.62KB
Generated cache for 'internal/modules/esm/default_resolve', size = 3.25KB, total = 610.87KB
Generated cache for 'internal/modules/esm/module_job', size = 2.28KB, total = 613.15KB
Generated cache for 'internal/modules/esm/module_map', size = 2.02KB, total = 615.17KB
Generated cache for 'internal/modules/esm/translators', size = 3.85KB, total = 619.02KB
Generated cache for 'internal/safe_globals', size = 1.98KB, total = 621.01KB
Generated cache for 'internal/net', size = 2.24KB, total = 623.25KB
Generated cache for 'internal/os', size = 1.29KB, total = 624.54KB
Generated cache for 'internal/priority_queue', size = 2.77KB, total = 627.30KB
Generated cache for 'internal/process/esm_loader', size = 2.59KB, total = 629.89KB
Generated cache for 'internal/process/main_thread_only', size = 6.70KB, total = 636.59KB
Generated cache for 'internal/process/next_tick', size = 5.02KB, total = 641.60KB
Generated cache for 'internal/process/per_thread', size = 6.98KB, total = 648.59KB
Generated cache for 'internal/process/promises', size = 2.37KB, total = 650.95KB
Generated cache for 'internal/process/stdio', size = 5.47KB, total = 656.42KB
Generated cache for 'internal/process/warning', size = 3.39KB, total = 659.81KB
Generated cache for 'internal/process/worker_thread_only', size = 1.25KB, total = 661.06KB
Generated cache for 'internal/querystring', size = 3.18KB, total = 664.24KB
Generated cache for 'internal/process/write-coverage', size = 1.35KB, total = 665.59KB
Generated cache for 'internal/readline', size = 2.52KB, total = 668.11KB
Generated cache for 'internal/repl', size = 2.20KB, total = 670.31KB
Generated cache for 'internal/repl/await', size = 2.98KB, total = 673.29KB
Generated cache for 'internal/socket_list', size = 2.52KB, total = 675.81KB
Generated cache for 'internal/test/binding', size = 1.29KB, total = 677.10KB
Generated cache for 'internal/test/unicode', size = 864.00B, total = 677.95KB
Generated cache for 'internal/timers', size = 3.00KB, total = 680.95KB
Generated cache for 'internal/tls', size = 960.00B, total = 681.88KB
Generated cache for 'internal/trace_events_async_hooks', size = 1.12KB, total = 683.00KB
Generated cache for 'internal/tty', size = 4.93KB, total = 687.93KB
Generated cache for 'internal/url', size = 18.53KB, total = 706.46KB
Generated cache for 'internal/util', size = 6.77KB, total = 713.23KB
Generated cache for 'internal/util/comparisons', size = 3.83KB, total = 717.05KB
Generated cache for 'internal/util/inspector', size = 1.37KB, total = 718.42KB
Generated cache for 'internal/util/types', size = 3.57KB, total = 721.99KB
Generated cache for 'internal/http2/core', size = 34.95KB, total = 756.94KB
Generated cache for 'internal/http2/compat', size = 13.67KB, total = 770.61KB
Generated cache for 'internal/http2/util', size = 11.24KB, total = 781.85KB
Generated cache for 'internal/v8_prof_polyfill', size = 3.05KB, total = 784.90KB
Generated cache for 'internal/v8_prof_processor', size = 2.57KB, total = 787.47KB
Generated cache for 'internal/validators', size = 4.35KB, total = 791.82KB
Generated cache for 'internal/stream_base_commons', size = 3.75KB, total = 795.57KB
Generated cache for 'internal/vm/module', size = 5.76KB, total = 801.33KB
Generated cache for 'internal/worker', size = 8.88KB, total = 810.21KB
Generated cache for 'internal/streams/lazy_transform', size = 1.95KB, total = 812.16KB
Generated cache for 'internal/streams/async_iterator', size = 3.13KB, total = 815.29KB
Generated cache for 'internal/streams/buffer_list', size = 3.14KB, total = 818.43KB
Generated cache for 'internal/streams/duplexpair', size = 2.13KB, total = 820.56KB
Generated cache for 'internal/streams/legacy', size = 1.57KB, total = 822.13KB
Generated cache for 'internal/streams/destroy', size = 2.15KB, total = 824.28KB
Generated cache for 'internal/streams/state', size = 2.06KB, total = 826.34KB
Generated cache for 'internal/streams/pipeline', size = 2.09KB, total = 828.43KB
Generated cache for 'internal/streams/end-of-stream', size = 1.55KB, total = 829.98KB
Generated cache for 'internal/wrap_js_stream', size = 3.84KB, total = 833.82KB
Generated cache for 'internal/deps/v8/tools/splaytree', size = 3.21KB, total = 837.03KB
Generated cache for 'v8/tools/splaytree', size = 1.19KB, total = 838.22KB
Generated cache for 'internal/deps/v8/tools/codemap', size = 4.98KB, total = 843.20KB
Generated cache for 'v8/tools/codemap', size = 1.18KB, total = 844.38KB
Generated cache for 'internal/deps/v8/tools/consarray', size = 1.55KB, total = 845.93KB
Generated cache for 'v8/tools/consarray', size = 1.19KB, total = 847.12KB
Generated cache for 'internal/deps/v8/tools/csvparser', size = 1.34KB, total = 848.46KB
Generated cache for 'v8/tools/csvparser', size = 1.19KB, total = 849.65KB
Generated cache for 'internal/deps/v8/tools/profile', size = 14.45KB, total = 864.09KB
Generated cache for 'v8/tools/profile', size = 1.18KB, total = 865.27KB
Generated cache for 'internal/deps/v8/tools/profile_view', size = 2.79KB, total = 868.06KB
Generated cache for 'v8/tools/profile_view', size = 1.20KB, total = 869.26KB
Generated cache for 'internal/deps/v8/tools/logreader', size = 2.06KB, total = 871.32KB
Generated cache for 'v8/tools/logreader', size = 1.19KB, total = 872.51KB
Generated cache for 'internal/deps/v8/tools/arguments', size = 1.83KB, total = 874.34KB
Generated cache for 'v8/tools/arguments', size = 1.19KB, total = 875.52KB
Generated cache for 'internal/deps/v8/tools/tickprocessor', size = 11.48KB, total = 887.01KB
Generated cache for 'v8/tools/tickprocessor', size = 1.20KB, total = 888.20KB
Generated cache for 'internal/deps/v8/tools/SourceMap', size = 4.76KB, total = 892.96KB
Generated cache for 'v8/tools/SourceMap', size = 1.19KB, total = 894.15KB
Generated cache for 'internal/deps/v8/tools/tickprocessor-driver', size = 2.71KB, total = 896.86KB
Generated cache for 'v8/tools/tickprocessor-driver', size = 1.22KB, total = 898.08KB
Generated cache for 'internal/deps/node-inspect/lib/_inspect', size = 4.77KB, total = 902.84KB
Generated cache for 'node-inspect/lib/_inspect', size = 1.21KB, total = 904.05KB
Generated cache for 'internal/deps/node-inspect/lib/internal/inspect_client', size = 4.00KB, total = 908.05KB
Generated cache for 'node-inspect/lib/internal/inspect_client', size = 1.25KB, total = 909.30KB
Generated cache for 'internal/deps/node-inspect/lib/internal/inspect_repl', size = 8.20KB, total = 917.51KB
Generated cache for 'node-inspect/lib/internal/inspect_repl', size = 1.24KB, total = 918.75KB
Generated cache for 'internal/deps/acorn/dist/acorn', size = 53.38KB, total = 972.13KB
Generated cache for 'internal/deps/acorn/dist/walk', size = 12.80KB, total = 984.94KB
Generated code cache C++ file to out/Release/obj/gen/node_code_cache.cc

After:

See logs
out/Release/node --expose-internals tools/generate_code_cache.js out/Release/obj/gen/node_code_cache.cc
Generated cache for 'internal/bootstrap/cache', size = 3.66KB, total = 3.66KB
Generated cache for 'async_hooks', size = 4.95KB, total = 8.61KB
Generated cache for 'assert', size = 8.68KB, total = 17.29KB
Generated cache for 'buffer', size = 17.08KB, total = 34.37KB
Generated cache for 'child_process', size = 6.02KB, total = 40.39KB
Generated cache for 'console', size = 10.13KB, total = 50.52KB
Generated cache for 'constants', size = 1.16KB, total = 51.68KB
Generated cache for 'crypto', size = 7.79KB, total = 59.47KB
Generated cache for 'cluster', size = 1.00KB, total = 60.47KB
Generated cache for 'dgram', size = 8.73KB, total = 69.20KB
Generated cache for 'dns', size = 7.94KB, total = 77.14KB
Generated cache for 'domain', size = 7.11KB, total = 84.25KB
Generated cache for 'events', size = 10.68KB, total = 94.93KB
Generated cache for 'fs', size = 27.88KB, total = 122.80KB
Generated cache for 'http', size = 2.26KB, total = 125.06KB
Generated cache for 'http2', size = 1.66KB, total = 126.73KB
Generated cache for '_http_agent', size = 4.36KB, total = 131.09KB
Generated cache for '_http_client', size = 6.70KB, total = 137.78KB
Generated cache for '_http_common', size = 4.01KB, total = 141.79KB
Generated cache for '_http_incoming', size = 2.22KB, total = 144.01KB
Generated cache for '_http_outgoing', size = 9.00KB, total = 153.01KB
Generated cache for '_http_server', size = 10.34KB, total = 163.35KB
Generated cache for 'https', size = 4.50KB, total = 167.85KB
Generated cache for 'inspector', size = 3.35KB, total = 171.20KB
Generated cache for 'module', size = 840.00B, total = 172.02KB
Generated cache for 'net', size = 21.53KB, total = 193.55KB
Generated cache for 'os', size = 6.84KB, total = 200.39KB
Generated cache for 'path', size = 8.84KB, total = 209.23KB
Generated cache for 'perf_hooks', size = 12.33KB, total = 221.55KB
Generated cache for 'process', size = 776.00B, total = 222.31KB
Generated cache for 'punycode', size = 3.83KB, total = 226.14KB
Generated cache for 'querystring', size = 5.75KB, total = 231.89KB
Generated cache for 'readline', size = 9.76KB, total = 241.65KB
Generated cache for 'repl', size = 10.91KB, total = 252.56KB
Generated cache for 'stream', size = 2.78KB, total = 255.34KB
Generated cache for '_stream_readable', size = 12.68KB, total = 268.02KB
Generated cache for '_stream_writable', size = 13.60KB, total = 281.63KB
Generated cache for '_stream_duplex', size = 3.05KB, total = 284.67KB
Generated cache for '_stream_transform', size = 2.69KB, total = 287.36KB
Generated cache for '_stream_passthrough', size = 1.32KB, total = 288.68KB
Generated cache for '_stream_wrap', size = 880.00B, total = 289.54KB
Generated cache for 'string_decoder', size = 3.37KB, total = 292.91KB
Generated cache for 'timers', size = 9.07KB, total = 301.98KB
Generated cache for 'tls', size = 5.08KB, total = 307.05KB
Generated cache for '_tls_common', size = 2.50KB, total = 309.55KB
Generated cache for '_tls_wrap', size = 12.98KB, total = 322.53KB
Generated cache for 'trace_events', size = 3.40KB, total = 325.93KB
Generated cache for 'tty', size = 4.25KB, total = 330.18KB
Generated cache for 'url', size = 8.33KB, total = 338.51KB
Generated cache for 'util', size = 19.46KB, total = 357.97KB
Generated cache for 'v8', size = 7.41KB, total = 365.38KB
Generated cache for 'vm', size = 7.10KB, total = 372.48KB
Generated cache for 'worker_threads', size = 1.21KB, total = 373.70KB
Generated cache for 'zlib', size = 11.85KB, total = 385.55KB
Generated cache for 'internal/assert', size = 2.70KB, total = 388.25KB
Generated cache for 'internal/async_hooks', size = 8.62KB, total = 396.87KB
Generated cache for 'internal/buffer', size = 9.55KB, total = 406.41KB
Generated cache for 'internal/cli_table', size = 2.39KB, total = 408.80KB
Generated cache for 'internal/child_process', size = 10.37KB, total = 419.17KB
Generated cache for 'internal/cluster/child', size = 3.28KB, total = 422.45KB
Generated cache for 'internal/cluster/master', size = 5.93KB, total = 428.38KB
Generated cache for 'internal/cluster/round_robin_handle', size = 1.99KB, total = 430.38KB
Generated cache for 'internal/cluster/shared_handle', size = 1.41KB, total = 431.78KB
Generated cache for 'internal/cluster/utils', size = 1.34KB, total = 433.12KB
Generated cache for 'internal/cluster/worker', size = 1.77KB, total = 434.88KB
Generated cache for 'internal/crypto/certificate', size = 2.13KB, total = 437.02KB
Generated cache for 'internal/crypto/cipher', size = 5.98KB, total = 442.99KB
Generated cache for 'internal/crypto/diffiehellman', size = 4.80KB, total = 447.79KB
Generated cache for 'internal/crypto/hash', size = 3.41KB, total = 451.20KB
Generated cache for 'internal/crypto/pbkdf2', size = 2.54KB, total = 453.73KB
Generated cache for 'internal/crypto/random', size = 2.72KB, total = 456.45KB
Generated cache for 'internal/crypto/scrypt', size = 2.68KB, total = 459.13KB
Generated cache for 'internal/crypto/sig', size = 3.25KB, total = 462.38KB
Generated cache for 'internal/crypto/util', size = 3.34KB, total = 465.72KB
Generated cache for 'internal/constants', size = 2.89KB, total = 468.61KB
Generated cache for 'internal/dns/promises', size = 6.07KB, total = 474.68KB
Generated cache for 'internal/dns/utils', size = 4.64KB, total = 479.32KB
Generated cache for 'internal/encoding', size = 23.37KB, total = 502.69KB
Generated cache for 'internal/errors', size = 33.83KB, total = 536.52KB
Generated cache for 'internal/error-serdes', size = 3.19KB, total = 539.70KB
Generated cache for 'internal/fixed_queue', size = 4.07KB, total = 543.77KB
Generated cache for 'internal/freelist', size = 1.69KB, total = 545.46KB
Generated cache for 'internal/fs/promises', size = 9.45KB, total = 554.91KB
Generated cache for 'internal/fs/read_file_context', size = 2.10KB, total = 557.01KB
Generated cache for 'internal/fs/streams', size = 4.63KB, total = 561.64KB
Generated cache for 'internal/fs/sync_write_stream', size = 1.98KB, total = 563.63KB
Generated cache for 'internal/fs/utils', size = 10.48KB, total = 574.11KB
Generated cache for 'internal/fs/watchers', size = 3.74KB, total = 577.85KB
Generated cache for 'internal/http', size = 1.41KB, total = 579.27KB
Generated cache for 'internal/inspector_async_hook', size = 2.05KB, total = 581.31KB
Generated cache for 'internal/linkedlist', size = 1.40KB, total = 582.71KB
Generated cache for 'internal/modules/cjs/helpers', size = 5.32KB, total = 588.03KB
Generated cache for 'internal/modules/cjs/loader', size = 23.29KB, total = 611.32KB
Generated cache for 'internal/modules/esm/loader', size = 2.91KB, total = 614.23KB
Generated cache for 'internal/modules/esm/create_dynamic_module', size = 1.66KB, total = 615.89KB
Generated cache for 'internal/modules/esm/default_resolve', size = 3.25KB, total = 619.14KB
Generated cache for 'internal/modules/esm/module_job', size = 2.28KB, total = 621.42KB
Generated cache for 'internal/modules/esm/module_map', size = 2.02KB, total = 623.45KB
Generated cache for 'internal/modules/esm/translators', size = 3.85KB, total = 627.30KB
Generated cache for 'internal/safe_globals', size = 3.93KB, total = 631.23KB
Generated cache for 'internal/net', size = 2.27KB, total = 633.49KB
Generated cache for 'internal/os', size = 1.29KB, total = 634.78KB
Generated cache for 'internal/priority_queue', size = 2.79KB, total = 637.57KB
Generated cache for 'internal/process/esm_loader', size = 2.77KB, total = 640.34KB
Generated cache for 'internal/process/main_thread_only', size = 6.72KB, total = 647.05KB
Generated cache for 'internal/process/next_tick', size = 5.05KB, total = 652.10KB
Generated cache for 'internal/process/per_thread', size = 7.23KB, total = 659.33KB
Generated cache for 'internal/process/promises', size = 2.40KB, total = 661.73KB
Generated cache for 'internal/process/stdio', size = 5.48KB, total = 667.21KB
Generated cache for 'internal/process/warning', size = 4.76KB, total = 671.97KB
Generated cache for 'internal/process/worker_thread_only', size = 1.25KB, total = 673.22KB
Generated cache for 'internal/querystring', size = 3.21KB, total = 676.43KB
Generated cache for 'internal/process/write-coverage', size = 1.35KB, total = 677.78KB
Generated cache for 'internal/readline', size = 2.88KB, total = 680.66KB
Generated cache for 'internal/repl', size = 2.20KB, total = 682.87KB
Generated cache for 'internal/repl/await', size = 2.98KB, total = 685.84KB
Generated cache for 'internal/socket_list', size = 2.52KB, total = 688.37KB
Generated cache for 'internal/test/unicode', size = 864.00B, total = 689.21KB
Generated cache for 'internal/timers', size = 3.02KB, total = 692.23KB
Generated cache for 'internal/tls', size = 960.00B, total = 693.16KB
Generated cache for 'internal/trace_events_async_hooks', size = 1.12KB, total = 694.28KB
Generated cache for 'internal/tty', size = 4.96KB, total = 699.24KB
Generated cache for 'internal/url', size = 18.56KB, total = 717.80KB
Generated cache for 'internal/util', size = 9.06KB, total = 726.87KB
Generated cache for 'internal/util/comparisons', size = 3.83KB, total = 730.70KB
Generated cache for 'internal/util/inspector', size = 1.37KB, total = 732.06KB
Generated cache for 'internal/util/types', size = 3.59KB, total = 735.66KB
Generated cache for 'internal/http2/core', size = 34.94KB, total = 770.59KB
Generated cache for 'internal/http2/compat', size = 13.67KB, total = 784.27KB
Generated cache for 'internal/http2/util', size = 11.24KB, total = 795.51KB
Generated cache for 'internal/validators', size = 4.38KB, total = 799.88KB
Generated cache for 'internal/stream_base_commons', size = 3.78KB, total = 803.66KB
Generated cache for 'internal/vm/module', size = 5.76KB, total = 809.42KB
Generated cache for 'internal/worker', size = 8.88KB, total = 818.30KB
Generated cache for 'internal/streams/lazy_transform', size = 2.31KB, total = 820.62KB
Generated cache for 'internal/streams/async_iterator', size = 3.13KB, total = 823.75KB
Generated cache for 'internal/streams/buffer_list', size = 3.17KB, total = 826.92KB
Generated cache for 'internal/streams/duplexpair', size = 2.13KB, total = 829.05KB
Generated cache for 'internal/streams/legacy', size = 1.59KB, total = 830.64KB
Generated cache for 'internal/streams/destroy', size = 2.16KB, total = 832.80KB
Generated cache for 'internal/streams/state', size = 2.09KB, total = 834.89KB
Generated cache for 'internal/streams/pipeline', size = 2.12KB, total = 837.01KB
Generated cache for 'internal/streams/end-of-stream', size = 1.57KB, total = 838.58KB
Generated cache for 'internal/wrap_js_stream', size = 3.99KB, total = 842.57KB
Generated code cache C++ file to out/Release/obj/gen/node_code_cache.cc
(node:71564) ExperimentalWarning: The http2 module is an experimental API.
/Users/joyee/.pyenv/versions/my-env/bin/python2.7 ./configure --code-cache-path out/Release/obj/gen/node_code_cache.cc
Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added the tools Issues and PRs related to the tools directory. label Jun 27, 2018
@joyeecheung
Copy link
Member Author

joyeecheung commented Jun 27, 2018

From #21563

As a sanity check, does the code cache size significantly if you create it after the script has been run?

@hashseed from the logs (posted in the OP) it does not look like the size changes much. Some of them changed about 1-2KB, but most of them are unchanged. I excluded the scripts under deps and the prof processor polyfills so the set of modules being cached is not exactly the same.

@hashseed
Copy link
Member

Seems like most scripts run during bootstrapping are straight-line code that only set up things and do not often call inner functions, which is why not a lot of lazily compiled functions get included, and therefore there is no benefit.

@hashseed
Copy link
Member

How come we don't see acorn being cached in the log for after?

(key) => NativeModule.isDepsModule(key) || key.startsWith('internal/deps')
);

// Modules with source code compiled in js2c that
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand this comment. Why can't these scripts be cached?

Copy link
Member Author

@joyeecheung joyeecheung Jun 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • config is a JSON string, it's not going to be wrapped
  • sys is deprecated and is redirected from util (executing it would trigger a deprecation warning)
  • internal/test/binding triggers a warning as well (also, it's not supposed to be loaded by users)
  • internal/v8_prof_polyfill is not supposed to be required directly (see the comments there)
  • internal/v8_prof_processor is an actual script (instead of a module) that is supposed to write a log file

The files in v8/tools are not really Node.js modules. They are mostly concatenated and run by internal/v8_prof_processor since they expose stuff to the global space and rely on that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thanks for the explanation.

@joyeecheung
Copy link
Member Author

How come we don't see acorn being cached in the log for after?

I've removed the internal/deps modules for now since there is no guarantee that they are Node.js modules that can be required (instead of scripts that are supposed to be run directly). We could add a white list for those that actually are.

if (cached && (cached.loaded || cached.loading)) {
const script = cached.script;
return script.createCachedData();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ unneeded var assignment return cached.script.createCachedData()

This patch makes it possible to generate the code cache
for the builtins directly from the original script object
(instead of compiling a new one) and after the script has
been run (via `NativeModule.require`). Before this patch
only the top level functions (the wrapped ones)
are included in the cache, after this patch the inner
functions in those modules will be included as well.

Also blacklists modules from dependencies like V8 and
node-inspect since we cannot guarantee that they are suitable
to be executed directly.
@joyeecheung
Copy link
Member Author

Rebased and addressed review from @jdalton. I think this is worth landing even though it does not affect the performance since it cleans up the code a bit and we no longer need to recompile the source.

@joyeecheung
Copy link
Member Author

@joyeecheung
Copy link
Member Author

Another CI is green: https://ci.nodejs.org/job/node-test-pull-request/16001/

@joyeecheung
Copy link
Member Author

Landed in 186c2fb, thanks!

joyeecheung added a commit that referenced this pull request Jul 27, 2018
This patch makes it possible to generate the code cache
for the builtins directly from the original script object
(instead of compiling a new one) and after the script has
been run (via `NativeModule.require`). Before this patch
only the top level functions (the wrapped ones)
are included in the cache, after this patch the inner
functions in those modules will be included as well.

Also blacklists modules from dependencies like V8 and
node-inspect since we cannot guarantee that they are suitable
to be executed directly.

PR-URL: #21567
Refs: #21563
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: John-David Dalton <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
targos pushed a commit that referenced this pull request Jul 31, 2018
This patch makes it possible to generate the code cache
for the builtins directly from the original script object
(instead of compiling a new one) and after the script has
been run (via `NativeModule.require`). Before this patch
only the top level functions (the wrapped ones)
are included in the cache, after this patch the inner
functions in those modules will be included as well.

Also blacklists modules from dependencies like V8 and
node-inspect since we cannot guarantee that they are suitable
to be executed directly.

PR-URL: #21567
Refs: #21563
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: John-David Dalton <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
@targos targos mentioned this pull request Jul 31, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tools Issues and PRs related to the tools directory.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants