Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More zlib/fwrite tracing for Solaris #4826

Merged
merged 3 commits into from
Dec 1, 2020
Merged

More zlib/fwrite tracing for Solaris #4826

merged 3 commits into from
Dec 1, 2020

Conversation

mattdowle
Copy link
Member

@mattdowle mattdowle commented Dec 1, 2020

As per #4099 (comment)
Also adds zlib version (both library and header) into test.data.table() output.

New output of test 1658.421 when working correctly locally. The end of the long lines has the new detail.

DT = data.table(a=rep(1:2,each=100), b=rep(1:4,each=25))
fwrite(DT, file=f1<-tempfile(fileext=".gz"), verbose=TRUE)
  OpenMP version (_OPENMP)       201511
  omp_get_num_procs()            12
  R_DATATABLE_NUM_PROCS_PERCENT  unset (default 50)
  R_DATATABLE_NUM_THREADS        unset
  R_DATATABLE_THROTTLE           unset (default 1024)
  omp_get_thread_limit()         2147483647
  omp_get_max_threads()          12
  OMP_THREAD_LIMIT               unset
  OMP_NUM_THREADS                unset
  RestoreAfterFork               true
  data.table is using 6 threads with throttle==1024. See ?setDTthreads.
No list columns are present. Setting sep2='' otherwise quote='auto' would quote fields containing sep2.
Column writers: 3 3 
args.doRowNames=0 args.rowNames=0 doQuote=-128 args.nrow=200 args.ncol=2 eolLen=1
maxLineLen=51. Found in 0.000s
Writing bom (false), yaml (0 characters) and column names (true) ... z_stream for header (1): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 41 5f d3 55 00 00 90 50 04 20 8e 7f 00 00 a0 50 04 20 8e 7f 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 state: 90 a3 ab 21 fe 7f 00 00 39 00 00 00 strm==0x7ffe21aba390 state->strm==0x7ffe21aba390 state->status==57 zalloc==0x7f8e20045090 zfree==0x7f8e200450a0 (s->strm==strm)==1 s->next_out==(nil) s->avail_in=0 s->next_in=(nil) deflates()'s checks would return -2 here
deflate input stream: 0x55d355f6c780 39 0x55d35e976760 4 z_stream: 60 67 97 5e d3 55 00 00 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80 c7 f6 55 d3 55 00 00 27 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 41 5f d3 55 00 00 90 50 04 20 8e 7f 00 00 a0 50 04 20 8e 7f 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 state: 90 a3 ab 21 fe 7f 00 00 39 00 00 00 strm==0x7ffe21aba390 state->strm==0x7ffe21aba390 state->status==57 zalloc==0x7f8e20045090 zfree==0x7f8e200450a0 (s->strm==strm)==1 s->next_out==0x55d355f6c780 s->avail_in=4 s->next_in=0x55d35e976760 deflates()'s checks would be ok here
deflate returned 1 with stream->total_out==24; Z_FINISH==4, Z_OK==0, Z_STREAM_END==1 z_stream: 64 67 97 5e d3 55 00 00 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 98 c7 f6 55 d3 55 00 00 0f 00 00 00 00 00 00 00 18 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 41 5f d3 55 00 00 90 50 04 20 8e 7f 00 00 a0 50 04 20 8e 7f 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 c5 10 97 24 00 00 00 00 00 00 00 00 00 00 00 00 state: 90 a3 ab 21 fe 7f 00 00 9a 02 00 00 strm==0x7ffe21aba390 state->strm==0x7ffe21aba390 state->status==666 zalloc==0x7f8e20045090 zfree==0x7f8e200450a0 (s->strm==strm)==1 s->next_out==0x55d355f6c798 s->avail_in=0 s->next_in=0x55d35e976764 deflates()'s checks would be ok here
z_stream for header (2): 64 67 97 5e d3 55 00 00 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 98 c7 f6 55 d3 55 00 00 0f 00 00 00 00 00 00 00 18 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 41 5f d3 55 00 00 90 50 04 20 8e 7f 00 00 a0 50 04 20 8e 7f 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 c5 10 97 24 00 00 00 00 00 00 00 00 00 00 00 00 state: 90 a3 ab 21 fe 7f 00 00 9a 02 00 00 strm==0x7ffe21aba390 state->strm==0x7ffe21aba390 state->status==666 zalloc==0x7f8e20045090 zfree==0x7f8e200450a0 (s->strm==strm)==1 s->next_out==0x55d355f6c798 s->avail_in=0 s->next_in=0x55d35e976764 deflates()'s checks would be ok here
done in 0.002s
Writing 200 rows in 1 batches of 200 rows (each buffer size 8MB, showProgress=1, nth=1)
zbuffSize=8391193 returned from deflateBound
z_stream for data (1): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 41 5f d3 55 00 00 90 50 04 20 8e 7f 00 00 a0 50 04 20 8e 7f 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 state: 70 a2 ab 21 fe 7f 00 00 39 00 00 00 strm==0x7ffe21aba270 state->strm==0x7ffe21aba270 state->status==57 zalloc==0x7f8e20045090 zfree==0x7f8e200450a0 (s->strm==strm)==1 s->next_out==(nil) s->avail_in=0 s->next_in=(nil) deflates()'s checks would return -2 here
z_stream for data (2): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 41 5f d3 55 00 00 90 50 04 20 8e 7f 00 00 a0 50 04 20 8e 7f 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 state: 70 a2 ab 21 fe 7f 00 00 39 00 00 00 strm==0x7ffe21aba270 state->strm==0x7ffe21aba270 state->status==57 zalloc==0x7f8e20045090 zfree==0x7f8e200450a0 (s->strm==strm)==1 s->next_out==(nil) s->avail_in=0 s->next_in=(nil) deflates()'s checks would return -2 here
deflate input stream: 0x55d36495bb10 8391193 0x55d363d76b80 800 z_stream: 80 6b d7 63 d3 55 00 00 20 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 10 bb 95 64 d3 55 00 00 19 0a 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 41 5f d3 55 00 00 90 50 04 20 8e 7f 00 00 a0 50 04 20 8e 7f 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 state: 70 a2 ab 21 fe 7f 00 00 39 00 00 00 strm==0x7ffe21aba270 state->strm==0x7ffe21aba270 state->status==57 zalloc==0x7f8e20045090 zfree==0x7f8e200450a0 (s->strm==strm)==1 s->next_out==0x55d36495bb10 s->avail_in=800 s->next_in=0x55d363d76b80 deflates()'s checks would be ok here
deflate returned 1 with stream->total_out==50; Z_FINISH==4, Z_OK==0, Z_STREAM_END==1 z_stream: a0 6e d7 63 d3 55 00 00 00 00 00 00 00 00 00 00 20 03 00 00 00 00 00 00 42 bb 95 64 d3 55 00 00 e7 09 80 00 00 00 00 00 32 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 41 5f d3 55 00 00 90 50 04 20 8e 7f 00 00 a0 50 04 20 8e 7f 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 f8 f0 0e c1 00 00 00 00 00 00 00 00 00 00 00 00 state: 70 a2 ab 21 fe 7f 00 00 9a 02 00 00 strm==0x7ffe21aba270 state->strm==0x7ffe21aba270 state->status==666 zalloc==0x7f8e20045090 zfree==0x7f8e200450a0 (s->strm==strm)==1 s->next_out==0x55d36495bb42 s->avail_in=0 s->next_in=0x55d363d76ea0 deflates()'s checks would be ok here
z_stream for data (3): a0 6e d7 63 d3 55 00 00 00 00 00 00 00 00 00 00 20 03 00 00 00 00 00 00 42 bb 95 64 d3 55 00 00 e7 09 80 00 00 00 00 00 32 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 41 5f d3 55 00 00 90 50 04 20 8e 7f 00 00 a0 50 04 20 8e 7f 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 f8 f0 0e c1 00 00 00 00 00 00 00 00 00 00 00 00 state: 70 a2 ab 21 fe 7f 00 00 9a 02 00 00 strm==0x7ffe21aba270 state->strm==0x7ffe21aba270 state->status==666 zalloc==0x7f8e20045090 zfree==0x7f8e200450a0 (s->strm==strm)==1 s->next_out==0x55d36495bb42 s->avail_in=0 s->next_in=0x55d363d76ea0 deflates()'s checks would be ok here
> 

@mattdowle mattdowle added this to the 1.13.3 milestone Dec 1, 2020
@codecov
Copy link

codecov bot commented Dec 1, 2020

Codecov Report

Merging #4826 (5b8f25b) into master (8480b6a) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #4826   +/-   ##
=======================================
  Coverage   99.47%   99.47%           
=======================================
  Files          73       73           
  Lines       14565    14579   +14     
=======================================
+ Hits        14489    14503   +14     
  Misses         76       76           
Impacted Files Coverage Δ
src/init.c 100.00% <ø> (ø)
R/test.data.table.R 100.00% <100.00%> (ø)
src/fwrite.c 97.79% <100.00%> (+0.04%) ⬆️
src/utils.c 98.36% <100.00%> (+0.02%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8480b6a...3f5920b. Read the comment docs.

@mattdowle
Copy link
Member Author

Travis CI seems stuck in queuing state. Proceeding as GLCI on will catch anything after merge.

@mattdowle mattdowle merged commit 958b693 into master Dec 1, 2020
@mattdowle mattdowle deleted the solaris_zlib branch December 1, 2020 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant