Incorrect code line count caused by ChildLanguage

[LeoQ7]

const demo: &str = "Hello, World!";

/// markdown? docstring?

Output:

> tokei poc.rs
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Rust                    1            0            0            0            0
 |- Markdown             1            1            0            1            0
 (Total)                              1            0            1            0
===============================================================================
 Total                   1            0            0            0            0
===============================================================================

Trace:

[2024-03-06T18:24:01Z TRACE tokei::language::language_type] const demo: &str = "Hello, World!";
[2024-03-06T18:24:01Z TRACE tokei::language::syntax] Start "\""
[2024-03-06T18:24:01Z TRACE tokei::language::syntax] End "\""
[2024-03-06T18:24:01Z TRACE tokei::language::syntax]

[2024-03-06T18:24:01Z TRACE tokei::language::syntax]

[2024-03-06T18:24:01Z TRACE tokei::language::syntax] /// markdown? docstring?
[2024-03-06T18:24:01Z TRACE tokei::language::syntax] Markdown found: " markdown? docstring?"
[2024-03-06T18:24:01Z TRACE tokei::language::language_type] markdown? docstring?
[2024-03-06T18:24:01Z TRACE tokei::language::syntax] ^ Skippable
[2024-03-06T18:24:01Z TRACE tokei::language::syntax] Comment No.1

Expected output:

> tokei poc.rs
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Rust                    1            3            1            1            1
 (Total)                              3            1            1            1
===============================================================================
 Total                   1            3            1            1            1
===============================================================================

[XAMPPRocky]

Thank you for your issue! However tokei is correct here. All doc comments in Rust are markdown and this code is invalid Rust because it needs to be attached to an item.

[LeoQ7]

Thank you for the clarification!

For the following code snippet, is the result from tokei also correct?

const demo: &str = "Hello, World!";

/// markdown? docstring?
const demo1: &str = "Hello, World!";

Output:

[2024-03-07T04:45:50Z TRACE tokei::language::language_type] const demo: &str = "Hello, World!";
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] Start "\""
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] End "\""
[2024-03-07T04:45:50Z TRACE tokei::language::syntax]

[2024-03-07T04:45:50Z TRACE tokei::language::syntax]

[2024-03-07T04:45:50Z TRACE tokei::language::syntax] /// markdown? docstring?

[2024-03-07T04:45:50Z TRACE tokei::language::syntax] Markdown found: " markdown? docstring?\n"
[2024-03-07T04:45:50Z TRACE tokei::language::language_type] markdown? docstring?
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] ^ Skippable
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] Comment No.1
[2024-03-07T04:45:50Z TRACE tokei::language::language_type] const demo1: &str = "Hello, World!";
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] Start "\""
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] End "\""
[2024-03-07T04:45:50Z TRACE tokei::language::language_type] const demo1: &str = "Hello, World!";
[2024-03-07T04:45:50Z TRACE tokei::language::language_type] Code No.1
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Rust                    1            1            1            0            0
 |- Markdown             1            1            0            1            0
 (Total)                              2            1            1            0
===============================================================================
 Total                   1            1            1            0            0
===============================================================================

IMHO, LoC should be 2 here?

A correct example is:

const demo: u8 = 42;

/// markdown? docstring?
const demo1: &str = "Hello, World!";

Output:

[2024-03-07T06:18:40Z TRACE tokei::language::language_type] Using Simple Parse on "const demo: u8 = 42;\n\n"
[2024-03-07T06:18:40Z TRACE tokei::language::language_type] /// markdown? docstring?
[2024-03-07T06:18:40Z TRACE tokei::language::syntax] /// markdown? docstring?

[2024-03-07T06:18:40Z TRACE tokei::language::syntax] Markdown found: " markdown? docstring?\n"
[2024-03-07T06:18:40Z TRACE tokei::language::language_type] markdown? docstring?
[2024-03-07T06:18:40Z TRACE tokei::language::syntax] ^ Skippable
[2024-03-07T06:18:40Z TRACE tokei::language::syntax] Comment No.1
[2024-03-07T06:18:40Z TRACE tokei::language::language_type] const demo1: &str = "Hello, World!";
[2024-03-07T06:18:40Z TRACE tokei::language::syntax] Start "\""
[2024-03-07T06:18:40Z TRACE tokei::language::syntax] End "\""
[2024-03-07T06:18:40Z TRACE tokei::language::language_type] const demo1: &str = "Hello, World!";
[2024-03-07T06:18:40Z TRACE tokei::language::language_type] Code No.1
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Rust                    1            3            2            0            1
 |- Markdown             1            1            0            1            0
 (Total)                              4            2            1            1
===============================================================================
 Total                   1            3            2            0            1
===============================================================================

[LeoQ7]

A similar issue is #713 , my guess is that the early return at https://github.com/XAMPPRocky/tokei/blob/master/src/language/syntax.rs#L250-L252 somehow affected the counting of previous lines when there is a quote.

[SorteKanin]

I also have seen the comment code count be kind of strange. On a project I have, I get this output when running tokei:

===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Dockerfile              2           46           22           12           12
 Markdown                7          267            0          190           77
 Shell                   4          527          383           70           74
 SQL                     4          747          357          214          176
 TOML                    1           92           82            4            6
 YAML                    1          148          114           21           13
-------------------------------------------------------------------------------
 Rust                   55         5676         4417          339          920
 |- Markdown            55          739            6          642           91
 (Total)                           6415         4423          981         1011
===============================================================================
 Total                  74         7503         5375          850         1278
===============================================================================

It seems like it is not counting the markdown inside Rust as comments, but I feel like it should?