🐛 Problem with multiple language per file support

[Sigmanificient]

Hi, when using tokei, file that include multiple language are not counted in the total.
I'm not sure if this is a bug or it has been intended that way.

Here is a vue project I work on, only 124 lines are counted for the vue part, instead of the 1969 ones.
This is the same for markdown, actually,

25 + 529 + 389 + 56 + 33 + 206 + 124 = 1362
Expected result (line section): 3209

===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 HTML                    2           25           23            1            1
 JavaScript             10          529          508            0           21
 PHP                     8          389          221           98           70
 Shell                   4           56           41            1           14
 SVG                     7           33           33            0            0
-------------------------------------------------------------------------------
 Markdown                3          206            0          160           46
 |- INI                  1            2            0            2            0
 (Total)                            208            0          162           46
-------------------------------------------------------------------------------
 Vue                    16          124           94            0           30
 |- CSS                 15          822          699            0          123
 |- HTML                16          329          307            0           22
 |- JavaScript          16          694          636            1           57
 (Total)                           1969         1736            1          232
===============================================================================
 Total                  50         1362          920          260          182
===============================================================================

[CameronNemo]

See also my experience with Svelte:

======================================================================================
 Language                   Files        Lines         Code     Comments       Blanks
======================================================================================
 Markdown                       1            1            0            1            0
--------------------------------------------------------------------------------------
 ./NOTES.md                                  1            0            1            0
--------------------------------------------------------------------------------------
 Svelte                         9          741          728            0           13
 |- JavaScript                  9         1507         1294          102          111
 (Total)                                  2248         2022          102          124
--------------------------------------------------------------------------------------
 ./ServiceExplanation.svelte
 |- Svelte                                  25           24            0            1
 |- JavaScript                               2            2            0            0
 ./ServiceExplanation.svelte                25           24            0            1
-- ./CamelOosTable.svelte ------------------------------------------------------------
 |- Svelte                                  65           63            0            2
 |- JavaScript                              71           57            6            8
 ./CamelOosTable.svelte                     65           63            0            2
-- ./CamelCircuitHistoryTable.svelte -------------------------------------------------
 |- Svelte                                  80           79            0            1
 |- JavaScript                              25           20            0            5
 ./CamelCircuitHistoryTable.svelte           80           79            0            1
-- ./EventImpactedServices.svelte ----------------------------------------------------
 |- Svelte                                  14           13            0            1
 |- JavaScript                               4            2            0            2
 ./EventImpactedServices.svelte             14           13            0            1
-- ./CamelAlarmHistoryTable.svelte ---------------------------------------------------
 |- Svelte                                  73           71            0            2
 |- JavaScript                             101           88            4            9
 ./CamelAlarmHistoryTable.svelte            73           71            0            2
-- ./SendAnnouncementForm.svelte -----------------------------------------------------
 |- Svelte                                  37           36            0            1
 |- JavaScript                              78           72            0            6
 ./SendAnnouncementForm.svelte              37           36            0            1
-- ./SciloEventTable.svelte ----------------------------------------------------------
 |- Svelte                                 114          112            0            2
 |- JavaScript                             245          199           23           23
 ./SciloEventTable.svelte                  114          112            0            2
-- ./GenerateTemplateForm.svelte -----------------------------------------------------
 |- Svelte                                 138          137            0            1
 |- JavaScript                             376          302           40           34
 ./GenerateTemplateForm.svelte             138          137            0            1
-- ./ManualComponentSelection.svelte -------------------------------------------------
 |- Svelte                                 195          193            0            2
 |- JavaScript                             605          552           29           24
 ./ManualComponentSelection.svelte          195          193            0            2
======================================================================================
 Total                         10          742          728            1           13
======================================================================================

[XAMPPRocky]

Thank you for your issue! Can you please check that this is also the case with when building from source? There have been some fixes but theres no release yet.

[CameronNemo]

Results when going from master:

a compile warning:

warning: package `crossbeam-channel v0.5.2` in Cargo.lock is yanked in registry `crates-io`, consider running without --locked
warning: package `crossbeam-utils v0.8.6` in Cargo.lock is yanked in registry `crates-io`, consider running without --locked
warning: package `dashmap v5.0.0` in Cargo.lock is yanked in registry `crates-io`, consider running without --locked

and this (seemingly correct) output:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Language            Files        Lines         Code     Comments       Blanks
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 JavaScript            104        15397        14203          485          709
 Markdown                1           17            0           16            1
───────────────────────────────────────────────────────────────────────────────
 Svelte                389        27450        26454           10          986
 |- JavaScript         389        42687        29592         8842         4253
 (Total)                          70137        56046         8852         5239
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Total                 494        85551        70249         9353         5949
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

[CameronNemo]

@XAMPPRocky are you planning a release soon? Seems like if there are fixes to release, might as well mark a patch release before doing major refactors.

[XAMPPRocky]

@CameronNemo See #911, there is no ETA for a new release. There are already breaking changes in main that prevent patch releases (tokei is a library as well as CLI tool), and I don't have the bandwidth to backport fixes. If you'd like a release to happen sooner, sponsoring me is best and most direct way to do that, as it enables to spend more time on projects like Tokei that I'm not being directly paid to maintain.