Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High memory traffic because of boxed SqlGuid struct #2300

Open
wilbit opened this issue Jan 17, 2024 · 9 comments · Fixed by #2306
Open

High memory traffic because of boxed SqlGuid struct #2300

wilbit opened this issue Jan 17, 2024 · 9 comments · Fixed by #2306
Labels
🙌 Up-for-Grabs Anyone interested in working on it can ask to be assigned

Comments

@wilbit
Copy link
Contributor

wilbit commented Jan 17, 2024

Is your feature request related to a problem? Please describe.

When our application is starting it loads a lot of data from DB and it causes memory consumption growth and high pressure to GC because of big memory traffic.
On the third place of killed objects is boxed System.Data.SqlTypes.SqlGuid struct.
These boxed values are 99.9% created when TdsParser is putting SqlGuid struct to SqlBuffer.SqlGuid property (see the image bellow).

image

Describe the solution you'd like

SqlBuffer already supports an efficient way to store Guid values. What I would like to do is:

  • change implementation of Microsoft.Data.SqlClient.SqlBuffer
    • remove SqlGuid value from StorageType enum (taking the second alternative approach, removing of SqlGuid value from enum looks like a bad idea);
    • save Guid representation of SqlGuid to _value._guid (similar to what Guid setter does, but with checking if SqlGuid value is null)
    • read SqlGuid value from _value._guid

Describe alternatives you've considered

  1. add a field of SqlGuid type to SqlBuffer, but it seems memory inefficient
  2. same as the main approach, but without removing SqlGuid value from StorageType enum, in case it is really important. The only place where it looks important to me is s_dbTypeToStorageType mapping array in Microsoft.Data.SqlClient.Server.ValueUtilsSmi, and I am not sure if it is safe to change the mapping to SqlBuffer.StorageType.Guid. It seems like it is not safe, because implementation of SqlBuffer.Guid does not support nulls.

Additional context

  • Microsoft.Data.SqlClient version is 5.1.0;
  • We use Microsoft.Data.SqlClient together with NHibernate;
  • Almost all our db tables contain Guid columns, so, memory traffic is really high in our case.
@wilbit wilbit changed the title High memory traffic because of boxing of SqlGuid High memory traffic because of boxed SqlGuid Jan 17, 2024
@wilbit
Copy link
Contributor Author

wilbit commented Jan 17, 2024

The changes are simple. So, I can do it if/when the issue is approved.

@wilbit wilbit changed the title High memory traffic because of boxed SqlGuid High memory traffic because of boxed SqlGuid struct Jan 17, 2024
@kf-gonzalez kf-gonzalez added 🙌 Up-for-Grabs Anyone interested in working on it can ask to be assigned and removed untriaged labels Jan 17, 2024
@kf-gonzalez
Copy link

@wilbit All contributors are welcomed to create PRs; we will review them and address them

@Wraith2
Copy link
Contributor

Wraith2 commented Jan 17, 2024

That's a bit strange. If you're calling IsDBNull it shouldn't be forcing the sql type it should be using the guid type and only creating the sql type if you request it. Can you provide a small repro?

@wilbit
Copy link
Contributor Author

wilbit commented Jan 17, 2024

@Wraith2 , have you seen the stack trace on the image and code of SqlDataReader.TryReadColumnInternal(i: int_value, readHeaderOnly: true, forStreaming: false), TdsParser.GetNullSqlValue and SqlBuffer.set_SqlGuid?
I think it is a straight forward from the code base, especially from SqlBuffer.set_SqlGuid.

@kf-gonzalez I will prepare PR tomorrow.

@Wraith2
Copy link
Contributor

Wraith2 commented Jan 17, 2024

I saw the trace. I just remember specifically optimizing guid in the past so that it doesn't allocate. It looks like this specific case slipped past me.

Looking at the code I think we should probably be calling SqlBuffer.SetToNullOfType(SqlBuffer.StorageType.SqlGuid) instead of using the SqlBuffer.set_SqlGuid property setter, that's going to box the SqlGuid.Null value every time we call because it's assigning the struct to the object member.
Can you try changing the two places in the codebase that call SqlBuffer.set_SqlGuid with SqlGuid.Null to use the setter and see if it works?

@wilbit
Copy link
Contributor Author

wilbit commented Jan 17, 2024

I will have a look, @Wraith2

wilbit added a commit to Intrigma/SqlClient that referenced this issue Jan 18, 2024
wilbit added a commit to Intrigma/SqlClient that referenced this issue Jan 18, 2024
@wilbit
Copy link
Contributor Author

wilbit commented Jan 18, 2024

Here is the PR: #2306, it removed boxing of SqlGuid 100%.

PS: I have also noticed, that SqlBinary has a very similar issue for SqlBinary.Null and not null values (it just affects us ~100 times less). I have a couple thoughts how to avoid boxing there too.

@Wraith2
Copy link
Contributor

Wraith2 commented Jan 18, 2024

I noticed the SqlBinary thing too. I think it's more complex than SqlGuid but if you're willing to dive in and improve things it'll be appreciated.

wilbit added a commit to Intrigma/SqlClient that referenced this issue Jan 21, 2024
wilbit added a commit to Intrigma/SqlClient that referenced this issue Jan 21, 2024
@wilbit
Copy link
Contributor Author

wilbit commented Jan 26, 2024

This is not fixed yet, because PR #2306 was reverted (#2311)

@David-Engel David-Engel reopened this Jan 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🙌 Up-for-Grabs Anyone interested in working on it can ask to be assigned
Projects
Status: Under Investigation
Development

Successfully merging a pull request may close this issue.

5 participants