Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] Fix Variant::construct of Object #90134

Merged
merged 1 commit into from
Jul 26, 2024

Conversation

AThousandShips
Copy link
Member

@AThousandShips AThousandShips commented Apr 2, 2024

Variant type was not updated correctly causing leaks in ref-counted

This caused errors as objects constructed this way would be typed NIL and therefore not be freed, even though they went through reference counting, might have caused further issues elsewhere, like extensions, but haven't tested, this should be a safe fix as it just clears and initializes the type

@AThousandShips AThousandShips added bug topic:core cherrypick:4.1 Considered for cherry-picking into a future 4.1.x release cherrypick:4.2 Considered for cherry-picking into a future 4.2.x release labels Apr 2, 2024
@AThousandShips AThousandShips added this to the 4.3 milestone Apr 2, 2024
@AThousandShips AThousandShips requested a review from a team as a code owner April 2, 2024 13:57
@@ -153,11 +153,12 @@ class VariantConstructor {
class VariantConstructorObject {
public:
static void construct(Variant &r_ret, const Variant **p_args, Callable::CallError &r_error) {
VariantInternal::clear(&r_ret);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this below matches the rest of the cases, where errors mean we don't touch it, can move it back above if desired, but take VariantConstructorNilObject for example

@AThousandShips
Copy link
Member Author

The reason it breaks GDScript in this case is that this code passes fine, as value is NIL, which is valid for that check, a NIL variant is a valid type for an object here, now it will correctly error, as the resulting variant is now a valid OBJECT so it will go through the checks and correctly determine the type is wrong:

if (err.error != Callable::CallError::CALL_OK || !member->data_type.is_type(value)) {

@AThousandShips

This comment was marked as outdated.

@AThousandShips
Copy link
Member Author

AThousandShips commented Apr 2, 2024

There's still a strange leak when using set directly, but unsure what is causing that, it would be in some part of the code unrelated to this, will dig further but seems a different case

Edit: Got a fix for this, part too, but will look at some other potential areas as well

Edit 2: Realized why, will improve

This issue was caused by that Variant::construct didn't clear properly with just change as it doesn't do anything if the types match, so when Variant value in GDScriptInstance::set was already assigned with p_value the change operation did nothing, so value was never cleared, and therefore leaked

@AThousandShips
Copy link
Member Author

AThousandShips commented Apr 2, 2024

There, new fix, this is perhaps not exactly the cleanest, but I prefer to use the methods available there instead of assigning directly to the type, an alterative solution would be to instead (diff on top of this PR):

diff --git a/core/variant/variant_construct.h b/core/variant/variant_construct.h
index 40e0d37360..b5cd7ff1e7 100644
--- a/core/variant/variant_construct.h
+++ b/core/variant/variant_construct.h
@@ -154,12 +154,10 @@ class VariantConstructorObject {
 public:
        static void construct(Variant &r_ret, const Variant **p_args, Callable::CallError &r_error) {
                if (p_args[0]->get_type() == Variant::NIL) {
-                       VariantInternal::clear(&r_ret);
                        VariantTypeChanger<Object *>::change(&r_ret);
                        VariantInternal::object_assign_null(&r_ret);
                        r_error.error = Callable::CallError::CALL_OK;
                } else if (p_args[0]->get_type() == Variant::OBJECT) {
-                       VariantInternal::clear(&r_ret);
                        VariantTypeChanger<Object *>::change(&r_ret);
                        VariantInternal::object_assign(&r_ret, p_args[0]);
                        r_error.error = Callable::CallError::CALL_OK;
@@ -171,7 +169,6 @@ public:
        }

        static inline void validated_construct(Variant *r_ret, const Variant **p_args) {
-               VariantInternal::clear(r_ret);
                VariantTypeChanger<Object *>::change(r_ret);
                VariantInternal::object_assign(r_ret, p_args[0]);
        }
@@ -201,13 +198,11 @@ public:
                        r_error.expected = Variant::NIL;
                }

-               VariantInternal::clear(&r_ret);
                VariantTypeChanger<Object *>::change(&r_ret);
                VariantInternal::object_assign_null(&r_ret);
        }

        static inline void validated_construct(Variant *r_ret, const Variant **p_args) {
-               VariantInternal::clear(r_ret);
                VariantTypeChanger<Object *>::change(r_ret);
                VariantInternal::object_assign_null(r_ret);
        }
diff --git a/core/variant/variant_internal.h b/core/variant/variant_internal.h
index dbd4a6a7ad..d867d08b2c 100644
--- a/core/variant/variant_internal.h
+++ b/core/variant/variant_internal.h
@@ -1493,13 +1493,13 @@ struct VariantDefaultInitializer<PackedColorArray> {
 template <typename T>
 struct VariantTypeChanger {
        static _FORCE_INLINE_ void change(Variant *v) {
-               if (v->get_type() != GetTypeInfo<T>::VARIANT_TYPE || GetTypeInfo<T>::VARIANT_TYPE >= Variant::PACKED_BYTE_ARRAY) { //second condition removed by optimizer
+               if (v->get_type() != GetTypeInfo<T>::VARIANT_TYPE || GetTypeInfo<T>::VARIANT_TYPE == Variant::OBJECT || GetTypeInfo<T>::VARIANT_TYPE >= Variant::PACKED_BYTE_ARRAY) { //second condition removed by optimizer
                        VariantInternal::clear(v);
                        VariantInitializer<T>::init(v);
                }
        }
        static _FORCE_INLINE_ void change_and_reset(Variant *v) {
-               if (v->get_type() != GetTypeInfo<T>::VARIANT_TYPE || GetTypeInfo<T>::VARIANT_TYPE >= Variant::PACKED_BYTE_ARRAY) { //second condition removed by optimizer
+               if (v->get_type() != GetTypeInfo<T>::VARIANT_TYPE || GetTypeInfo<T>::VARIANT_TYPE == Variant::OBJECT || GetTypeInfo<T>::VARIANT_TYPE >= Variant::PACKED_BYTE_ARRAY) { //second condition removed by optimizer
                        VariantInternal::clear(v);
                        VariantInitializer<T>::init(v);
                }

And just use the change method without clear like the other cases use, this would be slightly more invasive, but accomplish the same goal, either way works for me (though the current PR changes are cheaper in rebuilds)

Will see if I can add a test for leaks as well

Edit: Added a test for the leak, will push when CI is done

Variant type was not updated correctly causing leaks in ref-counted
@AThousandShips AThousandShips added cherrypick:4.3 Considered for cherry-picking into a future 4.3.x release and removed cherrypick:4.1 Considered for cherry-picking into a future 4.1.x release labels Jul 25, 2024
@AThousandShips AThousandShips modified the milestones: 4.3, 4.4 Jul 25, 2024
Copy link
Member

@vnen vnen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Should be safe to merge too.

@AThousandShips AThousandShips removed the cherrypick:4.3 Considered for cherry-picking into a future 4.3.x release label Jul 25, 2024
@AThousandShips AThousandShips modified the milestones: 4.4, 4.3 Jul 25, 2024
@akien-mga akien-mga merged commit 7805220 into godotengine:master Jul 26, 2024
18 checks passed
@akien-mga
Copy link
Member

Thanks!

@AThousandShips AThousandShips deleted the construct_fix branch July 26, 2024 11:52
@AThousandShips
Copy link
Member Author

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug cherrypick:4.2 Considered for cherry-picking into a future 4.2.x release topic:core
Projects
None yet
4 participants