Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

non UTF8 encoded string in enumerate_instance_extension_properties function #830

Closed
GummyGun opened this issue Nov 24, 2023 · 7 comments
Closed

Comments

@GummyGun
Copy link

There is a bug in the method enumerate_instance_extension_properties() where sometimes the last character in the field ExtensionProperties.extension_name is not properly encoded and may hold non-valid utf8 char.

@GummyGun
Copy link
Author

this bug to my findings is only present in the extension VK_EXT_direct_mode_display

@MarijnS95
Copy link
Collaborator

Can you share the u8 data that this array contains, i.e. the way that you ran into this issue?

@MarijnS95
Copy link
Collaborator

MarijnS95 commented Nov 24, 2023

After checking the ICD on my ArchLinux machine with the following snippet:

            for p in entry.enumerate_instance_extension_properties(None).unwrap() {
                let n = ::std::ffi::CStr::from_ptr(p.extension_name.as_ptr());

                if n.to_string_lossy() == "VK_EXT_direct_mode_display" {
                    dbg!(p.extension_name);
                }
            }

I get a sensible result out of the array:

[ash-examples/src/lib.rs:229] p.extension_name = [
    86,
    75,
    95,
    69,
    88,
    84,
    95,
    100,
    105,
    114,
    101,
    99,
    116,
    95,
    109,
    111,
    100,
    101,
    95,
    100,
    105,
    115,
    112,
    108,
    97,
    121,
    0,
    ...,
    0,
]

Note that Ash initially allocates uninitialized memory for these structs, and it's up to the ICD to optionally zero out the whole string buffer after the first NUL character. However, that might even be optional given the description at https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkExtensionProperties.html:

extensionName is an array of VK_MAX_EXTENSION_NAME_SIZE char containing a null-terminated UTF-8 string which is the name of the extension.

Typically (as far as I'm aware) implementations should stop reading data after the first NUL byte/character.

Are you sure that your test case isn't accidentally reading beyond the NUL character?

EDIT: For completeness #746 and its use of CStr::from_bytes_until_nul() might be useful to you. I'm willing to convert the debug APIs to it - and maybe even autogenerate CStr getters on structs utilizing it - once our MSRV allows for it.
You can use it in client code already though.

@GummyGun
Copy link
Author

I should use the CStr since it saves me a unsafe block and over all is a better practice and I'm pretty sure CStr would solve the problem. Either way I think it is worth fixing the this awkward behavior.

let extension_list = entry.enumerate_instance_extension_properties(None).unwrap();
 
for extension in &extension_list {
    println!("{:?}", extension);
    println!("{:?}", extension.extension_name);
    if extension.extension_name[255] != 0 {
        panic!("found nonValid utf8");
    }
}

The output of the code above:

ExtensionProperties { extension_name: "VK_EXT_direct_mode_display", spec_version: 1 }
[86, 75, 95, 69, 88, 84, 95, 100, 105, 114, 101, 99, 116, 95, 109, 111, 100, 101, 95, 100, 105, 115, 112, 108, 97, 121, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58]

This error is not present every run but I find it is pops out about half of the runs I do

@GummyGun
Copy link
Author

I will run its equivalent interfacing directly to the vulkan in C to see if I get a different behavior

@MarijnS95
Copy link
Collaborator

MarijnS95 commented Nov 24, 2023

Either way I think it is worth fixing the this awkward behavior.

What do you want to "fix"? There's no weird behaviour here. The driver provides a valid UTF-8 string and a NUL terminator. There is no requirement to set any bytes to NUL after that, and it seems the driver/ICD is off by one byte when trying that here (or Rust's uninitialized vector allocation happened to do that).

If you're testing on C, don't forget to set the array to uninitialized bytes first, to match ash behaviour.

MarijnS95 added a commit that referenced this issue Nov 25, 2023
It is a common operation to read and write NUL-terminated C string in
the Vulkan API, yet the only helpers for that were thus far open-coded
in the `Debug` printing implementations.

Move them to separate functions that are exposed to the user, in hopes
of helping them no longer misunderstand NUL-terminated strings (see
e.g. #830).

Important to note is that the array-copy for a static-sized `c_char`
array has also been replaced with a `CStr` wrapper: this forces the user
and our implementation to have a NUL-terminated string, but may now also
panic if the incoming `CStr` with NUL-terminator is too large for the
target `c_char` array.
MarijnS95 added a commit that referenced this issue Nov 25, 2023
It is a common operation to read and write NUL-terminated C string in
the Vulkan API, yet the only helpers for that were thus far open-coded
in the `Debug` printing implementations.

Move them to separate functions that are exposed to the user, in hopes
of helping them no longer misunderstand NUL-terminated strings (see
e.g. #830).

Important to note is that the array-copy for a static-sized `c_char`
array has also been replaced with a `CStr` wrapper: this forces the user
and our implementation to have a NUL-terminated string, but may now also
panic if the incoming `CStr` with NUL-terminator is too large for the
target `c_char` array.

Finally, change the `::std` reference to `core::` to slowly help out with
the `no_std` conversion.
MarijnS95 added a commit that referenced this issue Nov 25, 2023
It is a common operation to read and write NUL-terminated C string in
the Vulkan API, yet the only helpers for that were thus far open-coded
in the `Debug` printing implementations.

Move them to separate functions that are exposed to the user, in hopes
of helping them no longer misunderstand NUL-terminated strings (see
e.g. #830).

Important to note is that the array-copy for a static-sized `c_char`
array has also been replaced with a `CStr` wrapper: this forces the user
and our implementation to have a NUL-terminated string, but may now also
panic if the incoming `CStr` with NUL-terminator is too large for the
target `c_char` array.
MarijnS95 added a commit that referenced this issue Nov 25, 2023
It is a common operation to read and write NUL-terminated C string in
the Vulkan API, yet the only helpers for that were thus far open-coded
in the `Debug` printing implementations.

Move them to separate functions that are exposed to the user, in hopes
of helping them no longer misunderstand NUL-terminated strings (see
e.g. #830).

Important to note is that the array-copy for a static-sized `c_char`
array has also been replaced with a `CStr` wrapper: this forces the user
and our implementation to have a NUL-terminated string, but may now also
panic if the incoming `CStr` with NUL-terminator is too large for the
target `c_char` array.
MarijnS95 added a commit that referenced this issue Nov 25, 2023
It is a common operation to read and write NUL-terminated C string in
the Vulkan API, yet the only helpers for that were thus far open-coded
in the `Debug` printing implementations.

Move them to separate functions that are exposed to the user, in hopes
of helping them no longer misunderstand NUL-terminated strings (see
e.g. #830).

Important to note is that the array-copy for a static-sized `c_char`
array has also been replaced with a `CStr` wrapper: this forces the user
and our implementation to have a NUL-terminated string, but may now also
panic if the incoming `CStr` with NUL-terminator is too large for the
target `c_char` array.
MarijnS95 added a commit that referenced this issue Nov 25, 2023
It is a common operation to read and write NUL-terminated C string in
the Vulkan API, yet the only helpers for that were thus far open-coded
in the `Debug` printing implementations.

Move them to separate functions that are exposed to the user, in hopes
of helping them no longer misunderstand NUL-terminated strings (see
e.g. #830).

Important to note is that the array-copy for a static-sized `c_char`
array has also been replaced with a `CStr` wrapper: this forces the user
and our implementation to have a NUL-terminator at the end of the string,
and the setter returns `Err()` when the given `CStr (with NUL-terminator)
is too large for the static-sized array it has to be written to.
@MarijnS95
Copy link
Collaborator

@GummyGun did you end up checking whether your driver only writes a single NUL-terminator, or does it zero the entire array minus the last item (by accident)?

I should use the CStr since it saves me a unsafe block and over all is a better practice and I'm pretty sure CStr would solve the problem

On that note we've added CStr getters and setters to structs, to make this easier in the future: #831

@MarijnS95 MarijnS95 closed this as not planned Won't fix, can't repro, duplicate, stale Nov 27, 2023
MarijnS95 added a commit that referenced this issue Nov 28, 2023
…831)

It is a common operation to read and write NUL-terminated C string in
the Vulkan API, yet the only helpers for that were thus far open-coded
in the `Debug` printing implementations.

Move them to separate functions that are exposed to the user, in hopes
of helping them no longer misunderstand NUL-terminated strings (see
e.g. #830).

Important to note is that the array-copy for a static-sized `c_char`
array has also been replaced with a `CStr` wrapper: this forces the user
and our implementation to have a NUL-terminator at the end of the string,
and the setter returns `Err()` when the given `CStr (with NUL-terminator)
is too large for the static-sized array it has to be written to.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants