... as this can lead to a deadlock with the following sequence:
Time1: guest-thread-1: vkDestroyBuffer() called
Time2: VkEncoder grabs seqno 1
Time3: guest-thread-2: vkMapMemory() called
Time4: ResourceTracker::on_vkMapMemory() locks mLock
for using `info_VkDeviceMemory`
Time5: ResourceTracker::on_vkMapMemory() calls
enc->vkGetBlobGOOGLE()
Time6: VkEncoder grabs seqno 2
Time7: VkEncoder sends the vkGetBlobGOOGLE with seqno
2 via ASG to host
Time8: VkEncoder waits for the `VkResult` from the
host via `stream->read()`
Time9: guest-thread-1: VkEncoder calls sResourceTracker->destroyMapping()
->mapHandles_VkBuffer((VkBuffer*)&buffer);
which calls ResourceTracker::unregister_VkBuffer()
ResourceTracker::unregister_VkBuffer() tries to
locks mLock to erase the buffer's info struct
from `info_VkBuffer`
!!! DEADLOCKED HERE !!!
guest-thread-1 is stuck waiting on mLock (currently locked by
guest-thread-2) before it would `stream->flush();` to finishing
sending the vkDestroyBuffer() command to the host and potentially
ping its corresponding host-render-thread-1.
guest-thread-2 stuck waiting on the result from host-render-thread-2
but host-render-thread-2 won't progress until host-render-thread-1
finishes seqno 1 which needs guest-thread-1 to finish sending/pinging.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
This reverts commit 9eef6d0aefcf0aa1c07d42d9b307b1092a6deec9.
... as this does not yet have a way to generically convert Mesa
handles into Gfxstream handles in extension structs which causes
breakage in dEQP VK for many tests.
Moves the mesa based codegen to a separate `mesa_func_table`.
Reland fixes the end2end test guest vulkan ICD.
cts -m CtsDeqpTestCases
--module-arg CtsDeqpTestCases:include-filter:dEQP-VK.*
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
We have a situation that if an image memory is created with
VkMemoryDedicatedAllocateInfo, the following commands that refer to the
same image must happen in this exact order:
vkCreateImage
vkAllocateMemory (with dedicated image memory)
vkBindImageMemory
vkCreateImageView
Our previous dependency graph implementation uses Vulkan objects as
nodes and does not haven enough granularity to handle this situation.
vkCreateImageView can only be executed after an image is created and
bound to memory. Our previous dependency graph does not have the
notion of "bound to memory". We worked around it by adding
vkBindImageMemory as part of the image initialization steps.
To make sure we can bind image at initialization, we marked image to be
dependent on the memory it bounds to. When adding the new
vkAllocateMemory with dedicated image memory extension into the mix, it
introduced a circular dependency. vkAllocateMemory will say the memory
has dependency on the image. vkBindImageMemory will say the image has
dependency on the memory.
To properly resolve the issue, we will need to record the state of an
Vulkan object, and declare that vkCreateImageView not only depends on an
image, but also requires the image to be bound to memory. We do so by
introducing a "State" enum and use the pair <VkObjectHandle, State> as
the vertex in the deppendency graph. Currently State is an enum with 2
values: CREATED and BOUND_MEMORY. By default, after a VkCreate* command,
an object will be in CREATED state. When calling vkBindImageMemory or
vkBindBufferMemory it will transform the image or buffer into
BOUND_MEMORY state. Then vkCreateImageView will have dependency with
{VkImage, BOUND_MEMORY}.
Then we can create a dependency graph that looks like this:
VkImageView --> {VkImage,BOUND_MEMORY} --> VkMemory
| |
| ┌-------------------⅃
v v
VkImage
No more circular dependency.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
... as this does not yet have a way to generically convert Mesa
handles into Gfxstream handles in extension structs which causes
breakage in dEQP VK for many tests.
Moves the mesa based codegen to a separate `mesa_func_table`.
cts -m CtsDeqpTestCases
--module-arg CtsDeqpTestCases:include-filter:dEQP-VK.*
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
... as this is more generic and "VirglBlob" is not actually a blob.
find . -type f -name "*.cpp" -print0 | \
xargs -0 sed -i '' -e 's/VirtGpuBlob/VirtGpuResource/g'
find . -type f -name "*.h" -print0 | \
xargs -0 sed -i '' -e 's/VirtGpuBlob/VirtGpuResource/g'
find . -type f -name "*.cpp" -print0 | \
xargs -0 sed -i '' -e 's/createVirglBlob/createResource/g'
find . -type f -name "*.h" -print0 | \
xargs -0 sed -i '' -e 's/createVirglBlob/createResource/g'
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
There are 2 issues with the current implementation of VkReconstruct:
a. When deleting a child handle it does not erase itself from its
parent. When doing snapshot we enumerate children handles and check
if they are still "alive". But the handle pool is written in the way
that it could reuse handles that have been previously destroyed,
invalidating the "alive" check.
b. Previous dependency graph does not support one handle with multiple
parents.
In this commit we fix (a) by making child<->parent pointer
bidirectional. When deleting a child it will be erased from its parents.
We fix (b) by rewriting a significant part of dependency graph logic on
snapshot save.
The commit also makes all destroy command to be recursive (except for
vkDestroyShaderModule), which will be handled in later commits.
Also add dependency VkFrameBuffer -> VkImageView.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
We are trying to make vk snapshot load the google Chrome home page
without crashing. With this CL it does not crash during load but would
crash with a device lost after submitting a queue (which is
unsurprising because everything is missing to snapshot vk queue).
Content of this commit includes:
- Add dependencies for VkImageView, VkGraphicsPipelines, VkFramebuffer.
They are necessary to tell the snapshot module the order of vk object
creation.
- Add vkBindImageMemory into the loading sequencey of VkImage, so that
it would be executed before vkCreateImageView.
- Delay the destruction of VkShaderModule. This is because other
objects can still refer to it after it is destroyed.
- Initialize VK backend for color buffer properly on snapshot load.
- Save and load vk images in the same order by sorting them according
to their boxed handle.
- Record all the placeholder handles for vkCreateDescriptorPool. For
performance purpose this function creates a lot of extra handles
without real contents. We need to snapshot those handles as well.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
This builds the "standalone" GFXStream library that can be consumed by
Qemu 8.2
The bazel build allows us to transition rutabaga to a bazel based
build, which will unify the Qemu 8.2 build as used by the emulator.
It also opens up the path to migrate this to g3 if we decide to do this.
This change forces a few header changes, as bazel is more strict about
the relationship between headers and packages, and doesn't easily give
you fine grained control over the include paths.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
An user is now able to do:
export VK_ICD_FILENAMES=./amd64-build/guest/vulkan/gfxstream_vk_devenv_icd.x86_64.json
that allows vulkan apps without install the ICD.
If built with option `-Dgfxstream-build=guest-test`, that would go
into the gfxstream VK ICD and the test layer. Unfortunately,
the test layer hangs right now since I believe the gfxstream_backend's
vulkan loading logic is also affected by the environment variable.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
Sometimes guest renders to an AHB without calling
vkQueueSignalReleaseImageANDROIDAsyncGOOGLE. This would result in the
color buffer not being updated from Vulkan.
This commit tracks the situation that AHB is rendered to, and copies its
content to color buffer.
Note that it adds extra wait, which could hurt performance.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
... so that "guest gralloc", "guest egl", "guest vulkan", etc are
all using the same underlying emulation layer (RutabagaLayer).
This moves Gralloc and ANativeWindow into platform as these should
be hidden from GfxstreamEnd2EndTests.
Note: we still want to static-ify a lot of the guest libraries.
meson setup \
-Ddefault_library=static \
-Dgfxstream-build=both \
build
meson compile -C build
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
When allocating coherent memory (generally, suballocations) and looking
up existing VkDeviceMemory entries, the device is critical to finding
valid allocation blocks, otherwise allocations from other devices might
be chosen, which invalidate things.
This happens only in situations where a single process has multiple
VkDevices, so it was rare enough not to be caught until now.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
This commit snapshots vk image content by allocating a staging buffer
and copying the bytes on snapshot. It only works in the simplest setup.
Many situations are not considered in this commit, they include:
(1) the image does not support VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL
layout;
(2) the image does not support VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL
layout;
(3) the queue is dirty.
Also there is no performance optimization.
Implementation-wise, snapshot happens in VkDecoderGlobalState after
recording / playing back all create / bind commands. It borrows an
existing queue to run the extra vk copy commands. A temporary staging
buffer is also created for copying. Later we could optimize the code
by reusing most of the temporary objects.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
This commit tracks more functions related to coherent memory snapshot
and saves the memory content.
At this point we are not sure if we need to manually copy the memory
content, because they are supposed to be cloned in RAM snapshot. We
could revert that part if they turn out to be unnecessary in future.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
Similiar solution to aosp/2925036, but perhaps simpler.
- Creates the rcEncoder(..) when a new thread-local encoder
is initialized.
- keeps SetupInstanceForProcess improvement
- Always use kCapsetGfxStreamVulkan. This should make no
difference, since goldfish doesn't advertise
kCapsetGfxstreamVulkan and HostConnection::get(..) defaults
to kCapsetGfxstreamVulkan anyways to get ASG ring parameters.
- Move additional static global variables before functions
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
This makes the checks into a list. It also adds support for
uint16_t, which are hit with newer versions of vk.xml.
I'm not sure exactly why we need the list, only that codegen
errors occur if we don't don't generate a type here.
Maybe as we try to upstream the cerealgenerator, we can figure
out why and fix it.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
This change splits SetupInstance on the gfxstream vulkan layer into
SetupInstanceForProcess and SetupInstanceForThread, then forces
SetupInstanceForThread if the HostConnection for the current thread
hasn't been initialized when getConnection is called. Currently, this
will create an rcEncoder and include the PUID, which - when missing -
causes the host to get very confused.
This path is triggered by ANGLE, where a separate thread runs through
vulkan initialization than which ultimately uses the vulkan
components. In this case, none of the vulkan initialization code will
have independently called SetupInstanceForProcess on the current thread.
The semantics of SetupInstanceForProcess and SetupInstanceForThread are
somewhat muddled, because the sequence number pointer is a per-process
concept, but only set when initializing the rcEncoder per thread.
Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
