[webkit-gtk] webextensions, iframes and main gui process

[Gaute Hope]

Hi,

I am using WebKit2 with webextensions (WE) for parts of my GTK gui. I am
in the process of porting from WebKit1 to WebKit2 and the new
webextensions design [0].

The main GUI process adds the webkit widget to my GUI, and initializes
the webextension. The main GUI process and the WE-process communicate
using UNIX domain sockets and Google protobuf. The necessary socket
address is passed on WE-initialization.

The WE builds the page using DOM manipulation on top of a skeleton page.
The data which the WE is using for building the page is passed from the
main GUI process. Both user-input (keyboard, mouse GTK events) on the
main GUI, as well as outside events (timer-based) can cause the page to
be changed. These are passed from the main GUI process as messages to
the WE-process. The state of the rendered page (scroll, visible
elements) is passed back to the main GUI and is necessary for the
processing of the events.

A reader-thread on the WE-extensions listens for messages from the main
GUI thread. Whenever the DOM tree is changed this is done on the main GTK
thread on the WE.

In order to make sure that no race-conditions happen between the events
(e.g. if the state is in the process of being changed on the page from
the WE) the main GUI process locks the output stream (to safeguard
against concurrent writes on the pipe) and more importantly needs to
wait for the WE-GUI thread to finish the DOM manipulation. This is done
by waiting for an acknowledgement message from the WE containing the
updated state.

Now for the twist.

I wish to display parts of the data in IFRAME's to completely isolate it
from the rest of the page and other parts of the data. However, adding
an IFRAME and setting its src to something (data:text/html,... in my
case) causes a request to be made. This request needs to be handled on
the main GUI thread. Additionally, the WE-GUI blocks untill the request
has been processed. Naturally, the acknowledgment message is not
returned untill it is complete. But this causes the main GUI to block
and not process the request. So we have a deadlock. It is also not
enough to just make this request async, because if I send anther message
to the WE which waits for an acknowledgement racing against the IFRAME
handling this will also cause the main GUI to likely be blocked before
the IFRAME-request has been processed. This is not dependent on me
having a custom callback handler for the decision-policy.

Because of this request-callback-block from the IFRAME it becomes fairly
complicated for me to ensure against race-conditions from main GUI
events. Because anything that causes the main GUI to wait for a
response back from the WE could block the IFRAME request and cause a
deadlock.

Do you have any suggestions for how I could design or circumvent this?

Note that even if I guard against user-input untill the IFRAME has been
processed timer based events could still happen.

Also, I could not find any documentation on how the requests on the
IFRAME work. Or that it would make this on the main GUI. If this IFRAME
request was non-blocking on the WE-GUI thread I believe my problems
would be solved. Is this the case for image set src as well?

Best regards,
Gaute

[0] https://github.com/astroidmail/astroid/pull/455

[Michael Gratton]

Post by Gaute Hope
Do you have any suggestions for how I could design or circumvent this?

Everything involving I/O that needs to be done on the main loop should
be done asynchronously. That's precisely why GLib-based libs have such
good support for async operations: To avoid blocking the GUI.

In particular for WebKitGTK, if the data you need to exchange between
the GUI process and WebExtension can be trivially formatted and
re-parsed (GVariant helps a lot with that), then consider using
webkit_web_view_run_javascript() instead of your own custom protobuf
IPC.

If the data you need to load from the GUI process is large enough,
maybe consider registering a custom URI scheme handler using
webkit_web_context_register_uri_scheme(), then let WebKitGTK handle
getting the data to the WebProcess asynchronously for you.

If neither of those are a good fit, write an async protobuf wrapper
using a background thread, so that when it blocks, it doesn't block the
GUI as well.

Geary uses the first two of these approaches with WebKitGTK, and the
last approach for interacting asynchronously with SQLite, and they all
work quite well.

HTH,
//Mike

[Gaute Hope]

Post by Michael Gratton

Post by Gaute Hope
Do you have any suggestions for how I could design or circumvent this?

Everything involving I/O that needs to be done on the main loop should
be done asynchronously. That's precisely why GLib-based libs have such
good support for async operations: To avoid blocking the GUI.
In particular for WebKitGTK, if the data you need to exchange between
the GUI process and WebExtension can be trivially formatted and
re-parsed (GVariant helps a lot with that), then consider using
webkit_web_view_run_javascript() instead of your own custom protobuf
IPC.
If the data you need to load from the GUI process is large enough,
maybe consider registering a custom URI scheme handler using
webkit_web_context_register_uri_scheme(), then let WebKitGTK handle
getting the data to the WebProcess asynchronously for you.
If neither of those are a good fit, write an async protobuf wrapper
using a background thread, so that when it blocks, it doesn't block the
GUI as well.
Geary uses the first two of these approaches with WebKitGTK, and the
last approach for interacting asynchronously with SQLite, and they all
work quite well.

Hi Mike,

thanks for your answer. As you might have noticed I have found a lot of
inspiration in Geary when it comes to message rendering - thanks! I
noticed that I could use the set_inner_html() on the body from
iframe_get_content_document() without generating a request. This solves
my problem.

The necessicity to do all DOM manipulation on the main WE-thread
(sync'ed) and the interactivity between the webpage and the GUI allows
my design to become a lot simpler if I can do sync'ed requests between
my GUI thread and the WE-thread. Async-options would require me to keep
blocking user-events and making my GUI inactive when I am waiting for a
request to finish. I was hoping to avoid this since it would be really
hard to reliably guard against this. Especially since external events /
timers also need to be blocked untill the request is done. The last
option is what I am doing now (I just choose to do most things sync'ed).

- gaute

[Michael Gratton]

Hey Gaute,

Post by Gaute Hope
thanks for your answer. As you might have noticed I have found a lot
of inspiration in Geary when it comes to message rendering - thanks!
I noticed that I could use the set_inner_html() on the body from
iframe_get_content_document() without generating a request. This
solves my problem.

Good to hear!

Post by Gaute Hope
The necessicity to do all DOM manipulation on the main WE-thread
(sync'ed) and the interactivity between the webpage and the GUI
allows my design to become a lot simpler if I can do sync'ed requests
between my GUI thread and the WE-thread. Async-options would require
me to keep blocking user-events and making my GUI inactive when I am
waiting for a request to finish. I was hoping to avoid this since it
would be really hard to reliably guard against this. Especially since
external events / timers also need to be blocked untill the request
is done. The last option is what I am doing now (I just choose to do
most things sync'ed).

Yeah, async calls do make things more complicated, but quite often in
the long run it's usually worth it. I'm not completely sure how
WebKitGTK handles user events and other IO between the WebView and
WebProcess, but I'm pretty sure it is all async. If so handling user
input between the two is possible, at least.

//Mike

[Gaute Hope]

Post by Michael Gratton

Post by Gaute Hope
The necessicity to do all DOM manipulation on the main WE-thread
(sync'ed) and the interactivity between the webpage and the GUI
allows my design to become a lot simpler if I can do sync'ed requests
between my GUI thread and the WE-thread. Async-options would require
me to keep blocking user-events and making my GUI inactive when I am
waiting for a request to finish. I was hoping to avoid this since it
would be really hard to reliably guard against this. Especially since
external events / timers also need to be blocked untill the request
is done. The last option is what I am doing now (I just choose to do
most things sync'ed).

Yeah, async calls do make things more complicated, but quite often in
the long run it's usually worth it. I'm not completely sure how
WebKitGTK handles user events and other IO between the WebView and
WebProcess, but I'm pretty sure it is all async. If so handling user
input between the two is possible, at least.

Good point. Yeah for sure, must be possible.

Thanks, Gaute