Discussion:
[webkit-gtk] did 'glcolorconvert' element must be needed when play video ? seems low efficiency.
tugouxp
2018-06-09 08:32:00 UTC
Permalink
hi folks:

From the codes of webkitgtk, during the scenaric of playing streams , the 'glcolorconvert' would be linked to the pipeline unconditionally, which would do the YUV to RGB convert by software algo, it seems only support one diplay layer, the yuv picture must be mixed with GUI on one layer, it is a waste of cpu time.


why the webkitgtk suppports the mediaplayer function in the way like this? and if the display engine of the target platform has a seperate video layer which can directly show YUV format besides OSD layer, can the webkitgtk utilize
the hardware advantage for supporting two layers?


thanks for your kindly support.!


1054 GstElement* MediaPlayerPrivateGStreamerBase::createVideoSinkGL()
1055 {
1056 if (!webkitGstCheckVersion(1, 8, 0))
1057 Š return nullptr;
1058
1059 gboolean result = TRUE;
1060 GstElement* videoSink = gst_bin_new(nullptr);
1061 GstElement* upload = gst_element_factory_make("glupload", nullptr);
1062 GstElement* colorconvert = gst_element_factory_make("glcolorconvert", nullptr);
1063 GstElement* appsink = createGLAppSink();
1064
1065 if (!appsink || !upload || !colorconvert) {
1066 Š GST_WARNING("Failed to create GstGL elements");
1067 Š gst_object_unref(videoSink);
1068
1069 Š if (upload)
1070 Š Š gst_object_unref(upload);
1071 Š if (colorconvert)
1072 Š Š gst_object_unref(colorconvert);
1073 Š if (appsink)
1074 Š Š gst_object_unref(appsink);
1075
1076 Š return nullptr;
1077 }
1078
1079 gst_bin_add_many(GST_BIN(videoSink), upload, colorconvert, appsink, nullptr);
1080
1081 GRefPtr<GstCaps> caps = adoptGRef(gst_caps_from_string("video/x-raw(" GST_CAPS_FEATURE_MEMORY_GL_MEMORY "), format = (string) " GST_GL_CAPS_FORMAT));
1082
1083 result &= gst_element_link_pads(upload, "src", colorconvert, "sink");
1084 result &= gst_element_link_pads_filtered(colorconvert, "src", appsink, "sink", caps.get());
1085
1086 GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad(upload, "sink"));
1087 gst_element_add_pad(videoSink, gst_ghost_pad_new("sink", pad.get()));
Philippe Normand
2018-06-09 10:35:43 UTC
Permalink
Post by tugouxp
From the codes of webkitgtk, during the scenaric of playing
streams , the 'glcolorconvert' would be linked to the
pipeline unconditionally, which would do the YUV to RGB convert by
software algo, it seems only support one diplay layer, the yuv
picture must be mixed with GUI on one layer, it is a waste of cpu
time.
That's incorrect. glcolorconvert performs the color conversion in GPU,
using a shader.

The textures created in the GStreamer pipeline are directly reused by
WebKit, there shouldn't be any performance bottleneck there.

Philippe
Post by tugouxp
why the webkitgtk suppports the mediaplayer function in the way like
this? and if the display engine of the target platform has a seperate
video layer which can directly show YUV format besides OSD layer,
can the webkitgtk utilize
the hardware advantage for supporting two layers?
thanks for your kindly support.!
1054 GstElement* MediaPlayerPrivateGStreamerBase::createVideoSinkGL(
)
1055 {
1056 if (!webkitGstCheckVersion(1, 8, 0))
1057 ¦ return nullptr;
1058
1059 gboolean result = TRUE;
1060 GstElement* videoSink = gst_bin_new(nullptr);
1061 GstElement* upload = gst_element_factory_make("glupload", nullptr);
1062 GstElement* colorconvert = gst_element_factory_make("glcol
orconvert", nullptr);
1063 GstElement* appsink = createGLAppSink();
1064
1065 if (!appsink || !upload || !colorconvert) {
1066 ¦ GST_WARNING("Failed to create GstGL elements");
1067 ¦ gst_object_unref(videoSink);
1068
1069 ¦ if (upload)
1070 ¦ ¦ gst_object_unref(upload);
1071 ¦ if (colorconvert)
1072 ¦ ¦ gst_object_unref(colorconvert);
1073 ¦ if (appsink)
1074 ¦ ¦ gst_object_unref(appsink);
1075
1076 ¦ return nullptr;
1077 }
1078
1079 gst_bin_add_many(GST_BIN(videoSink), upload, colorconvert, appsink, nullptr);
1080
1081 GRefPtr<GstCaps> caps = adoptGRef(gst_caps_from_string("vi
deo/x-
raw(" GST_CAPS_FEATURE_MEMORY_GL_MEMORY "), format = (string) " GST_G
L_CAPS_FORMAT));
1082
1083 result &= gst_element_link_pads(upload, "src", colorconver
t, "sink");
1084 result &= gst_element_link_pads_filtered(colorconvert, "sr
c", appsink, "sink", caps.get());
1085
1086 GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad
(upload, "sink"));
1087 gst_element_add_pad(videoSink, gst_ghost_pad_new("sink", p
ad.get()));
_______________________________________________
webkit-gtk mailing list
https://lists.webkit.org/mailman/listinfo/webkit-gtk
tugouxp
2018-06-09 11:22:43 UTC
Permalink
hi normand:


Can the GUI and Video view be displayed on the two hardware layer on webkit? so the video data did not need to be converted from yuv to RGB, after all, the YUV format is common for video frame.


i dont how this implemented on andorid version webkit , but i think there are maybe has another implement ion without YUV convert because modern SOC display hardware usually has a separate YUV layer to play videos.


am i wrong?
thanks a lot.
Post by Philippe Normand
Post by tugouxp
From the codes of webkitgtk, during the scenaric of playing
streams , the 'glcolorconvert' would be linked to the
pipeline unconditionally, which would do the YUV to RGB convert by
software algo, it seems only support one diplay layer, the yuv
picture must be mixed with GUI on one layer, it is a waste of cpu
time.
That's incorrect. glcolorconvert performs the color conversion in GPU,
using a shader.
The textures created in the GStreamer pipeline are directly reused by
WebKit, there shouldn't be any performance bottleneck there.
Philippe
Post by tugouxp
why the webkitgtk suppports the mediaplayer function in the way like
this? and if the display engine of the target platform has a seperate
video layer which can directly show YUV format besides OSD layer,
can the webkitgtk utilize
the hardware advantage for supporting two layers?
thanks for your kindly support.!
1054 GstElement* MediaPlayerPrivateGStreamerBase::createVideoSinkGL(
)
1055 {
1056 if (!webkitGstCheckVersion(1, 8, 0))
1057 Š return nullptr;
1058
1059 gboolean result = TRUE;
1060 GstElement* videoSink = gst_bin_new(nullptr);
1061 GstElement* upload = gst_element_factory_make("glupload", nullptr);
1062 GstElement* colorconvert = gst_element_factory_make("glcol
orconvert", nullptr);
1063 GstElement* appsink = createGLAppSink();
1064
1065 if (!appsink || !upload || !colorconvert) {
1066 Š GST_WARNING("Failed to create GstGL elements");
1067 Š gst_object_unref(videoSink);
1068
1069 Š if (upload)
1070 Š Š gst_object_unref(upload);
1071 Š if (colorconvert)
1072 Š Š gst_object_unref(colorconvert);
1073 Š if (appsink)
1074 Š Š gst_object_unref(appsink);
1075
1076 Š return nullptr;
1077 }
1078
1079 gst_bin_add_many(GST_BIN(videoSink), upload, colorconvert, appsink, nullptr);
1080
1081 GRefPtr<GstCaps> caps = adoptGRef(gst_caps_from_string("vi
deo/x-
raw(" GST_CAPS_FEATURE_MEMORY_GL_MEMORY "), format = (string) " GST_G
L_CAPS_FORMAT));
1082
1083 result &= gst_element_link_pads(upload, "src", colorconver
t, "sink");
1084 result &= gst_element_link_pads_filtered(colorconvert, "sr
c", appsink, "sink", caps.get());
1085
1086 GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad
(upload, "sink"));
1087 gst_element_add_pad(videoSink, gst_ghost_pad_new("sink", p
ad.get()));
_______________________________________________
webkit-gtk mailing list
https://lists.webkit.org/mailman/listinfo/webkit-gtk
Loading...