[00/28] Venus updates

Message ID 20180424124436.26955-1-stanimir.varbanov@linaro.org
Headers show
Series
  • Venus updates
Related show

Message

Stanimir Varbanov April 24, 2018, 12:44 p.m.
Hello,

This patch set aims to:

* add initial support for Venus version 4xx (found on sdm845).

* introduce a common capability parser to enumerate better
  supported uncompressed formats, capabilities by codec,
  supported codecs and so on.

* also contains various cleanups, readability improvements
  and fixes.

* adds HEVC codec support for the Venus versions which has
  support for it.

* add multi-stream support (secondary decoder output), which
  will give as an opportunity to use UBWC compressed formats
  to optimize internal interconnect bandwidth on higher
  resolutions.

Comments are welcome!

regards,
Stan

Stanimir Varbanov (28):
  venus: hfi_msgs: correct pointer increment
  venus: hfi: preparation to support venus 4xx
  venus: hfi: update sequence event to handle more properties
  venus: hfi_cmds: add set_properties for 4xx version
  venus: hfi: support session continue for 4xx version
  venus: hfi: handle buffer output2 type as well
  venus: hfi_venus: add halt AXI support for Venus 4xx
  venus: hfi_venus: add suspend function for 4xx version
  venus: venc,vdec: adds clocks needed for venus 4xx
  venus: vdec: call session_continue in insufficient event
  venus: add common capability parser
  venus: helpers: make a commmon function for power_enable
  venus: core: delete not used flag for buffer mode
  venus: helpers: rename a helper function and use buffer mode from caps
  venus: add a helper function to set dynamic buffer mode
  venus: add helper function to set actual buffer size
  venus: delete no longer used bufmode flag from instance
  venus: helpers: add buffer type argument to a helper
  venus: helpers: add a new helper to set raw format
  venus: helpers,vdec,venc: add helpers to set work mode and core usage
  venus: helpers: extend set_num_bufs helper with one more argument
  venus: helpers: add a helper to return opb buffer sizes
  venus: vdec: get required input buffers as well
  venus: vdec: new function for output configuration
  venus: move frame size calculations in common place
  venus: implementing multi-stream support
  venus: add sdm845 compatible and resource data
  venus: add HEVC codec support

 .../devicetree/bindings/media/qcom,venus.txt       |   1 +
 drivers/media/platform/qcom/venus/Makefile         |   3 +-
 drivers/media/platform/qcom/venus/core.c           | 102 ++++
 drivers/media/platform/qcom/venus/core.h           |  91 ++--
 drivers/media/platform/qcom/venus/helpers.c        | 558 +++++++++++++++++++--
 drivers/media/platform/qcom/venus/helpers.h        |  23 +-
 drivers/media/platform/qcom/venus/hfi.c            |  12 +-
 drivers/media/platform/qcom/venus/hfi.h            |   9 +
 drivers/media/platform/qcom/venus/hfi_cmds.c       |  64 ++-
 drivers/media/platform/qcom/venus/hfi_helper.h     | 112 ++++-
 drivers/media/platform/qcom/venus/hfi_msgs.c       | 401 +++------------
 drivers/media/platform/qcom/venus/hfi_parser.c     | 290 +++++++++++
 drivers/media/platform/qcom/venus/hfi_parser.h     |  45 ++
 drivers/media/platform/qcom/venus/hfi_venus.c      |  69 +++
 drivers/media/platform/qcom/venus/hfi_venus_io.h   |  24 +
 drivers/media/platform/qcom/venus/vdec.c           | 324 +++++++-----
 drivers/media/platform/qcom/venus/venc.c           | 166 +++---
 17 files changed, 1641 insertions(+), 653 deletions(-)
 create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.c
 create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.h

-- 
2.14.1

Comments

Vikash Garodia May 4, 2018, 11:09 a.m. | #1
Hi Stanimir,

On 2018-05-03 17:06, Stanimir Varbanov wrote:
> Hi Vikash,

> 

> Thanks for the comments!

> 

> On  2.05.2018 09:26, Vikash Garodia wrote:

>> Hello Stanimir,

>> 

>> On 2018-04-24 18:14, Stanimir Varbanov wrote:

>>> Call session_continue for Venus 4xx version even when the event

>>> says that the buffer resources are not sufficient. Leaving a

>>> comment with more information about the workaround.

>>> 

>>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>

>>> ---

>>>  drivers/media/platform/qcom/venus/vdec.c | 8 ++++++++

>>>  1 file changed, 8 insertions(+)

>>> 

>>> diff --git a/drivers/media/platform/qcom/venus/vdec.c

>>> b/drivers/media/platform/qcom/venus/vdec.c

>>> index c45452634e7e..91c7384ff9c8 100644

>>> --- a/drivers/media/platform/qcom/venus/vdec.c

>>> +++ b/drivers/media/platform/qcom/venus/vdec.c

>>> @@ -873,6 +873,14 @@ static void vdec_event_notify(struct venus_inst

>>> *inst, u32 event,

>>> 

>>>              dev_dbg(dev, "event not sufficient resources (%ux%u)\n",

>>>                  data->width, data->height);

>>> +            /*

>>> +             * Workaround: Even that the firmware send and event for

>>> +             * insufficient buffer resources it is safe to call

>>> +             * session_continue because actually the event says that

>>> +             * the number of capture buffers is lower.

>>> +             */

>>> +            if (IS_V4(core))

>>> +                hfi_session_continue(inst);

>>>              break;

>>>          case HFI_EVENT_RELEASE_BUFFER_REFERENCE:

>>>              venus_helper_release_buf_ref(inst, data->tag);

>> 

>> Insufficient event from video firmware could be sent either,

>> 1. due to insufficient buffer resources

>> 2. due to lower capture buffers

>> 

>> It cannot be assumed that the event received by the host is due to 

>> lower capture

>> buffers. Incase the buffer resource is insufficient, let say there is 

>> a bitstream

>> resolution switch from 720p to 1080p, capture buffers needs to be 

>> reallocated.

> 

> I agree with you. I will rework this part and call session_continue

> only for case #2.


Even if the capture buffers are lower, driver should consider 
reallocation of capture
buffers with required higher count. Without this, it may happen that for 
a given video
frame, the decoded output will not be generated. The fact that the DPB 
buffer count is
same as capture buffers, will be lower than required. Hence the frame 
which needs YUV
reference beyond the DPB count, will get stuck as it cannot be decoded 
due to unavailability
of sufficient DPB buffers.
Say for ex. 10 DPB and capture buffers are allocated. For a given 
bitstream, firmware requested
the count to be 15. Frame 1 to 10 gets decoded and stored in DPB as 
references for future frame
decoding. Now when the 11th frame is queued to firmware, it can be 
decode but cannot be stored
as reference to decode future (12th) frame. Hence 11 frame will get 
stuck and will not be given
back to host driver.

>> 

>> The driver should be sending the V4L2_EVENT_SOURCE_CHANGE to client 

>> instead of ignoring

>> the event from firmware.

> 

> The v4l2 event is sent always to v4l clients.

> 

> regards,

> Stan
Hans Verkuil May 7, 2018, 10:41 a.m. | #2
On 24/04/18 14:44, Stanimir Varbanov wrote:
> Hello,

> 

> This patch set aims to:

> 

> * add initial support for Venus version 4xx (found on sdm845).

> 

> * introduce a common capability parser to enumerate better

>   supported uncompressed formats, capabilities by codec,

>   supported codecs and so on.

> 

> * also contains various cleanups, readability improvements

>   and fixes.

> 

> * adds HEVC codec support for the Venus versions which has

>   support for it.

> 

> * add multi-stream support (secondary decoder output), which

>   will give as an opportunity to use UBWC compressed formats

>   to optimize internal interconnect bandwidth on higher

>   resolutions.


I'm a bit confused about this: is this a purely driver-internal thing,
or is this exposed somehow to userspace as well? It seems to be purely
internal.

Regards,

	Hans

> 

> Comments are welcome!

> 

> regards,

> Stan

> 

> Stanimir Varbanov (28):

>   venus: hfi_msgs: correct pointer increment

>   venus: hfi: preparation to support venus 4xx

>   venus: hfi: update sequence event to handle more properties

>   venus: hfi_cmds: add set_properties for 4xx version

>   venus: hfi: support session continue for 4xx version

>   venus: hfi: handle buffer output2 type as well

>   venus: hfi_venus: add halt AXI support for Venus 4xx

>   venus: hfi_venus: add suspend function for 4xx version

>   venus: venc,vdec: adds clocks needed for venus 4xx

>   venus: vdec: call session_continue in insufficient event

>   venus: add common capability parser

>   venus: helpers: make a commmon function for power_enable

>   venus: core: delete not used flag for buffer mode

>   venus: helpers: rename a helper function and use buffer mode from caps

>   venus: add a helper function to set dynamic buffer mode

>   venus: add helper function to set actual buffer size

>   venus: delete no longer used bufmode flag from instance

>   venus: helpers: add buffer type argument to a helper

>   venus: helpers: add a new helper to set raw format

>   venus: helpers,vdec,venc: add helpers to set work mode and core usage

>   venus: helpers: extend set_num_bufs helper with one more argument

>   venus: helpers: add a helper to return opb buffer sizes

>   venus: vdec: get required input buffers as well

>   venus: vdec: new function for output configuration

>   venus: move frame size calculations in common place

>   venus: implementing multi-stream support

>   venus: add sdm845 compatible and resource data

>   venus: add HEVC codec support

> 

>  .../devicetree/bindings/media/qcom,venus.txt       |   1 +

>  drivers/media/platform/qcom/venus/Makefile         |   3 +-

>  drivers/media/platform/qcom/venus/core.c           | 102 ++++

>  drivers/media/platform/qcom/venus/core.h           |  91 ++--

>  drivers/media/platform/qcom/venus/helpers.c        | 558 +++++++++++++++++++--

>  drivers/media/platform/qcom/venus/helpers.h        |  23 +-

>  drivers/media/platform/qcom/venus/hfi.c            |  12 +-

>  drivers/media/platform/qcom/venus/hfi.h            |   9 +

>  drivers/media/platform/qcom/venus/hfi_cmds.c       |  64 ++-

>  drivers/media/platform/qcom/venus/hfi_helper.h     | 112 ++++-

>  drivers/media/platform/qcom/venus/hfi_msgs.c       | 401 +++------------

>  drivers/media/platform/qcom/venus/hfi_parser.c     | 290 +++++++++++

>  drivers/media/platform/qcom/venus/hfi_parser.h     |  45 ++

>  drivers/media/platform/qcom/venus/hfi_venus.c      |  69 +++

>  drivers/media/platform/qcom/venus/hfi_venus_io.h   |  24 +

>  drivers/media/platform/qcom/venus/vdec.c           | 324 +++++++-----

>  drivers/media/platform/qcom/venus/venc.c           | 166 +++---

>  17 files changed, 1641 insertions(+), 653 deletions(-)

>  create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.c

>  create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.h

>