avfilter/bwdif: account for chroma sub-sampling in min size calculation
The current logic for detecting frames that are too small for the
algorithm does not account for chroma sub-sampling, and so a sample
where the luma plane is large enough, but the chroma planes are not
will not be rejected. In that event, a heap overflow will occur.
This change adjusts the logic to consider the chroma planes and makes
the change to all three bwdif implementations.
Fixes #10688
Signed-off-by: Cosmin Stejerean <cosmin@cosmin.at>
Reviewed-by: Thomas Mundt <tmundt75@gmail.com>
Signed-off-by: Philip Langdale <philipl@overt.org>
(cherry picked from commit
737ede405b11a37fdd61d19cf25df296a0cb0b75)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
git.videolan.org / ffmpeg.git / commit