See: https://gist.github.com/Jim-Bar/3cbba684a71d1a9d468a6711a6eddbeb
Here we focus on implementing YUV cropping for I420 images. In term of speed, it appears that YUV cropping is legit when the image resolution is very high, otherwise just cvtColor then crop.
g++ -std=c++17 yuv_crop.cpp -o yuv_crop `pkg-config --cflags --libs opencv4`