Getting Started
Read, Show & Save Images
OpenCV reads images in BGR order (not RGB) — a common pitfall when interoperating with matplotlib or PIL. cv2.waitKey(0) blocks until a key is pressed; always call destroyAllWindows() to close windows. JPEG quality is 0-100.
import cv2
import numpy as np
# read an image (BGR format by default)
img = cv2.imread("photo.jpg")
print(img.shape) # (height, width, channels)
# read as grayscale
gray = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
# read with alpha channel
rgba = cv2.imread("logo.png", cv2.IMREAD_UNCHANGED)
# display
cv2.imshow("window", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
# save
cv2.imwrite("output.jpg", img, [cv2.IMWRITE_JPEG_QUALITY, 90])Install & Import
opencv-python ships only the main modules; opencv-contrib-python adds extra (often patent-encumbered) modules like SIFT/SURF and ArUco. Use the headless variant in servers/Docker to avoid pulling X11 dependencies. Despite the name cv2, OpenCV 4.x still imports as cv2.
# Install OpenCV (Python bindings)
pip install opencv-python # main modules only
pip install opencv-contrib-python # includes extra modules (SIFT, SURF, aruco...)
pip install opencv-python-headless # no GUI (servers / Docker)
# The universal alias is cv2
import cv2
print(cv2.__version__) # e.g. 4.9.0
# NumPy is required — images are ndarray
import numpy as npImage as NumPy Array
OpenCV images are NumPy arrays, so everything you know about NumPy applies. The coordinate convention is img[y, x] (row, col) — the opposite of (x, y) image coordinates, a frequent source of bugs. Color images are (H, W, 3) in BGR; grayscale are (H, W).
import cv2
import numpy as np
img = cv2.imread("photo.jpg") # BGR uint8, shape (H, W, 3)
print(type(img)) # <class 'numpy.ndarray'>
print(img.shape, img.dtype) # (480, 640, 3) uint8
# A grayscale image is 2-D: (H, W)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
print(gray.shape) # (480, 640)
# Coordinate convention: img[y, x] (row, col)
px = img[100, 200] # BGR pixel at (x=200, y=100)
blue, green, red = img[100, 200]
# Vectorized access to a region (ROI)
roi = img[50:150, 100:300] # rows 50-149, cols 100-299Pixel Access & ROI
Per-pixel access via img[y, x] is slow and discouraged in loops — always prefer NumPy slicing. .copy() when assigning a ROI so later changes to the source don't bleed in. Modifying img[:, :, channel] is the fastest way to operate on a single color channel.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
# access a single pixel (slow)
b, g, r = img[100, 200]
# access a single channel faster
blue = img[100, 200, 0]
img[100, 200] = [255, 255, 255] # set to white
# numpy slicing is much faster than item-by-item
roi = img[100:200, 200:400].copy()
# copy ROI to another location
img[0:100, 0:200] = roi
# modify a whole channel
img[:, :, 2] = 0 # zero out the red channelData Types & Conversion
Most OpenCV functions expect uint8 (0-255), but algorithms like SIFT, optical flow, and edge detection often need float32. Never cast directly with .astype() after subtraction — uint8 overflow turns -1 into 255. Use cv2.absdiff or convert to float first. Always cast back to uint8 before cv2.imwrite / imshow.
import cv2
import numpy as np
img = cv2.imread("photo.jpg") # uint8, 0-255
print(img.dtype, img.min(), img.max()) # uint8 0 255
# convert to float32 (0.0-1.0 for many algorithms)
f = img.astype(np.float32) / 255.0
# convert back to uint8 for display/save
u = (f * 255).astype(np.uint8)
# clamp then cast (avoid overflow when subtracting)
diff = cv2.absdiff(img, img) # safe subtraction
print(diff.dtype) # uint8
# CV_32F / CV_64F / CV_8U constants (used in some functions)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
print(gray.dtype) # uint8Video Capture
Open Webcam
VideoCapture(0) opens the default webcam; use 1, 2... for additional cameras. Always call cap.release() to free the device — forgetting it can leave the camera busy for other apps. waitKey(1) waits ~1ms; the & 0xFF trick ensures compatibility on 64-bit systems where waitKey returns more than 8 bits.
import cv2
cap = cv2.VideoCapture(0) # 0 = default camera
print(cap.isOpened()) # True if camera opened
while True:
ret, frame = cap.read() # ret is False at end of stream
if not ret:
break
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cv2.imshow("frame", gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()Read Video File
CAP_PROP_FRAME_COUNT is often approximate and unreliable for some codecs — don't rely on it for exact seeking. waitKey(int(1000/fps)) plays at roughly the correct speed; for precise timing use a real clock. Some compressed videos won't open without the right backend (FFmpeg).
import cv2
cap = cv2.VideoCapture("movie.mp4")
# video properties
fps = cap.get(cv2.CAP_PROP_FPS)
width = cap.get(cv2.CAP_PROP_FRAME_WIDTH)
height = cap.get(cv2.CAP_PROP_FRAME_HEIGHT)
count = cap.get(cv2.CAP_PROP_FRAME_COUNT)
print(f"{width}x{height} @ {fps} fps, {count} frames")
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
cv2.imshow("video", frame)
if cv2.waitKey(int(1000 / fps)) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()Save / Write Video
The FourCC code is platform- and codec-dependent: 'mp4v' is widely portable, 'XVID' produces .avi, 'avc1' gives H.264 (may not be available without proprietary codecs). The frame size passed to VideoWriter MUST match the frames you write, otherwise the file is corrupted. Always call out.release() to flush and close the file.
import cv2
cap = cv2.VideoCapture(0)
fourcc = cv2.VideoWriter_fourcc(*'mp4v') # or 'XVID', 'avc1'
fps = 20.0
size = (640, 480)
out = cv2.VideoWriter("out.mp4", fourcc, fps, size)
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
frame = cv2.resize(frame, size)
out.write(frame)
cv2.imshow("rec", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
out.release()
cv2.destroyAllWindows()Camera Settings
Camera properties are hardware- and driver-dependent — many webcams silently ignore unsupported settings or snap to nearest valid values, so always read back the actual value. CAP_PROP_EXPOSURE uses log2 of seconds (-4 = 1/16 s). Auto-exposure must be disabled (set to 1) before manual exposure takes effect on most cameras.
import cv2
cap = cv2.VideoCapture(0)
# set resolution
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
# set exposure / focus / brightness
cap.set(cv2.CAP_PROP_EXPOSURE, -4) # log2 exposure time
cap.set(cv2.CAP_PROP_BRIGHTNESS, 128)
cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, 1) # 1=manual, 3=auto
# disable autofocus and set manual focus
cap.set(cv2.CAP_PROP_AUTOFOCUS, 0)
cap.set(cv2.CAP_PROP_FOCUS, 50)
print(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) # actual width may differFrame Rate & Timing
cap.get(CAP_PROP_FPS) returns the configured FPS, not the actual capture rate — measure with a real clock. A 30-frame moving average smooths jitter. The biggest FPS killers are cv2.imshow (GUI thread) and per-frame cv2.imwrite; for benchmarking, disable display. For deterministic timing, accumulate frames in a thread and process from a queue.
import cv2
import time
cap = cv2.VideoCapture(0)
prev = time.time()
fps_hist = []
while True:
ret, frame = cap.read()
if not ret:
break
now = time.time()
dt = now - prev
prev = now
if dt > 0:
fps_hist.append(1.0 / dt)
fps = sum(fps_hist[-30:]) / min(len(fps_hist), 30)
cv2.putText(frame, f"FPS: {fps:.1f}", (10, 30),
cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
cv2.imshow("fps", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()Drawing
Line & Rectangle
All drawing functions mutate the image in place — pass a copy if you want to keep the original. Coordinates are (x, y), opposite to NumPy's [y, x] indexing. Colors are BGR tuples. Use thickness=-1 for filled shapes and cv2.LINE_AA for smooth (anti-aliased) lines on diagonal edges.
import cv2
import numpy as np
img = np.zeros((400, 400, 3), dtype=np.uint8)
# line(img, pt1, pt2, color, thickness)
cv2.line(img, (10, 10), (390, 10), (0, 255, 0), 2)
# rectangle(img, pt1, pt2, color, thickness)
cv2.rectangle(img, (50, 50), (200, 150), (255, 0, 0), 3)
# filled rectangle (thickness = -1)
cv2.rectangle(img, (220, 50), (370, 150), (0, 0, 255), -1)
# anti-aliased line (cv2.LINE_AA)
cv2.line(img, (10, 380), (390, 200), (255, 255, 0), 1, cv2.LINE_AA)
cv2.imshow("img", img)
cv2.waitKey(0)Circle & Ellipse
ellipse's axes parameter is (halfWidth, halfHeight) — full width is 2 * axes[0]. angle rotates the whole ellipse; startAngle/endAngle draw partial arcs (0° = right, counterclockwise). Drawing partial arcs is the standard way to render pie-chart slices or motion-direction indicators.
import cv2
import numpy as np
img = np.zeros((400, 400, 3), dtype=np.uint8)
# circle(img, center, radius, color, thickness)
cv2.circle(img, (100, 100), 50, (0, 255, 255), 3)
cv2.circle(img, (300, 100), 50, (255, 0, 255), -1) # filled
# ellipse(img, center, axes, angle, startAngle, endAngle, color, thickness)
cv2.ellipse(img, (200, 250), (100, 50), 30, 0, 360, (0, 255, 0), 2)
# partial arc (0 to 180 degrees)
cv2.ellipse(img, (200, 250), (100, 50), 30, 0, 180, (0, 0, 255), 3)
cv2.imshow("img", img)
cv2.waitKey(0)Polygon & Polylines
polylines accepts a list of point arrays, so you can draw multiple polygons in one call. The points array must be int32 and reshaped to (-1, 1, 2). Use fillPoly for filled shapes; for translucent fills, draw onto an overlay and blend with addWeighted. The isClosed flag connects the last point back to the first.
import cv2
import numpy as np
img = np.zeros((400, 400, 3), dtype=np.uint8)
# polygon points: shape (N, 1, 2), int32
pts = np.array([[50, 50], [200, 30], [350, 200],
[300, 350], [80, 300]], dtype=np.int32)
pts = pts.reshape((-1, 1, 2))
# outline only (not closed)
cv2.polylines(img, [pts], False, (0, 255, 0), 2)
# closed polygon outline
cv2.polylines(img, [pts], True, (255, 0, 0), 3)
# filled polygon
cv2.fillPoly(img, [pts], (0, 0, 255))
cv2.imshow("img", img)
cv2.waitKey(0)Text
OpenCV's built-in fonts are Hershey vector fonts — they don't support CJK/Chinese characters. For non-Latin text, render with PIL (Pillow) ImageFont and convert back, or use cv2.freetype from opencv-contrib. getTextSize returns ((width, height), baseline) — useful to center text or draw a background box behind it.
import cv2
import numpy as np
img = np.zeros((300, 600, 3), dtype=np.uint8)
# putText(img, text, org, fontFace, fontScale, color, thickness)
cv2.putText(img, "Hello OpenCV", (10, 50),
cv2.FONT_HERSHEY_SIMPLEX, 1.2, (255, 255, 255), 2)
# fonts
fonts = [cv2.FONT_HERSHEY_SIMPLEX, cv2.FONT_HERSHEY_PLAIN,
cv2.FONT_HERSHEY_DUPLEX, cv2.FONT_HERSHEY_COMPLEX,
cv2.FONT_HERSHEY_TRIPLEX, cv2.FONT_ITALIC]
for i, f in enumerate(fonts):
cv2.putText(img, f"font {i}", (10, 100 + i*30), f, 0.8, (0, 255, 0), 1)
# measure text size before drawing
(text_w, text_h), baseline = cv2.getTextSize(
"Measured", cv2.FONT_HERSHEY_SIMPLEX, 1.0, 2)
cv2.rectangle(img, (10, 260), (10 + text_w, 260 + text_h + baseline),
(0, 0, 255), 1)Arrow & Marker
arrowedLine's tipLength is a fraction of the line length, so longer arrows have proportionally bigger tips — set it explicitly if you want consistent arrowhead sizes. drawMarker is convenient for highlighting key points (e.g. detected features or corners); MARKER_TILTED_CROSS gives an X-shaped marker.
import cv2
import numpy as np
img = np.zeros((400, 400, 3), dtype=np.uint8)
# arrowedLine(img, pt1, pt2, color, thickness, tipLength)
cv2.arrowedLine(img, (50, 50), (350, 50), (0, 255, 0), 2, tipLength=0.05)
cv2.arrowedLine(img, (200, 200), (200, 380), (0, 0, 255), 3, tipLength=0.1)
# drawMarker (cross, tilted cross, circle, square, diamond)
cv2.drawMarker(img, (100, 300), (255, 0, 255),
markerType=cv2.MARKER_CROSS, markerSize=30, thickness=2)
cv2.drawMarker(img, (300, 300), (255, 255, 0),
markerType=cv2.MARKER_DIAMOND, markerSize=30, thickness=2)
cv2.imshow("img", img)
cv2.waitKey(0)Overlay & Annotation
The overlay technique (draw on a copy, then addWeighted) is the standard way to draw translucent fills — OpenCV has no native alpha for shapes. Always draw solid text AFTER blending so it stays crisp; drawing text on the overlay before blending makes it blurry. This pattern is everywhere in detection/segmentation visualization.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
overlay = img.copy()
# draw a translucent rectangle on the overlay
cv2.rectangle(overlay, (50, 50), (250, 150), (0, 0, 255), -1)
cv2.putText(overlay, "REGION", (60, 110),
cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2)
# blend overlay onto the original (alpha=0.4)
out = cv2.addWeighted(overlay, 0.4, img, 0.6, 0)
cv2.imshow("out", out)
cv2.waitKey(0)Image Arithmetic
Add & Subtract
OpenCV's cv2.add saturates at 255 (250 + 10 = 255), while NumPy's + wraps around (250 + 10 = 8 due to uint8 overflow). For images, always use cv2.add / cv2.subtract — wrapping creates visible black/white artifacts. absdiff is the workhorse for background subtraction and motion detection.
import cv2
import numpy as np
a = cv2.imread("a.jpg")
b = cv2.imread("b.jpg")
b = cv2.resize(b, (a.shape[1], a.shape[0])) # match sizes
# OpenCV addition: saturated (caps at 255)
add_cv = cv2.add(a, b)
# NumPy addition: wraps around (250 + 10 = 264 -> 8)
add_np = a + b
# subtract (saturated, never goes below 0)
sub = cv2.subtract(a, b)
# absolute difference (handy for motion detection)
diff = cv2.absdiff(a, b)Blending & AddWeighted
addWeighted requires both images to have identical size and dtype — resize first. The gamma term adds a constant to every pixel, useful for brightness control. The classic fade/cross-dissolve between two images is just addWeighted with alpha sweeping from 0 to 1.
import cv2
import numpy as np
a = cv2.imread("a.jpg")
b = cv2.imread("b.jpg")
b = cv2.resize(b, (a.shape[1], a.shape[0]))
# dst = alpha * a + beta * b + gamma
blend = cv2.addWeighted(a, 0.7, b, 0.3, 0)
# brighten by adding a constant (gamma)
bright = cv2.addWeighted(a, 1.0, a, 0.0, 50)
# fade between two images over N steps
for i in range(11):
alpha = i / 10.0
frame = cv2.addWeighted(a, alpha, b, 1 - alpha, 0)
cv2.imshow("fade", frame)
cv2.waitKey(200)Bitwise Operations
Bitwise ops are the standard way to apply a mask: cv2.bitwise_or(src, dst, mask=mask) copies src into dst only where mask is non-zero. The mask must be a single-channel 8-bit image where non-zero means 'keep'. This is the foundation of ROI extraction and background replacement.
import cv2
import numpy as np
a = np.zeros((200, 200), dtype=np.uint8)
cv2.rectangle(a, (20, 20), (120, 120), 255, -1)
b = np.zeros((200, 200), dtype=np.uint8)
cv2.circle(b, (100, 100), 60, 255, -1)
# bitwise AND / OR / XOR / NOT
and_ = cv2.bitwise_and(a, b)
or_ = cv2.bitwise_or(a, b)
xor = cv2.bitwise_xor(a, b)
not_ = cv2.bitwise_not(a)
# use a mask to copy only part of an image
mask = np.zeros((200, 200), dtype=np.uint8)
cv2.circle(mask, (100, 100), 50, 255, -1)
src = np.full((200, 200, 3), (0, 255, 0), dtype=np.uint8)
dst = cv2.imread("photo.jpg")
dst = cv2.resize(dst, (200, 200))
result = cv2.bitwise_or(src, dst, mask=mask)Image Difference & Motion
Frame differencing is the simplest motion detector: threshold the absolute difference between consecutive frames. It fails when the camera moves and ignores slow motion — for real systems use background subtraction (cv2.createBackgroundSubtractorMOG2) or optical flow. Morphological opening removes noise; dilation merges nearby blobs.
import cv2
import numpy as np
prev = cv2.imread("frame1.jpg", cv2.IMREAD_GRAYSCALE)
curr = cv2.imread("frame2.jpg", cv2.IMREAD_GRAYSCALE)
# absolute difference
diff = cv2.absdiff(prev, curr)
# threshold to get a binary motion mask
_, mask = cv2.threshold(diff, 25, 255, cv2.THRESH_BINARY)
# clean up with morphology
kernel = np.ones((5, 5), np.uint8)
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
mask = cv2.dilate(mask, kernel, iterations=2)
# find moving contours
contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
for c in contours:
if cv2.contourArea(c) > 500:
x, y, w, h = cv2.boundingRect(c)
cv2.rectangle(curr, (x, y), (x+w, y+h), (0, 255, 0), 2)ROI & Masking
bitwise_and with a mask is the idiomatic way to extract irregular regions. Boolean NumPy indexing (img[mask]) is more flexible but returns a flat array of pixels, not a shaped image. Always .copy() ROIs you plan to reuse — slicing returns a view that aliases the original.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
h, w = img.shape[:2]
# build a mask: white circle on black background
mask = np.zeros((h, w), dtype=np.uint8)
cv2.circle(mask, (w // 2, h // 2), 150, 255, -1)
# keep only the masked region, zero elsewhere
masked = cv2.bitwise_and(img, img, mask=mask)
# extract ROI by bounding box
roi = img[100:300, 200:500].copy()
# put ROI back (or to a different location)
img[0:200, 0:300] = roi
# boolean mask (NumPy) — also works, but converts to 0/255
bool_mask = np.zeros((h, w), dtype=bool)
bool_mask[100:300, 200:500] = True
img_masked = img.copy()
img_masked[~bool_mask] = 0Saturation vs Wrapping
The uint8 overflow trap: NumPy wraps (300 -> 44), OpenCV saturates (300 -> 255). For any arithmetic that may exceed 0-255, either use cv2.add/subtract or upcast to int16/float32 first and clip back. Mixing cv2 and NumPy ops in the same pipeline is the #1 source of mysterious dark/bright pixel artifacts.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
# WRONG: NumPy arithmetic wraps around (200 + 100 -> 44)
bad = img + 100
# RIGHT: cv2.add saturates at 255 (200 + 100 -> 255)
good = cv2.add(img, np.full_like(img, 100))
# subtracting a bright image: cv2 clamps to 0
dark = cv2.subtract(img, np.full_like(img, 100))
# manual saturation (equivalent to cv2.add)
manual = np.clip(img.astype(np.int16) + 100, 0, 255).astype(np.uint8)
# mixing cv2 and numpy is a common bug source:
# half = img // 2 # NumPy: fine (no overflow)
# half_then_add = half + half # might equal original (ok)
# but bright = img + img # wraps! 150+150=300 -> 44Geometric Transforms
Resize / Scale
INTER_AREA is best for downscaling (avoids aliasing), INTER_CUBIC or INTER_LINEAR for upscaling. INTER_NEAREST is fastest but blocky — used for label/mask images where you must not introduce new pixel values. The dsize tuple is (width, height), the opposite of img.shape which is (height, width).
import cv2
img = cv2.imread("photo.jpg")
h, w = img.shape[:2]
# resize to a fixed size
small = cv2.resize(img, (320, 240))
# resize by a scale factor (fx, fy)
big = cv2.resize(img, None, fx=2.0, fy=2.0)
# resize keeping aspect ratio
scale = 0.5
new = cv2.resize(img, (int(w * scale), int(h * scale)))
# interpolation methods
down = cv2.resize(img, (w // 2, h // 2),
interpolation=cv2.INTER_AREA) # shrinking
up = cv2.resize(img, (w * 2, h * 2),
interpolation=cv2.INTER_CUBIC) # enlargingRotation
cv2.rotate is much faster than warpAffine for 90° multiples. getRotationMatrix2D's angle is in degrees, positive = counterclockwise. To rotate without clipping, recompute the bounding box (new_w, new_h) and shift the rotation matrix — otherwise the corners of a rotated large image get cut off.
import cv2
img = cv2.imread("photo.jpg")
h, w = img.shape[:2]
# rotate 90/180/270 (fast, no interpolation)
rot90 = cv2.rotate(img, cv2.ROTATE_90_CLOCKWISE)
rot180 = cv2.rotate(img, cv2.ROTATE_180)
rot270 = cv2.rotate(img, cv2.ROTATE_90_COUNTERCLOCKWISE)
# arbitrary-angle rotation
center = (w // 2, h // 2)
angle = 45
scale = 1.0
M = cv2.getRotationMatrix2D(center, angle, scale)
rot = cv2.warpAffine(img, M, (w, h))
# rotate and expand the canvas to fit the whole image
cos = abs(M[0, 0]); sin = abs(M[0, 1])
new_w = int(h * sin + w * cos)
new_h = int(h * cos + w * sin)
M[0, 2] += (new_w - w) / 2
M[1, 2] += (new_h - h) / 2
rot_fit = cv2.warpAffine(img, M, (new_w, new_h))Affine Transform
An affine transform preserves parallelism and ratios of distances — defined by 3 point correspondences (6 degrees of freedom). warpAffine uses forward mapping implicitly via inverse lookup, so M maps src->dst but the function inverts it internally. Use borderMode=BORDER_REFLECT or BORDER_CONSTANT to control how out-of-bounds regions are filled.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
h, w = img.shape[:2]
# affine: parallel lines stay parallel (6 DOF)
src = np.float32([[50, 50], [200, 50], [50, 200]])
dst = np.float32([[10, 100], [200, 50], [100, 250]])
M = cv2.getAffineTransform(src, dst)
warped = cv2.warpAffine(img, M, (w, h))
# inverse mapping (destination -> source)
inv = cv2.invertAffineTransform(M)
# translation is a special affine
T = np.float32([[1, 0, 100], [0, 1, 50]]) # shift right 100, down 50
shifted = cv2.warpAffine(img, T, (w, h))
# black border is default; use borderMode for other fill
reflected = cv2.warpAffine(img, T, (w, h),
borderMode=cv2.BORDER_REFLECT)Perspective Transform
Perspective (homography) is defined by 4 non-collinear point pairs — 8 DOF. This is the standard 'document scanner' transform: pick the 4 corners of a card/paper in the photo and warp to a rectangle. getPerspectiveTransform needs exactly 4 points; for an overdetermined fit use cv2.findHomography with RANSAC.
import cv2
import numpy as np
img = cv2.imread("card.jpg")
h, w = img.shape[:2]
# 4 point correspondences define a perspective (homography)
src = np.float32([[73, 110], [320, 78], [380, 350], [40, 320]])
dst = np.float32([[0, 0], [w, 0], [w, h], [0, h]])
M = cv2.getPerspectiveTransform(src, dst)
warped = cv2.warpPerspective(img, M, (w, h))
# invert the homography to map back
M_inv = cv2.getPerspectiveTransform(dst, src)
restored = cv2.warpPerspective(warped, M_inv, (w, h))Flip & Mirror
flip is a cheap in-memory operation — much faster than warpAffine. flipCode sign convention: positive flips horizontally, 0 vertically, negative both. cv2.transpose swaps axes and is sometimes used as a fast 90° rotation when combined with a flip.
import cv2
img = cv2.imread("photo.jpg")
# flip(img, flipCode)
# 0 -> vertical flip (around x-axis) (upside-down)
# 1 -> horizontal flip (around y-axis) (mirror)
# -1 -> both (180° rotation equivalent)
v = cv2.flip(img, 0)
h = cv2.flip(img, 1)
both = cv2.flip(img, -1)
# transpose (swap x and y axes) — like a 90° rotation + flip
t = cv2.transpose(img)Remap (Custom Mapping)
remap is the most general geometric transform — resize, rotate, and even barrel/lens distortion correction are all expressible as remap. Always build the maps with NumPy vectorization, not Python loops (the loop version above is for illustration only — it's ~100x slower). INTER_LINEAR is the default; for sub-pixel accuracy on edges use INTER_CUBIC.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
h, w = img.shape[:2]
# build coordinate maps: map_x, map_y say where each output pixel comes from
map_x = np.zeros((h, w), dtype=np.float32)
map_y = np.zeros((h, w), dtype=np.float32)
for y in range(h):
for x in range(w):
# mirror the image
map_x[y, x] = w - 1 - x
map_y[y, x] = y
remapped = cv2.remap(img, map_x, map_y, cv2.INTER_LINEAR)
# vectorized version (much faster) — wave distortion
xs, ys = np.meshgrid(np.arange(w), np.arange(h))
map_x = xs.astype(np.float32) + 10 * np.sin(ys / 20.0)
map_y = ys.astype(np.float32) + 10 * np.cos(xs / 20.0)
wave = cv2.remap(img, map_x, map_y, cv2.INTER_LINEAR,
borderMode=cv2.BORDER_REFLECT)Image Filtering
Average Blur (Box Filter)
Average blur is the simplest low-pass filter — it replaces each pixel with the mean of its neighborhood. Kernels must be odd and positive. Larger kernels blur more but also destroy detail faster. Use it when you specifically want uniform weighting; for natural images Gaussian blur is almost always a better choice.
import cv2
img = cv2.imread("photo.jpg")
# 3x3 box blur
blur3 = cv2.blur(img, (3, 3))
# larger kernel = more blur
blur9 = cv2.blur(img, (9, 9))
# boxFilter with normalized=False gives the sum (not average)
sum_ = cv2.boxFilter(img, -1, (5, 5), normalize=False)
# normalized=True is equivalent to cv2.blur
avg = cv2.boxFilter(img, -1, (5, 5), normalize=True)Gaussian Blur
Gaussian blur is the default smoothing kernel — it weights neighbors by a Gaussian, preserving edges better than a box filter. If you pass ksize=(0,0) and sigmaX, OpenCV auto-computes the kernel size. Gaussian blur is a required pre-step before Canny, downsampling, and Laplacian to suppress noise amplification.
import cv2
img = cv2.imread("photo.jpg")
# GaussianBlur(img, ksize, sigmaX)
g = cv2.GaussianBlur(img, (5, 5), 0)
# specifying sigma explicitly
g2 = cv2.GaussianBlur(img, (5, 5), sigmaX=1.5, sigmaY=1.5)
# odd kernel size required; (0, 0) auto-derives sigma from ksize
g3 = cv2.GaussianBlur(img, (0, 0), sigmaX=2.0)
# Gaussian is the standard pre-processing step before edge detection
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
edges = cv2.Canny(blurred, 50, 150)Median Blur
Median blur is the go-to filter for salt-and-pepper (impulse) noise — it replaces each pixel with the median of its neighborhood, so isolated outliers are removed rather than spread. Unlike Gaussian, it preserves edges well. The kernel size must be odd. It's slower than Gaussian/box because sorting is involved.
import cv2
img = cv2.imread("noisy.jpg")
# median blur — great for salt-and-pepper noise
m = cv2.medianBlur(img, 5)
# larger kernel removes more noise but blurs edges
m2 = cv2.medianBlur(img, 7)
# compared to Gaussian on the same image
g = cv2.GaussianBlur(img, (5, 5), 0)Bilateral Filter
Bilateral filtering smooths flat regions while keeping sharp edges — it's a non-linear, edge-preserving filter that weighs neighbors by BOTH spatial proximity and color similarity. It's much slower than Gaussian (often 10x). Repeat-application is the classic 'cartoonifier' preprocessing step; large sigmaColor values blend across more colors.
import cv2
img = cv2.imread("photo.jpg")
# bilateralFilter(img, d, sigmaColor, sigmaSpace)
# d = neighborhood diameter
# sigmaColor = how much colors can differ and still blend
# sigmaSpace = how far pixels influence each other (spatial)
b = cv2.bilateralFilter(img, d=9, sigmaColor=75, sigmaSpace=75)
# strong edge-preserving smoothing (cartoon effect)
b_strong = cv2.bilateralFilter(img, d=15, sigmaColor=150, sigmaSpace=150)
# repeat several times for a "cartoon / oil painting" look
cartoon = img.copy()
for _ in range(7):
cartoon = cv2.bilateralFilter(cartoon, d=9, sigmaColor=75, sigmaSpace=75)Sharpening (Kernel)
Sharpening kernels emphasize high frequencies — but they also amplify noise, so apply them after denoising. The classic 3x3 sharpening kernel sums to 1 (preserves brightness); a sum > 1 brightens. The unsharp-mask technique (subtract a blurred copy) is more controllable and is what most photo editors actually use.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
# sharpening kernel: center boosts, neighbors subtracted
kernel = np.array([[ 0, -1, 0],
[-1, 5, -1],
[ 0, -1, 0]], dtype=np.float32)
sharp = cv2.filter2D(img, -1, kernel)
# stronger sharpening
kernel_strong = np.array([[-1, -1, -1],
[-1, 9, -1],
[-1, -1, -1]], dtype=np.float32)
sharp2 = cv2.filter2D(img, -1, kernel_strong)
# unsharp mask: image + (image - blurred) * amount
blur = cv2.GaussianBlur(img, (0, 0), 3)
unsharp = cv2.addWeighted(img, 1.5, blur, -0.5, 0)Custom Kernel (filter2D)
filter2D applies an arbitrary correlation kernel — ddepth=-1 keeps the source depth. Kernels can be float for negative coefficients; if the result goes negative, switch ddepth=cv2.CV_16S or CV_32F then convertScaleAbs back. Separable kernels (outer product of 1D kernels) are much faster via sepFilter2D.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
# emboss kernel
emboss = np.array([[-2, -1, 0],
[-1, 1, 1],
[ 0, 1, 2]], dtype=np.float32)
out = cv2.filter2D(img, -1, emboss)
# edge enhance (Laplacian-like)
edge = np.array([[ 0, 1, 0],
[ 1, -4, 1],
[ 0, 1, 0]], dtype=np.float32)
edges = cv2.filter2D(img, -1, edge)
# box blur done manually
box = np.ones((5, 5), dtype=np.float32) / 25.0
blurred = cv2.filter2D(img, -1, box)
# separable filter (Gaussian) — faster than 2D
kx = cv2.getGaussianKernel(5, 1.0) # 5x1
ky = cv2.getGaussianKernel(5, 1.0) # 5x1
g2d = kx @ ky.T # 5x5
out2 = cv2.filter2D(img, -1, g2d)Morphological Operations
Erosion & Dilation
Erosion shrinks foreground (white) regions and removes small noise specks; dilation grows them and fills small holes. The kernel shape matters: RECT for general use, ELLIPSE for rounder features, CROSS for thinning/connecting along axes. iterations=N repeats the operation N times, equivalent to using a larger kernel.
import cv2
import numpy as np
img = cv2.imread("shape.png", cv2.IMREAD_GRAYSCALE)
_, binary = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)
kernel = np.ones((5, 5), np.uint8)
# erosion: shrinks white regions (removes small noise)
eroded = cv2.erode(binary, kernel, iterations=1)
# dilation: grows white regions (fills small holes)
dilated = cv2.dilate(binary, kernel, iterations=1)
# custom kernel shapes
rect = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))
cross = cv2.getStructuringElement(cv2.MORPH_CROSS, (5, 5))
ellipse = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))Opening & Closing
Opening (erode→dilate) removes small bright specks without changing the size of larger objects. Closing (dilate→erode) fills small dark holes. They're idempotent — applying twice is the same as once. The classic denoise pattern is close-then-open (or open-then-close) to handle both kinds of defects.
import cv2
import numpy as np
img = cv2.imread("noisy.png", cv2.IMREAD_GRAYSCALE)
_, binary = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)
kernel = np.ones((5, 5), np.uint8)
# opening = erode then dilate -> removes small foreground noise
opened = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel)
# closing = dilate then erode -> fills small holes in foreground
closed = cv2.morphologyEx(binary, cv2.MORPH_CLOSE, kernel)
# typical denoise pipeline: open then close
clean = cv2.morphologyEx(binary, cv2.MORPH_CLOSE, kernel)
clean = cv2.morphologyEx(clean, cv2.MORPH_OPEN, kernel)Morphological Gradient
Morphological gradient (dilation - erosion) produces a thick outline of object boundaries — useful as a quick edge detector or as a mask for segmentation. Unlike Canny it gives filled, wide edges, so it's good for highlighting regions rather than measuring exact edge positions.
import cv2
import numpy as np
img = cv2.imread("shape.png", cv2.IMREAD_GRAYSCALE)
_, binary = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)
kernel = np.ones((5, 5), np.uint8)
# gradient = dilation - erosion -> outlines of objects
gradient = cv2.morphologyEx(binary, cv2.MORPH_GRADIENT, kernel)
# on a grayscale image (works too)
gray_grad = cv2.morphologyEx(img, cv2.MORPH_GRADIENT, kernel)Top Hat & Black Hat
Top-hat extracts small bright structures smaller than the kernel (e.g. text on a dark document); black-hat extracts small dark structures. Together they're used for shadow/highlight extraction and as a fast uneven-illumination corrector (the divide-by-background trick is common in document scanners).
import cv2
import numpy as np
img = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
kernel = np.ones((15, 15), np.uint8)
# top hat = image - opening -> small bright features on dark bg
tophat = cv2.morphologyEx(img, cv2.MORPH_TOPHAT, kernel)
# black hat = closing - image -> small dark features on bright bg
blackhat = cv2.morphologyEx(img, cv2.MORPH_BLACKHAT, kernel)
# use case: correct uneven illumination
# background = large closing; divide image by background
bg = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel)
corrected = cv2.divide(img, bg, scale=255)Skeletonization
Morphological skeletonization thins a binary shape to a 1-pixel-wide medial axis — useful for OCR, fingerprint, and vessel analysis. The iterative erode-subtract-open loop runs until the image is empty. For production use, scikit-image's medial_axis or thinning is faster and gives cleaner results.
import cv2
import numpy as np
img = cv2.imread("shape.png", cv2.IMREAD_GRAYSCALE)
_, img = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)
# skeleton via iterative morphological opening
kernel = np.ones((3, 3), np.uint8)
skeleton = np.zeros_like(img)
while True:
eroded = cv2.erode(img, kernel)
opened = cv2.morphologyEx(eroded, cv2.MORPH_OPEN, kernel)
temp = cv2.subtract(eroded, opened)
skeleton = cv2.bitwise_or(skeleton, temp)
img = eroded
if cv2.countNonZero(img) == 0:
breakHit-or-Miss
Hit-or-miss is the morphological pattern matcher — it finds exact configurations of foreground and background pixels, useful for detecting specific shapes like corners or T-junctions. The kernel uses 1 for required foreground, -1 for required background, 0 for don't-care. It's niche but unbeatable for exact pattern location.
import cv2
import numpy as np
img = cv2.imread("pattern.png", cv2.IMREAD_GRAYSCALE)
_, binary = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)
# hit-or-miss finds exact patterns
# kernel encodes foreground (1), background (-1), don't-care (0)
kernel = np.array([[ 0, 1, -1],
[ 1, -1, -1],
[ 1, 1, 0]], dtype=np.int32)
hits = cv2.morphologyEx(binary, cv2.MORPH_HITMISS, kernel)
# count matching locations
ys, xs = np.where(hits > 0)
print(f"found {len(xs)} matches")Edge Detection
Canny Edge Detector
Canny is the most-used edge detector: Gaussian smoothing → Sobel gradients → non-maximum suppression → hysteresis (two thresholds). Pixels above the high threshold are edges; those between low and high are edges only if connected to a strong edge. The auto-threshold trick (median × 0.7/1.3) works well for varied images.
import cv2
img = cv2.imread("photo.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
# Canny(image, threshold1, threshold2, apertureSize, L2gradient)
edges = cv2.Canny(blurred, 50, 150)
# auto-tuned thresholds from the median intensity
v = np.median(blurred)
lower = int(max(0, 0.7 * v))
upper = int(min(255, 1.3 * v))
auto = cv2.Canny(blurred, lower, upper)
# L2 gradient (more accurate, slower)
edges_l2 = cv2.Canny(blurred, 50, 150, L2gradient=True)Sobel Operator
Sobel computes the first derivative — use ddepth=cv2.CV_64F (or CV_16S) to capture negative gradients, otherwise uint8 overflow turns dark-to-bright transitions invisible. ksize=3 is standard; ksize=5 gives smoother but wider edges. The magnitude (sqrt(sx²+sy²)) is the basis of Canny and many feature detectors.
import cv2
import numpy as np
img = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
# Sobel x (vertical edges)
sx = cv2.Sobel(img, cv2.CV_64F, 1, 0, ksize=3)
# Sobel y (horizontal edges)
sy = cv2.Sobel(img, cv2.CV_64F, 0, 1, ksize=3)
# gradient magnitude
mag = np.sqrt(sx**2 + sy**2)
mag = np.uint8(255 * mag / mag.max())
# gradient direction (for non-max suppression)
angle = np.arctan2(sy, sx)
# convert back to visible uint8
sx_abs = cv2.convertScaleAbs(sx)
sy_abs = cv2.convertScaleAbs(sy)Laplacian
Laplacian is the second derivative — it's very sensitive to noise, so always blur first. It detects zero-crossings (sign changes), giving thin, precise edges but no direction. Most often used in Marr-Hildreth edge detection and as the 'L' channel input in image sharpening (unsharp mask). Use CV_64F because the second derivative swings hard negative.
import cv2
import numpy as np
img = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
blurred = cv2.GaussianBlur(img, (3, 3), 0)
# Laplacian: second derivative in both directions
lap = cv2.Laplacian(blurred, cv2.CV_64F)
lap_abs = cv2.convertScaleAbs(lap)
# zero-crossing detection (sharper edges)
# Manual Laplacian kernel
kernel = np.array([[0, 1, 0],
[1, -4, 1],
[0, 1, 0]], dtype=np.float32)
edges = cv2.filter2D(blurred, -1, kernel)Scharr Filter
Scharr is a Sobel variant designed for 3x3 windows — when you must use a small kernel, it gives more accurate gradient direction than Sobel ksize=3. For larger kernels (5x5, 7x7) Sobel is fine. cv2.Sobel with ksize=-1 is equivalent to Scharr. Scharr is preferred when gradient direction matters (HOG, edge orientation).
import cv2
import numpy as np
img = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
# Scharr: a Sobel variant optimized for 3x3 kernels
# better rotational symmetry than 3x3 Sobel
sx = cv2.Scharr(img, cv2.CV_64F, 1, 0)
sy = cv2.Scharr(img, cv2.CV_64F, 0, 1)
mag = cv2.magnitude(sx, sy)
mag_u8 = cv2.convertScaleAbs(mag)
# equivalent to Sobel with ksize=-1
sx2 = cv2.Sobel(img, cv2.CV_64F, 1, 0, ksize=-1)Edge Detection Pipeline
The grayscale → denoise → Canny → dilate → findContours pipeline covers a huge fraction of practical edge tasks. Dilation before contour-finding closes broken edges that would otherwise split one object into many. Tuning the Canny thresholds and the denoise kernel size is where most of the engineering effort goes.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
# 1. grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# 2. denoise (Gaussian or bilateral)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
# 3. auto-thresholded Canny
v = np.median(blurred)
edges = cv2.Canny(blurred, int(0.7 * v), int(1.3 * v))
# 4. dilate to close gaps, then find contours
edges = cv2.dilate(edges, np.ones((3, 3), np.uint8), iterations=1)
contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
# 5. filter by area
out = img.copy()
for c in contours:
if cv2.contourArea(c) > 100:
cv2.drawContours(out, [c], -1, (0, 255, 0), 2)
cv2.imshow("edges", edges)
cv2.imshow("contours", out)
cv2.waitKey(0)Contours
Find Contours
findContours mutates the source image in OpenCV 3.x — pass a copy if you need the binary afterward. RETR_EXTERNAL returns only the outermost contours (fastest); RETR_TREE gives the full hierarchy for nested shapes (holes inside objects). CHAIN_APPROX_SIMPLE discards redundant points along straight lines, saving memory.
import cv2
img = cv2.imread("shapes.png")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, binary = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)
# findContours(image, mode, method)
# mode: RETR_EXTERNAL (outer only), RETR_LIST, RETR_TREE, RETR_CCOMP
# method: CHAIN_APPROX_SIMPLE compresses horizontal/vertical/diagonal segments
contours, hierarchy = cv2.findContours(binary, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
print(f"found {len(contours)} contours")
# hierarchy is shape (1, N, 4): [next, prev, child, parent]
# OpenCV 3 returns (image, contours, hierarchy)
# OpenCV 4 returns (contours, hierarchy)Draw Contours
drawContours(image, contours, contourIdx, color, thickness): contourIdx=-1 draws all; a specific index draws one. Thickness=-1 fills the contour. Drawing onto a blank image is the standard way to build a contour mask for downstream operations like mean color or masked histogram.
import cv2
img = cv2.imread("shapes.png")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, binary = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)
contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
# draw all contours (-1), color, thickness
cv2.drawContours(img, contours, -1, (0, 255, 0), 2)
# draw a single contour by index
cv2.drawContours(img, contours, 0, (0, 0, 255), 3)
# filled contour (thickness = -1)
cv2.drawContours(img, contours, -1, (255, 0, 0), -1)
# draw on a blank canvas (mask)
mask = np.zeros_like(gray)
cv2.drawContours(mask, contours, -1, 255, -1)Contour Properties
contourArea is in pixels and ignores any holes (use the contour with RETR_CCOMP hierarchy for net area). The moments dictionary gives the centroid (m10/m00, m01/m00) — guard against division by zero if m00 is 0 (degenerate contours). Aspect ratio and extent are the simplest shape descriptors for shape classification.
import cv2
img = cv2.imread("shapes.png")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, binary = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)
contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
for c in contours:
area = cv2.contourArea(c) # area in pixels
peri = cv2.arcLength(c, True) # perimeter (closed=True)
M = cv2.moments(c) # spatial moments
cx = int(M['m10'] / M['m00']) # centroid x
cy = int(M['m01'] / M['m00']) # centroid y
# bounding box
x, y, w, h = cv2.boundingRect(c)
aspect = w / float(h) # aspect ratio
extent = area / float(w * h) # filled area ratio
print(f"area={area:.0f} peri={peri:.1f} center=({cx},{cy}) aspect={aspect:.2f}")Bounding Rectangles
boundingRect gives an axis-aligned box (fast, simple); minAreaRect gives the smallest rotated rectangle (useful for measuring an object's true orientation/size). boxPoints converts the rotated rect to 4 corner points you can pass to drawContours. minEnclosingCircle is the smallest circle containing the contour.
import cv2
img = cv2.imread("shapes.png")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, binary = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)
contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
for c in contours:
# axis-aligned bounding rectangle
x, y, w, h = cv2.boundingRect(c)
cv2.rectangle(img, (x, y), (x+w, y+h), (0, 255, 0), 2)
# minimum area (rotated) rectangle
rect = cv2.minAreaRect(c) # ((cx, cy), (w, h), angle)
box = cv2.boxPoints(rect) # 4 corner points
box = np.int0(box)
cv2.drawContours(img, [box], 0, (0, 0, 255), 2)
# minimum enclosing circle
(cx, cy), r = cv2.minEnclosingCircle(c)
cv2.circle(img, (int(cx), int(cy)), int(r), (255, 0, 0), 2)Contour Approximation
approxPolyDP simplifies a contour to a polygon with fewer vertices using the Douglas-Peucker algorithm. The epsilon (typically 1-5% of perimeter) controls tolerance: smaller keeps more detail. Counting vertices is the classic hand-rolled shape classifier — triangles=3, rectangles=4, circles=many.
import cv2
img = cv2.imread("shapes.png")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, binary = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)
contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
for c in contours:
peri = cv2.arcLength(c, True)
# approxPolyDP: simplify the contour to fewer vertices
eps = 0.04 * peri # 4% of perimeter
approx = cv2.approxPolyDP(c, eps, True)
n = len(approx)
if n == 3:
label = "triangle"
elif n == 4:
label = "rectangle"
elif n == 5:
label = "pentagon"
else:
label = f"{n}-gon"
cv2.drawContours(img, [approx], -1, (0, 255, 0), 2)
cv2.putText(img, label, tuple(approx[0][0]),
cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)Convex Hull & Defects
The convex hull is the smallest convex shape enclosing the contour — useful for measuring object compactness and for hand/finger detection via convexity defects. Each defect's depth measures how 'inward' the contour dips relative to the hull; deep defects on a hand silhouette typically correspond to gaps between fingers.
import cv2
img = cv2.imread("hand.png")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, binary = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)
contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
c = max(contours, key=cv2.contourArea)
# convex hull (smallest convex polygon enclosing the contour)
hull = cv2.convexHull(c)
cv2.drawContours(img, [hull], -1, (0, 255, 0), 2)
# convexity defects (gaps between contour and hull) — needs returnPoints=False
hull_idx = cv2.convexHull(c, returnPoints=False)
defects = cv2.convexityDefects(c, hull_idx)
# defects[i] = (start, end, far, depth) — depth in 1/256 of peri
# is the contour convex?
is_convex = cv2.isContourConvex(c)Thresholding
Basic Threshold
THRESH_BINARY: above->255, below->0. THRESH_TRUNC: above->capped at thresh, below->unchanged. THRESH_TOZERO: above->unchanged, below->0. The returned ret equals the input threshold for plain thresholding but becomes meaningful for Otsu/Triangle. Pick the type by how you want out-of-range pixels handled.
import cv2
img = cv2.imread("doc.png", cv2.IMREAD_GRAYSCALE)
# threshold(src, thresh, maxval, type)
# returns (retval, binary_image)
ret, binary = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)
# threshold types
_, inv = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY_INV)
_, trunc = cv2.threshold(img, 127, 255, cv2.THRESH_TRUNC)
_, tozero = cv2.threshold(img, 127, 255, cv2.THRESH_TOZERO)
_, tozero_inv = cv2.threshold(img, 127, 255, cv2.THRESH_TOZERO_INV)
# ret is the threshold actually used (useful with Otsu)Adaptive Threshold
Adaptive thresholding computes a different threshold for each pixel based on its local neighborhood — essential for images with uneven lighting (e.g. photographs of documents). blockSize must be odd (typically 11-31) and larger than the feature size. C subtracts a constant from the local mean; raise it to suppress low-contrast noise.
import cv2
img = cv2.imread("doc.png", cv2.IMREAD_GRAYSCALE)
img = cv2.medianBlur(img, 5)
# adaptiveThreshold(src, maxValue, adaptiveMethod, thresholdType, blockSize, C)
# adaptiveMethod:
# ADAPTIVE_THRESH_MEAN_C -> mean of neighborhood minus C
# ADAPTIVE_THRESH_GAUSSIAN_C -> weighted mean minus C
adap_mean = cv2.adaptiveThreshold(img, 255,
cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 11, 2)
adap_gauss = cv2.adaptiveThreshold(img, 255,
cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2)Otsu's Method
Otsu's method finds the optimal global threshold by maximizing the variance between foreground and background classes — pass thresh=0 and add the THRESH_OTSU flag. Always blur first; Otsu is sensitive to histogram noise. The Triangle method works better on images with a single dominant peak (e.g. microscopy). The returned ret is the computed threshold.
import cv2
img = cv2.imread("doc.png", cv2.IMREAD_GRAYSCALE)
blurred = cv2.GaussianBlur(img, (5, 5), 0)
# Otsu: automatically picks the threshold that maximizes between-class variance
ret, otsu = cv2.threshold(blurred, 0, 255,
cv2.THRESH_BINARY + cv2.THRESH_OTSU)
print(f"Otsu threshold: {ret}")
# Triangle method (good for skewed histograms / single peak)
ret_t, tri = cv2.threshold(blurred, 0, 255,
cv2.THRESH_BINARY + cv2.THRESH_TRIANGLE)
print(f"Triangle threshold: {ret_t}")Color Threshold (inRange)
inRange is the standard color-segmentation tool — convert to HSV first because it separates color (hue) from brightness, making thresholds robust to lighting changes. Red hue wraps around 0/180, so you need two masks OR'd together. Always blur before inRange to reduce noise speckles in the mask.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# detect a range of green pixels
lower = np.array([35, 50, 50])
upper = np.array([85, 255, 255])
mask = cv2.inRange(hsv, lower, upper)
# apply mask to the original
result = cv2.bitwise_and(img, img, mask=mask)
# hue wraps around for red (0-10 and 170-180)
mask1 = cv2.inRange(hsv, np.array([0, 50, 50]), np.array([10, 255, 255]))
mask2 = cv2.inRange(hsv, np.array([170, 50, 50]), np.array([180, 255, 255]))
red_mask = cv2.bitwise_or(mask1, mask2)Threshold Pipeline
The denoise → threshold → morphology pipeline handles most binarization tasks. Pick Otsu for even lighting, adaptive for uneven. The morphological close-then-open removes both interior holes and exterior specks. Always inspect the binary mask — threshold parameters are almost always image-specific and need tuning per dataset.
import cv2
import numpy as np
img = cv2.imread("doc.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# 1. denoise
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
# 2. choose a method
# - flat lighting -> Otsu
# - uneven lighting -> adaptive
# - document scan -> adaptive Gaussian
if lighting == "flat":
_, binary = cv2.threshold(blurred, 0, 255,
cv2.THRESH_BINARY + cv2.THRESH_OTSU)
else:
binary = cv2.adaptiveThreshold(blurred, 255,
cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 21, 5)
# 3. morphology cleanup
kernel = np.ones((3, 3), np.uint8)
binary = cv2.morphologyEx(binary, cv2.MORPH_CLOSE, kernel)
binary = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel)Binarization for OCR
For OCR, adaptive Gaussian almost always beats global Otsu on photographed documents because of uneven lighting. Match blockSize to the character height (rule of thumb: 1-2× the stroke width × 4). Most OCR engines (Tesseract) prefer black text on white background — invert if your binary is the opposite.
import cv2
img = cv2.imread("text.png", cv2.IMREAD_GRAYSCALE)
# adaptive Gaussian with a block size matched to character height
# (e.g. char ~25 px -> blockSize ~31)
binary = cv2.adaptiveThreshold(img, 255,
cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 31, 10)
# invert if needed (OCR often prefers black-on-white)
binary_inv = cv2.bitwise_not(binary)
# Otsu on a preprocessed image as an alternative
blur = cv2.GaussianBlur(img, (3, 3), 0)
_, otsu = cv2.threshold(blur, 0, 255,
cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)Color Spaces
BGR / RGB / Gray
OpenCV reads images as BGR by default; most other libraries (matplotlib, PIL, scikit-image) use RGB. Forgetting cvtColor is the most common cause of 'red and blue swapped' images. The grayscale conversion uses perceptual weights (Y = 0.299R + 0.587G + 0.114B), not a simple average.
import cv2
bgr = cv2.imread("photo.jpg") # OpenCV default: BGR
# BGR -> RGB (for matplotlib, PIL, etc.)
rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
# BGR -> grayscale (luminance-weighted)
gray = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY)
# RGB -> BGR (reverse)
bgr2 = cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR)
# grayscale -> BGR (replicates channel, color image with no color)
bgr_from_gray = cv2.cvtColor(gray, cv2.COLOR_GRAY2BGR)BGR to HSV
OpenCV's HSV uses H in 0-179 (not 0-360) so it fits in uint8 — a frequent source of off-by-2x bugs when porting code from other libraries. S and V are 0-255. HSV is the go-to space for color-based segmentation because hue is roughly invariant to brightness changes — use it for object tracking by color.
import cv2
import numpy as np
bgr = cv2.imread("photo.jpg")
hsv = cv2.cvtColor(bgr, cv2.COLOR_BGR2HSV)
# split channels
h, s, v = cv2.split(hsv)
# HSV ranges in OpenCV: H: 0-179, S: 0-255, V: 0-255
# (H is halved so it fits in 8-bit — full hue circle is 0-360)
print(h.max(), s.max(), v.max()) # 179 255 255
# back to BGR
bgr2 = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)BGR to LAB
CIELAB approximates human perception — equal numeric changes look roughly equally different, which is why it's used for color-difference metrics (ΔE). The L channel encodes lightness only, so editing L is the cleanest way to brighten an image without shifting colors. OpenCV's LAB uses 0-255 for all channels, not the standard 0-100 for L.
import cv2
bgr = cv2.imread("photo.jpg")
# BGR -> LAB (perceptually uniform color space)
lab = cv2.cvtColor(bgr, cv2.COLOR_BGR2LAB)
L, A, B = cv2.split(lab)
# L: lightness (0-255), A: green<->red, B: blue<->yellow
# useful for separating luminance from color
# LAB is great for color-difference metrics (delta-E)
# and for editing lightness without affecting color
import numpy as np
lab_bright = lab.copy()
lab_bright[:, :, 0] = np.clip(lab_bright[:, :, 0].astype(int) + 30, 0, 255)
bgr_bright = cv2.cvtColor(lab_bright, cv2.COLOR_LAB2BGR)Color Tracking (HSV)
HSV-based color tracking is the simplest object tracker — pick the hue range, threshold, find the largest blob. It fails under poor lighting (low saturation), with similar-colored background objects, and on reflective surfaces. For robust tracking combine with camshift/meanshift (uses a hue histogram backprojection) or a Kalman filter to smooth motion.
import cv2
import numpy as np
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
if not ret:
break
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
# track a blue object
lower = np.array([100, 80, 80])
upper = np.array([130, 255, 255])
mask = cv2.inRange(hsv, lower, upper)
# find the largest blob
contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
if contours:
c = max(contours, key=cv2.contourArea)
if cv2.contourArea(c) > 500:
(x, y), r = cv2.minEnclosingCircle(c)
cv2.circle(frame, (int(x), int(y)), int(r), (0, 255, 255), 2)
cv2.imshow("frame", frame)
cv2.imshow("mask", mask)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()Split & Merge Channels
split/merge are convenient but slower than NumPy indexing — img[:, :, 0] is the blue channel and modifying it is faster than split. Always pass channels to merge in the correct order for your target color space. The 'zero out two channels' trick is the classic way to visualize the contribution of a single color channel.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
# split into B, G, R channels (each is a 2-D array)
b, g, r = cv2.split(img)
# merge channels back (order matters!)
merged = cv2.merge([b, g, r])
# zero out one channel to visualize
zeros = np.zeros_like(b)
blue_only = cv2.merge([b, zeros, zeros])
green_only = cv2.merge([zeros, g, zeros])
red_only = cv2.merge([zeros, zeros, r])
# swap channels (e.g. fake RGB->BGR by swapping R and B)
swapped = cv2.merge([r, g, b])Other Color Spaces
YCrCb separates luminance (Y) from chroma (Cr, Cb) — the basis of JPEG and MPEG compression, which discards chroma detail because humans are less sensitive to color than brightness. HLS differs from HSV in that white is at L=1 regardless of saturation, making HLS better for some shading effects. Pick the space that makes your operation invariant to what you don't care about.
import cv2
bgr = cv2.imread("photo.jpg")
# YCrCb (used in JPEG compression)
ycrcb = cv2.cvtColor(bgr, cv2.COLOR_BGR2YCrCb)
# HLS (Hue, Lightness, Saturation — alternative to HSV)
hls = cv2.cvtColor(bgr, cv2.COLOR_BGR2HLS)
# YUV (used in video encoding)
yuv = cv2.cvtColor(bgr, cv2.COLOR_BGR2YUV)
# grayscale -> all spaces start from BGR, so chain conversions
gray_from_ycrcb = cv2.cvtColor(ycrcb, cv2.COLOR_YCrCb2BGR)
gray = cv2.cvtColor(gray_from_ycrcb, cv2.COLOR_BGR2GRAY)Histograms
Calculate Histogram
calcHist takes a list of images, a list of channels, an optional mask (None = whole image), the bin count, and the value range. For multi-dimensional histograms (e.g. 2-D hue-saturation) pass two channels and two bin sizes — used in histogram backprojection for color tracking. Each channel's histogram is computed independently.
import cv2
import numpy as np
img = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
# calcHist(images, channels, mask, histSize, ranges)
hist = cv2.calcHist([img], [0], None, [256], [0, 256])
print(hist.shape) # (256, 1)
# normalize to 0-1
hist /= hist.sum()
# multi-channel histogram (color)
bgr = cv2.imread("photo.jpg")
colors = ('b', 'g', 'r')
for i, col in enumerate(colors):
h = cv2.calcHist([bgr], [i], None, [256], [0, 256])Plot Histogram
Matplotlib is the easiest way to visualize histograms — but remember to convert BGR to RGB for the legend colors, or your plot colors won't match the channel labels. A histogram skewed to the left is a dark image, to the right a bright one; a narrow spike means low contrast. Use this to diagnose exposure and white-balance issues.
import cv2
import numpy as np
import matplotlib.pyplot as plt
img = cv2.imread("photo.jpg")
colors = ('b', 'g', 'r')
for i, col in enumerate(colors):
hist = cv2.calcHist([img], [i], None, [256], [0, 256])
plt.plot(hist, color=col)
plt.xlim([0, 256])
plt.title("Color Histogram")
plt.xlabel("Pixel value")
plt.ylabel("Frequency")
plt.show()Histogram Equalization
Histogram equalization stretches pixel intensities to use the full 0-255 range — great for low-contrast images but can blow out highlights and amplify noise. NEVER equalize all three BGR channels directly — it shifts colors. Always convert to a luminance/chrominance space (YCrCb, LAB, HSV), equalize the luminance, and convert back.
import cv2
img = cv2.imread("dark.jpg", cv2.IMREAD_GRAYSCALE)
# global histogram equalization
eq = cv2.equalizeHist(img)
# equalization on color images: convert to YCrCb, equalize Y only
bgr = cv2.imread("color.jpg")
ycrcb = cv2.cvtColor(bgr, cv2.COLOR_BGR2YCrCb)
ycrcb[:, :, 0] = cv2.equalizeHist(ycrcb[:, :, 0])
eq_color = cv2.cvtColor(ycrcb, cv2.COLOR_YCrCb2BGR)CLAHE (Adaptive Equalization)
CLAHE is the production-quality alternative to global equalization — it operates on small tiles so it works on images with uneven lighting, and clipLimit caps the contrast amplification to avoid blowing up noise. tileGridSize of 8x8 is the default; smaller tiles give more local adaptation. The go-to choice for medical imaging and document enhancement.
import cv2
img = cv2.imread("dark.jpg", cv2.IMREAD_GRAYSCALE)
# CLAHE: Contrast Limited Adaptive Histogram Equalization
# equalizes small tiles, limiting amplification of noise
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
enhanced = clahe.apply(img)
# on color: equalize the L channel of LAB
bgr = cv2.imread("color.jpg")
lab = cv2.cvtColor(bgr, cv2.COLOR_BGR2LAB)
clahe = cv2.createCLAHE(clipLimit=3.0, tileGridSize=(8, 8))
lab[:, :, 0] = clahe.apply(lab[:, :, 0])
enhanced_color = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)Histogram Backprojection
Histogram backprojection answers 'where do pixels look like my ROI's color distribution?'. Build a 2-D hue-saturation histogram from a sample, then backproject onto a target image to get a probability map. This is the basis of meanShift/camShift tracking — far more robust than a fixed HSV range because it learns the target's actual color distribution.
import cv2
import numpy as np
img = cv2.imread("scene.jpg")
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# take a small ROI of the target color (the object to track)
roi = hsv[100:150, 100:150]
roi_hist = cv2.calcHist([roi], [0, 1], None, [180, 256], [0, 180, 0, 256])
cv2.normalize(roi_hist, roi_hist, 0, 255, cv2.NORM_MINMAX)
# backproject: each pixel = probability of belonging to the ROI's color
backproj = cv2.calcBackProject([hsv], [0, 1], roi_hist, [0, 180, 0, 256], 1)
# denoise and threshold
disc = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
backproj = cv2.filter2D(backproj, -1, disc)
_, mask = cv2.threshold(backproj, 50, 255, cv2.THRESH_BINARY)
result = cv2.bitwise_and(img, img, mask=mask)Feature Detection
Harris Corner Detector
Harris detects corners by checking whether moving a small window in any direction causes a large intensity change — corners change in all directions, edges in one, flat regions in none. The output is a corner-response map; threshold relative to the max (e.g. 0.01 * max). blockSize is the neighborhood for the structure-tensor computation.
import cv2
import numpy as np
img = cv2.imread("chessboard.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# cornerHarris(gray, blockSize, ksize, k)
# blockSize: neighborhood for corner detection
# ksize: Sobel aperture (3, 5, 7)
# k: Harris free parameter (0.04 - 0.06)
dst = cv2.cornerHarris(gray, 2, 3, 0.04)
# dilate to mark corners
dst = cv2.dilate(dst, None)
# threshold and mark in red
img[dst > 0.01 * dst.max()] = [0, 0, 255]Shi-Tomasi (goodFeaturesToTrack)
Shi-Tomasi improves on Harris by using min(eigenvalue1, eigenvalue2) as the score, which gives more stable corners for tracking. maxCorners caps the count; qualityLevel is a fraction of the strongest corner response (0.01 = 1% of best); minDistance enforces spatial separation between detected corners. This is the default input for the Lucas-Kanade optical flow tracker.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# goodFeaturesToTrack returns the N strongest corners
corners = cv2.goodFeaturesToTrack(gray, maxCorners=100,
qualityLevel=0.01,
minDistance=10)
corners = np.int0(corners)
for c in corners:
x, y = c.ravel()
cv2.circle(img, (x, y), 3, (0, 0, 255), -1)SIFT (Scale-Invariant Features)
SIFT finds scale- and rotation-invariant keypoints and a 128-D descriptor for each — robust to lighting, viewpoint, and partial occlusion. Patented for years, it became free in OpenCV 4.4. Use SIFT for high-accuracy matching where speed is secondary. DRAW_RICH_KEYPOINTS draws the scale and orientation of each keypoint.
import cv2
img = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
# SIFT was patented; free since OpenCV 4.4 (contrib not required)
sift = cv2.SIFT_create()
# detect keypoints
kp = sift.detect(img, None)
# detect and compute descriptors
kp, des = sift.detectAndCompute(img, None)
print(len(kp), des.shape) # N, (N, 128)
# draw keypoints
out = cv2.drawKeypoints(img, kp, None,
flags=cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)ORB (FAST + BRIEF)
ORB combines the FAST keypoint detector with the BRIEF descriptor — both patent-free and ~10x faster than SIFT, making ORB the standard choice for real-time matching on mobile/embedded. Descriptors are binary (32 bytes vs SIFT's 128 floats), so matching uses Hamming distance. Slightly less robust than SIFT to scale/rotation, but excellent for the cost.
import cv2
img = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
# ORB: free, fast alternative to SIFT
orb = cv2.ORB_create(nfeatures=1000)
kp, des = orb.detectAndCompute(img, None)
print(len(kp), des.shape) # N, (N, 32) — binary descriptors
# draw keypoints
out = cv2.drawKeypoints(img, kp, None, color=(0, 255, 0),
flags=0)
# FAST alone (detector only, no descriptor)
fast = cv2.FastFeatureDetector_create(threshold=20)
kp = fast.detect(img, None)FAST Detector
FAST tests whether a circle of 16 pixels around a candidate has N contiguous pixels all brighter or all darker than the center — extremely fast because it short-circuits. It returns keypoints only, no descriptors (use BRIEF or ORB for descriptors). Lower threshold = more (and noisier) corners; non-max suppression removes duplicates at the local maxima.
import cv2
img = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
# FAST: speed-optimized corner detector (detector only)
fast = cv2.FastFeatureDetector_create(threshold=20,
nonmaxSuppression=True)
kp = fast.detect(img, None)
print(f"{len(kp)} keypoints")
# tune the threshold
fast.setThreshold(30)
print(f"threshold is now {fast.getThreshold()}")
# turn off non-max suppression to get more (clustered) corners
fast.setNonmaxSuppression(False)
kp_all = fast.detect(img, None)MSER (Maximally Stable Extremal Regions)
MSER finds regions that stay stable across a range of intensity thresholds — ideal for text detection (where letters are blob-like), license plates, and scene text. It handles uneven lighting well because each region is detected at its own threshold. Filter results by aspect ratio and size to remove obvious non-text.
import cv2
img = cv2.imread("text.jpg", cv2.IMREAD_GRAYSCALE)
# MSER: detects stable connected regions across thresholds
mser = cv2.MSER_create()
regions, _ = mser.detectRegions(img)
# regions is a list of point arrays
out = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
for pts in regions:
x, y, w, h = cv2.boundingRect(pts)
if 5 < h < 50: # filter by likely text size
cv2.rectangle(out, (x, y), (x+w, y+h), (0, 255, 0), 1)Feature Matching
Brute Force Matching
Brute-force matching tries every descriptor pair — exact but O(N*M). For ORB's binary descriptors use NORM_HAMMING; for SIFT/SURF float descriptors use NORM_L2. crossCheck=True keeps only mutual best matches (A's best is B and vice versa), which removes many false positives. Each match has a .distance — smaller is better.
import cv2
img1 = cv2.imread("box.png", cv2.IMREAD_GRAYSCALE)
img2 = cv2.imread("scene.png", cv2.IMREAD_GRAYSCALE)
orb = cv2.ORB_create()
kp1, des1 = orb.detectAndCompute(img1, None)
kp2, des2 = orb.detectAndCompute(img2, None)
# for ORB (binary descriptors) use Hamming distance
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des1, des2)
# sort by distance (best first)
matches = sorted(matches, key=lambda m: m.distance)
# draw top 20
out = cv2.drawMatches(img1, kp1, img2, kp2, matches[:20], None,
flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)FLANN Matching
FLANN (Fast Library for Approximate Nearest Neighbors) builds a KD-tree (for SIFT/SURF) or LSH (for ORB) to find approximate matches — much faster than brute-force on large descriptor sets at the cost of occasional misses. The trees and checks parameters trade accuracy for speed. Use FLANN when you have thousands of features; brute-force is fine for small sets.
import cv2
import numpy as np
img1 = cv2.imread("box.png", cv2.IMREAD_GRAYSCALE)
img2 = cv2.imread("scene.png", cv2.IMREAD_GRAYSCALE)
sift = cv2.SIFT_create()
kp1, des1 = sift.detectAndCompute(img1, None)
kp2, des2 = sift.detectAndCompute(img2, None)
# FLANN parameters for SIFT (float descriptors)
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5)
search_params = dict(checks=50)
flann = cv2.FlannBasedMatcher(index_params, search_params)
matches = flann.knnMatch(des1, des2, k=2)KNN Matching & Ratio Test
Lowe's ratio test is the standard filter for feature matching: keep a match only if the best match is significantly closer than the second-best (typical threshold 0.7-0.8). The intuition: if both candidates are similarly close, the match is ambiguous and probably wrong. This single test removes the vast majority of false positives.
import cv2
img1 = cv2.imread("box.png", cv2.IMREAD_GRAYSCALE)
img2 = cv2.imread("scene.png", cv2.IMREAD_GRAYSCALE)
sift = cv2.SIFT_create()
kp1, des1 = sift.detectAndCompute(img1, None)
kp2, des2 = sift.detectAndCompute(img2, None)
bf = cv2.BFMatcher(cv2.NORM_L2)
matches = bf.knnMatch(des1, des2, k=2) # 2 nearest neighbors
# Lowe's ratio test: keep only unambiguous matches
good = []
for m, n in matches:
if m.distance < 0.75 * n.distance:
good.append(m)
out = cv2.drawMatches(img1, kp1, img2, kp2, good, None,
flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)Homography (RANSAC)
findHomography with RANSAC computes the perspective transform that maps src to dst while ignoring outliers — the inlier mask tells you which matches agree with the model. The reprojection threshold (5.0) is in pixels; lower = stricter. You need at least 4 good matches. Use this to locate a known object in a scene or to align overlapping photos.
import cv2
import numpy as np
img1 = cv2.imread("box.png", cv2.IMREAD_GRAYSCALE)
img2 = cv2.imread("scene.png", cv2.IMREAD_GRAYSCALE)
sift = cv2.SIFT_create()
kp1, des1 = sift.detectAndCompute(img1, None)
kp2, des2 = sift.detectAndCompute(img2, None)
bf = cv2.BFMatcher(cv2.NORM_L2)
matches = bf.knnMatch(des1, des2, k=2)
good = [m for m, n in matches if m.distance < 0.75 * n.distance]
# build point correspondences
src = np.float32([kp1[m.queryIdx].pt for m in good]).reshape(-1, 1, 2)
dst = np.float32([kp2[m.trainIdx].pt for m in good]).reshape(-1, 1, 2)
# RANSAC homography
H, mask = cv2.findHomography(src, dst, cv2.RANSAC, 5.0)
inliers = mask.ravel().tolist()
print(f"{sum(inliers)} inliers / {len(good)} matches")Object Localization
This is the classic object-localization pipeline: detect features in both images, match them, fit a homography with RANSAC, then project the object's bounding box into the scene. It works for planar objects or far-away 3D objects (where perspective dominates). It fails on textureless, symmetric, or highly occluded objects — for those use deep-learning detectors.
import cv2
import numpy as np
img1 = cv2.imread("box.png", cv2.IMREAD_GRAYSCALE)
img2 = cv2.imread("scene.png", cv2.IMREAD_GRAYSCALE)
sift = cv2.SIFT_create()
kp1, des1 = sift.detectAndCompute(img1, None)
kp2, des2 = sift.detectAndCompute(img2, None)
bf = cv2.BFMatcher(cv2.NORM_L2)
good = [m for m, n in bf.knnMatch(des1, des2, k=2)
if m.distance < 0.75 * n.distance]
src = np.float32([kp1[m.queryIdx].pt for m in good]).reshape(-1, 1, 2)
dst = np.float32([kp2[m.trainIdx].pt for m in good]).reshape(-1, 1, 2)
H, _ = cv2.findHomography(src, dst, cv2.RANSAC, 5.0)
# project the object's corners into the scene
h, w = img1.shape
corners = np.float32([[0, 0], [w-1, 0], [w-1, h-1], [0, h-1]]).reshape(-1, 1, 2)
scene_corners = cv2.perspectiveTransform(corners, H)
out = cv2.cvtColor(img2, cv2.COLOR_GRAY2BGR)
cv2.polylines(out, [np.int32(scene_corners)], True, (0, 255, 0), 3)Face Detection
Haar Cascade Classifier
Haar cascades are the classic Viola-Jones detector — fast on CPU and good for frontal faces, but less accurate than modern deep-learning detectors (MTCNN, SSD, RetinaFace). scaleFactor (>1) controls multi-scale search — 1.1 = 10% scale step; smaller = slower but more thorough. minNeighbors filters overlapping detections; raise it to reduce false positives.
import cv2
# load a pretrained cascade (shipped with OpenCV)
cascade_path = cv2.data.haarcascades + "haarcascade_frontalface_default.xml"
face_cascade = cv2.CascadeClassifier(cascade_path)
print(face_cascade.empty()) # False = loaded ok
img = cv2.imread("people.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# detectMultiScale(gray, scaleFactor, minNeighbors, minSize, maxSize)
faces = face_cascade.detectMultiScale(
gray, scaleFactor=1.1, minNeighbors=5, minSize=(30, 30))
for (x, y, w, h) in faces:
cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)Detect Faces in Image
Detecting eyes (or smile) inside the face ROI is the standard efficiency trick: searching the whole image for eyes gives many false positives, but inside a confirmed face ROI they're rare. cv2.data.haarcascades points to the bundled XML directory — other cascades (eye, smile, full body, plate) are available. Crop the ROI BEFORE calling the cascade.
import cv2
face_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades + "haarcascade_frontalface_default.xml")
eye_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades + "haarcascade_eye.xml")
img = cv2.imread("people.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.3, 5)
for (fx, fy, fw, fh) in faces:
cv2.rectangle(img, (fx, fy), (fx+fw, fy+fh), (255, 0, 0), 2)
# search for eyes INSIDE the face region (faster, fewer false positives)
roi_gray = gray[fy:fy+fh, fx:fx+fw]
roi_color = img[fy:fy+fh, fx:fx+fw]
eyes = eye_cascade.detectMultiScale(roi_gray)
for (ex, ey, ew, eh) in eyes:
cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 255, 0), 2)Detect Faces in Video
Real-time face detection on webcam is feasible on CPU thanks to Haar cascades' speed. The ROI-blur trick (GaussianBlur on the face region only) is a simple privacy/anonymization technique. To track faces across frames (avoid flicker), associate detections by IoU and smooth with a Kalman filter — cascades alone have no temporal continuity.
import cv2
face_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades + "haarcascade_frontalface_default.xml")
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
if not ret:
break
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.1, 5)
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
# blur the face for privacy
frame[y:y+h, x:x+w] = cv2.GaussianBlur(
frame[y:y+h, x:x+w], (25, 25), 30)
cv2.imshow("face", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()Profile / Side Face
Haar cascades are pose-specific — the frontal cascade misses side faces, so combine it with the profile cascade. Since the profile cascade is trained on one side, detect the other side by flipping the image horizontally and mirroring the resulting coordinates back. This still misses 3/4 views; modern detectors (DNN, MTCNN) handle all poses in one model.
import cv2
profile_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades + "haarcascade_profileface.xml")
img = cv2.imread("people.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# detect left-facing profiles
profiles = profile_cascade.detectMultiScale(gray, 1.1, 5)
for (x, y, w, h) in profiles:
cv2.rectangle(img, (x, y), (x+w, y+h), (0, 0, 255), 2)
# also detect right-facing by flipping the image
flipped = cv2.flip(gray, 1)
profiles_r = profile_cascade.detectMultiScale(flipped, 1.1, 5)
h, w = gray.shape
for (x, y, fw, fh) in profiles_r:
cv2.rectangle(img, (w - x - fw, y), (w - x, y + fh), (255, 0, 0), 2)DNN Face Detection
OpenCV's dnn module can run pretrained Caffe/TensorFlow/ONNX models — SSD and ResNet-based face detectors are far more accurate than Haar cascades and handle all poses in one model. blobFromImage does mean subtraction and scaling in one shot. Detections are normalized [0,1] — multiply by (w, h, w, h) to get pixel boxes. Models: SSD (Caffe), YuNet (ONNX, via cv.FaceDetectorYN).
import cv2
import numpy as np
model = cv2.dnn.readNetFromCaffe(
"deploy.prototxt", "weights.caffemodel")
img = cv2.imread("people.jpg")
h, w = img.shape[:2]
# build a 300x300 blob (mean subtraction + scale)
blob = cv2.dnn.blobFromImage(img, 1.0, (300, 300), (104, 177, 123))
model.setInput(blob)
detections = model.forward() # shape (1, 1, N, 7)
for i in range(detections.shape[2]):
conf = detections[0, 0, i, 2]
if conf > 0.5:
box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
x1, y1, x2, y2 = box.astype(int)
cv2.rectangle(img, (x1, y1), (x2, y2), (0, 255, 0), 2)
cv2.putText(img, f"{conf:.2f}", (x1, y1 - 5),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 1)Object Detection
HOG Descriptor
HOG (Histogram of Oriented Gradients) describes local shape via histograms of edge directions — combined with a linear SVM it's a classic object detector. The pretrained people detector works for full-body pedestrians in upright poses. winStride and scale trade speed for recall. Modern deep detectors (YOLO, SSD) are far more accurate but HOG runs on CPU without a model file.
import cv2
img = cv2.imread("person.jpg", cv2.IMREAD_GRAYSCALE)
# create HOG descriptor with the default detector
hog = cv2.HOGDescriptor()
hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())
# detectMultiScale returns (rectangles, weights)
boxes, weights = hog.detectMultiScale(
img, winStride=(8, 8), padding=(4, 4), scale=1.05)
for (x, y, w, h) in boxes:
cv2.rectangle(img, (x, y), (x+w, y+h), (0, 255, 0), 2)HOG Feature Visualization
The standard HOG window is 64x128 pixels (the INRIA pedestrian size). Parameters: window (64x128), block (16x16), block stride (8x8), cell (8x8), bins (9). Each descriptor is 3780-D. To classify custom objects, extract HOG features for many positive/negative samples and train a linear SVM (scikit-learn), then set the detector with setSVMDetector.
import cv2
img = cv2.imread("person.jpg", cv2.IMREAD_GRAYSCALE)
img = cv2.resize(img, (64, 128)) # standard HOG window
hog = cv2.HOGDescriptor((64, 128), (16, 16), (8, 8), (8, 8), 9)
descriptor = hog.compute(img)
print(descriptor.shape) # (3780, 1)
# 9 bins per cell, 7x15 cells, 4 cells per block -> 3780 features
# this is the input vector to a linear SVM classifierTemplate Matching
Template matching slides the template across the image and computes a similarity score — works only for exact (or near-exact) matches with no rotation or scale change. TM_CCOEFF_NORMED is the most robust method; results are in [-1, 1]. For multiple matches, threshold the response map and apply non-maximum suppression to avoid overlapping detections.
import cv2
import numpy as np
img = cv2.imread("scene.png", cv2.IMREAD_GRAYSCALE)
template = cv2.imread("template.png", cv2.IMREAD_GRAYSCALE)
h, w = template.shape
# matchTemplate returns a correlation map
res = cv2.matchTemplate(img, template, cv2.TM_CCOEFF_NORMED)
# find the best match
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)
top_left = max_loc
bottom_right = (top_left[0] + w, top_left[1] + h)
out = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
cv2.rectangle(out, top_left, bottom_right, (0, 255, 0), 2)
# find all matches above a threshold
loc = np.where(res >= 0.8)
for pt in zip(*loc[::-1]):
cv2.rectangle(out, pt, (pt[0] + w, pt[1] + h), (0, 0, 255), 1)Multi-Scale Template Matching
Multi-scale template matching brute-forces a range of scales — slow but works when the template's size in the image is unknown. It still can't handle rotation; for that you'd need to also sweep over angles (very slow) or use feature matching instead. For production object detection use a trained detector (YOLO, SSD) rather than template matching.
import cv2
import numpy as np
img = cv2.imread("scene.png", cv2.IMREAD_GRAYSCALE)
template = cv2.imread("template.png", cv2.IMREAD_GRAYSCALE)
best = (-1, None, None) # (score, location, scale)
for scale in np.linspace(0.5, 1.5, 20):
resized = cv2.resize(template, None, fx=scale, fy=scale)
rh, rw = resized.shape
if rh > img.shape[0] or rw > img.shape[1]:
continue
res = cv2.matchTemplate(img, resized, cv2.TM_CCOEFF_NORMED)
_, max_val, _, max_loc = cv2.minMaxLoc(res)
if max_val > best[0]:
best = (max_val, max_loc, scale)
score, (x, y), scale = best
h, w = template.shape
out = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
rw, rh = int(w * scale), int(h * scale)
cv2.rectangle(out, (x, y), (x + rw, y + rh), (0, 255, 0), 2)DNN Object Detection (YOLO)
OpenCV's dnn module can run YOLO/SSD/Faster-RCNN exported weights — useful when you can't install a full DL framework. YOLO output parsing is verbose: each detection has center, size, objectness, and per-class scores. Always apply NMSBoxes (non-maximum suppression) to collapse overlapping boxes into one. For real-time use YOLOv4-tiny or swap to ONNX + a faster backend.
import cv2
import numpy as np
# load a YOLO model (weights + cfg)
net = cv2.dnn.readNet("yolov3.weights", "yolov3.cfg")
with open("coco.names") as f:
classes = [line.strip() for line in f.readlines()]
layer_names = net.getLayerNames()
out_layers = [layer_names[i - 1] for i in net.getUnconnectedOutLayers()]
img = cv2.imread("street.jpg")
blob = cv2.dnn.blobFromImage(img, 1/255.0, (416, 416), swapRB=True, crop=False)
net.setInput(blob)
outputs = net.forward(out_layers)
# parse YOLO output: each detection is [cx, cy, w, h, obj, class_scores...]
h, w = img.shape[:2]
boxes, confs, ids = [], [], []
for out in outputs:
for det in out:
scores = det[5:]
cls = int(np.argmax(scores))
conf = scores[cls]
if conf > 0.5:
cx, cy, bw, bh = (det[:4] * [w, h, w, h]).astype(int)
boxes.append([cx - bw//2, cy - bh//2, bw, bh])
confs.append(float(conf))
ids.append(cls)
idxs = cv2.dnn.NMSBoxes(boxes, confs, 0.5, 0.4) # non-max suppressionImage Segmentation
K-Means Clustering
K-means clusters pixels by color — a simple way to posterize or segment an image into K dominant colors. Pass float32 pixels as (N, 3) (or (N, 5) if you add xy coordinates for spatial clustering). Increasing K gives finer color separation. For spatial coherence, augment each pixel with its (x, y) coordinates; for pure color quantization, use only BGR.
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
h, w = img.shape[:2]
# reshape to a list of pixels (N, 3) in float32
pixels = img.reshape(-1, 3).astype(np.float32)
# kmeans(data, K, criteria, attempts, flags)
K = 4
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 20, 1.0)
_, labels, centers = cv2.kmeans(pixels, K, None, criteria, 10,
cv2.KMEANS_RANDOM_CENTERS)
# rebuild the image with cluster centers
centers = np.uint8(centers)
seg = centers[labels.flatten()].reshape((h, w, 3))GrabCut
GrabCut is an interactive foreground extractor — you give a bounding rectangle and it iteratively models foreground/background with Gaussian Mixture Models. For finer results, refine with GC_INIT_WITH_MASK after manually labeling pixels (definite fg / definite bg). Works well for a single salient object against a relatively simple background.
import cv2
import numpy as np
img = cv2.imread("person.jpg")
h, w = img.shape[:2]
# initialize mask (all probable background)
mask = np.zeros((h, w), np.uint8)
# define a rectangle around the foreground object
rect = (50, 50, w - 100, h - 100)
# internal scratch arrays
bgd = np.zeros((1, 65), np.float64)
fgd = np.zeros((1, 65), np.float64)
# run GrabCut: 5 iterations
cv2.grabCut(img, mask, rect, bgd, fgd, 5, cv2.GC_INIT_WITH_RECT)
# mask values: 0=bg, 1=fg, 2=probable bg, 3=probable fg
fg_mask = np.where((mask == 1) | (mask == 3), 255, 0).astype(np.uint8)
result = cv2.bitwise_and(img, img, mask=fg_mask)Watershed
Watershed treats the image as a topographic map (intensity = elevation) and 'floods' basins from seed markers — without good seeds it produces severe over-segmentation. The distance-transform trick generates sure-foreground seeds for touching objects (e.g. overlapping coins). Markers: 0=unknown, 1=background, 2..N=objects, -1=boundary. This is the classic 'separate touching objects' pipeline.
import cv2
import numpy as np
img = cv2.imread("coins.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# threshold to separate coins from background
_, binary = cv2.threshold(gray, 0, 255,
cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
# noise removal
kernel = np.ones((3, 3), np.uint8)
opening = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel, iterations=2)
# sure background area
sure_bg = cv2.dilate(opening, kernel, iterations=3)
# sure foreground: distance transform + threshold
dist = cv2.distanceTransform(opening, cv2.DIST_L2, 5)
_, sure_fg = cv2.threshold(dist, 0.5 * dist.max(), 255, 0)
sure_fg = np.uint8(sure_fg)
# unknown region (between sure fg and sure bg)
unknown = cv2.subtract(sure_bg, sure_fg)
# marker labelling
_, markers = cv2.connectedComponents(sure_fg)
markers = markers + 1
markers[unknown == 255] = 0
# watershed modifies markers in place
markers = cv2.watershed(img, markers)
img[markers == -1] = [255, 0, 0] # boundaries in redFlood Fill
floodFill is the magic-wand tool — start from a seed pixel and fill all connected pixels whose color is within loDiff/upDiff of either the seed (FIXED_RANGE) or the previous pixel. The mask must be 2 pixels larger than the image (a 1-pixel border on each side). Useful for region growing, simple background replacement, and interactive selection.
import cv2
img = cv2.imread("photo.jpg")
h, w = img.shape[:2]
mask = np.zeros((h + 2, w + 2), np.uint8)
# floodFill fills a connected region similar to the seed pixel
# (image, mask, seedPoint, newVal, loDiff, upDiff, flags)
cv2.floodFill(img, mask, (100, 100), (0, 255, 0),
loDiff=(20, 20, 20), upDiff=(20, 20, 20),
flags=8 | cv2.FLOODFILL_FIXED_RANGE)
# FLOODFILL_FIXED_RANGE: compare to the seed pixel (strict)
# 4-connectivity (4) or 8-connectivity (8)
# extract the filled mask
filled_mask = mask[1:-1, 1:-1] # the mask is padded by 1Connected Components
connectedComponents labels each white region with a unique integer — faster than findContours when you only need count, area, and centroids. connectedComponentsWithStats also returns bounding boxes and areas. Label 0 is always the background. Default connectivity is 8 (diagonals included); pass connectivity=4 for stricter. Use this for particle counting and blob analysis.
import cv2
import numpy as np
img = cv2.imread("blobs.png", cv2.IMREAD_GRAYSCALE)
_, binary = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)
# connectedComponents returns (num_labels, labels)
num, labels = cv2.connectedComponents(binary)
print(f"found {num - 1} objects (label 0 is background)")
# connectedComponentsWithStats also gives area and bounding box
num, labels, stats, centroids = cv2.connectedComponentsWithStats(binary)
for i in range(1, num):
x, y, w, h, area = stats[i]
cx, cy = centroids[i]
cv2.rectangle(img, (x, y), (x+w, y+h), (0, 255, 0), 1)
cv2.putText(img, str(area), (x, y - 5),
cv2.FONT_HERSHEY_SIMPLEX, 0.4, (0, 0, 255), 1)Mean-Shift / PyrMeanShiftFiltering
pyrMeanShiftFiltering is a non-parametric segmentation that 'flattens' regions of similar color while preserving edges — often used as a preprocessing step before contour-based segmentation. sp (spatial radius) controls how far the algorithm looks spatially; sr (color radius) controls the color tolerance. Larger values produce larger, smoother regions.
import cv2
img = cv2.imread("photo.jpg")
# pyrMeanShiftFiltering: edge-preserving smoothing that homogenizes regions
# (spatialRadius, colorRadius) — larger values merge more
filtered = cv2.pyrMeanShiftFiltering(img, sp=20, sr=40)
# now segment on the filtered image
gray = cv2.cvtColor(filtered, cv2.COLOR_BGR2GRAY)
_, binary = cv2.threshold(gray, 0, 255,
cv2.THRESH_BINARY + cv2.THRESH_OTSU)
contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
out = img.copy()
cv2.drawContours(out, contours, -1, (0, 255, 0), 2)Camera Calibration
Chessboard Calibration
Camera calibration finds the intrinsic matrix (focal length, principal point) and lens distortion coefficients from images of a known pattern (chessboard is standard because corners are easy to detect). You need ~10-20 images from varied angles. cornerSubPix refines corners to sub-pixel accuracy — essential for a good calibration. The pattern size passed to findChessboardCorners is interior corners (squares-1).
import cv2
import numpy as np
import glob
# prepare object points (0,0,0), (1,0,0), ..., for a 7x6 board
objp = np.zeros((6 * 7, 3), np.float32)
objp[:, :2] = np.mgrid[0:7, 0:6].T.reshape(-1, 2)
objpoints = [] # 3-D real-world points
imgpoints = [] # 2-D image points
for fname in glob.glob("calib/*.jpg"):
img = cv2.imread(fname)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# findChessboardCorners returns corners (or None)
ret, corners = cv2.findChessboardCorners(gray, (7, 6), None)
if ret:
# refine to sub-pixel accuracy
corners2 = cv2.cornerSubPix(gray, corners, (11, 11), (-1, -1),
(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001))
objpoints.append(objp)
imgpoints.append(corners2)
# calibrate
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(
objpoints, imgpoints, gray.shape[::-1], None, None)
print("camera matrix:\n", mtx)Undistort Image
undistort removes radial and tangential lens distortion using the calibration parameters. For video, pre-compute the maps once with initUndistortRectifyMap and apply remap each frame — much faster than calling undistort repeatedly. getOptimalNewCameraMatrix with alpha=1 keeps all pixels (with black borders); alpha=0 crops to the valid region.
import cv2
import numpy as np
# mtx, dist come from calibrateCamera
img = cv2.imread("distorted.jpg")
h, w = img.shape[:2]
# method 1: simple undistort
undistorted = cv2.undistort(img, mtx, dist)
# method 2: pre-compute the maps for speed (run once, reuse many times)
newcameramtx, roi = cv2.getOptimalNewCameraMatrix(
mtx, dist, (w, h), 1, (w, h))
mapx, mapy = cv2.initUndistortRectifyMap(
mtx, dist, None, newcameramtx, (w, h), cv2.CV_16SC2)
undistorted2 = cv2.remap(img, mapx, mapy, cv2.INTER_LINEAR)
# crop to the valid ROI
x, y, rw, rh = roi
undistorted2 = undistorted2[y:y+rh, x:x+rw]Pose Estimation (ArUco)
ArUco markers are square fiducial markers (like QR codes) used for pose estimation — given the marker's physical size and the camera calibration, you get its 3-D rotation (rvec) and translation (tvec) relative to the camera. Available in opencv-contrib. drawFrameAxes draws the marker's local X/Y/Z axes — red=X, green=Y, blue=Z, length in meters.
import cv2
# load a predefined ArUco dictionary
aruco_dict = cv2.aruco.getPredefinedDictionary(cv2.aruco.DICT_4X4_50)
params = cv2.aruco.DetectorParameters()
detector = cv2.aruco.ArucoDetector(aruco_dict, params)
img = cv2.imread("marker.png")
corners, ids, rejected = detector.detectMarkers(img)
if ids is not None:
cv2.aruco.drawDetectedMarkers(img, corners, ids)
# estimate pose (marker size in meters, camera matrix, distortion)
rvecs, tvecs, _ = cv2.aruco.estimatePoseSingleMarkers(
corners, 0.05, mtx, dist)
for rvec, tvec in zip(rvecs, tvecs):
cv2.drawFrameAxes(img, mtx, dist, rvec, tvec, 0.05)Fiducial Pose (SolvePnP)
solvePnP (Perspective-n-Point) recovers the camera pose from N 3-D-to-2-D correspondences — needs at least 4 points. Use SOLVEPNP_ITERATIVE (default) for accuracy or SOLVEPNP_IPPE for planar targets. The rvec is a rotation vector (axis × angle); use Rodrigues to convert to a 3x3 matrix. This is the foundation of marker-based AR and 6-DOF tracking.
import cv2
import numpy as np
# 3-D coordinates of known object points (e.g. a cube corner)
obj_points = np.array([
[0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0],
[0, 0, -1], [1, 0, -1], [1, 1, -1], [0, 1, -1],
], dtype=np.float32)
# corresponding 2-D image points (detected via feature matching etc.)
img_points = np.array([
[120, 90], [220, 88], [225, 190], [115, 195],
[80, 60], [240, 55], [248, 210], [70, 215],
], dtype=np.float32)
# solvePnP(objectPoints, imagePoints, cameraMatrix, distCoeffs)
ret, rvec, tvec = cv2.solvePnP(obj_points, img_points, mtx, dist)
print("rotation:", rvec.ravel())
print("translation:", tvec.ravel())
# project the 3-D points back to verify
proj, _ = cv2.projectPoints(obj_points, rvec, tvec, mtx, dist)Reprojection Error
Reprojection error is the gold standard for calibration quality — it measures how far the projected 3-D points land from the detected 2-D points. A good calibration is below 0.5 px average; above 1 px usually indicates a bad image set, wrong pattern size, or motion blur. Per-image error helps you find and discard bad calibration images.
import cv2
import numpy as np
# after calibrateCamera or solvePnP, check accuracy
mean_err = 0
total = 0
for i in range(len(objpoints)):
# project the 3-D points using the estimated pose
proj, _ = cv2.projectPoints(objpoints[i], rvecs[i], tvecs[i], mtx, dist)
err = cv2.norm(imgpoints[i], proj, cv2.NORM_L2) / len(proj)
mean_err += err
total += 1
print(f"mean reprojection error: {mean_err / total:.3f} px")
# good calibrations are typically < 0.5 px per cornerDepth & Stereo
Stereo Disparity Map
StereoBM computes a disparity map (pixel offset between left/right) for a rectified stereo pair — closer objects have larger disparity. numDisparities must be a multiple of 16; blockSize must be odd. The output is int16 scaled by 16 (divide by 16.0 for pixel disparity). StereoSGBM (semi-global block matching) is slower but much more accurate.
import cv2
# stereo pair (rectified: corresponding points on the same row)
imgL = cv2.imread("left.png", cv2.IMREAD_GRAYSCALE)
imgR = cv2.imread("right.png", cv2.IMREAD_GRAYSCALE)
# StereoBM (block matching, fast)
stereo = cv2.StereoBM_create(numDisparities=16, blockSize=15)
disparity = stereo.compute(imgL, imgR)
# normalize for display
disp_vis = cv2.normalize(disparity, None, 0, 255,
cv2.NORM_MINMAX, cv2.CV_8U)StereoSGBM
StereoSGBM enforces smoothness across scanlines (semi-global matching) — far more accurate than StereoBM at the cost of ~5x runtime. P1/P2 penalize disparity changes of 1 and >1 respectively; rule of thumb is P1=8*C*blockSize² and P2=32*C*blockSize² where C is the channel count. speckle* parameters filter small noisy disparity regions.
import cv2
imgL = cv2.imread("left.png", cv2.IMREAD_GRAYSCALE)
imgR = cv2.imread("right.png", cv2.IMREAD_GRAYSCALE)
sgbm = cv2.StereoSGBM_create(
minDisparity=0,
numDisparities=128,
blockSize=5,
P1=8 * 3 * 5 * 5,
P2=32 * 3 * 5 * 5,
disp12MaxDiff=1,
uniquenessRatio=10,
speckleWindowSize=100,
speckleRange=32,
)
disparity = sgbm.compute(imgL, imgR)
disp_vis = cv2.normalize(disparity, None, 0, 255,
cv2.NORM_MINMAX, cv2.CV_8U)Stereo Calibration & Rectification
Stereo matching requires the images to be rectified — epipolar lines aligned horizontally so corresponding points share a row. The pipeline is: calibrate each camera, stereo-calibrate to find R and T between them, stereoRectify to compute the rectification transforms, then remap each frame once. After rectification, disparity computation is a 1-D search along rows.
import cv2
import numpy as np
# calibrate each camera separately, then stereo-calibrate
ret, mtxL, distL, rvecsL, tvecsL = cv2.calibrateCamera(
objpoints, imgpointsL, grayL.shape[::-1], None, None)
ret, mtxR, distR, rvecsR, tvecsR = cv2.calibrateCamera(
objpoints, imgpointsR, grayR.shape[::-1], None, None)
# stereo calibration: finds the rigid transform between cameras
flags = cv2.CALIB_FIX_INTRINSIC
ret, mtxL, distL, mtxR, distR, R, T, E, F = cv2.stereoCalibrate(
objpoints, imgpointsL, imgpointsR, mtxL, distL, mtxR, distR,
grayL.shape[::-1], flags=flags)
# stereo rectification: aligns epipolar lines to be horizontal
R1, R2, P1, P2, Q, _, _ = cv2.stereoRectify(
mtxL, distL, mtxR, distR, grayL.shape[::-1], R, T)
# pre-compute the rectification maps
mapL1, mapL2 = cv2.initUndistortRectifyMap(
mtxL, distL, R1, P1, grayL.shape[::-1], cv2.CV_16SC2)
mapR1, mapR2 = cv2.initUndistortRectifyMap(
mtxR, distR, R2, P2, grayR.shape[::-1], cv2.CV_16SC2)
rectL = cv2.remap(imgL, mapL1, mapL2, cv2.INTER_LINEAR)
rectR = cv2.remap(imgR, mapR1, mapR2, cv2.INTER_LINEAR)Depth from Disparity
Depth is inversely proportional to disparity: depth = focal × baseline / disparity. The baseline (camera separation) and focal length must be in consistent units (typically pixels for focal, meters for baseline gives depth in meters). Disparity is noisy for far-away objects (small disparity, large depth error) — that's why stereo cameras need a wide baseline for far range.
import cv2
import numpy as np
# given a disparity map and the stereo baseline + focal length:
# depth = (focal_length * baseline) / disparity
focal = 700.0 # pixels (from camera matrix)
baseline = 0.1 # meters (distance between cameras)
disparity = sgbm.compute(rectL, rectR).astype(np.float32) / 16.0
# avoid division by zero
disparity[disparity <= 0] = 0.1
depth = (focal * baseline) / disparity # in meters
# cap to a sensible range
depth = np.clip(depth, 0.1, 100.0)
depth_vis = cv2.normalize(depth, None, 0, 255, cv2.NORM_MINMAX, cv2.CV_8U)Optical Flow (Lucas-Kanade)
Lucas-Kanade computes sparse optical flow — it tracks a set of keypoints across frames assuming small local motion. Fast and accurate for moderate motion; fails for large or fast motion (use a pyramid, which calcOpticalFlowPyrLK already does). status==1 means the point was tracked successfully. For dense per-pixel flow use calcOpticalFlowFarneback — much slower but produces a full motion field.
import cv2
import numpy as np
cap = cv2.VideoCapture("traffic.mp4")
ret, prev = cap.read()
prev_gray = cv2.cvtColor(prev, cv2.COLOR_BGR2GRAY)
# detect good features to track
prev_pts = cv2.goodFeaturesToTrack(prev_gray, 100, 0.3, 7)
while True:
ret, frame = cap.read()
if not ret:
break
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# sparse optical flow (Lucas-Kanade)
next_pts, status, err = cv2.calcOpticalFlowPyrLK(
prev_gray, gray, prev_pts, None)
good_new = next_pts[status == 1]
good_old = prev_pts[status == 1]
for new, old in zip(good_new, good_old):
a, b = new.ravel()
c, d = old.ravel()
cv2.line(frame, (int(a), int(b)), (int(c), int(d)), (0, 255, 0), 2)
cv2.circle(frame, (int(a), int(b)), 3, (0, 0, 255), -1)
cv2.imshow("flow", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
prev_gray = gray.copy()
prev_pts = good_new.reshape(-1, 1, 2)
cap.release()Utilities & Tips
Performance Measurement
TickMeter is the cleanest way to benchmark OpenCV calls — getTimeMilli returns wall-clock milliseconds. getTickCount/getTickFrequency give cycle-accurate timing. Always average over many iterations to amortize jitter, and disable cv2.imshow when benchmarking (GUI rendering dominates). setNumThreads controls the underlying parallel backend (TBB/OpenMP).
import cv2
import numpy as np
img = cv2.imread("photo.jpg")
# TickMeter: high-precision timer
tm = cv2.TickMeter()
tm.start()
for _ in range(100):
blurred = cv2.GaussianBlur(img, (5, 5), 0)
tm.stop()
print(f"avg: {tm.getTimeMilli() / 100:.3f} ms")
# getTickCount / getTickFrequency
t0 = cv2.getTickCount()
edges = cv2.Canny(img, 50, 150)
t1 = cv2.getTickCount()
print(f"Canny: {(t1 - t0) / cv2.getTickFrequency() * 1000:.2f} ms")
# set USE_AVX / thread count
cv2.setNumThreads(4)
print(cv2.getNumThreads())Mouse Events
setMouseCallback registers a handler for mouse events on a named window — useful for interactive ROI selection, annotation tools, and simple paint apps. Common events: LBUTTONDOWN, LBUTTONUP, MOUSEMOVE, RBUTTONDOWN. The handler must be a top-level (or global) function; globals are the standard way to share state because the callback signature is fixed.
import cv2
import numpy as np
drawing = False
ix, iy = -1, -1
def on_mouse(event, x, y, flags, param):
global drawing, ix, iy
if event == cv2.EVENT_LBUTTONDOWN:
drawing = True
ix, iy = x, y
elif event == cv2.EVENT_MOUSEMOVE and drawing:
cv2.rectangle(img, (ix, iy), (x, y), (0, 255, 0), -1)
elif event == cv2.EVENT_LBUTTONUP:
drawing = False
cv2.rectangle(img, (ix, iy), (x, y), (0, 255, 0), -1)
img = np.zeros((512, 512, 3), np.uint8)
cv2.namedWindow("canvas")
cv2.setMouseCallback("canvas", on_mouse)
while True:
cv2.imshow("canvas", img)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cv2.destroyAllWindows()Trackbar
Trackbars let you tune parameters interactively without restarting the script — invaluable for finding good thresholds, kernel sizes, and HSV ranges. The callback is called on every change (a no-op is fine if you poll getTrackbarPos in the main loop). Trackbars must be created on a named window and read with getTrackbarPos(window, name).
import cv2
import numpy as np
def nothing(x):
pass
img = np.zeros((300, 512, 3), np.uint8)
cv2.namedWindow("image")
# create trackbars
cv2.createTrackbar("R", "image", 0, 255, nothing)
cv2.createTrackbar("G", "image", 0, 255, nothing)
cv2.createTrackbar("B", "image", 0, 255, nothing)
cv2.createTrackbar("switch", "image", 0, 1, nothing)
while True:
cv2.imshow("image", img)
if cv2.waitKey(1) & 0xFF == 27: # Esc to quit
break
r = cv2.getTrackbarPos("R", "image")
g = cv2.getTrackbarPos("G", "image")
b = cv2.getTrackbarPos("B", "image")
s = cv2.getTrackbarPos("switch", "image")
if s == 0:
img[:] = 0
else:
img[:] = [b, g, r]
cv2.destroyAllWindows()ROI Selector
selectROI opens an interactive window where the user drags a rectangle and presses Enter/Space — far easier than writing your own mouse callback. Press 'c' to cancel (returns all zeros). selectROIs lets you pick multiple rectangles one after another. The returned tuple is (x, y, w, h); crop with img[y:y+h, x:x+w].
import cv2
img = cv2.imread("photo.jpg")
# built-in ROI selector (drag a rectangle)
roi = cv2.selectROI("select", img, showCrosshair=True, fromCenter=False)
cv2.destroyAllWindows()
x, y, w, h = roi
cropped = img[y:y+h, x:x+w]
# multiple ROIs
rois = cv2.selectROIs("select", img, showCrosshair=True, fromCenter=False)
for r in rois:
x, y, w, h = r
cv2.imshow("roi", img[y:y+h, x:x+w])
cv2.waitKey(0)
cv2.destroyAllWindows()Common Pitfalls
These five pitfalls cause the majority of OpenCV bugs: BGR/RGB confusion (red and blue swap), uint8 overflow (dark/bright artifacts), coordinate order (off-by-one ROIs), missing-file None (subsequent operations crash with cryptic errors), and forgetting waitKey (windows don't update). Internalize them early — every OpenCV developer hits each one at least once.
import cv2
import numpy as np
# 1. BGR vs RGB — imshow expects BGR; matplotlib expects RGB
img = cv2.imread("p.jpg")
# plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB)) # correct
# plt.imshow(img) # colors look wrong
# 2. uint8 overflow — use cv2 ops or upcast
a = np.array([200], np.uint8)
print(a + 100) # [44] (wraps, WRONG)
print(cv2.add(a, 100)) # [[255]] (saturates, RIGHT)
# 3. (x, y) vs (row, col) — drawing uses (x,y), NumPy uses [y,x]
cv2.rectangle(img, (10, 20), (110, 120), (0, 255, 0), 2) # x=10..110
roi = img[20:120, 10:110] # rows 20..120, cols 10..110 (same region)
# 4. None returned by imread when the file is missing
img = cv2.imread("missing.jpg")
if img is None:
raise FileNotFoundError("could not read image")
# 5. waitKey is mandatory for imshow to actually render
cv2.imshow("img", img)
cv2.waitKey(1) # without this, the window may stay blank関連する OpenCV スニペット
Copy-paste ready code for common tasks.
Read and Write Images
Load, display, and save images in various formats.
Resize and Crop
Resize with interpolation and crop regions of interest.
Color Conversion
Convert between BGR, RGB, HSV, and grayscale.
Blur and Filter
Smooth and sharpen images with kernels.
Edge Detection
Detect edges with Canny, Sobel, and Laplacian.
Contours
Find, draw, and measure contours.
Face Detection
Detect faces with a Haar cascade classifier.
Threshold
Apply binary, adaptive, and Otsu thresholding.
Was this helpful?