Computer Vision Tutorial: Learn Image Processing from Scratch (2026)
I built my first face detector using OpenCV's Haar cascades in 2015 and was amazed that 200 lines of C++ could find faces in real time. Nine years later, deep learning has transformed the field — modern vision systems recognize objects, segment scenes, estimate poses, and generate images. But the fundamentals still matter: pixels, filters, features, and transformations.
This tutorial takes a practical path through computer vision, starting with image processing basics and progressing to modern deep learning approaches using PyTorch and OpenCV. You will build a complete pipeline: load images, preprocess, extract features, train a detector, and evaluate.
Images as Tensors
A digital image is a 3D tensor: height × width × channels (HWC). Grayscale has 1 channel (intensity), RGB has 3 (red, green, blue), and RGBA adds alpha transparency. Each channel contains integer values 0-255 (uint8) for standard 8-bit images, or floats 0.0-1.0 in normalized form.
Operations like flipping, cropping, and resizing are tensor manipulations. OpenCV loads images as BGR (not RGB) by default — a common source of bugs when displaying with Matplotlib. Always convert with cv2.cvtColor(img, cv2.COLOR_BGR2RGB) before display.
import cv2
import numpy as np
img = cv2.imread('photo.jpg')
img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
print(f'Shape: {img_rgb.shape}, dtype: {img_rgb.dtype}')
img_norm = img_rgb.astype(np.float32) / 255.0
img_flip = cv2.flip(img_rgb, 1)Convolution and Edge Detection
Convolution is the core operation of image processing. A small kernel (e.g., 3x3) slides over the image, computing a weighted sum at each position. Different kernels produce different effects: Sobel detects edges, Gaussian blurs smooth noise, and sharpening kernels enhance detail.
The Canny edge detector combines Gaussian blur, gradient computation, non-maximum suppression, and hysteresis thresholding into a robust edge-finding pipeline. The two thresholds define sensitivity: edges above the high threshold are kept, those below the low threshold are discarded, and those in between are kept only if connected to a strong edge.
gray = cv2.cvtColor(img_rgb, cv2.COLOR_RGB2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 1.0)
edges = cv2.Canny(blurred, threshold1=50, threshold2=150)
magnitude = np.sqrt(cv2.Sobel(gray, cv2.CV_64F, 1, 0, ksize=3)**2 +
cv2.Sobel(gray, cv2.CV_64F, 0, 1, ksize=3)**2)Feature Detection: SIFT and ORB
Keypoints are distinctive locations in an image (corners, blobs) that remain identifiable under scale and rotation changes. SIFT (Scale-Invariant Feature Transform) detects scale-space extrema and computes a 128-dimensional descriptor. ORB (Oriented FAST and Rotated BRIEF) is a free, faster alternative that works well for real-time applications.
Feature matching finds correspondences between two images by comparing descriptors. Lowe's ratio test filters matches by comparing the best match to the second-best — if they are too close, the match is ambiguous and should be discarded. This is the backbone of image stitching, panorama creation, and object tracking.
orb = cv2.ORB_create(nfeatures=1000)
kp1, des1 = orb.detectAndCompute(img1, None)
kp2, des2 = orb.detectAndCompute(img2, None)
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = sorted(bf.match(des1, des2), key=lambda x: x.distance)
result = cv2.drawMatches(img1, kp1, img2, kp2, matches[:30], None)Object Detection with YOLO
You Only Look Once (YOLO) treats object detection as a single regression problem: divide the image into a grid, predict bounding boxes and class probabilities per cell. YOLOv8 (Ultralytics) supports detection, segmentation, classification, and pose estimation in a single framework. It runs at 100+ FPS on a modern GPU.
The model outputs: bounding boxes, confidence scores, and class IDs. Apply non-maximum suppression (NMS) with IoU threshold 0.5 to remove duplicate detections. Training a custom YOLO model typically takes 2-10 hours on a single GPU.
from ultralytics import YOLO
model = YOLO('yolov8n.pt')
results = model('street_scene.jpg')
for r in results:
for box in r.boxes:
x1, y1, x2, y2 = box.xyxy[0].tolist()
conf = box.conf[0].item()
cls = int(box.cls[0].item())
print(f'Class: {model.names[cls]}, Conf: {conf:.2f}')Segmentation: U-Net and SAM
Segmentation assigns a class label to every pixel. Semantic segmentation (all cars are 'car') is distinct from instance segmentation (car 1, car 2, ...). U-Net is the classic architecture: an encoder-decoder with skip connections that preserve spatial detail. It excels in medical imaging where training data is limited.
Meta's Segment Anything Model (SAM) is a foundation model for segmentation that generalizes zero-shot to objects it has never seen. Given an image and a prompt (point, box, or text), SAM outputs a mask. SAM is transformative for interactive annotation tools.
from segment_anything import SamPredictor, sam_model_registry
sam = sam_model_registry['vit_h'](checkpoint='sam_vit_h_4b8939.pth')
predictor = SamPredictor(sam)
predictor.set_image(cv2.imread('photo.jpg'))
input_box = np.array([100, 150, 400, 500])
masks, scores, logits = predictor.predict(
point_coords=None,
point_labels=None,
box=input_box[None, :],
multimask_output=True
)Video Processing and Tracking
Video is a sequence of frames — essentially a 4D tensor (T×H×W×C). Processing video efficiently means reusing computations across frames. Object tracking algorithms like SORT and DeepSORT assign unique IDs to detected objects and maintain identity across frames using Kalman filters and appearance features.
For real-time performance, skip frames (process every Nth frame) and use lightweight trackers (CSRT or KCF). When accuracy matters more than speed, detect with YOLO every 10 frames and interpolate the rest using optical flow.
cap = cv2.VideoCapture('video.mp4')
tracker = cv2.TrackerCSRT_create()
ret, frame = cap.read()
bbox = cv2.selectROI('Select Object', frame, False)
tracker.init(frame, bbox)
while True:
ret, frame = cap.read()
if not ret:
break
success, bbox = tracker.update(frame)
if success:
x, y, w, h = [int(v) for v in bbox]
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
cv2.imshow('Tracking', frame)
if cv2.waitKey(30) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()Frequently Asked Questions
Should I use OpenCV or PyTorch for computer vision?
Both. OpenCV handles IO, preprocessing, and traditional algorithms. PyTorch handles deep learning models. The common pipeline: OpenCV loads and transforms images -> converts to tensors -> feeds into a PyTorch model -> OpenCV visualizes results.
How much data do I need to train a custom object detector?
For YOLO, 500-2000 labeled images per class is a good baseline. With transfer learning from pretrained weights, you can get usable results with as few as 100 images per class. Data augmentation effectively multiplies your dataset size by 5-10x.
What is the best way to label images for segmentation?
Label Studio, CVAT, and Roboflow are the most popular tools. CVAT (open-source) supports polygon, brush, and keypoint annotation. For large datasets, use SAM as a pre-labeling tool: annotate a few images manually, fine-tune SAM, then auto-label the rest.
How do I handle class imbalance in my dataset?
Class imbalance is common. Use Focal Loss (modifies cross-entropy to down-weight easy examples), oversample minority classes during training, or use hard negative mining. Evaluate with mAP (mean Average Precision) rather than accuracy.
Originally published on Ayodhyyya. Last updated June 1, 2026.