Handwritten LaTeX Formula Recognition

Upload or draw any handwritten mathematical equation to extract accurate, copy-ready LaTeX code powered by Deep Learning.

1. Upload Image
2. Preprocess & Otsu
3. Neural Inference
4. LaTeX Result

Upload Handwritten Formula

Drag & drop your handwritten formula image here

or click to browse files (PNG, JPG, WEBP, BMP up to 10MB)
Try Sample Preset Formula:
5

Recognized Formula Result

Rendered mathematical formula will appear here...
\text{Recognized LaTeX code will appear here}

Formula Recognition History

All your previously recognized handwritten equations saved locally.

About the Project

Handwritten Formula Analyzer: Handwritten Mathematical Formula Recognition using EfficientNet-B0 and Transformer

Handwritten mathematical expression recognition is a challenging problem in computer vision and deep learning because mathematical formulas contain a complex combination of symbols, superscripts, subscripts, fractions, square roots, summations, and spatial relationships that are difficult for traditional Optical Character Recognition (OCR) systems to interpret accurately.

Handwritten Formula Analyzer aims to develop an intelligent system capable of recognizing handwritten mathematical expressions from images and converting them into their corresponding LaTeX representation. The system combines state-of-the-art deep learning techniques with computer vision algorithms to provide accurate recognition of mathematical formulas.

The proposed system first uses OpenCV image processing techniques to detect and isolate handwritten mathematical expressions from large images or scanned documents. Once the equation region is extracted, it is passed to an EfficientNet-B0 encoder, which extracts rich visual features while maintaining computational efficiency. These extracted features are then processed by a Transformer Decoder, which generates the corresponding LaTeX sequence token by token using an attention mechanism.

End-to-End System Workflow

Step 1: Preprocessing & Detection

Applies OpenCV Gaussian Adaptive Thresholding and Horizontal Projection Profiling (HPP) to segment handwritten ink strokes from ruled notebook paper and crop individual equation lines.

Step 2: Visual Feature Extraction

Fine-tuned EfficientNet-B0 backbone extracts spatial feature maps ($128 \times 512$ canvas) projected into a 256-dimensional feature space.

Step 3: Neural Sequence Generation

A 4-layer Transformer Decoder with 8 attention heads generates token-by-token LaTeX code using Beam Search ($k=5$) decoding over a 241-symbol vocabulary.

Step 4: Symbolic Solving & Plotting

SymPy Symbolic Engine evaluates roots, integrals, and summations, while Matplotlib renders dark-mode function curve plots.

Key Technologies Used

  • Python 3.10+: Core Programming Language
  • PyTorch: Deep Neural Network Training & Inference
  • OpenCV: Adaptive Thresholding & Line Segmentation
  • EfficientNet-B0: Pretrained Transfer Learning Backbone
  • Transformer Decoder: Multi-Head Sequence Generation
  • HME100K Dataset: 100,000+ Handwritten Math Expressions
  • SymPy: Symbolic Math Computation Engine
  • Matplotlib & NumPy: Function Plotting & Curve Generation
  • FastAPI & Uvicorn: High-Performance REST API Server
  • KaTeX & Vanilla CSS: Real-Time LaTeX Rendering & Web Client

Key System Features

  • Detects handwritten equations from notebook paper images.
  • Automatically crops equation regions using OpenCV projection.
  • Extracts deep visual features using EfficientNet-B0.
  • Generates LaTeX expressions using a Transformer Decoder.
  • Supports fractions, powers, roots, Greek symbols, integrals, and limits.
  • Employs transfer learning for higher accuracy and faster convergence.
  • Provides a FastAPI-based REST API for seamless external integration.
  • Saves best-performing model weights with checkpoint support.

Real-World Applications

EdTech & E-Learning

Automated homework grading, interactive learning platforms, and digital math tutoring systems.

Academic & Research Digitization

Converting handwritten research notes, whiteboard derivations, and lecture slides into publication-ready LaTeX.

Stylus Note-Taking Apps

Integration into digital tablet apps (iPad, Android stylus, e-ink devices) for instant formula recognition.

Copied to clipboard!