Environmental Sound Classification Using Deep Learning
For my degree project, I built an end-to-end environmental sound classification system. The work covered signal processing, a custom CNN model, model evaluation and deployment into a real-time desktop application.
A full pipeline from raw sound to live classification
The system classifies environmental sounds such as sirens, rain, engines, alarms and human activity. Instead of feeding raw waveforms directly into the network, the audio is converted into Mel-spectrograms so the CNN can learn time-frequency patterns.
The thesis did not stop at offline model accuracy. The trained model was converted to ONNX and used in a desktop application that captures audio continuously, runs inference in a separate thread and visualizes predictions in real time.
Audio preparation
ESC-50 recordings were converted to mono, normalized and augmented with pitch shifting, time stretching, temporal shifting and low-level Gaussian noise.
Feature extraction
Each clip was converted into a 128 x 216 Mel-spectrogram using 128 Mel bins, FFT window size 2048 and hop length 512.
Model training
I developed ESCNet, a compact four-block 2D CNN trained with cross-entropy loss, Adam, dropout, batch normalization and early stopping.
Real-time deployment
The trained model was exported to ONNX and integrated into a C++ desktop application using PortAudio, ONNX Runtime and OpenGL.
ESCNet: a compact CNN built for Mel-spectrograms
ESCNet was designed as a lightweight four-block 2D convolutional neural network. The goal was to keep the model strong enough to learn useful acoustic patterns, but compact enough to reduce overfitting on ESC-50, which only contains 2,000 recordings.
Results were measured with cross-validation, not a single lucky split
I evaluated the model using the official five-fold ESC-50 split. The final reported performance is the average across all folds, which gives a more reliable estimate than one train-test split on a small dataset.
The strongest classes were distinctive sounds like gunshots, sirens and vacuum cleaners. The main errors were also explainable: rain vs thunderstorm, drilling vs jackhammer and birds vs crickets.
The model was deployed into a real-time local application
The desktop application was written in C++. PortAudio captures microphone or system audio into a ring buffer. A separate inference thread reads the latest audio window, converts it into a Mel-spectrogram and runs ONNX Runtime predictions roughly every half second.
OpenGL handles the visual feedback, including spectrum display, predicted class and confidence. Keeping capture, inference and rendering separate prevented the UI or model from blocking the audio stream.
Local-first by design
The application processes audio locally on the computer. No recordings are stored or sent over the network, which is important for privacy when building software that continuously listens to the environment.
Open thesis PDF