Point your phone at a printed page of music and AntiTranscript reads it back as a MIDI file.
It works on a normal photo despite distortion, shadows, and ink bleed. No more need for clean scans!
The first version
This was my CS 131 final project, where the first version tried to read music by cutting the page into individual symbols and classifying each one. The problem is that everything downstream depends on getting those cuts right, so a single bad split throws everything off. On clean renders it was fine, but on real photos the segmentation kept on failing and the exact pitch accuracy was 10%.
The switch-up
After the class I rebuilt it around an end to end CRNN with a CTC head, which drops the segmentation step entirely. It instead scans the whole staff left to right and decodes the note sequence directly. The setup now reads rhythm and not just pitch, since the recurrent layers see the horizontal spacing between notes.
Closing the gap
The thing that actually fixed real world accuracy was the training data. I couldn't collect thousands of labeled phone photos, so I made them. I took clean rendered staves and degraded them with perspective warps, uneven lighting, blur, JPEG artifacts, and ink bleed. The model trains on these synthetic photos and acts like a real camera already expecting the noise. That moved real-photo accuracy from 10% to 95% and it still trains on a laptop GPU in minutes.
Built with Python, PyTorch, OpenCV, a CRNN+CTC model, and the PRIMUS dataset.
