About SOVIS
We draw the sounds that captions leave out
Which way footsteps are coming from. Whether a siren is going off. Whether someone is talking to you. Hearing accessibility has grown around conveying dialogue, through captions and sign language. Sounds other than dialogue are still left out.
SOVIS is a team of students at Dongguk University. In spring 2026, we built SoundVisualizer for Windows in an open source software project course. After testing it twice with deaf and hard of hearing students, we brought the same design to phones and made the Android app.
People registered with a hearing disability in Korea at the end of 2024. That is 16.8% of the 2.63 million registered people with disabilities, the second-largest group after physical disabilities.
Ministry of Health and Welfare, Status of Registered Persons with Disabilities 2024The share of people with a hearing disability among everyone newly registered with a disability in 2024. It is the largest group among new registrations.
Ministry of Health and Welfare, Status of Registered Persons with Disabilities 2024For every moment when sound is hard to hear, or can't be turned on
It turns the direction and type of sounds that aren't dialogue, such as footsteps, gunshots and sirens, into something you can see, so movies, dramas, videos, music and games can be enjoyed more fully.
It isn't tied to one app or game. Any sound that plays on your device, from YouTube and streaming services to music and games, is drawn right over what you are watching.
On the subway, in a lecture hall, late at night, in a room where a child is sleeping. Even when you can't play sound freely, you can see where a sound comes from and what it is.
Turn on External sound mode on Android and the app listens through the microphone. It alerts you to fire alarms, sirens or someone calling you, on screen and with vibration.
We chose features from what deaf and hard of hearing students told us, not from our own guesses. We tested twice in May 2026 and ran a field trial of the Android app in October.
Measuring loudness and telling sound types apart all happens on your phone or computer. Sound is never saved to a file or sent to a server.
It supports 18 languages, and a tutorial shows you how to use it the first time you open it. The Windows version also works with a keyboard and screen readers.
It started in a one-semester course and became two apps, for phones and computers
Three Dongguk University students share development, AI and design
Team lead · Development · Design
AI
Development
Contact
SoundVisualizer grows with feedback from the people who use it. Your impressions, sounds it drew wrong, ideas for working together: all are welcome.
nawwhod@gmail.comWe're looking for partners to run usability tests with deaf and hard of hearing people, or to advise us. We provide the install files and a user guide, and share the results with you.
Tell us about sounds it got wrong or anything that gets in your way. You can report straight from the app's Help tab, with your device details included.
The source code for both apps is on GitHub. Issues and suggestions are welcome.
Draws the sound playing on your phone, from videos, music, games and more, around the edge of the screen, and gives each sound type its own vibration. Turn on External sound mode and it also listens to the sounds around you through the microphone.
Downloads so far102
v1.8.4 · Released 2026.11.11 · Apache 2.0 · How to install · Try the demo · Intro video (opens in a new tab)
Without leaving your video, music or game, it shows the direction and type of each sound with color, motion and vibration
Five scenes drawn around the edge of a phone screen. The color changes with the sound type, and the edge on the side the sound comes from is drawn larger.
Ambient
Human sounds
Danger
← Left
Right →Choose Wave, Pad, Circle or Outline. The graphics float transparently over other apps, and touches pass straight through to the app below. Size, Opacity, Sensitivity, Speed and Glow are saved separately for each mode.
It compares the left and right channels of the sound playing on your phone, and draws which side the sound comes from and how loud it is. Phone audio has only two channels, so it can't tell front from back.
Sorts sounds into Ambient, Human sounds and Danger, and shows each in its own color. Warning sounds such as sirens, alarms, explosions and horns count as Danger more readily, so they aren't missed even under music or game audio.
The 521 sounds the AI recognizes are shown in groups of similar sounds. For each group (or each sound, in Advanced mode) you choose which type it counts as. Your changes apply to the AI right away.
For each sound type, choose a pattern (Off, Slow, Medium, Fast or Continuous) and a strength from 10 to 100%. At first only Danger is on, set to ‘Medium’. It vibrates while the sound lasts and stops when the sound stops.
If a Danger sound is heard while your phone is locked, an alert appears on the lock screen and the screen turns on for a moment. The alert makes no sound.
Instead of the phone's own sound, it listens to the sounds around you through the microphone and alerts you with the same graphics and vibration. Use it to notice fire alarms, sirens or someone calling you. Microphone sensitivity has 16 steps. This mode doesn't show direction.
From the running notification, stop the visualizer or switch the visual mode. With the Quick Settings tile, turn it on and off without opening the app.
The first time you open it, a tutorial shows how it works with the real drawing engine and real vibration. You can report problems straight from the Help tab. It supports 18 languages.
Select a sound and tap the screen. The sound is drawn the same way SoundVisualizer draws it
The real apps capture and analyze the sound playing on your phone or computer. Direction and loudness come from comparing the volume of each channel, and a YAMNet-based AI identifies the type of sound. A browser can't capture other apps' sound, so in this demo you set where the sound comes from yourself. 7.1 surround is available in the Windows version; the Android version tells left from right in the phone's sound. The demo's colors are chosen to match this page; in the apps you set the color for each type yourself. Which sounds count as Danger can also be changed for each sound in the app's Sounds tab, so for this wildlife documentary scene the demo puts the bear growl under Danger.
Install it from the APK file on a phone with Android 10 or later.
To listen to the sounds around you, turn on the External sound mode switch on Home before you start. It can't be changed while running.
If your question isn't here, email us
Choose the version that fits your device
| Feature | Android | Windows |
|---|---|---|
| Sound source | Sound playing on the phone, or sounds around you through the microphone | All sound playing on the computer |
| Direction | Left and right (sounds around you appear in the center, without direction) | Stereo: left and right. 5.1 and 7.1: front, back and sides too |
| Sound types | Ambient, Human sounds and Danger. You can change which type each sound counts as in the Sounds tab | |
| Visual modes | Wave, Pad, Circle and Outline. Size, Opacity and Sensitivity are saved separately for each mode | |
| Vibration | Pattern and strength for each sound type | None |
| Turning on and off | App, running notification, Quick Settings tile | Launcher, tray icon |
| Languages | 18 languages, including Korean, English, Japanese and Chinese | |
| Installation | Install the APK file | Unzip and run |
| License | Apache 2.0 | AGPL-3.0 |
Captures all the sound playing on your computer and draws its direction on the screen, from stereo up to 7.1 channels. No installation: unzip it and run.
Downloads so far58
v2.1.1 · Released 2026.11.11 · AGPL-3.0 · How to install · Try the demo · Intro video (opens in a new tab)
It draws every sound playing on your computer, from videos, music and games to video calls. With 7.1-channel output, it also tells front, back and sides apart
Five scenes drawn along the edges of a computer screen. The color changes with the sound type, and the edge on the side the sound comes from is drawn larger.
Ambient
Human sounds
Danger
← Left
Right →Choose Wave, Pad, Circle or Outline. Whatever maximum frame rate you set, from 30 FPS up to your monitor's refresh rate, the graphics move at the same speed.
Detects stereo, 5.1 and 7.1 output automatically and draws each channel in its place. When the output device changes, it adjusts to that device, and you can also change it yourself.
Sorts sounds into Ambient, Human sounds and Danger, and shows each in its own color. With 5.1 or 7.1, the AI listens to the rear and side channels too, not just the front.
Without minimizing a full-screen video or game, press F2 to switch the sound mode and F3 to switch the visual mode. You can change the hotkeys to any key combination in the Settings tab.
Press F4 and boundary lines appear on the screen. Drag them to set the size and limits of the drawing area. Settings such as color and opacity can be changed right in the panel.
The 521 sounds the AI recognizes are shown in the same groups as in the Android version. For each group (or each sound, in Advanced mode) you choose which type it counts as.
From the icon in the notification area, turn the visualizer on or off, open the launcher or exit the program.
Every control in the launcher can be reached with the Tab key. Ctrl + Tab moves between tabs, and Esc closes windows. Screen readers such as Narrator and NVDA read out buttons and their states.
The first time you open it, a tutorial shows how it works with the real drawing engine. You can report problems straight from the Help tab. It supports 18 languages and starts in your Windows display language.
A portable package for Windows 10/11 (64-bit). You don't need to install .NET separately.
Settings are saved in settings.json next to SoundVisualizer.exe, so you can move or copy the whole folder.
If your question isn't here, email us
Choose the version that fits your device
| Feature | Android | Windows |
|---|---|---|
| Sound source | Sound playing on the phone, or sounds around you through the microphone | All sound playing on the computer |
| Direction | Left and right (sounds around you appear in the center, without direction) | Stereo: left and right. 5.1 and 7.1: front, back and sides too |
| Sound types | Ambient, Human sounds and Danger. You can change which type each sound counts as in the Sounds tab | |
| Visual modes | Wave, Pad, Circle and Outline. Size, Opacity and Sensitivity are saved separately for each mode | |
| Vibration | Pattern and strength for each sound type | None |
| Turning on and off | App, running notification, Quick Settings tile | Launcher, tray icon |
| Languages | 18 languages, including Korean, English, Japanese and Chinese | |
| Installation | Install the APK file | Unzip and run |
| License | Apache 2.0 | AGPL-3.0 |
Technology
The sound playing on your device goes through four steps before it is drawn on the screen. Every calculation happens on the device.
No extra equipment, and no help needed from content creators
It takes in the sound playing on your phone or computer as it is. There's no need to connect each app separately, whether it's YouTube, a streaming service, music or a game. On Android, External sound mode takes in the sounds around you through the microphone.
It measures and compares the volume of each channel to work out direction and loudness. Windows detects 2.0, 5.1 and 7.1 channels by itself; Android tells left from right.
Every 0.25 seconds, the last second or so of sound is turned into a 16 kHz mono log-mel spectrogram, and a YAMNet-based ONNX model sorts it into Ambient, Human sounds or Danger.
Loudness becomes size, direction becomes position, and type becomes color. What you're watching stays as it is, and Android also alerts you with vibration.
The four screens below are being drawn live right now. Size, Opacity and Sensitivity can be set separately for each mode
Fills in along the edges of the screen, reaching further for louder sounds. You can read direction and loudness together, which suits watching videos.
Shows sound with pads fixed at eight points around the edge. It covers less of the screen while showing direction clearly.
A ring in the middle of the screen swells toward the side the sound comes from. You can read direction while keeping your eyes on the center.
Keeps only the outline of the wave. Of the four modes, it covers the least of the screen.
An app that handles sound needs extra care
Sound type detection isn't always right. So direction and loudness are calculated separately from the AI and always shown, and sounds you must not miss, like sirens and alarms, are caught as Danger more readily. When safety is at stake, don't rely on this app as your only safeguard.
The graphics float transparently over other apps, mostly along the edges of the screen. You decide how much of the screen they cover by changing the mode, size and opacity.
The 521 sounds the AI recognizes are shown in groups of similar sounds, and for each group you decide whether it counts as Ambient, Human sounds or Danger. You can change the colors too, and on Android, the vibration.
Measuring loudness and telling sound types apart all happens on your device. Sound is never saved to a file or sent to a server, and it works without the internet.
Research
In May 2026 we tested the Windows version twice. After making changes based on what we heard in round 1, we checked again with the same students
They tried it themselves, then answered a 5-point survey. The information they wanted most was the direction of sound; their biggest worry was drawing their eyes away from what they were watching.
We made the AI's decisions steadier and added more game sound effect data. We tuned how it responds to match loudness, and widened the range of settings. We also changed the defaults to a screen that stays comfortable over long viewing.
Every student answered positively on all four items, and the biggest rise was in willingness to actually use it. After the test, the Windows version was installed in the school's computer lab.
Round 1 shows answers from all 20 participants; round 2, from the same 6 deaf and hard of hearing students.
In October 2026, we ran a field trial of the Android app with 6 deaf and hard of hearing students
At Seoul National School for the Deaf, they tried the Android app with the phone's own sound and in External sound mode, then answered a survey. The information they said they needed most was which direction a sound comes from.
We were advised to make the settings simpler, with pictures instead of text.
We made the Simple settings view, where you choose by looking at pictures. Coughs and sneezes now also count as Human sounds, and these are shown sooner.
“Just having it shown visually was a big help,” one student said. We're looking for ways to improve the lowest-rated item, the left–right direction display (3 of 6).
We're looking for institutions and organizations to join our next field trial.
Releases, awards and contests, research, and mentions of SOVIS elsewhere, all in one place