There are at least a dozen apps out there right now that offer ad skipping for podcasts. The problem is they all work the same way, and that approach has some undesirable tradeoffs.
Currently, these apps do some variation of the following:
- Transcribe the podcast audio (sometimes 1+ hours)
- Pass transcribed text to an LLM asking it to find the ads
- Map detected ads back to audio and skip those parts
Since steps one and two are compute-heavy, developers of the current generation of podcast ad skippers have opted to do this work on a server and deliver the results to mobile clients.
This makes the engineering very easy. Essentially something that could be done with API glue code. However, servers cost money and APIs cost money. Someone has to pay for these services, so typically it lands on the end user in the form of a recurring subscription or maintaining a credit balance. At that point, if recurring payment is required, users would be better off just paying for an ad-free feed (assuming it's available) and the content creator would be better off as well. This business model just enriches API and compute providers at the expense of content creators.
To add insult to injury, modern podcasts have dynamic ad insertion, which means the length and timing of ads may change from download to download. So in order to get accurate ad detections the user's client has to download the audio from the podcast then upload it to the app's servers for processing. Or what is more common: the server downloads the audio for the user, processes it, and redistributes it. In worse cases they redistribute their one download to all of their users who listen to that podcast. I am not a lawyer, but it seems to me this is copyright infringement and also reduces the podcasts' download counts, which actively harms content creators.
Thinking differently
For earsay we wanted to do something better. We didn't want to change the economics for listeners by making them pay a recurring fee, and we didn't want to impact content creators' download counts. We wanted to fit into the currently acceptable niche where users can tap forward on their app to skip ahead past an ad, but we simply wanted to automate that action for our users.
This is an entirely different engineering problem. Apple offers both on-device transcription and an on-device LLM. However, if you have ever used these you know they're not made for processing large pieces of content. The model's context window is only 4096 tokens! For transcription, all iOS users know how "great" Apple's dictation feature is.
This required training our own machine learning model on millions of real-world examples, and a great deal of trial-and-error. Throughout a typical podcast we sample the model thousands of times by passing it chunks of an episode. We then have a complex post-processing pipeline to refine uncertainties using hysteresis, among other techniques.
While our model is not perfect, it can approximate and often exceed the experience of skipping over ads yourself. We are continually training and improving our model.
Experience not exclusion
In the minds of users, the villain in the story of obnoxious ads is often the sponsors themselves. We do not subscribe to this theory. At the end of the day, sponsors pay for some of the content we enjoy. We may not agree with the price (loud incongruous interruptions) but we still think they should be heard. This is an area where Apple's on-device LLM is helpful. earsay transcribes detected ad segments using Apple's dictation and asks Apple's on-device LLM to extract any websites, phone numbers, calls to action, and offers from the detected ad. earsay then takes the output and automatically creates what is essentially a display ad based on the audio ad. If a domain or URL is given in the audio, the display ad is linked up and UTM parameters crediting the podcast are sent along.
A complete story
Many peers of earsay rely solely on ad skipping to draw in users. In many cases the podcast client itself leaves a lot to be desired. At earsay we are definitely more engineers than designers, so we do not claim that earsay, by any measure, is the most beautiful or refined podcast player (yet!). However, we did put a great deal of effort into supporting important creature comforts like a listening queue, sleep timers, deep iOS integration, and more. These features and experiences are just as important to us as the ad skipping.
Conclusion
Our hope is that earsay can mend the broken relationship many listeners now have with the podcast ecosystem while not negatively impacting content creators' download numbers. We think we are hitting this mark, though the job is not yet done.
