CaTe: A Caption Template calculation program
In a joint project with the Australian
Captioning Center, we explore the use of MPEG video and audio
analysis software to support captioners (caption = subtitles for
the deaf). To that end, we use MPEG Maaate, the Mediaware
Systems video analysis library and some additional heuristic
rules to create a segmentation of the audio track that resembles
closely to the blocks that captioners produce for filling in the
text. We thus create blank caption files (EBU file format) which
can be uploaded into any existing captioning software to enter the
caption text itself.
Sports Highlights extraction program
CSIRO digital media information systems researchers have used
MPEG Maaate functionality in the development of a system
to extract sports highlights from sports footage. The system
uses both audio and video features of footage to determine
segments containing a highlight such as scoring a goal or an
attack. It uses crowd cheers detected via segmentation of noisy
regions plus camera operations such as pan and zoom to determine
the highlights with every sport having a different set of
heuristic rules that determine a highlight.
More application ideas:
- Automatic segmentation of large sound recordings e.g. story
segmentation of news
- Sound search engine based on specific similarities to a
given sample file
- Automatic classification of large sound file collections
into specific classes e.g. different animals
- Browsing large sound file collections based on similarities
or sound models
- Support for sound editing tools e.g. for filtering out crazy
noises
Here's a radio interview of Silvia about the use of audio
analysis technology for online music searching:
|
