JAudIO ß library.

Its been a while since I updated the blog. I've been busy working on improvements and adding more features to it. I will try to list  major / minor changes, feature additions to the API here.
There have been several performance improvements, especially, while reading large uncompressed files. Consequently, now the API can now read and render files upto 3 hours of uncompressed PCM encoded audio files in less than 2 mins ( I will do a benchmarking in the coming weeks here).  

Now, these handy tools are a part of library called JAudIO^ß

The code samples are updated at JAudIO@GitHub.

MultiChannelAudioUI

I have been pondering on an idea about having multiple visualizations for the same underlying piece of audio. The ability to able to visualize all the channels or having different displays for the same channel, at the same time having the same set of controls.

MultiChannelAudioUI or MUI does exactly this job. Although, it doesn't do anything by it self. It acts just as a wrapper around the AudioUI.  Here is how it works,


In this example, we are going to display two channels for a stereophonic audio ( 2 channeled audio) .

Lets start by doing the initial setups for AudioUIs one for each channel,

// initial setup for audioui-1

ch1ui = new AudioUI();

ch1ui.attachUIComponent(AudioUI.UIComponent.CONTAINER,ch1Panel); ch1ui.setChannelToRead(1); // read channel 1 

 // intial setup for audioui-2

ch2ui = new AudioUI();

ch2ui.attachUIComponent(AudioUI.UIComponent.CONTAINER, ch2Panel);

ch2ui.setChannelToRead(2); // read channel 2

Since MUI is acts a wrapper, the initial settings for individual channels has to be setup here.  

Although, it is still valid to attach the controllers for individual audioUIs, for eg,

ch1ui.attachUIComponent(AudioUI.UIComponent.PLAY, playAudio);

ch1ui.attachUIComponent(AudioUI.UIComponent.PAUSE, pauseAudio);

ch1ui.attachUIComponent(AudioUI.UIComponent.SEEKER, audSlider);

Since, the idea is to have one common set of controls for both the channels, a sort of "tying" both channels with the same controls, we use MUI for the setting the controls,

MultiChannelAudioUI mui = new MultiChannelAudioUI();

 // configure multichannel ui

mui.attachUIComponent(MultiChannelAudioUI.UIComponent.PLAY, playAudio);

mui.attachUIComponent(MultiChannelAudioUI.UIComponent.PAUSE, pauseAudio);

mui.attachUIComponent(MultiChannelAudioUI.UIComponent.SEEKER, audioSlider);

NOTE:
Any existing controls to the AudioUIs will be overwritten when the tracks are added to this MUI.

Earlier, in my blog post, I mentioned that the updated API expects the type display before interacting with the displays, for eg,

// Configure myAudio to use container display type "waveform"
 myAudio.setContainerDisplay(AudioDisplay.TYPE.WAVEFORM);


In this case, the initialization of the individual channels takes place "inside" MUI. Hence, there is no need to have separate setting the display for individual channels. This is can be done while adding tracks to MUI. In this case,

 // add audio uis

mui.addTrack(ch1ui, AudioDisplay.TYPE.WAVEFORM);

mui.addTrack(ch2ui, AudioDisplay.TYPE.WAVEFORM);

The audio source for each channel is done by using MUI but not the individual UIs.

mui.setAudioFile(mySterioAudioFile);


that is, it doesnot make sense in using,

ch1ui.setAudioFile(mySterioAudioFile);
ch2ui.setAudioFile(mySterioAudioFile);

Thats it.

All the interactions with the channels are performed using the individual UIs. For example,

// Set channel1 to mute 
ch1ui.setAudioMute(true);
// Zoom in channel 2 ui
ch2ui.getDisplay(AudioDisplay.TYPE.WAVEFORM).zoomIn();



 Here, is a small demo

Feature List / Updates / Changes

NonTrivialAudio:

•  Added method to retrieve raw data:

int[] channelRawData = instanceOfNonTrivialAudio.getAudioDataRaw(channel);

• Added method to retrieve data from all channel at once.  

Double[][] allChannelNormData = instanceOfNonTrivialAudio.getAllChannelAudioNormData(); 

NOTE:
Retrieving data from all channels requires a relatively large heap space. Make sure to increase heap space before hand to avoid OutOfMemoryException 

• Added method to retrieve data utilizing all the available CPU cores.  

 Double[] channelNormData = instanceOfNonTrivialAudio.getAudioNormData_multicore(channel,chunk_dur_secs); 

This is one of the major improvements that I've been working on since some time already. I finally managed to finish it off.  I am going to talk a lot about this on a separate blog post.

• Added method to mute the audio. 

 instanceOfNonTrivialAudio.setMute(bool_state);

I received a suggestion from a user suggesting to add volume controls. I thought it would be a good addition to the existing framework. As a result, this method would mute the audio that currently is being played. 

I also wrote a method to explicitly adjust the volume via setVolumeLevel(norm_vol_level)but, its experimental. I will finalize it in the coming versions.

 

 AudioUI:

• Now, the type of display needs to be specified before interacting with the display.

// Get the file 

audFile = new File("/path/to/file/someFile.wav");

// Channel to read 

myAudio.setChannelToRead(channel);

// Attach the file 

 myAudio.setAudioFile(audFile); 

 // Suggest the type of display desired -- MUST call!

 // Must be called after the intial UI setup myAudio.setContainerDisplay(AudioDisplay.TYPE.WAVEFORM); 

 // Setup the display

myAudio.getDisplay(AudioDisplay.TYPE.WAVEFORM).setTimePrecision(4);

myAudio.getDisplay(AudioDisplay.TYPE.WAVEFORM).setZoomLevel(1);

... 

• Initial UI setup is now performed uniformly using attachUIComponent

//     Initial UI Setup

Before,

myAudio.setDisplayContainer(myPanel);

myAudio.setUIPlay(playAudio);  

myAudio.setUIPause(stopAudio);

myAudio.setUISeeker(audSlider);
      

Now,   

        myAudio.attachUIComponent(AudioUI.UIComponent.CONTAINER, myPanel);
        myAudio.attachUIComponent(AudioUI.UIComponent.PLAY, playAudio);
        myAudio.attachUIComponent(AudioUI.UIComponent.PAUSE, stopAudio);
        myAudio.attachUIComponent(AudioUI.UIComponent.SEEKER, audSlider);

• Fixed issues with the rendering. 

Now, AudioUI can render short bits as well as long audio files (<100 ms  to 3 hours ). 

AudioUI automatically decides to utilize a single core or all the cores available based on its duration to ensure fast rendering of the data. It uses NonTrivialAudio#getAudioNormData_multicore for large audio files. 

• Added setMute, isMute and toggleMute option. 

• Other minor bugfixs and performance updates.   

Created MultiChannelAudioUI for interacting with multiple channel / displays with a common set of controls. Read all about it here

Multicore data reader

To be updated.