Distributed Real-Time Low-Latency Audio I/O Framework for Mobile Devices

­Technology Description:

      We present a system and method of a distributed real-time low-latency audio input/output (I/O) framework for mobile devices. The framework relates generally to real-time audio I/O mobile devices featuring a processing unit and some type of wireless communication capabilities that can be used for real-time wireless transmission among a group of multiple similar devices without the requirement of internet access. The purpose of the framework is to simulate an irregular and flexible microphone array using two or more mobile devices. The proposed framework is a purely software solution that does not require any additional hardware, and is able to synchronize and process the real-time audio input of multiple devices into real-time output, while achieving the lowest possible I/O latency allowed by the hardware/software specifications of the devices.

Value Proposition:

Provides an alternative to other potentially expensive and unpractical solutions for beamforming-based algorithms.

Key Benefits:

• Speed: ability to synchronize and process real-time audio input of multiple devices into real-time output, while achieving the lowest possible I/O latency allowed by the hardware/software specifications of the devices, noting that several modern mobile devices guarantee continuous audio round-trip latencies below 20ms.
• Cost: No additional components beyond the network devices are required to be obtained as the system is a software-only solution.  In fact, no external server is required, in contrast to some contemporary solutions.
• Foundational: The framework can aid the design of high performing sound source localization (SSL) and speech enhancement algorithms based on multi-channel audio processing.

Applications:

• Interactive smartphone applications: The proposed framework can be used as a building block for developing interactive smartphone applications based on multi-channel audio signal processing.
• Beamforming. Can be used in any application employing beamforming techniques.
• Hearing aids: Low latency (real-time) This makes it possible to use mobile devices for improving hearing aid as well as many other applications.

IP Status: US Provisional (Filed/Pending)

ID: 21029

Primary Inventor: Dr. Issa Panahi

Contact: otc@utdallas.edu

Additional Information: www.utdallas.edu/~issa.panahi/

Fig. 1: Demo of the proposed framework as an Android application running on two different Android devices. The device on the left is operating in slave mode. The device on the right is operating in master mode. Both devices are connected through Wi-Fi Direct and each device includes three built-in microphones. Hence, in this example, the master device has synchronous access to six irregularly spaced microphones. As a demonstration of the functionality of proposed system, the user interface (UI) of the master device allows the user to listen to the synchronized multi-channel audio inputs in real-time and with minimal I/O latency. The application also allows the option of saving different data, including the recordings, to file for offline analysis.

Fig S1. Master/slave smartphone-based distributed audio I/O example. In this example, there is one master (middle) and four slaves. The geometry of the network, as well as the number of slaves and masters is flexible and not limited to this example. The devices can be moving or in a fixed position.