Audio Inference

Leading diverse human-robot or human-AI teams is slowly becoming the new normal. As automation and AI become integral to many industries, leaders must learn how to manage teams that include humans and AI/robots. Although there is plenty of training on how to use the technology, however, no one is teaching leaders how to lead the technology, thus creating skills gaps and workforce adaptability issues. As organisations face digital disruption, leaders must integrate these digital innovations into their operations and inspire their teams to become ‘technologically savvy’. However, despite the pressing need to better understand how to cultivate cohesive team dynamics amidst human and AI entities, research in this field is limited, which is problematic as leaders lack evidencebased solutions to deal with these novel challenges. Therefore, this project aims to unlock the potential of human-AI collaboration by:

1) uncovering the essential leadership qualities needed to effectively lead human-AI teams; 2) optimizing the design of virtual teams, here humans and AI work symbiotically; 3) pioneering the use of the newest AI for accurately assessing individual traits through natural language analysis; 4) investigating how feedback acceptance varies when it comes from human or AI sources. This research integrates the scientific fields of leadership, organisational psychology and IT, which pushes the boundaries of traditional research domains. Moreover, it is built on the newest technological trends and employs cutting-edge deep neural network technologies offering a state-of-the-art exploration of the subject. All in all, by achieving these objectives, this project will reshape the future of work, empowering organizations to harness the full potential of human/AI teams, enhance leadership capabilities, and optimize team dynamics in this ‘brave new world’ of workplace digitalisation.

Current advances in artificial intelligence (AI) are supported by artificial neural networks (ANNs). Such networks are inspired by the connectivity in biological networks, but communicate using real valued mathematical functions. By contrast, real biological networks are known to communicate using short electrical bursts, or
spikes. Networks constructed to emulate this mechanism are known as spiking neural networks (SNNs). SNNs have arisen from the neuroscience community as a means to understand the function of the brain, and as tempting solutions to physiological functions such as speech and image processing. They have also been addressed by the neuromorphic community, hoping to build circuits that mimic the brain whilst also being power
efficient.

MORPhyN arises from work at Idiap building on these concepts applied to speech processing, showing that SNNs can be freely mixed with ANNs. We have found that SNNs present two distinct mathematical problems that have not been fully addressed. The first concerns the large number of hyperparameters; it is beyond what can be reasonably handled by hand. The second is the tendency for the training to get stuck due to the binary activation function.

Bayesian optimisation (BO) presents an attractive solution to the hyperparameter problem in SNNs. It is a principled approach to searching the multi-dimensional hyperparameter space that has already shown promise in enhancing the performance and efficiency of ANNs.
However, spike-based learning rules and discrete events introduce challenges in modelling and optimising parameters. Research is required to address these challenges and, e.g., to determine the most suitable Gaussian process kernels for effectively modelling non-stationarity within SNNs.
The core of MORPhyN is built around the application of the BO expertise from Mahidol to the SNNs for which expertise exists at Idiap.
In addition, the Idiap side brings expertise in Bayesian learning, which promises to address the parameters themselves to smooth the loss landscape associated with the binary nature of SNNs. The Mahidol side also brings expertise in the vehicle routing problem, a classical NP-hard application, essentially a generalisation of the travelling salesman problem. ANNs have been shown to be good heuristic solutions to routing via their graph processing abilities; SNNs promise to improve on this.
In MORPhyN, we propose to combine the above four techniques in three threads representing three objectives:
1. In a core collaborative thread, we aim to enable BO for SNNs. By better handling the hyperparameters, we aim advance the state of the art in both fields, giving SNNs the same or better performance compared to current ANNs.
2. In an application oriented thread, researchers primarily at Mahidol will enable the use of both ANNs and SNNs in the context of vehicle routing. Routing is more analytic than speech recognition, promising deeper insight into SNN function. A more general aim will be to show that SNNs are beneficial for applications without the physiological bases of speech and image processing.
3. In a more theoretical thread, researchers primarily at Idiap will apply Bayesian learning techniques to SNNs to both smooth out error landscapes and better interface to BO. This will yield not only better SNN performance, but also a complete theory of their operation.
Overall, we aim to put SNNs on a par with ANNs in terms of utility and accessibility, allowing the community to freely choose between ANN and SNN architectures depending on the application. This in turn will not only allow new applications being particularly suited to SNNs, but will allow more energy efficient implementation of current applications on neuromorphic hardware.

In addition to being complementary in terms of technical capabilities, the partners take advantage of the SPIRIT program to address gender equality. Whilst Idiap, as an ICT-focused institute, suffers from male bias, the partnership with Mahidol brings the opportunity of a female PI and a more balanced student cohort.

The main objective of the present proposal is to advance the state of the art in spiking neural networks (SNNs), in particular for generative models.  It arises from work done under a recently completed SNSF project Neural Architectures for Speech Technology (NAST), which was concerned with SNNs and text to speech synthesis (TTS, a particular application of generative modelling), albeit separately.

SNNs have potentially better representational capability than their artificial counterparts, and much lower power consumption.  Under the SNN thread of NAST, we were able to show that state of the art performance currently depends on the concept of surrogate gradients.  Such surrogates are important when the gradients associated with spikes are undefined or flat.  Under the TTS thread of NAST, we were also able to show that state of the art performance depends on denoising approaches, notably diffusion.  Diffusion defines a forward process that gradually warps a complex distribution into a simple one such as Gaussian noise; the generative process is then the reverse of this.

Recent literature has shown that well defined noise implies a concrete definition of surrogate gradients, which are otherwise quite ad-hoc.  This and the observations above lead to a first concrete goal of the present proposal: to show that generative modelling using spiking implies a much tighter integration of the noise associated with surrogates and with diffusion.  This in turn has two potential outcomes.  A first is state of the art generative modelling that can benefit from the advantages of spiking.  The other is a principled approach to spiking that should improve the performance of SNNs in general applications.

Another observation from NAST is that the membrane potential inside the neurons has a similar impulse response to the muscle models previously studied by the applicant.  Such muscle models were shown to lead to more natural sounding speech synthesis.  Generally, the literature on use of spiking for regression (hence generative modelling) is quite sparse.  A key issue is how to convert the binary spikes of the SNN into continuous outputs; approaches such as convolution layers are available, but ad-hoc.  This leads to a second concrete goal of the proposal: to show that natural speech synthesis can arise from the principled use of the continuous signals available inside the spiking neurons, or from physiologically derived models.  The potential outcome, dependent on the first goal above, is good natural TTS using SNNs.

A third, more speculative, goal arises from the literature and observations from NAST, that the current state of the art in generative modelling is based on score matching and probability flow.  These related concepts can be seen as continuous versions of the discrete diffusion described above.  Typically, they require differential equation solvers to discretise the continuous process.  The goal, then, is to combine the results above with these state of the art techniques to yield truly state of the art performance.  At the outset, we note that the equation solvers use the same (Euler) methods as the SNN neurons and the diffusion processes, suggesting that advances could be built on these relationships.

The research plan is written as three task groups that broadly map onto the three goals above.  Each task group has baseline and  development phases, leading to two, more speculative, final tasks.  This enables a graded risk profile with the potential to truly innovate, whilst supporting mitigation by backing off to development tasks or focusing on just one speculative task.

Most tasks have both practical and theoretical components.

The overall relevance and impact of the proposal is detailed in terms of five areas: scientific output, software, collaboration, teaching, and industrial transfer.