Emeritus Professor of Physics,
President Advisor, Tianjin University-NUS Joint Institute,
Fuzhou, China
Emeritus Executive Vice-President
National University of Singapore
MY LATEST RESEARCH
Wei Wenbo, Chong Jia Le Nicholas, Lai Choy heng, Feng Ling
​Multiple Descents in Deep Learning as a Sequence of Order-Chaos Transitions
AI4X 2025, Singapore, 8-11 July 2025
​
In this study, we introduce a novel "multipledescent" phenomenon in Long Short-Term Memory (LSTM) networks, where test loss cycles repeatedly, increasing slowly and then abruptly decreasing during training. We found out that that these cycles are linked to phase transitions between order and chaos in the network’s dynamics, where the abrupt drop of test loss is often due to the sudden chaos order transition. Our research focuses on an LSTM trained on the Large Movie Review Dataset for sentiment analysis, revealing that the optimal performance occurs at the first transition from order to chaos, where the network’s "edge of chaos" is widest, facilitating the best exploration of weight configurations.
​
Lin Zhang, Ling Feng, Kan Chen & C H Lai​
Asymptotic Edge of Chaos as Guiding Principle for Neural Network Training
International Journal of Artificial Intelligence and Robotics Research, Vol. 01, 2350001 (2024)
​
It has been recently demonstrated that optimal neural networks operate near the asymptotic edge of chaos for state-of-the-art feed-forward neural networks, where its generalization power is maximal due to the highest number of asymptotic metastable states. However, how to leverage this principle to improve the model training process remains open. Here, by mapping the model evolution during training to the phase diagram in the classic analytic result of Sherrington–Kirkpatrick model in spin glasses, we illustrate on a simple neural network model that one can provide principled training of the network without manually tuning the training hyper-parameters. In particular, we provide a semi-analytical method to set the optimal weight decay strength, such that the model will converge toward the edge of chaos during training. Consequently, such hyper-parameter setting leads the model to achieve the highest test accuracy. Another benefit for restricting the model at the edge of chaos is its robustness against the common practical problem of label noise, as we find that it automatically avoids fitting the shuffled labels in the training samples while maintaining good fitting to the correct labels, providing simple means of achieving good performance on noisy labels without any additional treatment.
​
S Ma, L Feng, C P Monterola and C H Lai
Importance of small degree nodes in assortative networks with degree-weight correlations
Physical Review E 96, 042308
It has been known that assortative network structure plays an important role in spreading dynamics for unweighted networks. Yet its influence on weighted networks is not clear, in particular when weight is strongly correlated with the degrees of the nodes as we empirically observed in Twitter. Here we use the self-consistent probability method and revised non-perturbative HMF method to investigate this influence on both SIR and SIS spreading dynamics. Both our simulation and theoretical results show that while the critical threshold is not significantly influenced by the assortativity, the prevalence in the supercritical regime shows a cross-over under different degree-weight
correlations. In particular, unlike the case of random mixing networks, in assortative networks, the negatively degree-weight correlation leads to higher prevalence in their spreading beyond the critical transmissivity than that of the
positively correlated. In addition, the previously observed inhibition effect on spreading velocity by assortative structure is not apparent in negatively degree-weight correlated networks, while enhanced for that of the positively correlated. Detailed investigation into the degree distribution of the infected nodes reveals that small degree nodes play essential roles in the supercritical phase of both SIR and SIS spreadings. Our results have direct implications on viral information spreading over online social networks and epidemic spreading over contact networks.
H S Sugiarto, J S Lansing, N N Chung, C H Lai, S A Cheong and L Y Chew,
Social cooperation and disharmony in communities mediated through common pool resource exploitation
Physical Review Letters 118, 208301 (2017)
Phys.Org News: https://phys.org/news/2017-05-phase-transitions-rice-farmers-insight.html
It was theorized that when a society exploits a shared resource, the system can undergo extreme phase
transition from full cooperation in abiding by a social agreement, to full defection from it. This was shown
to happen in an integrated society with complex social relationships. However, real-world agents tend to
segregate into communities whose interactions contain features of the associated community structure. We
found that such social segregation softens the abrupt extreme transition through the emergence of multiple
intermediate phases composed of communities of cooperators and defectors. Phase transitions thus now
occur through these intermediate phases which avert the instantaneous collapse of social cooperation within a society. While this is beneficial to society, it nonetheless costs society in two ways. First, the return to full cooperation from full defection at the phase transition is no longer immediate. Community linkages have rendered greater societal inertia such that the switch back is now typically stepwise rather than a single change. Second, there is a drastic increase in social disharmony within the society due to the greater tension in the relationship between segregated communities of defectors and cooperators. Intriguingly, these results on multiple phases with its associated phenomenon of social disharmony are found to characterize the level of cooperation within a society of Balinese farmers who exploit water for rice production.
​
N N Chung, L Y Chew, W Chen, R M D'Souza and C H Lai
​Susceptible individuals drive active social contagion
Physical Review Research 1, 033125 (2019)
​
The catalyzing role that influential individuals play in driving large-scale social contagion has been the focus of many recent empirical and theoretical studies. Nevertheless, complementary studies suggest that the success of social contagion depends largely on having a group of highly susceptible individuals. Here we show that being influential is not due solely to having susceptible peers and that the relative importance of influential versus susceptible individuals depends on the underlying mechanism driving the contagion. Our mathematical models show that for passive social influence both influential and susceptible individuals catalyze the spread of the contagion progressively and equitably. In contrast, for active social influence, susceptible individuals facilitate rapid initial contagion, but then influential individuals are necessary for sustaining the growth of the contagion. We show that active influence characterizes certain online social networks, like retweeting behavior on Twitter, while passive influence characterizes other online social networks, like adoption of whom to follow on the Digg network.
