HK Tech Forum on
Data Science and AI

Presented by: Prof Nick SAHINIDIS
Georgia Institute of Technology

This talk presents recent theoretical, algorithmic, and methodological advances for black-box optimization problems for which optimization must be performed in the absence of an algebraic formulation, i.e., by utilizing only data originating from simulations or experiments. We investigate the relative merits of optimizing surrogate models based on generalized linear models and deep learning. Additionally, we present new optimization algorithms for direct data-driven optimization. Our approach combines model-based search with a dynamic domain partition strategy that guarantees convergence to a global optimum. Equipped with a clustering algorithm for balancing global and local search, the proposed approach outperforms existing derivative-free optimization algorithms on a large collection of problems.

Presented by: Prof Dacheng TAO
Senior Vice President of JD.com

Big data contains a tremendous amount of dark knowledge. The community has realized that effectively exploring and using such knowledge is essential to achieving superior intelligence. How can we effectively distill the dark knowledge from ultra-large-scale data? One possible answer is: "through Transformers". Transformers have proven their prowess at extracting and harnessing the dark knowledge from data. This is because more is truly different when it comes to Transformers. In this talk, I will showcase our recent work on transformers named ViTAE, on many dimensions of “more” including: model parameters, labeled and unlabeled data, prior knowledge, computing resource, tasks, and modalities. Specifically, ViTAE has more model parameters and more input modality support; ViTAE can absorb and encode more data to extract more dark knowledge; ViTAE is able to adopt more prior knowledge in the form of biases and constraints; ViTAE can be easily adapted to larger-scale parallel computing resources to achieve faster training. ViTAE has been applied to many computer vision tasks and has proven its promise, such as image recognition, object detection, semantic segmentation, image matting, pose estimation, scene text understanding, and remote sensing. You can find the source code for this work at https://urldefense.com/v3/__https://github.com/ViTAE-Transformer__;!!KjDnqvtInNPT!i3k4KWhRFknZtIFfMTpzLqUDfieMXqgmXkORXFmLHAcjGZvAv7BghfKgV3Yw-Splx2SLPXR89JqAANOLPKK06dM6$ .

Presented by: Prof Qiang YANG
Hong Kong University of Science and Technology

Federated learning is an important intersection of AI, big data and privacy computing. How to make federated learning safe, trustworthy and efficient at the same time is the focus of future industry and research. In my lecture, I will systematically review the progress and challenges of AI and introduce federated learning as a secure distributed approach to AI modeling. I will then discuss several important research and application directions.

Presented by: Prof Jun WANG
City University of Hong Kong, China

The past four decades witnessed the birth and growth of neurodynamic optimization, which has emerged as a potentially powerful problem-solving tool for constrained optimization due to its inherent nature of biological plausibility and parallel and distributed information processing. Despite the success, almost all existing neurodynamic approaches a few years ago worked well only for optimization problems with convex or generalized convex functions. Effective neurodynamic approaches to optimization problems with nonconvex functions and discrete variables are rarely available. In this talk, a collaborative neurodynamic optimization framework will be presented. Multiple neurodynamic optimization models with different initial states are employed in the framework for scatter local search. In addition, a meta-heuristic rule in swarm intelligence (such as PSO) is used to reposition neuronal states upon their local convergence to escape local minima toward global optima. Experimental results will be elaborated to substantiate the efficacy of several specific paradigms in this framework for nonnegative matrix factorization, supervised learning, vehicle-task assignment, portfolio selection, and energy load dispatching.

Presented by: Prof Kay Chen TAN
University of Victoria, Canada

It is known that the processes of learning and transfer of what has been learned are important to humans for solving complex problems. However, the study on optimization methodologies via learning from existing solutions and the transfer of what has been learned to help on solving related or unseen problems, has been under-explored in the context of evolutionary computation. This talk will give an overview of evolutionary transfer optimization (ETO), which is an emerging research direction that integrates evolutionary algorithm solvers with knowledge learning and transfer across different problem domains to achieve better optimization efficiency and performance. It will present some recent research work in ETO for solving multi-objective and large-scale optimization problems via high-performance computing. Some discussions on future ETO research directions, including topics such as theoretical analysis and real-world applications, will also be given.

Presented by: Prof Yingying FAN
University of Southern California, USA

As a flexible nonparametric learning tool, random forests algorithm has been widely applied to various real applications with appealing empirical performance, even in the presence of high-dimensional feature space. Unveiling the underlying mechanisms has led to some important recent theoretical results on the consistency of the random forests algorithm and its variants. However, to our knowledge, all existing works concerning random forests consistency in high dimensional setting were established for various modified random forests models where the splitting rules are independent of the response. In light of this, in this paper we derive the consistency rates for the random forests algorithm associated with the sample CART splitting criterion, which is the one used in the original version of the algorithm (Breiman2001), in a general high-dimensional nonparametric regression setting through a bias-variance decomposition analysis. Our new theoretical results show that random forests can indeed adapt to high dimensionality and allow for discontinuous regression function. Our bias analysis characterizes explicitly how the random forests bias depends on the sample size, tree height, and column subsampling parameter. Some limitations on our current results are also discussed.

Presented by: Dr Chuchu FAN
Massachusetts Institute of Technology (MIT), USA

The introduction of machine learning (ML) creates unprecedented opportunities for achieving full autonomy. However, learning-based methods in building autonomous systems can be extremely brittle in practice and are not designed to be verifiable. In this talk, I will present several of our recent efforts that combine ML with formal methods and control theory to enable the design of provably dependable and safe autonomous systems. I will introduce our techniques to generate safety certificates and certified decision and control for complex large-scale multi-agent autonomous systems, even when the agents follow nonlinear and nonholonomic dynamics and need to satisfy high-level specifications.

Presented by: Prof Peng SHI
University of Adelaide, Australia

Cyber-physical systems (CPSs) are the mechanisms controlled or monitored by computer-based algorithms, tightly integrated with the internet and its users. CPSs are the central research topic in the era of Industrial 4.0, and continue to be in the forthcoming Industrial 5.0, which have attracted a lot attention in the past years. Undergoing an ever-enriching cognitive process, CPSs deeply integrates control, communication, computation, cloud and cognition. In this talk, we firstly review some basic knowledge with respect to the concepts, history, and some viewpoints on CPS security. Next, some commonly appeared malicious threats will be introduced.

Presented by: Dr Qingpeng ZHANG
City University of Hong Kong, China

In this talk, I will introduce an end-to-end deep learning framework based on a protein–protein interaction (PPI) network to make synergistic anticancer drug combination predictions. The framework, namely GraphSynergy, adapts a spatial-based Graph Convolutional Network component to encode the high-order topological relationships in the PPI network of protein modules targeted by a pair of drugs, as well as the protein modules associated with a specific cancer cell line. The pharmacological effects of drug combinations are explicitly evaluated by their therapy and toxicity scores. An attention component is also introduced in GraphSynergy, which aims to capture the pivotal proteins that play a part in both PPI network and biomolecular interactions between drug combinations and cancer cell lines. GraphSynergy outperforms the classic and state-of-the-art models in predicting synergistic drug combinations on the 2 latest drug combination datasets. Specifically, GraphSynergy achieves accuracy values of 0.7553 (11.94% improvement compared to DeepSynergy, the latest published drug combination prediction algorithm) and 0.7557 (10.95% improvement compared to DeepSynergy) on DrugCombDB and Oncology-Screen datasets, respectively. Furthermore, the proteins allocated with high contribution weights during the training of GraphSynergy are proved to play a role in view of molecular functions and biological processes, such as transcription and transcription regulation. This research indicates that introducing topological relations between drug combination and cell line within the PPI network can significantly improve the capability of synergistic drug combination identification.

Presented by: Prof Qingfu ZHANG
City University of Hong Kong, China

Many optimization problems in the real world, by nature, have multiple conflicting objectives. Unlike a single optimization problem, multiobjective optimization problem has a set of Pareto optimal solutions (Pareto front) which are often required by a decision maker. Evolutionary algorithms are able to generate an approximation to the Pareto front in a single run, and many traditional optimization methods have been also developed for dealing with multiple objectives. Combination of evolutionary algorithms and traditional optimization methods should be a next generation multiobjective optimization solver. Decomposition techniques have been well used and studied in traditional multiobjective optimization. Over the last decade, a lot of effort has been devoted to build efficient multiobjective evolutionary algorithms based on decomposition (MOEA/D). In this talk, I will describe main ideas and techniques and some recent development in MOEA/D. I will also discuss some possible research issues in multiobjective evolutionary computation.

Presented by: Prof Ruth MISENER
Imperial College London, UK

This talk introduces OMLT (https://github.com/cog-imperial/OMLT), an open source software package incorporating surrogate models, which have been trained using machine learning, into larger optimisation problems. Computer science applications include maximizing a neural acquisition function and verifying neural networks. Engineering applications include the use of machine learning models to replace complicated constraints in larger design/operations problems. OMLT 1.0 supports GBTs through an ONNX (https://github.com/onnx/onnx) interface and NNs through both ONNX and Keras interfaces. We discuss the advances in optimisation technology that made OMLT possible and show how OMLT seamlessly integrates with the python-based algebraic modeling language Pyomo (http://www.pyomo.org). The literature often presents different optimization formulations as competitors, but in OMLT, competing formulations become alternatives: users can select the best for a specific application. We provide examples including neural network verification, autothermal reformer optimization, and Bayesian optimization.

This work is joint with the Imperial Computational Optimisation Group (Francesco Ceccon, Ruth Misener, Alexander Thebelt, Calvin Tsay), Sandia National Laboratories (Jordan Jalving, Joshua Haddad), and Carnegie Mellon (Carl Laird).

Presented by: Prof Fengqi YOU
Cornell University, USA

Quantum computing is attracting growing interest due to its unique capabilities and disruptive potential. This presentation will briefly introduce quantum computing and its potential applications to systems optimization and machine learning. We will introduce several new algorithms and methods that exploit the strengths of quantum computing techniques to address the computational challenges of classically intractable optimization problems. Applications include molecular design, manufacturing systems operations, and supply chain optimization. In the second half of the presentation, we will focus on quantum machine learning and the emerging hybrid classical-quantum computing paradigm that exploit the strengths of quantum computing techniques to address the computational challenges of important AI-related problems. The presentation will conclude with a novel deep learning model and quantum computing algorithm for efficient and effective fault diagnosis in manufacturing and electric power systems.

Presented by: Prof Yiran CHEN
Duke University, USA

Federated learning has become a popular distributed machine learning paradigm for developing on-device AI applications. However, the data residing across devices is intrinsically statistically heterogeneous (i.e., non-IID data distribution) and mobile devices usually have limited communication bandwidth to transfer local updates. Such statistical heterogeneity and communication limitation are two major bottlenecks that hinder applying federated learning in practice. In addition, recent works have demonstrated that sharing model updates makes federated learning vulnerable to inference attacks and model poisoning attacks. In this talk, we will present our recent works on novel federated learning frameworks to address the scalability and heterogeneity issues simultaneously. In addition, we will also reveal the essential reason the of adversarial vulnerability of deep learning models and the privacy leakage in federated learning procedures, and provide the defense mechanisms accordingly towards trustworthy federated learning.