Black-box optimization and machine learning
A principal challenge in optimization practice is how to optimize in the absence of an algebraic model of the system to be optimized. We are interested in problems for which algebraic models are (1) intractable to conventional optimization software (for instance, due to discontinuities, non-smoothness, or excessive computational cost of a function evaluation), or are (2) entirely unavailable (as in the case of many experimental systems and operating processes to be optimized). We refer to systems of this sort as ``black-box systems'' and assume that the black box can be queried through a simulation or experimental measurements that provide a system output for specified values of system inputs. Our work in this area focuses on the development of methodologies that rely on statistical and machine learning techniques to handle experimental and simulation data in conjunction with deterministic optimization methods to aid decision-making and model-building for black-box systems. The methods we develop have been applied to problems in bioinformatics, portfolio optimization, protein-ligand docking, carbon capture and sequestration, parameter tuning for optimization solvers, antenna optimization, optimization of polymerase chain reactions, protein structure alignment, and powder diffraction.
Derivative-free optimization (DFO)
Obtaining derivative information for many complex and expensive simulations is impractical. To tackle such systems, we maintain a comprehensive library of existing derivative-free algorithms, and perform extensive studies of their performance in various domains. Toward this end, we have performed a comparison of 22 software implementations for continuous DFO problems and a comparison of 13 software implementations for mixed-integer DFO problems. In addition, we are developing novel algorithms to address classes of derivative-free optimization problems while performing the fewest number of experiments.
Learning process models
We are developing a model generation methodology that uses derivative-based and derivative-free optimization alongside machine learning and statistical techniques to learn algebraic models of detailed simulations and experimental systems. Once a candidate set of models is defined, they are tested, exploited, and improved through the use of derivative-free solvers to adaptively sample new points. Of particular importance is the algorithm's ability to generate models that are simple yet accurate. We have implemented this strategy in ALAMO, a code for the Automatic Learning of Algebraic MOdels.
Simulation optimization
An added complication in the case of discrete-event simulations is the inherent stochasticity associated with their outputs. Our goals in this area are to (1) compile and compare existing methods for simulation optimization on new problem testbeds, and to (2) provide algorithms and implementations that handle uncertainty in data in a robust manner while locating optimal solutions.
Related Publications
- Ma, K., L. M. Rios, A. Bhosekar, N. V. Sahinidis and S. Rajagopalan, Branch-and-Model: A derivative-free global optimization algorithm, Computational Optimization and Applications, 85, 337-367, 2023.
- Sauk, B. and N. V. Sahinidis, Hyperparameter tuning of programs with HybridTuner, Annals of Mathematics and Artificial Intelligence, 91, 133-151, 2023.
- Engle, M. and N. V. Sahinidis, Deterministic symbolic regression with derivative information: General methodology and application to equations of state, AIChE Journal, 68, e17457, 2022.
- Ma, K., N. V. Sahinidis, R. Bindlish, S. J. Bury, R. Haghpanah and S. Rajagopalan, Data-driven strategies for extractive distillation unit optimization, Computers and Chemical Engineering, 167, 107970, 2022.
- Ma, K., N. V. Sahinidis, S. Amaran, R. Bindlish, S. J. Bury, D. Griffith and S. Rajagopalan, Data-driven strategies for optimization of integrated chemical plants, Computers and Chemical Engineering, 166, 107961, 2022.
- Zheng, C., X. Chen, T. Zhang, N. V. Sahinidis and J. J. Siirola, Learning process patterns via multiple sequence alignment, Computers and Chemical Engineering, 159, 107676, 2022.
- Na, J., J. H. Bak and N. V. Sahinidis, Efficient Bayesian inference using adversarial machine learning and low-complexity surrogate model, Computers and Chemical Engineering, 151, 107322, 2021.
- Ma, K., N. V. Sahinidis, S. Rajagopalan, S. Amaran and S. J. Bury, Decomposition in derivative-free optimization, Journal of Global Optimization, 81, 269-292, 2021.
- Ploskas, N., and N. V. Sahinidis, Review and comparison of algorithms and software for mixed-integer derivative-free optimization, Journal of Global Optimization, 2021.
- Ma, K., N. V. Sahinidis, S. Rajagopalan, S. Amaran and S. J. Bury, Decomposition in derivative-free optimization, Journal of Global Optimization, 81, 269-292, 2021.
- Sarwar, O., B. Sauk and N. V. Sahinidis, A discussion on practical considerations with sparse regression methodologies, Statistical Science, 35, 593-601, 2020.
- Ploskas, N., C. Laughman, A. U. Raghunathan, and N. V. Sahinidis, Optimization of circuitry arrangements for heat exchangers using derivative-free optimization, Chemical Engineering Research and Design, 131, 16-28, 2018.
- Amaran, S., N. V. Sahinidis, B. Sharda, and S. J. Bury, Simulation optimization: A review of algorithms and applications, Annals of Operations Research, 240, 351–380, 2016.
- Rios, L. M. and N. V. Sahinidis, Derivative-free optimization: A review of algorithms and comparison of software implementations, Journal of Global Optimization, 56, 1247–1293, 2013.
- Shah, S. B. and N. V. Sahinidis, SAS-Pro: Simultaneous residue assignment and structure superposition for protein structure alignment, PLoS ONE, 7(5): e37493, 2012.