This volume is concerned with optimal control problems governed by ordinary differential systems and partial differential equations. The emphasis is on first-order necessary conditions of optimality and the construction of optimal controllers in feedback forms. These subjects are treated using some new concepts and techniques in modern optimization theory, such as Clarke's generalized gradient, Ekeland's variational principle, viscosity solution to the Hamilton--Jacobi equation, and smoothing processes for optimal control problems governed by variational inequalities. A substantial part of this book is devoted to applications and examples. A background in advanced calculus will enable readers to understand most of this book, including the statement of the Pontriagin maximum principle and many of the applications. This work will be of interest to graduate students in mathematics and engineering, and researchers in applied mathematics, control theory and systems theory.