He attended Duke University, where he received his Bachelor's degree in Mechanical Engineering in 1946 and his Ph.D in Mathematics in 1953. Cocke spent his entire career as an industrial researcher for IBM, from 1956 to 1992.
Perhaps the project where his innovations were most noted was in the IBM 801 minicomputer, where his realization that matching the design of the architecture's instruction set to the relatively simple instructions actually emitted by compilers could allow high performance at a low cost.
Cocke won the ACM Turing Award in 1987 "for significant contributions in the design and theory of compilers, the architecture of large systems and the development of reduced instruction set computers (RISC); for discovering and systematizing many fundamental transformations now used in optimizing compilers including reduction of operator strength, elimination of common subexpressions, register allocation, constant propagation, and dead code elimination". This contribution is notable for its breadth across many fields.
He also received the National Medal of Science in 1994 amongst many other honours.