Important branches include proof theory, model theory, and mathematical logic. The original meaning of David Hilbert is closest to proof theory.
Many issues regarding the foundations of mathematics (there is no longer necessarily considered to be any one "problem") and the philosophy of mathematics touch on or use ideas from metamathematics. The working assumption of metamathematics is that mathematical content can be captured in a formal system.
On the other hand, quasi-empiricism in mathematics, the cognitive science of mathematics, and ethno-cultural studies of mathematics, which focus on scientific method, quasi-empirical methods or other empirical methods used to study mathematics and mathematical practice by which such ideas become accepted, are non-mathematical ways to study mathematics.