(Note the first two of this are instruments without optionality, the term vanilla is applied to these too).
Calibration and hedging are key factors in determining which instruments are called vanilla. For example a bermudan swaption may be hedged by a collection of european swaptions. Thus a trader in the bermudan market will likely think of european swaptions as the vanilla. Further a model developed to price bermudan options may be calibrated (i.e. have its parameters chosen) such that the model gives the same prices as those found in the market for european swaptions. That is, the model takes the european prices as given and uses them to determine a price for the more complex bermudan product. Again the term vanilla describes the european swaptions and exotic describes the bermudan.
Even with the above caveats there are instruments almost universally termed vanilla in various markets. In the foreign exchange and stock markets the most vanilla options are european call and put options. In the interest rate market swaps and caps are the vanilla instruments, although the former does not actually involve any optionality. European swaptions are sufficiently liquid to be called vanilla. The credit derivatives market is less mature. Here the three most traded instruments are total return swaps, credit default swaps and credit spread options. These are widely expected to become the mainstream vanilla products of the market in the coming years.