Major nations have organizations which act in this role. The methods of how this is done can be through surveillance of suspects and their communications, undercover agents and similar means.
When spying is discovered, those agencies do have the power to arrest, but it is often more productive to keep a careful eye on them to see what they know. Furthermore, disinformation can be used to fool the spies and their sponsors, or make them cease their activities if they learn their information has become unreliable and/or their secrecy has been compromised.
Counter-Intelligence Agencies