The encrypted messaging service owned by Facebook acknowledged that a security flaw allowed hackers to install spyware on mobile phones.