There really is no technical moat. Any new architectures are going to be published because that's 100% the culture and AI folks won't work somewhere where that's not true. Training details/hyperparameters/model "build-ons" aren't published but those are a very weak moat.
The only moat that is meaningful is data and they've got that more than any other player save maybe google. Publishing models doesn't erode that moat, and it's not going anywhere as long as facebook/whatsapp/instagram rule "interactive" social.
The only moat that is meaningful is data and they've got that more than any other player save maybe google. Publishing models doesn't erode that moat, and it's not going anywhere as long as facebook/whatsapp/instagram rule "interactive" social.