@rjculley -- good question. Training only locks in the parameters of the model, but it does not guarantee that the model that is actually performing the inference on device is the same model. An attack may compromise the model on-device so that it no longer behaves the way you've trained it. Watermarking allows you to "check" the model after the model is trained and deployed.

I worked at a company that worked on a neural network to evaluate the consistancy of a material based on the work used to stir the material. With this we developed coefficents that where loaded into the code after the training process was complete. If my neural network is no longer learning how can there be an attack on the device that would need watermarking?