Thanks @cfanatic, this seems to be the easiest solution and could be done for multiple images at once.
This did work after I made sure, the original was rejected again:
exiftool -all= -tagsfromfile @ -all:all -unsafe xxxx.JPG
It did produce two warnings:
Warning: Bad length ICC_Profile (length 632)
Warning: ICC_Profile deleted. Image colors may be affected