To replicate...

• Create an Assignment in a course with at least one student
• Enable file upload in submissions and 'Annotate PDF' in Feedback
• Upload a file (either as the student or on behalf of, doesn't matter)
• Go to the grading screen, endure the version for annotation appears and make some marks/comments on the page
• Save grading page
• Go back to 'show all submissions' and note that the annotated version is now available
• Click edit and remove submission
• Annotated version remains

It seems to be that removing the submission should also remove the annotated version.