Should I worry about code plagiarism if it's my code?

If it's patented then it doesn't matter that you independently came up with it. Most software is not patented, though. Most developers do not even think about patents when writing code. In theory you could spend your time keeping up to date on software patents so that you avoid infringing. But this probably only makes sense for specialists who would want to read the latest patents in their field anyway.

All software is protected by literary copyright, though. In this case they will try to prove that you did not independently created it, and did copy it. Whelan v. Jaslow in 1986 ruled that structure, sequence and organization of a computer program were protected by copyright. So you could be liable even if you did not just copy and paste sections of code. But even if you're worried the court would get things wrong and think you copied a program when you never even saw it in the first place, there is really nothing you can do about it.