Are there open-source scientific libraries which use modern Fortran with OOP?

I agree with the comment by @JackPoulson. I am not aware of any open source libraries written in modern Fortran. So the answer to your specific question about needing to switch to C++ to learn by example is an unfortunate "yes." But, to be clear, I do know that at least one library exists that was written almost exclusively in modern Fortran with an eye towards OOP, so it is possible (and worthwhile). This library is not open source, though, so it will not help you achieve your learning goal.

However, you are also likely to be reluctant to throw away what you have already completed. If you are still interested in pursuing Fortran for this and have specific development questions, I'd encourage you to ask them on StackOverflow. There are a few highly skilled Fortran developers active there that are more than happy to help.

If you want to learn by example I would recommend rosettacode, a programming chrestomathy site. There you can find many example problems comparatively written in quite a few different languages.

Maybe even more interesting for you would be Fortran wiki, and here is a section on OOP in Fortran which might be useful.

Fortan is doing just fine - let's not give up on it!

There are modern CFD codes which you can look into. For example ...

Fluidity: General purpose multiphase CFD (FE) code; Even does fully unstructured AMR

WRF: Next-generation numerical model weather prediction system from NCAR

Code Saturne: General purpose CFD (FV) code; Some features listed on Wikipedia

There are similar modern Fortran codes for structural analysis.

But they may not use full F2003 capabilities due to lack of widespread compiler support. Most new compilers do support large parts of F2003 (see the ACM SIGPLAN Fortran Forum papers) but it takes time to have a wide user base. Debian stable for example has GCC 4.4.5 so generally speaking you should not expect your users to have anything better (i.e., 4.5/4.6/4.7).

When F2003 compilers become widespread you will see libs/code that use the F2003 capabilities.

For now you can look at this book by Damian Rouson on how to use the OOP capabilities.

I don't know which Fortran dialect it's written in, but there is William Mitchell's PHAML code that is a general purpose Fortran FEM code.

I will admit that I'm not impartial in the discussion of Fortran vs C++ but I do like to point out that you difficulty in finding a modern Fortran-based FEM code while at the same time finding a whole lot of C++-based codes tells something of what the community as a whole thinks is the way to go. There is of course nothing wrong with trying to build your own code in a language you like and are familiar with, but your brief survey already shows you that you're building an island. You may want to take that into account.

I'm a little late on this one, but you should have a look at version 3 of the parallel linear algebra library PSBLAS, which takes full advantage of the modern Fortran features such as inheritance and polymorphism. There's also a paper here where they compared the performance of that version with their older versions, which stuck to F90/F95. Surprisingly enough, the speed difference was negligible overall.

There is the open source fortran library forDat by ffr Engineering which uses oop 2003 features