Access PETSc data in totalview?

PETSc even recommends TotalView as a parallel debugger, so you should be fine using it. However, you must ensure you are using a version of the library with debug symbols in it:

Most code development for PETSc codes should be done on one processor; hence, using a standard debugger such as dbx, gdb, xdbx, etc. is fine. For debugging parallel runs we suggest Totalview if it is available on your machine. Otherwise, you can run each process in a separate debugger; this is not the same as using a parallel debugger, but in most cases it is not so bad. The PETSc run-time options -start_in_debugger [-display xdisplay:0] will open separate windows and debuggers for each process. You should debug using the debugging versions of the libraries (run ./configure with the additional option --with-debugging (the default)).

Even though TotalView is a very nice parallel debugger, more common serial debuggers might serve you better. Especially since the debugging is commonly done on relatively small examples. So, I would use gdb for most of my debugging, unless I am interested in the debugging of parallel features (say some MPI communications I coded up myself).