TikZ: Complicated Flow Chart

To answer the questions in order:

1. Rotating a figure is relatively easy. Add usepackage{rotating} and change the figure environment to a sidewaysfigure one.

2. You can add most things within a TikZ node. For your particular question, I would suggest something along the lines of:

node[text width=5cm,rectangle,draw,rounded corners] (outerNode) {
    begin{tikzpicture}
        node[text width=2cm,draw] (innerNode1) {$A$};
        
        node[text width=2cm,draw,above of=innerNode1] (innerNode2) {$B$};
        
        path[line] (innerNode1) -- (innerNode2);
        path[line] (innerNode2) -- (innerNode1);
    end{tikzpicture}
};

which will result in something along the lines of:

3. Again, similar to the above answer, you can add a lot of different things to a node. In this case I would suggest usepackage{tcolorbox}, and then having:

    node[text width=5cm] (node:1) {
        begin{tcolorbox}[%
            title={Node title},
            width=textwidth
        ]
            Box interior text.
        end{tcolorbox}
    };

which would result in something like:

There are lots of different skins, etc. you can alter the tcolorbox by to suit your styles.

For my answer (and these are just examples on how one could do it), you need

• my positioning-plus library (on TeX.sx),
• my node-families library (on TeX.sx),
• the fit library (loaded by positioning-plus),
• the backgrounds library to draw stuff behind other stuff,
• the calc library for some funny coordinate calculations and the
• shapes.geometric library for the ellipse shape.

Code

documentclass[tikz]{standalone}
usetikzlibrary{shapes.geometric,backgrounds,positioning-plus,node-families,calc}
tikzset{
  basic box/.style={
    shape=rectangle, rounded corners, align=center,
    draw=#1, fill=#1!25},
  header node/.style={
    Minimum Width=header nodes,
    font=strutLargettfamily,
    text depth=+0pt,
    fill=white, draw},
  header/.style={%
    inner ysep=+1.5em,
    append after command={
      pgfextra{letTikZlastnodetikzlastnode}
      node [header node] (header-TikZlastnode) at (TikZlastnode.north) {#1}
      node [span=(TikZlastnode)(header-TikZlastnode)] at (fit bounding box) (h-TikZlastnode) {}
    }
  },
  hv/.style={to path={-|(tikztotarget)tikztonodes}},
  vh/.style={to path={|-(tikztotarget)tikztonodes}},
  fat blue line/.style={ultra thick, blue}
}
begin{document}
begin{tikzpicture}[node distance=1cm, thick, nodes={align=center}, >=latex]
  node[Minimum Width=loop, shape=ellipse, fill=red] (imp-sol)       {ellipsoid box};
  node[Minimum Width=loop, fill=yellow, below=of imp-sol] (rec-box) {rectangular box, and very wiiiiiiiiiiiiiiide2nd line};
  node[shift=(left:.5*x_node_dist)] at ($(imp-sol.west|-imp-sol.south)!.5!(rec-box.north west)$) (for-1) {formula 1};
  node[shift=(right:.5*x_node_dist)] at ($(imp-sol.east|-imp-sol.south)!.5!(rec-box.north east)$) (for-2) {formula 2};
  begin{scope}[on background layer]
    node[fit=(for-1)(for-2)(imp-sol)(rec-box), basic box=blue, header=DMFT loop] (dmft-l) {};
  end{scope}
  path[very thick, blue, hv] (rec-box) edge[->] (for-1) edge[<-] (for-2)
                              (imp-sol) edge[->] (for-2) edge[<-] (for-1);

  node[east above=of dmft-l, basic box=green, header=DMFT prelude] (dmft-p)
    {Math and text math and text math and text math and text math and text math and text};
  node[north left=of dmft-l, basic box=green, header=$rho$ update, shift=(down:y_node_dist)] (rho)
    {Much more text much more textmuch more text much more text};
  node[basic box=blue, header=DFT part, anchor=north] at (dmft-p.north-|rho) (dft)
    {So much text so much text so much textI think I need texttt{tikz-lipsum}or something like that.};
  node[basic box=green, anchor=north] at ($(dft.north east)!.5!(dmft-p.north west)$) (upd) {update$math$};
  path[fat blue line, <-, dashed, vh] (rho) edge ({$(rho.south)!.5!(dmft-l.south)$}-|dmft-l.south west);
  path[fat blue line, ->] ({$(upd.south)!.5!(dmft-p.south)$}-|dmft-p.south west) coordinate (@)
    edge[<-, solid] coordinate[pos=.2] (@s) coordinate[pos=.8] (@e) (@-|dft.east)
    {[every edge/.append style=dashed, vh] (@s) edge[<-] (upd) (@e) edge (upd)}
    (h-rho) edge[dashed] (dft)
    ($(dmft-p.south)!.5!(dmft-p.south east)$) coordinate (@) edge (@|-dmft-l.north);
end{tikzpicture}
end{document}

Output