Understanding the profiler: Is pattern matching/de-structuring expensive?

I was profiling some code and I am trying to understand what the results mean. Suppose we have the very idiomatic,

f[t_, s_] := t - s
Table[f[i, j], {i, 1000}, {j, 1000}] // RuntimeTools`Profile

I get that,

What piques me is the nearly 6 seconds lost between the two arrows. Please correct me if I am reading the profiling results wrong, but I am interpreting it as the 1 million calls to f[i,j] took 10.7 seconds to execute and the 1 million calls to f[t_,s_] took 4.7 seconds to execute. Is that an accurate way to read it? So where did those 6 seconds go?

Changing it to read:

f[{t_, s_}] := t - s
xx = Join @@ Table[{i, j}, {i, 1000}, {j, 1000}];
f /@ xx ; // RuntimeTools`Profile

results in:

which is a two-second improvement. So pre-allocating the array saved some time, but that still doesn’t explain the ~3.7 seconds between the 1 call to Map and the 1m calls to f[{t_,s_}]. There is also a 3 seconds difference between the time to execute the 1m calls to f[{t_,s_}] and 1m executions of t-s. What is that single extra call to f at the bottom?

So what is going on? 1 million subtractions should definitely not be taking that long on a modern PC. So I tried

Table[i - j, {i, 1000}, {j, 1000}]; // RuntimeTools`Profile

and

xx = Join @@ Table[{i, j}, {i, 1000}, {j, 1000}];
Map[#[[1]] - #[[2]] &, xx]; // RuntimeTools`Profile

which one can verify run readily under a second.

So is pattern-matching of the arguments/definitions of f an expensive process or is this just profiler overhead?

EDIT

---

I do understand one case is giving a list of lists and the other is a single list (that is a bit oversight on my part) but I think the essential question remains unchanged vis-a-vis f.