Substituting oil in baking?

I use extra virgin olive oil in cakes all the time because that's usually all I have on hand it it works just fine. Never tasted a difference and never had anyone else taste a difference. I expect it'd be the same with peanut or canola, which don't even have a strong fragrance like olive oil.

You probably would not notice much difference substituting peanut or canola oil for "vegetable oil", because of the amount of sugar in a cake. On the other hand if you substitute with extra virgin olive oil you will notice a difference, not so much in the flavour, but in the texture. Unlike the other oils mentioned EVOO contains natural mono- and di-glycerides, which are emulsifiers which will give the cake a fine, dense crumb, and help it to hang onto moisture, so that it remains fresh longer. There is a long tradition of using EVOO instead of butter in cakes in places like Italy and France.

As an aside, in the case of pie crusts, which do not contain sugar, I have noticed a flavour difference between oils (if you have not tried making pie crust with oil, do yourself a favour and give it a try; they are super easy). Avocado oil has a slight fruity flavour, and canola had a slight fishy flavour (maybe because it is high in Omega 6) which I find unpleasant. My favourite is peanut oil which has a rich taste (highest in saturated fat) and seems the closest to butter. Unfortunately I did not include EVOO in my side-by-side testing but I am guessing it would produce a very tender crust similar to ones made with shortening, since they both contain mono- and diglycerides.