How can I determine how much improvement (dollar value wise) I can put on an urban lot?

So, I’m in the position of considering where (within a given city/real-estate-market) I and my family want to build our next house (which will also be our first experience as homeowners — we’ve lived in houses for some time now, but always as tenants). So far, we know that your typical suburban (new plat/subdivision) lot is out for three reasons:

1. Our current market is not exactly saturated with transit service, and most of what exists is constrained to more urban areas by the realities of transit efficiency.
2. I, personally, have no wish to contribute to sprawl by being a unit of demand for suburban real estate.
3. Builder attached lots are of no use to me, nor are HOA’ed lots (an HOA on a SFR has no upside and significant downside for me)

However, I have been doing some research into both appraisal/valuation in general and into valuations in the areas I am looking at, and that raises the spectre of a killer word for many house plans: overimprovement. In particular, while I’m not looking to put tiger maple and granite countertops in a 1000sf econobox house or build a 5000sf mansion in a neighborhood of 1000sf econoboxes, I have a different problem.

You see, I have no affinity for ordinary "stick" construction, and am strongly considering higher-end structure and envelope techniques as a result. While one can get some money back on envelope performance improvements (better HERS rating) based on the limited research I’ve done, it seems much easier to overimprove on the structural side when building a house than it does in commercial work, where structural build classification (ISO/IBC/M&S construction type) is explicitly accounted for in the valuation process.

This is compounded by the factor that I have no need for a supercomplex floor plan with protuberances and funky angles everywhere, nor am I seeking particularly high-end finishes, fixtures, or amenities. In fact, complex floorplanning is a negative to me, because I am trying to fit the house to urban lot constraints instead of sprawling out over a low-density suburban lot, and that favors simple massing and intelligent use of vertical space.

As a result, the question of "how much house can I put on a given lot?" is a major constraint for me, so how can I figure that out for my real estate market? Furthermore, are there things I can do at this stage to limit the aforementioned overimprovement risk without sacrificing construction? (Say, by being cautious about neighborhood selection, or by facilitating "house hacking" in the design of this house.)